Leading technology advisors are urging
hoteliers to open up to artificial intelligence (AI) search platforms by giving
greater access to their crawlers and bots.
Hoteliers should be adding new types of
unique content too, as large language models (LLMs) accelerate as a source of
direct bookings.
Hospitality companies are in a favorable
position in the new “zero-click” environment—where results appear in generative
AI summaries on Google or an LLM search such as ChatGPT. A recent Bain study
found 80% of consumers rely on zero-click results at least 40% of the time.
While opening up to crawlers can boost
visibility and potentially direct bookings, questions remain over security and
compliance risks and the longer-term benefits as search-and-book habits evolve.
Access all areas
As Propellic’s
Brennen Bliss has previously noted, LLMs are changing how
the web is indexed, processed and served to users. His advice? “Serve a
crawler-friendly version of your site.”
AI crawlers scour websites and follow
links, read publicly accessible content, store information for analysis and
generally use what they find to train their LLMs and make sense of the world.
It’s different to Google’s Googlebot, for example, which indexes content for
search results.
Websites often have a robots.txt file in
place, which are publicly available files that tell web crawlers and bots what
exactly they are allowed (or not allowed) to access on that site.
Ira Vouk, founder of Hospitality 2.0
Consulting, said larger hotel groups are not open
enough to these AI crawlers, leaving franchisee properties underexposed.
That was her conclusion after analyzing
several hotel chain websites. In a recent LinkedIn post, she argued these chains were “blocking most AI bots in your
robots.txt, which makes you practically invisible on all major AI platforms.”
These files can be viewed by adding
robots.txt to the end of a domain name, like https://www.hilton.com/robots.txt
or https://www.marriott.com/robots.txt.
Although a rudimentary analysis, it has
stimulated debate: Just how far should hotels open up to crawlers?
Vouk believes online travel agencies (OTAs)
such as Expedia and Booking.com are more receptive to crawlers and so have an
advantage.
“Hotels are losing direct bookings right
now because they're not allowing AI bots,” she told PhocusWire. For example, if
someone asks ChatGPT about a specific property, Vouk reckons the bot is more
likely to source information and a link from the likes of Expedia or
Booking.com.
“They're really not showing up in the
conversation,” added Pedro Colaco, president and CEO of GuestCentric.
“And that is very bizarre, that they're leaving this to third parties.”
PhocusWire contacted Hilton and a number of
other hotel groups for comment.
An international spokesperson for Choice
Hotels said it is working with its content distribution network provider to
allow LLM agents to search its sites.
“We’ve also working with LLM providers to
ensure that our sites are visible and searchable on their platforms. We pride
ourselves on being industry innovators and embrace technologies that meet
travelers wherever they search for hotels to help our franchisees succeed.”
Line of questioning
Another strategy hotels can adopt to grab
the attention of crawlers and boost visibility among AI platforms is to
incorporate unique information that OTAs are unlikely to feature.
While adding a frequently asked questions
(FAQ) section can boost a website’s organic traffic, one expert suggested
creating an “IFAQ”—or infrequently asked questions section.
“You should be looking at infrequently
asked questions that only you at the hotel can answer,” said Sanjay Vakil, CEO
and co-founder at DirectBooker, a PhocusWire Hot 25 Travel Startup for 2026.
“We've typically gone down this (FAQ) list
and cut it off and said: These questions are no longer important, they are too
long tail,” he said during a recent webinar hosted by direct booking platform Triptease.
“I would encourage people to flip that
around. Long-tail queries should be recast as high-intent queries. You can make
that information available to them easily within an AI. [Visitors] are going to
be more likely to book.”
For example, someone may want to know
whether the hotel’s pet policy has a weight limit.
“That's a reasonable question,” Vakil said.
Other examples include the temperature of the Jacuzzi.
Handle with care
Granting access to such granular, and
sometimes sensitive, data may optimize searchability, but large organizations
have their reasons to restrict or reduce access to AI bots.
This can include preventing unauthorized
price scrapping, content copying and competitive intelligence gathering,
according to Olivier Delaunoy, a hospitality technology expert and former CTO at
Bloc Hotels.
“Robots.txt files for groups like Hilton
and Marriott are only the visible surface,” he said. “Large hotel brands
typically run multiple layers of bot management behind the scenes to block
unwanted scraping.”
Meanwhile, disclosing bot-filtering methods
in publicly available files effectively helps malicious systems work around
them: “A robots.txt file may look simple, but operational security drives the
rest,” Delaunoy said.
He added that key players likely have many
secondary URL and related microsites, franchise sites, independent hotel pages
and older domains still indexed. “These can be discovered by bots even if the
main root domain blocks them, leaving them open to AI crawlers,” he said.
However, he admitted visibility can be
reduced by blocking AI-driven models, notably because modern AI platforms do
not behave like traditional web crawlers.
According to reports, even ChatGPT’s can
now pass the “I am not a robot” verification test, and Amazon recently sued Perplexity over its agentic shopping tool,
arguing the AI company disguised automated activity as human browsing.
“The best practice is to tailor the hotel
bot strategy to block harmful scrapers and allow for beneficial AI crawlers and
engage with friendly and approved data-sharing programs, where the hotelier can
control accuracy, pricing and brand image,” he added. “Detailed filtering is a
skill, and everyone is applying different filters.”
The challenge for hospitality companies is
knowing how to detect LLM crawler activity and identify content optimization
opportunities. Further ahead, the travel industry needs to work out new types
of visibility metrics to assess effectiveness.
Reputation is everything
At the same time, not all businesses are
keen to open their doors to AI bots. In July, Internet security provider
Cloudflare defaulted to blocking AI crawlers for new domains and introduced a pay-per-crawl model, said Felix Shpilman,
CEO of Emerging
Travel Group, which owns the RateHawk, ZenHotels and Roundtrip
brands.
“This meant property owners, whether large
chains or independent hosts, had to manually adjust their settings to allow AI
bots access. Many hoteliers are likely unaware of this default setting,” he
said.
He also said that while AI platforms
can serve as a “promising channel” for direct distribution, they carry a high
risk of unauthorized reselling by third parties. “To mitigate this risk, many
hotels prefer traditional, controlled channels such as bedbanks and DMCs, which
offer clear terms and established agreements,” Shpilman added.
Some media companies have also blocked OpenAI’s GPTBot web crawler from accessing their content.
And will “screen scraping” ever ditch its
bad reputation? Only a year ago, a U.S. court ruled Booking.com had violated
the Computer Fraud and Abuse Act in a case brought against the OTA by Ryanair, which claimed Booking.com “screen scraped” its website
without permission.
Chase ‘fit and context’
Among all the buzz around AI optimization,
GuestCentric’s Colaco offered a word of caution. He feels this new world of
search and discoverability is in the phase that SEO was in the beginning,
where people were writing keywords in transparent fonts at the end of pages.
“Just like people started to search on
Booking.com or on Google, now they start on ChatGPT or Gemini. And it's really
a discovery tool, so we should feed it as much information about our hotels as
possible,” he said.
As a result, he advises hotels to get back
to the basics and tell stories.
“If hotels are telling their story well on
their website, everybody that talks about them, whether it's the media or user
reviews, if they can surface what the experience is at the hotel, then they'll
show up,” he said.
He cited one hotel client, Memmo
Alfama in Lisbon, Portugal, which had seen a drop in website visits
but an increase in conversion and direct bookings.
“I said they should go back to the original
strategy when they launched the hotel, which was about 10 years ago. It was
about the neighborhood and its artisans,” Colaco added.
“We're going back to that, where people are
going to bring in the overall neighborhood experience into the story and tell
that story well. Because people just want to know: What am I going to
experience? Travel starts with the discovery piece.”
It’s difficult to find universal agreement
on the right strategy regarding AI search, but most will agree that a new
chapter is in the process of being written—one that doesn’t rely on traditional
search behaviors.
As Barry Diller, chairman and senior
executive of IAC and Expedia Group, noted at The Phocuswright Conference in November, “Now
that AI has a chance, that search monopoly experience of Google is going to
end. What its consequences are to Google, I can't say.”