The airline industry is tightly braced for the ramp-up of
air travel as the warmer months approach. And it needs every penny of revenue
to compensate for colossal pandemic-inflicted losses.
In the effort to tackle that challenge, data-scraping bots that help operators
streamline a massive real-time database of travel listings to optimize
competitive pricing could be on the chopping block. They have taken a
considerable toll on airlines, many of which have very humble profits on the
actual flight tickets themselves. Most airlines’ revenue comes from in-flight
upgrades and affiliate commissions on related bookings made through the airline
website, such as car rentals or hotel accommodations.
The problem with eliminating bots, however, is they actually help travelers
save by optimizing the booking process, making it easier and cheaper to find
the deal most suitable for them.
Travel bots: Can’t live with them, can’t live without them
Despite the turbulence of the last few years, the travel industry remains a
highly competitive business environment. And like many others, the industry has
come to view data as a form of currency that holds the key to driving sales,
reach and value.
It is perhaps to the detriment of the industry that online
travel agencies such as Expedia, Kayak and Skyscanner realized this
better and earlier than most, becoming the predominant force in today's travel
For most airlines, OTAs distribute and sell their flights, minimizing the
losses on low occupancy. To do this, OTAs and aggregators scrape data about
flight information from an airline under agreed-upon terms, in exchange for
Subscribe to our newsletter below
To scrape the data, OTAs use an automated script to access
the airline or booking engine through an application programming interface
(API), which is usually left open by the airline. Such scripts are otherwise
known as bots and generally account for more than 90% of the
traffic on an airline’s website.
Consequently, tickets aren’t booked directly with the airline, but with the
OTA. The airline ends up sharing its negligent profit with the OTA, paying it
an affiliation fee. Consider the absurdity for a moment: the airline pays the
OTA for using the airline’s own data to make travelers book elsewhere.
And since OTAs don't need to share data with airlines until check-in, they control
much of the leverage if any changes occur throughout the customer process,
while the airline is usually, and ironically, the one who ultimately shoulders
the blame for customer travails.
This abundance of bot traffic creates another conundrum - it
skews important business metrics designed for airlines to understand website
and business performance, such as the look-to-book ratio, which is defined as
the number of times a flight is requested per reservation made. But surely OTAs
and airlines have combed out a better equilibrium by now, right?
Not exactly. As it has evolved, this open, scrape-friendly approach to driving
business has exposed airlines to exploitation by both OTAs and malicious
attackers alike. Collectively, from unauthorized scraping and seat spinning to
loyalty programs’ misuse and the coordinated mass bookings of flight seats,
bots create a significant headache for airlines. In 2017, only one industry – gambling
- had a higher
proportion of bad bots. And that’s before we even mention the struggles with
The question, then, is what to do about these bots.
Traditional bot mitigation does more harm than good
Technology has gradually caught up to provide platforms with increasingly
robust bot mitigation approaches, which wasn’t always the case.
Initially, security inspections like CAPTCHA were
implemented with the aim of separating the human from the bot. Yet over time,
these implementations have arguably done more harm than good, particularly with
the user experience. Impatient customers began aborting purchases out of
frustration, due to tedious security checks, while the users who make it
through the CAPTCHA are often still, it turns out, bots. Ironically, they even
The need for a better equilibrium
Despite the minefield they create for security management, bots serve a crucial
role in streamlining customer-friendly processes. As much as the industry
reviles them, it relies on them in equal measure. The market needs a more
harmonious equilibrium where humans get the most convenient and frictionless
service and malicious bots are gradually weeded out - but not to the point of
suffocating the OTA businesses.
And the signs are encouraging. Today’s bot mitigation approaches and
technologies are beginning to identify bad bots through improvements in
detecting abnormal or suspicious behaviors. A promising strategy is to focus on
the way the user is interacting with the web page and sending additional
checkup challenges to users who display suspicious patterns. This is a good way
to tell humans from bots, which is a prerequisite from telling good bots from
Intensifying the focus on the user’s device can also prove valuable. Telemetry
analysis can identify bot emulators designed to simulate real users more
succinctly, while indicators such as whether a device has been “jailbroken”
flag suspicious or illicit activity.
This high-definition device fingerprint, instrumental in
helping us understand certain characteristics of the user’s browser (header
requests, browser extensions, use of graphic libraries) is becoming harder and
harder for bot operators, or emulators, to spoof.
Ironically, one of the most difficult things for these state-of-the-art bots to
achieve is to emulate the randomness of real human users. Take cursor motion
for example: A human pattern won’t ever have perfect straight lines, while bot
patterns are all about those. Analyses of key presses and mouse movements are
making real progress in this regard, but there’s still a long way to go.
Striking the right balance between safety and friction has traditionally been a
game of give and take, but by using advances in technology and detection,
platforms can start forging a path that takes the best of both worlds.
This would be a most welcome move, since the airline
industry has for too long pushed customers to the brink. As the relationship
becomes healthier, such practices like airlines capitalizing on “calculated
misery,” where they make their baseline products and services so
low-quality and unpleasant that people will pay more to avoid them, can
fortunately begin to dwindle.
For the airline industry, let's not forget the importance of safeguarding
websites, mobile apps and APIs from malicious bots that love to disrupt its
business. More importantly, let's remember that bots, whether bad or not, do
not corrupt the ecosystem enough to the point where they damage both the
airline and the customer. With the right innovation, travel and bots can
About the author...
Dr. Eduardo Rocha is senior
sales engineer and security analyst at GlobalDots