If you are in the online travel business, you’re a bot target. That’s pretty much a fact of life today - airlines, hotels, rental car companies, and online travel agencies are fair game.
NB: This is an analysis by Courtney Brady, director of marketing at Distil Networks.
If you’ve seen our webinar on web scraping, you know that it’s easy for competitors to scrape prices and promotions off your website, run them through a filter, shave a few points off, and repost them as their own.
It’s all automatic, and the tools are easy to buy or build, so the barrier to entry for these guys is extremely low.
They get to rise above you in the search engine results and steal your business from price-conscious travelers and you probably don’t even know it’s happening.
DreamCheaper - a new breed of botdriven threat
Here at Distil Networks , we spend a lot of time chasing down bad bots and blocking them from websites. We’ve seen it all. But even we were taken aback when we came across DreamCheaper, which has just emerged from stealth mode.
Their basic approach is similar to other web scraping bots, but with an added twist: the company specifically targets the flexible booking options offered by hotels.
What DreamCheaper does is to cancel travelers’ hotel bookings once its algorithms have found a cheaper equivalent price at the same hotel and makes a replacement booking – and it keeps doing this until cancellation penalties start to kick in.
The company makes its money by taking a percentage of the savings delivered to the customer over the cost of the original reservation.
The targeting and persistence of this approach pretty much ensures that hotels will lose business on a global scale (although DreamCheaper executives claim otherwise – see TechCrunch’s assessment ).
Answer these questions to gauge the impact bots have on your bottom line
Bots can be programmed to do almost anything their “master” wants, so it’s definitely helpful to have some expertise of your own to defend your online travel business. Ask yourself these questions to assess the impact on your site:
- Those spikes in traffic coming through your online ads are they genuine potential customers or click fraud
- How much is it costing you to provide the infrastructure to support malicious website traffic?
- How much is your site performance being degraded by “bot storms” that aggressively scrape your site or overwhelm your registration system with spam?
- How much revenue are you leaving on the table by letting competitors scrape your website instead of entering into an API licensing agreement?
Bot detection and mitigation requires a purpose-built solutionThis is not the kind of solution that can be delivered as a feature on a Web Application Firewall (WAF).
A typical WAF will catch maybe 20% of the bots hitting your website, because it’s not doing much more than IP address blocking and user agent validation. IP address blocking also has the disadvantage of blocking legitimate users that might be sharing the same IP addresses as the bots.
The advantage of a purpose-built solution such as Distil Networks is our ability to accurately identify human, good bot (e.g., Google and Bing) and bad bot visitors across your website.
How does Distil Networks accurately detect the bad bots?
Our inline fingerprinting technology sticks to the bot even if it tries to reconnect from random IP addresses or anonymous proxies. We combine this with behavioral modeling of your human traffic to detect anomalies.
All of this data goes to create the world’s largest Known Violators Database, based on the collective intelligence of all Distil protected sites, which is updated in real time to deliver 99.99% protection against bad bots without affecting legitimate users.
Whether you’re a startup or a Fortune 500 business, we’d be happy to show you what’s crawling under the surface of your website with an extended free trial (deployed in cloud or as an appliance) to determine the vulnerability of your online travel website.
It’s fast and simple to set up, won’t interfere with your business – and will safeguard your investment against the likes of DreamCheaper.
NB: This is an analysis by Courtney Brady, director of marketing at Distil Networks. It appears here as part of Tnooz's sponsored content initiative.
NB2: Distil Networks hosted a TLearn Webinar in February titled " How to clean up travel website traffic from bots and spammers." A recording of the webinar is available. Click here to register for access
NB3: Image by Shutterstock.