Aerospike (the fastest read/write database that operates in-memory, on flash, as a hybrid and on disk as well) is a huge deal in the travel sector.
NB: This is part one of a two-part viewpoint from Max Rayner, a partner at Hudson Crossing. In this article he assesses some of the technologies in the market and how they affect travel distribution, particularly hotels.
The technology is now open source under the Free Software Foundation’s AGPL License, while its client software SDK became available under the Apache license.
Why is it so important in our sector?
This is mainly because it presents a modern open source alternative to ancient and fairly expensive tech that has powered a number of caches in legacy travel infrastructure… but also sucked the lifeblood out of many a company.
At the same time, it provides hot read/write performance far exceeding open projects in common usage such as Mongo and Cassandra, often by a factor of 10 or more.
The good, the bad and the ugly… and not necessarily in that order
As part of our advisory work at Hudson Crossing, we routinely suggest to clients that the increasing demands for dynamic rate marketing, continuous rate management optimizations, cross-sell and personalization create a problem that is best addressed with modern internet-scale tools.
Tools that can disappoint in this new world (either in technical performance, development efficiency or cost-effectiveness terms) can be broken down to legacy big-software stacks and various open source components.
Among the big-software stacks:
- Oracle’s ATG + Endeca + Coherence and SAP’s Hybris + Solr + HANA. These Product Information Management (PIM) + indexing + in-memory tier options are nice for static e-commerce inventory where prices change fairly slowly… like grocery stores and physical goods e-commerce sites, but can otherwise be expensive and clunky to tune, even when the PIM is not included.
- While it is possible, and some would argue "safe", to build a large-scale solution with them, we’ve never seen true dynamic read/write travel projects use one of these and fail to disappoint, be late, or escalate the bill into several million dollars in licenses alone, before customization piles on about double the cost.
Among the open source components: Cassandra, Mongo, Couchbase, Redis:
- These are a bit like a complex Swiss knife. If you know what you’re doing they could do the job, but most people end up cutting their fingers instead. And those whose companies succeed and catch explosive growth find limits much sooner than they’d like.
- One of the aspects of using these that is often under-appreciated is the cost of hiring and keeping a high quality team that can take fairly raw ingredients and add the enterprise features needed for long term success. This is why we see examples of other companies such as MongoDB Inc. (formerly 10gen) and DataStax succeeding in the "commercial open source" model.
One alternative to both big-software legacy stuff and to other open source components that has proven itself under the most stringent circumstances is Aerospike, which has had success among clients with truly challenging loads.
Among its endorsers you’ll find some of the largest real-time ad bidding ecosystem platforms: AppNexus, BlueKai, SnapDeal, eBay, and many others.
Up to now, the main objection to Aerospike’s use had been its code not being open source. Now that’s dispensed with, and this opens the door for a lot of potential uses in travel.
Here's a few examples of why dynamic pricing and inventory in travel (coupled with lopsided look-to-book ratios) is an area calling for a core technology like Aerospike (both for its lightning speed and the economic advantages of being able to deliver that speed from direct addressing of flash instead of more expensive DRAM.
It may be helpful to consider examples in travel (with names withheld to protect the guilty and with slightly obfuscated numbers to protect the innocent).
How does this affect a distributor of travel goods?
We have seen cases where companies were routinely victimized with more than 150,000:1 look-to-book ratios. You might well say "Quoi, mais c’est pas possible!" (What, that can't be possible!)
The complete exploration of the reasons why is a long story for a another day but here’s a short version:
Whether it’s air, hotel, car rental or other services, end users don’t ask for a price just once, but many times, in many internet storefronts and from many devices.
All those storefronts, desperate to get the sale, are in turn shopping one another to verify their price and availability position relative to competitive distributors and to the original travel brands themselves.
In addition, analytics/BI companies are shopping everybody… you get the picture.
Additionally, while 150,000:1 sounds ridiculous, look at it another way: suppose that on a per-destination basis you are at the receiving end of 7500:1 in broad based requests for a destination. In a destination where at least 20 hotels out of 200 require individual pull searches you would metastasize into (20 * 7500 + 1 bulk local search) or 150,001:1.
Clearly, a good case for heuristically optimized caching, but that also means the ability to be updating a very large cache with ongoing reads and writes.
At this point you might say, yes, but how is an airplane ticket or a hotel night different from buying a toy on Amazon or bananas from Walmart?
Part of the difference is this: PIM-driven e-commerce is relatively tame:
Amazon knows its stock position in most items, and also the related supply chain capabilities.
The price is for all practical purposes fixed or changing rather slowly and therefore everything about a specific product can be cached: descriptions, prices, availability.
Even better, as stocks deplete more copies can easily be made, just as more bananas can be ordered.
Hotel beds and airplane seats are a different matter:
For starters, having the bed or seat on the appointed time and day is crucial. So the commodity is as perishable as each ticking second and date-specific availability can change very fast.
Additionally, hotel beds and airplane seats are revenue managed dynamically, and in fact increasingly so. From proven existing leaders like IDeaS to startups like Duetto everybody is taking revenue management from overnight batches to a real time dynamic world.
To make the implications simple, let’s compare a couple of simplified and indeed simplistic uses cases in physical goods e-commerce vs. hospitality.
Traditional e-commerce outfits, even gigantic ones like Amazon which might serve up prices and availability on something like one billion products worldwide, set a price and let it ride for a while. Even if the stock vanishes, Amazon can put up a note inviting users to per-order for a future date, and nobody thinks much of it.
The net of it is that everything in this case is eminently cache-able. Pricing and availability can be cached, and when a purchase happens, decreasing availability is not rocket science.
Hospitality is dicey and full of under-appreciated complexities
For one thing a lot of distributors are not selling their own inventory, nor do they have fulfillment control over other people’s inventory. So distributors end up having to handle multiple properties either directly or indirectly through brand level systems.
One single individual hotel may have 200 rooms broken into five room types (such as king, two queens, studio, junior suite and suite). Additionally it may have defined 20 rate plans (such as AAA discount, corporate negotiated rates, weekend deals, advance purchase rates, rates with bundled amenities such as Wi-Fi or breakfast or both, etc).
Then, it may have stay pattern rules affecting prices and availability depending on the days or arrival and departure, and other rules around number of occupants, the applicable taxes depending on location and so on.
Our friendly travel distributor is dealing not just with that hotel, but also with all the possible hotels in each particular destination.
So every time a shopper asks about price and availability in NYC for two people from August 1st to 5th, 2014, you have 600 or more hotels involved with not only constantly changing availability but also constantly changing prices which have to be exploded into all the room type * rate plan * stay patterns permutations.
NB: This is part one of a two-part viewpoint from Max Rayner, a partner at Hudson Crossing. Read part two: Airline availability and solving the travelling salesman problem.
NB2: Disclosure - the author has a minor equity stake in Aerospike.
NB3: Online search image via Shutterstock.