In early April, SimilarWeb, a traffic analytics startup, revised its algorithm.
The effects are only now apparent. Web visitor estimates have been slashed by roughly half for many consumer content travel websites, among others.
The advertisers and investors who rely on this data for competitive analysis may be disappointed by the newly revealed, actual performance of many sites.
Exhibit A: Yahoo's U.S. travel portal now appears to be down 52% in monthly visitors. In March, it had 4.9 million visitors; in April, 2.8 million.
That apparent drop is a mirage. Yahoo Travel was never that high to start. But the algorithm only corrected the latest numbers, making it appear that there was a drop.
Here are SimilarWeb's estimates of the number of monthly website visits via desktop between March (old) and April (revised).
Who cares?
SimilarWeb is one of several traffic analytics services, such as Compete.com, Quantcast, Alexa, and (without public free versions) ComScore and Experian Hitwise, that measure traffic to websites.
It has been considered by some travel industry commentators to be one of the least inaccurate of the traffic-measuring services. Its data comes from Internet service providers, its web crawlers, and "the largest user panel of any web measurement service", according to the Israeli startup, which has received $40 million in investment.
Who uses SimilarWeb and its rivals? Advertisers, venture capital firms, destination marketing companies, ad networks, and enterprise companies subscribe to analytics services to get details on the marketing strategies, mobile and website traffic, and investment performance of competitor sets within digital verticals like travel.
For instance, while an advertiser may request a website's internal analytics data, third-party services can put that internal data into the context of competitors' relative performance.
No clothes
Like a receding tide that reveals some swimmers are naked, SimilarWeb's recalculations suggest that some travel content sites may not have been doing as well as was previously suggested.
The algorithm revision hit consumer travel content sites disproportionately.
Large ecommerce sites like Expedia, Booking.com, Kayak, and TripAdvisor saw no drop during the six months.
Smarter Travel, a consumer travel content website owned by TripAdvisor, has seen its overall level of traffic, as recorded by SimilarWeb, fail to rise on a rolling average -- whether you include mobile traffic or not:
A source at Smarter Travel said off the record that its internal data reports an overall rise in the site, on average, instead. That seems plausible.
Historically, April and May have been growth months, traffic-wise, for US travel content sites, as that's when American travelers plan for their summer vacations.
Tnooz spoke to managers of a half-dozen travel content websites, all of whom said off-the-record that the apparent drops were wildly inaccurate. All but one said they were seeing year-on-year improvements in traffic.
The confusing thing about the SimilarWeb update is that it makes the charts seem as if many travel sites saw their traffic drop off a cliff. To repeat: That's not the case.
The confusion could be cleared up if the old data was revised downward. But the company has made conflicting statements about whether it will revise the older data.
Complicating the picture is that Google is believed to have introduced an algorithm revision around the same time, nicknamed Phantom 2.
We asked SimilarWeb if the sharp drops could be attributed to Phantom 2 instead. The answer from the spokesperson:
"The changes are due to the SimilarWeb algorithm change."
What has changed in SimilarWeb's algorithm? Is it now better able to filter out spam/fake/bot traffic, for instance?
"Not explicitly. However, the new algorithm provides higher weights to traffic sources that are more reliable, hence it's expected to contain less spam traffic."
The smaller the site, larger the margin of error
The analytics services vary in methodology and in the type of data they make available for free.
Compete says that Smarter Travel's traffic has increased in a hockey-stick curve, more than doubling since the winter, from 479,000 to 1.1 million unique visitors in mid-May.
Quantcast also estimates "the number of people accessing the property via desktop and mobile web in aggregate". By that measure, it says Smarter Travel had 400,000 visitors in mid-May -- with a temporary bump to 600,000 unique visitors in March.
So Compete's and Quantcast's data on unique visitors appear to be contradictory. It seems curious that unique visitor numbers would increase in a hockey-stick pattern while overall visits would be flat except for a mid-March bump.
Murkiness like this is common with smaller websites. The web analytics services tend to rely on panel measurements that study the behaviors of large groups of Internet protocol addresses — a statistical method that loses its accuracy when measuring smaller companies.
For large sites, the services do better. For instance, all three services have roughly identical patterns describing the traffic of giant consumer sites. SimilarWeb's estimate of Expedia's traffic dovetails with Quantcast's.
Yet even then there can be discrepancies. Quantcast says Expedia's flagship US site has about 6 million US unique visitors a month. Compete says it has between 14 and 16 million people visiting Expedia.com monthly. SimilarWeb counts 43 million visits per month, which includes repeat visitors.
Estimates tend to be limited to, or only robust in, the US — a problem for travel websites with global audiences.
The estimates also tend to be less accurate around app-based or mobile traffic, which is where the traffic growth is instead of desktop.
In their defense, the services invite companies to feed in their Google Analytics data or, alternatively, to place javascript code on their sites for using cookies to track visitors and correlate those numbers with its panels.
But travel companies have been skittish about making such data publicly available.
Quantcast has had the most success in getting household name publishers, such as Gawker, to feed their real data to it. But we couldn’t find any significant travel brand that had done so.
What's an advertiser to do? As one former proprietor of a content site said: Advertisers at least can fall back on one reliable metric: how often their ads are loaded on pages.
What about Google Analytics?
Internal data derived from Google Analytics is still seen as the gold standard for web data. Code sits on every webpage produced, reporting if the page has been viewed and any details about the interaction, such as the device used to access the page.
Google Analytics also the gold standard because it sits within Google Webmaster tools, the most widely used tool set for web functionality.
Yet Google Analytics's conclusions can be wildly off if it's not set up correctly. It also calculates metrics in particular ways that may not reflect data that other services calculate.
In short web analytics sites can be directionally correct, if not precisely correct on actual numbers for any given site.
As ever, when it comes to third party analytics, remember Ronald Reagan's slogan: Trust, but verify.