Online reviews are a vital part of how consumers pick hotels, but the rating system used by TripAdvisor encourages small hotel owners to game it with fake reviews, says an academic study.
Published last week, the study looks at a half-million reviews posted as of October 2011 for 3,082 hotels cross-matched among TripAdvisor, Expedia, and Smith Travel Research, with additional cross-corroboration on Orbitz. You can read the study, here, as a PDF.
The researchers write:
"The net gains from promotional reviewing are likely to be highest for independent hotels that are owned by single-unit owners and lowest for branded chain hotels that are owned by multi-unit owners.
Furthermore, we show that the hotel neighbors of hotels with a high incentive to fake have more one- and two-star (negative) reviews on TripAdvisor relative to Expedia."
The empirical analysis focuses on just two travel websites: TripAdvisor and Expedia.
Anyone can post a review on TripAdvisor, while a consumer may only post a review of a hotel on Expedia if he or she actually books at least one night at the hotel through the agency. (Expedia spun off TripAdvisor last year.)
Study says: Watch out for the franchised properties of name brand hotel chains
The study focuses on hotels that may seem the same to the average traveler yet actually have different ownership or management.
For example, the study says regional franchisees and independent owners have an incentive to artificially boost ratings for their own hotels.
Small owners are about 10% more likely to receive five-star reviews on TripAdvisor than they are on Expedia, relative to hotels owned by large corporations.
The researchers also claim the owners have an incentive to post bad reviews on hotels owned by competing regional hoteliers. The study found these owners have 16% more one- and two-star ratings than those with hotels owned by large companies located up to half a kilometer away.
A hotel that is located next to an independent hotel owned by a small owner will have 5 more fake negative reviews compared to an isolated hotel -- on average.
The authors estimate that, on average, "an independent hotel owned by a small owner will generate 7 more fake positive reviews (out of 120)."
The authors say they found "relatively more positive manipulation than negative manipulation, even though the order of magnitude of the two is similar."
The authors of the study were Judith A. Chevalier (Yale School of Management; National Bureau of Economic Research); Dina Mayzlin (USC Marshall School of Business); and Yaniv Dover (Tuck School of Business, Dartmouth College).
TripAdvisor responds
TripAdvisor's Kevin Carter responded to the study by e-mail:
Anecdotally, we know from TripAdvisor travelers that the personalized service, value and quality
that certain smaller properties offer resonates with them, as is often expressed in their reviews and opinions.
We take the authenticity of our reviews very seriously and have numerous methods to manage the legitimacy of the content on TripAdvisor, including automated screening tools that are constantly upgraded.
A team of quality assurance specialists investigates suspicious reviews which are flagged by our proprietary tools or the passionate TripAdvisor community of more than 50 million monthly visitors.
We know that the sheer volume of reviews on TripAdvisor allows travelers to get the complete picture of a property, and make an educated decision based on the opinions of many, before they book their trips.
According to a recent PhoCusWright study commissioned by TripAdvisor, 98% of respondents have found TripAdvisor hotel reviews to be accurate of the actual experience.
It’s because of the valuable advice on TripAdvisor that travelers keep coming back.
Note: The July 2011 PhoCusWright study is not available to the public, but was a survey of 3,641 respondents solicited at random through a pop-up invitation link on TripAdvisor's US site.
TripAdvisor made a broader case for its reviews in its post on its company blog last March.
In April Robb, senior manager of content at TripAdvisor, told Tnooz:
"The majority of reviews on the site are positive. Three-quarters of all TripAdvisor reviews are rated “very good” or “excellent”, and the average rating on TripAdvisor is just over four out of a possible five."
To put that statistic in context, TripAdvisor's reviews are somewhat less positive on average than Expedia's, where only users who book rooms are allowed to review properties.
The academic study claims that "hotels with a high incentive to post fake reviews have a greater share of five star (positive) reviews on TripAdvisor relative to Expedia."
Expedia responds, too
Given that Expedia's reviews were also examined in the study, we asked the company for its view. Sarah Keeling, director, public relations and social media at Expedia, said by e-mail:
"We believe that consumers put a lot of trust in us to provide them with the best information possible when booking their trip.
That's why Expedia reviews are vetted and verified. Expedia takes multiple steps to ensure the validity of customer reviews, including the step of ensuring that the reviews are posted by travelers who have paid for a room in the hotel they are reviewing."
What else could TripAdvisor and Expedia do? Another new study suggests an option
Could TripAdvisor and Expedia do more to battle fake reviews? Possibly. This summer, researchers revealed how math can be used to detect false reviews.
Researchers at State University of New York, Stony Brook, developed a formula that can detect when fradulent reviews distort the standard statistical distribution for hotel scores.
Translation: The statistical technique can't tell if an individual review is fake, but it can tell if there is a high likelihood that some of the reviews of a particular hotel are fake. A computer could flag a hotel for having a non-standard distribution of star ratings for a manual review.
Researchers said they found fraudulent reviews 72% of the time. (The study can be read here.)
TripAdvisor hasn't said if it will adopt this specific method of mathematic sleuthing -- which was first reported in Technology Review -- or any similar investigative method.
TripAdvisor's Kevin Carter, manager, business & trade public relations at TripAdvisor responded to the second study by e-mail:
TripAdvisor uses frequently evolving and highly sophisticated filters (more than 25) to scan reviews for biased material, monitoring a wide range of attributes associated with electronic correspondence.
Suspicious activity is then flagged for further inspection by our team of dedicated agents who use a variety of additional confidential investigative methods designed to identify potential fraud.
So, for more background, we have three primary methods to manage the legitimacy of reviews:
1. Systems. Reviews are systematically screened by our proprietary site tools that are frequently upgraded.
2. Community. Our large and passionate community of more than 50 million monthly visitors help report suspicious content.
3. Quality Assurance Teams. An international team of quality assurance specialists investigate suspicious reviews that are flagged by our proprietary tools or community.
Also of note, in April, TripAdvisor did appoint Andrew Marane to the new position of Director of Content Integrity.
There's no magic cure for policing online reviews, no technique to prescribe that will be foolproof.
In the meantime, the US Federal Trade Commission has stepped up its investigations into companies paying for positive and negative reviews online in violation of national regulations.
The Advertising Standards Authority in the UK upheld a complaint against TripAdvisor from online reputation firm Kwikchex and two hotels in January this year.
The complaint argued wording on TripAdvisor’s UK website, such as “reviews you can trust…from real travellers…and trusted advice from real travellers", was misleading because the company could not prove reviews were genuine or from real travellers.
NB: Image of villain with laptop from Shutterstock.