For years TripAdvisor has kept how it vets its reviews a closely guarded secret, prompting calls for verified reviews and column inches from the media on supposed mistreated hotels.
Now, the review giant is shedding a little more light on how it keeps the boosters, vandals and optimizers at bay.
The company feels strongly about the verified reviews route and says it believes its scale of the reviews is what maintains its content integrity and that verified reviews are still open to manipulation.
The process it takes follows a general pattern - review is submitted and goes through an automated process with about 50 filters for integrity and moderation issues.
It then falls into one of three paths:
- no issues are identified and it is published.
- definite issues are identified and it is rejected e.g. for profanity - the system scores words and gets more accurate at removing inappropriate reviews.
- issues are highlighted and the review needs closer scrutiny by an analyst to determine whether it meets guidelines. It is then published or rejected.
TripAdvisor says it has a bank of 300 content specialists, many of who have backgrounds in law enforcement, credit card fraud and even forensic computing.
Business owners also have a role to play and can report a potential blackmail situation i.e upgrade or bad review. TripAdvisor senior vice president, global product, Adam Medros, says 80% of the time no review materialises.
He adds that of the 139 reviews the company receives every minute, the percentage that is problematic or fraudulent is in the "low, low single digits."
He argues that the gain is not worth the risk and the threat of penalties alone is enough to put most people off.
That said the three most common problems are boosting, vandals and optimization companies and this is where the process gets interesting.
As part of the vetting process TripAdvisor uses visual analytics tools which highlight normal review behaviour versus fraudulent behaviour.
The image shows healthy reviews on a property on the top left versus dubious activity on the bottom right.
Without giving too much away, Medros says signals are gathered including IP address, reviewer history, information gleaned from owners and the community
"By understanding what normal behaviour looks like from a forensic perspective, TripAdvisor is able to identify suspicious activity."
He goes on to talk about connections in the data being made along the way such as the device being used and the time of day which go further in alerting the company to signals it should be concerned about.
In March 2014, TripAdvisor received a review about a hotel from the property's own IP address. Straight away an email was sent to the reviewer for verification, the user did not respond and the review was not published.
Later, in July, a second review was submitted on the hotel from the same reviewer and hotel IP address and again, an email was sent to verify. This time the reviewer responded.
The TripAdvisor team then put his/her name into publicly used systems such as Facebook to reveal that it was a member of staff.
"So now we have other reviews that start to form clusters of behaviour. As we start to dig in the weak signals start to become strong."
While the above example set out to boost the property's standing on TripAdvisor, the removal of a fraudulent review has the opposite effect.
Next comes vandalism whereby properties seek to increase their own ranking by lowering that of the competition around them.
Finally, there are optimization companies out there which promise to write reviews for a price, say $500 for 10 reviews and TripAdvisor responds in a number of ways when dealing with these companies.
- It can examine the reviews submitted for a particular property and identify other clients being written for.
- Where it spots review writing jobs being advertised it applies for them enabling it to write some reviews, identify the property and from the sort of text required, uncover other properties
- It can act as an optimization firm itself
Medros points out that with when it catches a property working with optimization firm and starts to take down reviews, the property's ranking slips down.
"More effort and focus should be put around the optimization firms. There are regulations but little enforcement. We don't mind being held to a standard but others have to be equally pursued."
One recent example resulted in 150 properties being penalised for working with a single optimization company.
But, why has TripAdvisor decided to open up all of a sudden?
According to Medros there as been internal debate for some and one reason is to simply give the content integrity team credit for its work.
A further reason is to do with the discussions happening around content law and who owns the copyright on review content and therefore has the power to take it down
"We can't talk about the right to write without all the safeguards we put in place."
NB: Crime scene tape via Shutterstock.