Airlines are always talking up the awesomeness of their apps, but sometimes that self-confidence sits uneasily with what their users are actually saying.
NB: This is a viewpoint by Dr Piotr Spiewanowski, co-founder of flexponsive.
KLM's mobile app has ranked lowest among the major European full service carriers as a recent study reveals. In this follow-up article we take a deeper look to see what the reviews could tell us about the specific issues around KLM's app and whether or not a relaunched version addressed these concerns.
In the words of the airline’s e-commerce SVP Tjalling Smit:
"With this new app we have reached the next level in anticipating our customers’ needs, keeping them one step ahead of their journey. And this is only the beginning."
Unfortunately, the users disagree, as it can be seen on the plot below presenting the average app ratings over time. The average app user satisfaction in fact plummeted for both iOS and Android after the new launch (gray area).
[caption id="attachment_157275" align="aligncenter" width="550"]
KLM mobile app ratings on iTunes and Google Play[/caption]
To understand the key problems that KLM app designers need to address, we have developed new methodology outlined in detail on UXBooth.
What went wrong?
In short, we have read and assessed mobile app feedback from all the App Store reviews written in English over 12 months between October 2014 and October 2015.
We have identified eight app features crucial for all airline apps – personalisation, booking management, check in, flight info, account management, compatibility, performance, design – and analysed user sentiment towards each feature, based on the reviews. Then we summarised user sentiment and the frequency of user comments on two plots, to compare the old app (before version 5.3.2) and the new (5.3.2 and later).
We present the results on a two-dimensional matrix on which we focus on sentiment and frequency separately for the old and new version of the app. To make the analysis more actionable we divide the plane into four quadrants:
[caption id="attachment_157276" align="aligncenter" width="550"]
KLM mobile app feedback before the relaunch[/caption]
As we can see the strongest side of the old version was the design. More than 40% of users mentioned the looks or the ease of navigation. With the average score of 0.76 those opinions were mostly positive. Check-in and flight information were also an asset. These features were not so talked about, but passengers just expect the app to work.
The app’s biggest problem was performance. More than 30% of users were complaining about the app speed and instability. This should clearly have been the main focus of the upgrades.
Other crucial airline app features – personalisation, compatibility and account management were also rather shaky, but some positive feedback was also received.
How has the upgrade change users’ satisfaction?
[caption id="attachment_157277" align="aligncenter" width="550"]
KLM mobile app feedback after the relaunch[/caption]
Nearly all features became much worse.
Not a single feature gets positive feedback. Performance is not the biggest issue any more, but one can’t tell if it is because the speed and stability improved or just other features became much more annoying and users just don’t bother to complain about performance either.
Conclusion: A lot to do
App review analysis can easily be included in an airline usability audit, where in addition, usability metrics are tracked in clickstream data and live experiments are performed continuously.
But this analysis alone clearly reveals that a lot needs to be done not to disappoint over 1 million regular customers that have downloaded the app.
NB: This is a viewpoint by Dr Piotr Spiewanowski, co-founder of flexponsive.
NB2 Tnooz invited KLM to comment prior to publication but it declined our request.