Competing to be the best: How reliable are survey readings?

Courtesy Cathay Pacific

Courtesy Cathay Pacific


SKYTRAX has named Cathay Pacific as the world’s best airline in 2014, displacing last year’s winner, Emirates. In second and third place are Qatar Airways and Singapore Airlines (SIA) respectively. Asian and Middle East carriers dominated the ranks of the top ten: Emirates (4th), Turkish Airlines (5th), All Nippon Airways (6th), Garuda Indonesia (7th), Asiana Airlines (8th), Etihad Airways (9th) and Lufthansa (10th). No American carrier was placed.

Are those really the world’s best airlines?

The winning airlines are unlikely to question the validity of any survey, as you can see how many of them are listing awards from all and sundry like a laundry list as endorsement of their good reputation. The corollary must be that if you accept the accolade willy nilly, so must you recognize one and all sideswipes.

Which leads to the next question: Is Skytrax the standard?

Skytrax claims its World Airline Awards to be “the global benchmarks of airline excellence”. The winners are decided by 18.85 million travellers from over 160 countries, and that should take care of any misgiving about the survey having an inadequate population and most importantly, the bias factor or its susceptibility to political influence.

Cathay CEO Ivan Chiu said: “The World’s Best Airline award is particularly important to us because it was decided by the votes of close to 19 million travellers from around the world.” Cathay was placed sixth last year and has won the award four times, previously in 2003, 2005 and 2009.

Emirates president Tim Clark said: “These awards are widely regarded as the industry’s benchmark for excellence. To be voted ‘World’s Best Airline’ by millions of discerning travellers is something… to be proud of.”

Qatar CEO Akbar Al Baker said: “These awards are highly rewarding as they are judiciously voted by passengers a true account of the overall experience felt by customers who have travelled with the airline.” Qatar won in 2011 and 2012.

Courtesy Etihad Airways

Courtesy Etihad Airways


However, Etihad’s withdrawal from participation apparently over differences in the methodology may tell a different story. Although it had never won, Etihad was consistently placed in the top ten in the past five years, ahead of Emirates in some years. Despite its withdrawal, Etihad was still ranked in this year’s survey because according to Skytrax, “an airline cannot be withdrawn from the World Airline Awards since these results are directly decided by customers.” That statement should add to the survey’s credibility, yet without taking sides and arguing the toss about fairness, one can only suspect and understand that the subjective nature of the survey (and of any survey) is naturally exposed to dissatisfaction, whether baseless or with reasons which may well be valid, the way that the Oscars results do not sit as squarely with a lot of people. Now and then you get an outstanding actor declaring his or her disinterest in the awards.

The issue is usually one of weightage and relevance of selection. However designed, the respondents may to some degree be steered by what is being asked. Take, as matter of curiosity, the 2014 Skytrax survey readings for the top ten. SIA is ranked ahead of Cathay for inflight entertainment, cabin cleanliness, First Class amenities, First Class cabin overall, seats in First, Business and Economy, and First Class meals; but close behind Cathay in other areas except for its noted absence for airport services, Business Class amenities and Business Class meals. Yet Cathay takes the cake.

It is encouraging to see breakthroughs by airlines such as Turkish and Garuda in a game dominated by the familiar big names. Interestingly, Turkish ranks above everyone else except Emirates and SIA for inflight entertainment. It is no surprise that Garuda tops for cabin crew, the epitome of Asian service culture, in a category swept by Qatar (6th) and nine other Asian carriers: Cathay (2nd), SIA (3rd), Asiana (4th), Malaysia Airlines (5th), EVA Air (7th), ANA (8th), Thai Airways (9th) and Hainan Airlines (10th). In like fashion, with the exception of KLM (8th) and Qantas (9th), the airport services category belonged to Asian carriers: ANA (1st), EVA (2nd), Thai (3rd), Asiana (4th), Cathay (5th), Korean Air (6th), Garuda (7th) and Dragonair (10th).

Yet, giving credit where it is due, one may question the appropriateness of comparing a carrier having limited global presence with others that are more exposed in the global arena, and how a population of largely local respondents compares with the wider global population. Hence it may be more meaningful to look at niche rankings, but we all love the sweeping titles of the best overall, don’t we? Even regionalized readings must be viewed in their proper context. The Qantas Group went ga-ga over Jetstar Airways’ win as best low-cost airlines in Australia/Pacific over AirAsia X (2nd), Scoot (3rd) and Tiger Airways (4th), but the world’s best is AirAsia followed by AirAsia X in second place ahead of Jetstar Airways (4th). Note how the preferences change when the population mix changes.

Who then really is the best overall? It may be difficult to say for sure one definite airline, and under the circumstances a wider reading of the top three or five or up to ten may be a more sensible assessment. The contest is to get into that magic circle of the elite.

Courtesy TODAY

Courtesy TODAY


Equally significant is the consistency over time. Airlines such as Cathay, Emirates, Qatar and SIA may pat themselves on the back for being there long enough to deserve their stripes. Narrow that down further, and you will see that only two airlines – Qatar and SIA – have been consistently placed in the top three in the past five years. Asiana had a good run from 2010 to 2012. Cathay was just outside in 4th place until it tumbled to 6th last year and bounced back to be this year’s winner. The wider reading should lead some airlines such as Qantas to ask why it has dropped out of the respectable club.

One survey alone cannot be definitive, hence winning across notable surveys may strengthen the reading. Compare the Skytrax results with Conde Nast Traveler’s assessment by its readers – based on the same principle of uninfluenced feedback – and you will begin to understand why. In its ranking for foreign carriers (outside America), Etihad is placed 4th behind Emirates (2nd) and ahead of Qatar (7th). Cathay is 7th, and the winner is SIA. Korean Air (8th) did better than rival Asiana (18th), and so did Japan Airlines (16th) over ANA (21st). The Conde Nast top ten includes Virgin Atlantic (3rd), Air New Zealand (5th) and Swiss International (10th).

Then there is the annual Airline of the Year award given by the Air Transport World (ATW) magazine. The criteria take into consideration financial performance (which debunks the myth that the world’s favourite airline is not necessarily the most profitable or even profitable) and visible leaps forward in services. However, naming only one winner can often lead to suspicions of political influence (the way that some beauty pageants are said to be when a winner is crowned) and the tendency to pass the honour around although airlines such as ANA (2007 and 2013) and Air New Zealand (2010 and 2012) had been named twice. Cathay (2006), SIA (2008) and Asiana (2009) had all had their turns. Delta Air Lines is ATW’s Airline of the Year 2014.

Several other magazines also dish out their own annual awards, which may be based on their readers’ feedback, or assessed by a panel of judges or arrived at combining the two methods. Some of them target niche markets such as awards that recognize the best airline for business travel. That in a way avoids spillover or halo effects and sectarian prejudices as, for example, an airline that impresses in First and Business Class may pay scant attention to what happens in Economy.

Nevertheless, surveys are useful tools in maintaining competition. Everyone loves to win, unless you do not give a hoot about how the world sees it and how that may affect your bottom line. So too, everybody loves a winner; but that is no guarantee that the traveller will necessarily fly with the named best airline. Without downplaying their influence on the market, such awards probably mean more to the airlines than the travellers.

This article was first published in Aspire Aviation.

Advertisements

About David Leo
David Leo has more than 30 years of aviation experience, having served in senior management in one of the world's best airlines and airports. He continues to maintain a keen interest in the business, writes freelance and provides consultancy services in the field.

One Response to Competing to be the best: How reliable are survey readings?

  1. Matt says:

    Looks like these surveys mean very little, as they managed to place Norwegian third in budget airlines. Looks like they failed to check the repeated failings of Norwegian. Constant delays/cancellations, lost/stolen baggage, charging of customers for their mistakes and zero customer support when needed. The Twitter hashtag #NeverFlyNorwegian is trending with daily complaints, along with the Facebook group “Norwegian Airlines Experiences From Hell” growing daily in members. Norwegian are day by day making Ryanair seem like a five star choice, compared to their services.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: