Humanitarian organizations may easily succumb to the temptation to misuse numbers and statistics in order to promote their own causes. Does the end justify the means?

Illustration: Espen Friberg / Morgenbladet

Disasters are most dangerous for moms reported Save the Children’s Carolyn S Miles in Huffington Post when presenting the organization’s State of the World’s Mothers report for 2014. The claim was followed by a number: women and children are ‘14 times more likely to die in a disaster than men’. A sky-high number when one is talking about differences in death rates and a colossal injustice if the information is reliable. But it’s not.

Fight for attention

Humanitarian organizations fight for attention for their good causes in a challenging everyday media environment. Attaching a number to a dramatic claim is an effective way of endowing it with trustworthiness. And numbers also form the basis for headlines. As a result there may be a considerable temptation to select the highest and most dramatic numbers, even though the researchers behind the numbers may urge extreme caution. Sometimes organizations use numbers that have no basis in research whatsoever. Often these ‘mythical numbers’ are reproduced from other organizations’ reports and go from strength to strength as they circulate between humanitarian and international organizations, the media, and politicians.

Mythical numbers

The claim that women and children are up to 14 times more likely than men to die in a disaster is a classic example of a ‘mythical number’. It took fewer than five minutes to find and cross-check the source. Save the Children was citing a report published by Plan International in 2013. Both Save the Children and Plan refer to what at first glance seems to be to an article published in a research periodical, Natural Hazards Observer, in 1997. The article turns out, however, to be a two-page opinion piece authored by a pastor associated with Church World Service, a US ecumenical organization. Pastor Kristina Peterson does not provide any sources to back up her claim. Interestingly, both Plan International and Save the Children have qualified the original claim by adding the words ‘up to’ in their own reports, even though these words do not appear in the original article – apparently suggesting that the original claim went too far.

My target here is not primarily the way in which Save the Children cites its sources, but its uncritical use of numbers that simply should have been checked and rejected. It is difficult to understand how a large, professional humanitarian organization can fail thoroughly to check numbers that it uses so directly for marketing purposes. And although the claim referred to above is the most serious error, it is not an isolated case of a number that has shaky foundations. Another key claim in Save the Children’s press release is that ‘statistically, it is more dangerous to be a woman or a child in the Democratic Republic of the Congo than an armed fighter’. This claim is impossible to substantiate. In order to make such a comparison, Save the Children would need to know at the very least how many soldiers there are in the DRC and how many have died both on and off the battlefield. It would also need equivalently reliable figures for mortality among women in the DRC. No such figures exist, and accordingly neither Save the Children nor anyone else can verify this claim. The report also claims that more than 5.4 million people have lost their lives as a result of war in the DRC. This is another ‘mythical number’. While it does indeed originate from extensive surveys conducted by another humanitarian organization, the International Rescue Committee, the methods used by the latter organization to calculate its figures have been strongly criticised. Much of this criticism is summarised in the 2009/10 Human Security Report

300,000 child soldiers?

Save the Children is by no means alone, but shares the company of a succession of humanitarian and international organizations when it comes to the use of mythical numbers. One frequently heard claim is that there are 300,000 child soliders worldwide. This number has been used by a range of children’s rights organizations in order to draw global attention to the suffering of child soldiers. The number was advanced originally by Rachel Brett of the Quaker UN Office. It has been promulgated worldwide by organizations like UNICEF, but is not supported by any solid evidence. Like many other mythical numbers, it has remained remarkably constant despite changing circumstances. Several wars involving a high number of child soldiers, such as the wars in Angola, Liberia and Nepal, have now ended. This has not, however, caused any reduction in the number. As far as we are aware, no one has made a serious attempt to calculate the actual number of child soldiers. Probably this is impossible.

200 million climate refugees?

The same applies to the much-cited claim that there will be 200 million climate refugees by 2050. The number originates back to a report written in the mid-1990s by Norman Myers, a British environmentalist. Myers has no research expertise in the field of human migration and provides no documentation to demonstrate how he arrived at this number. Nevertheless it is frequently referred to both by humanitarian organizations and by international governmental organizations, and took on a new lease of life when it was reproduced in the highly influential Stern Review in 2006. Thus endowed with renewed authority, the number has been presented repeatedly by the media as a ‘new finding’ from a ‘research report’. This is despite the fact that all references to it can be traced back – via just a few detours – to Myers’ 1995 report. In reality there is no clear definition of the term ‘climate refugee’. And few people ask whether this undocumented number is really as dramatic as the reports indirectly suggest. The answer is no, not necessarily. Many ‘climate refugees’ are very likely among the people who relocate from increasingly difficult living conditions in rural areas in order to try their luck in cities, which in the vast majority of cases will be within their own countries. Even though estimates for human migration in general are uncertain, as many as 200 million people worldwide may have relocated from rural areas to cities in the course of just 10 years, between 2005 and 2015, according to the UN’s Population Division’s ‘World Urbanization Prospects 2005’ (page 3).

Temptation too great?

Why do humanitarian organizations choose to use numbers that they know – or ought to know – are wrong? I would hazard to suggest that this has nothing to do with the competence of the organizations’ analytical departments. Rather, the temptation not to discard an attention-grabbing number is sometimes simply too great, even though one may doubt its reliability. Specialist expertise or knowledge of advanced statistical methods is seldom required in order to ascertain whether a number is mythical. What one does need is a fundamentally critical approach to sources and some understanding of what constitutes proper research evidence. When organizations choose to put forward numbers that they know – or ought to know – are undocumented, this may create an impression that they believe that it is justifiable to lie when they consider the circumstances to be in any event precarious and the objective to be sufficiently important and good; that it is not so important for all the details to be accurate so long as the overall message gets through. There are three main reasons why this conclusion is troubling.

• Firstly, global humanitarian and international organizations are important contributors to the policymaking of governments worldwide. Their aim is to get governments and others to act. If the evidentiary basis for their arguments is weak or distorted, this will reduce their effectiveness.
• Secondly, the use of exaggeratedly large numbers may cause the general public to become jaded. This may contribute to a vicious circle where dramatic numbers and spectacular campaigns become increasingly necessary in order to attract attention and justify action.
• Thirdly, this is a problem for the organizations themselves. Humanitarian organizations generally enjoy special trust as the providers of international aid. They should not take this trust for granted.

Humanitarian organizations can themselves take steps to cultivate a critical approach to numbers within their own analytical and informational departments. A process of quality assurance for numbers that are used aggressively for marketing should be a minimum requirement. There is never a good reason not to check one’s facts one last time, but it does require the inclination to change.

Henrik Urdal is a Research Professor at the Peace Research Institute Oslo (PRIO) and is directing a research project on conflict trends in collaboration with the Norwegian Ministry of Foreign Affairs.

This text was first published in Norwegian in the weekly newspaper Morgenbladet 13 June 2014: Er det greit å lyve i det godes tjeneste?