‘Value for money’ or ‘Results Obsession Disorder’?

“intussen… BLOG” verwelkomt Marcus Leroy als gastauteur. Marcus, ontwikkelingsdeskundige sinds jaar en dag, heeft een boeiende tekst geschreven (in het Engels) over de nood aan aantoonbare resultaten en “waar voor geld”: twee concepten die thans opgang maken binnen ontwikkelingssamenwerking, en waarvan het uiteindelijk doel het behoud van publieke steun is voor de sector. Hij komt daarbij tot verrassende inzichten. Zoals steeds zijn we benieuwd naar jou reacties op de tekst. En bij uitbreiding naar jouw ideeën over hoe OS er in de toekomst zal uitzien?  

‘Value for money’ or ‘Results Obsession Disorder’?

For many decades development aid of western donors has been pretty well shielded from probing questions by the public opinion and politicians. Development aid was, and to some extend still is, essentially seen as “helping poor people”, a charitable activity that is inherently good and respectable and doesn’t need any further examination. A number of factors – the appearance of new actors on the aid scene, budget constraints, aid fatigue, the shift to the right of the political spectrum – have changed this. More than ever before the aid industry is now confronted by the incisive and pressing question: what is the use of all these billions of aid money?

Faced with this question, two reactions are a priori possible: the ‘William-Easterly-Dambisa-Moyo reaction’: “let’s face it, it’s awfully difficult to find undisputable evidence of positive outcomes” or “let’s make sure we can produce results”. Not surprisingly, the aid industry has chosen the latter alternative. And so we see all western donors gradually taking the road of “performance indicators”, “aid that makes a difference”, “measurable results” and “value for money”. On this road DFID has resolutely taken the lead. In its “Common Agenda for Development Results” of January 2011 DFID says it wants to thoroughly reorient its development policies and put results and value for money at the heart of everything it does. Though DFID acknowledges that measuring results is not simple, this consideration is vigorously wiped off the table with “We will not let complexity stop us from determining whether we are achieving value for money.” Since then “value for money” is definitely at the core of DFID’s aid policies. As quite often, where DFID leads, others follow. And DFID is unmistakably making full use of its intellectual firepower, unequalled in the world of international aid, to push other donors, bilaterals and multilaterals, in the same direction.

“Value for money”, a return to the countability of the 1980s?
With this agenda, knowingly or not, donors are going back to the “New Public Management” movement that, starting in the 1980s, intended to modernize public services by adopting elements from the private sector, especially its emphasis on cost-benefit analysis and measurable performance. This is highly surprising for there is a wealth of scientific work demonstrating the various limitations of this approach. Such literature consistently points out that public services should comply with other norms and values than private businesses. As for performance, it shows the (unintended) perverse effects of excessive focus on quantifiable results.

Reduced to its essence, the emphasis on measurable performance that is at the core of “value for money” is based on a contract relationship between a customer and a supplier; that is a relationship in which the parties have opposing interests and, at bottom, do not trust each other. That’s why the contract defines what the one party has to deliver, how this will be assessed – indicators – and what the other party will do in return. Obviously, such arrangements fit well in the delivery of physical goods. They are hardly workable however in areas such as quality of health, institution building, furthering accountability and good governance, promotion of entrepreneurship and the like. And to extend them into the field of development aid with its very unequal and asymmetrical relationship between donor and local partner is asking for dire consequences.

In order to look deeper into this, we make the somewhat artificial distinction between the conceptual and the operational level. But before doing so we have to point out that our argument is not about the pursuit of results in itself – evidently, no human action is ever undertaken without the intention of getting results – but about measurability, about what, how and when to measure, about ‘whose results?’ and about ‘who measures?’

“Value for money”, a new wave of reductionism?
At the conceptual level the “value for money” approach takes a lot after nineteenth century utilitarianism. We may even be tempted to see it as an afterthought of it.

As much as utilitarianism, “value for money” is fundamentally consequentialist. That means acts, in this case activities of an aid donor, are judged by their results, not by any preset ethic rules. Indeed, the “value for money” approach is inspired by the idea that we can get rid of the political or moral dimension of the issues involved. It wants to translate every objective into a “deliverable”, a rational objective that has nothing to do with ideology. Results tends to be judged independently from the process to get there and apart from considerations of fairness, accountability or participation of the people involved. On this point much of the criticism that was recently formulated with regard to utilitarianism by thinkers like John Rawls and Amartya Sen is also valid for excessive focus on measurable results. In particular Sen’s insistence on the role of humans as the actors of their own development and his preference for actual opportunities of living as opposed to the means of living are highly relevant here.

Therefore, at the conceptual level, “value for money” seems to miss the essence of what development really is. We find an excellent illustration of this in a DFID health sector program in Uganda where payments are made based on a performance based contract: the donor funds are released according to the number of women and children under 5 years old that are provided with medical care. Regardless of the many operational issues such system evokes, one cannot but exclaim: that is not what development is about! What people want is not getting as much medical care as possible. What they want is leading a good and healthy life. And it is highly questionable whether statistics on the administration of medical care are a reliable proxy for that.

This example shows how the “value for money” approach appears to ignore the essence of development in a very fundamental sense. For good public service is not just a question of quantity – how many children go to school? – not even just a question of quality – do we provide sound education? – but essentially a matter of fairness, accountability, participation and creating opportunities for people to go their own way… all elements that are absent from “value for money” thinking.

In other words “value for money” is ultimately based on a schematized and incomplete representation of the development process. This entails the additional risk of deceiving politicians and the public opinion. Indeed, showing results based on such a defective picture of reality may create the impression that development is simple and straightforward, and that aid is a gigantic machine of money-in development-out.

“Value for money”, donor-centrism revitalized?
At the operational level, the “value for money” approach – as much as its “New Public Management” predecessor of a few decades ago – inevitably entails a number of important risks. As aid agencies will be assessed on their measurable results, they will be inclined to opt for activities whose results are easy to measure, regardless of their sustainable impact on fundamental developments in the society. This is outright deplorable because we know that those developments that are most precisely and easily measured are the least transformational while those that are most transformational are the least measurable. But even where agencies choose for programs that aim at fundamental societal transformations, there will be a high risk of indicatorism, that is indicators will tend to become autonomous goals. In practice agencies will tend to focus on immediate, small-scale output rather than on long-term macro-outcome. And as they are aware they will be assessed on “value for money”, they will tend to avoid risks and opt as much as possible for activities for which the results are assured.

The most pernicious shortcomings of “value for money” however are due to its strong donor-centric rationale. Accountability to a donor’s politicians and public opinion is no doubt a legitimate concern, but it tends to overlook the fact that outcomes of development aid are and ought to be first and foremost the concern and the responsibility of the local partner. The contract relationship between donor and local partner that is at the core of “value for money” will undermine this in several ways. The reason is that the power balance between a donor – who has the money – and a local partner – who is dependent on it – is always very asymmetric. At the end of the day, it’s the donor who will assess whether or not there is “value for money”. And he will measure in accordance with his standards, his policy preferences and, ultimately, his ideology. For the same reason, “value for money” will tend to cripple the sense of responsibility and initiative with local partners. Indeed, they will direct all efforts towards obtaining the agreed figures within the given time-span, because that is how the donor will measure their performance (and eventually disburse his money). On a more fundamental level, this will tend to weaken the sense of accountability to local populations.

Unfortunately we see more and more how, forced by the steamroller of “value for money”, donors are imposing on their local partners pressing demands in terms of reporting against quantifiable indicators, often with little regard to real needs and scant understanding of issues on the ground. So far local actors more often than not end up accepting such conditions. We should be aware however that they do so mainly because of the money that comes with it, not because they feel the reporting on measurable results is a sensible approach. So, in the end “value for money” risks damaging the credibility and eventually even the relevance of western donors .

When confronted with the above ideas, aid agencies attempt to put our mind at rest by underlining that “value for money” is not as mechanistic and technocratic an approach as it may appear at first glance. It will go, they say, well beyond the purely quantitative approach and there will be ample consideration for process and qualitative issues, for long-term outcomes and for systematic learning from past experiences. To this end DFID, for one, has considerably expanded and improved its research and evaluation capacities and methods. These are no doubt hopeful words, but we all know where the proof of the pudding is to be found. And so far the evidence is far from reassuring as can be seen e.g. in the unwarranted use of “cash on delivery” mechanisms, the pressure on practitioners in the field to produce quantified results and the donor-centric approach in DFID’s Multilateral Aid review of March 2011.

The value of partnership
Should all this lead to the conclusion that we should not work towards results? Throw our indicators in the wastebasket? Abolish monitoring? Just do things? Of course not! Of course we should count and measure. But counting and measuring must be part of a dynamic process among partners, of joint learning and adjusting, continuous communication and interaction that goes well beyond the measurable elements of an aid program. The relationship with a partner in development must not be reduced to a contractual relationship based on distrust. That is the antithesis of concerted action. Development cooperation is about patiently building trust and creating a shared sense of responsibility for common goals. And that takes time.

Marcus Leroy