Follow/Share


Author Topic: Conversation with Marc Robinson on performance budgeting  (Read 17162 times)

0 Members and 1 Guest are viewing this topic.

Napodano

  • *****
  • Posts: 609
Conversation with Marc Robinson on performance budgeting
« on: September 09, 2012, 05:46:02 GMT »
Dear PFM Boarders,

Our past fire-side online conversations have been one of the most successful features on the Board (recording around ten thousand views).

The main reason is no doubt the international caliber of our interviewees. This is also the case for  our fifth fire- side conversation which gives the opportunity to members to talk to Marc Robinson. He is a former IMF Fiscal Affairs Department economist, now a successful PFM consultant. He is also an influential blogger on PFM through his blog http://blog.pfmresults.com/wordpress/  and a regular contributor to the IMF PFM blog at http://blog-pfm.imf.org/ .

One of his recent works included a collaboration with the CLEAR Initiative (http://www.theclearinitiative.org/ ) for which he developed the Manual on Performance-Based Budgeting (attached below - you need to register to download it).

If you are a registered member of the Board, you have the possibility to pose a question to Marc online. The focus of the conversation with him is 'performance budgeting – its role in the overall PFM reform and its practice in transition and development countries ’.

The time allotted for questions is  Monday 10th to Wednesday 19th September 2012. After this period the interview will be closed and remain in the archive for future reference.
« Last Edit: September 19, 2012, 17:57:10 GMT by Napodano »

Napodano

  • *****
  • Posts: 609
Re: Conversation with Marc Robinson on performance budgeting
« Reply #1 on: September 09, 2012, 11:31:45 GMT »
Hi, Marc;

let me start our conversation with two questions:

Question #1
What does it take to have a performance in budget? I have seen many levels of indicators besides the output/outcome ones, are those necessary elements of a performance budget? Can an activity-based budget be considered a performance budget?  When is appropriate to talk about unit costs?

Question #2
Recently there are have been many voices against performance budgeting,  due to its perceived complexity and unwielding results in countries where there have been applied (see for instance http://blog-pfm.imf.org/pfmblog/2012/09/performance-budgeting-facing-up-to-the-hard-questions.html ). What do you reply to these critics? Would you recommend developing countries to launch budget reforms that include the introduction of a performance framework?


« Last Edit: September 09, 2012, 11:47:18 GMT by Napodano »

Glen Wright

  • *
  • Posts: 59
Re: Conversation with Marc Robinson on performance budgeting
« Reply #2 on: September 10, 2012, 11:54:26 GMT »
I think this will be a very interesting discussion.   I did look through the PB manual and find it quite useful in defining the PB approach.  Just to follow on with my main concerns about this approach which has been confusing to me.

Question # 3
First is the question of how interlinked does Program Budgeting and Performance Budgeting have to be.  It seems to me in the literature these two approaches have been treated as two separate, and maybe equal approaches.  Sometimes I am not sure whether an author is describing program budgeting or performance budgeting. This has gotten more complicated with the introduction of medium term budgeting and how program budgeting and performance budgeting fits into this approach.  I notice that in your manual you begin with description of performance budgeting and it is not until chapter 8 that you start to describe program budgeting.  This makes me think that performance budgeting is a level above program budgeting.  To my mind you need first the program budget structure before you can get to performance budgeting.  Can you clarify this relationship of performance budgeting to program budgeting and how they should be developed.

Question # 4
Second,  just based on your experience what examples of well developed performance budgeting can you identify, either on country basis or a sector level basis that would be useful to examine.

« Last Edit: September 10, 2012, 14:24:31 GMT by Napodano »

Marc Robinson

  • *
  • Posts: 18
Re: Conversation with Marc Robinson on performance budgeting
« Reply #3 on: September 10, 2012, 15:57:43 GMT »
Hi colleagues,

Thanks very much to Mauro and PFM Board for the opportunity to discuss these important topics with so many other PFM professionals.

I believe that performance budgeting has a great deal to offer, but at the same time it mustn't be seen as a magic solution to either budgeting or performance problems. Rather, it is part of a broader set of reforms, and will only work with them.

A key first point is that performance budgeting is not just about the use of performance indicators in the budget. Performance indicators, no matter how good they are, do not tell you conclusively whether programs are effective or not, nor do they conclusively demonstrate the degree of efficiency of a program or process. Performance indicators simply provide the raw data for performance analysis – starting point for program evaluation, policy analysis, benchmarking et cetera. It is entirely to be expected that those who believe that simply injecting a whole lot of performance indicators into the budget would change things radically were going to be disappointed. Without the development of better evaluation and, more generally expenditure analysis – making use of good performance indicators – it is inevitably difficult to link budgeting and performance systematically.

In short, performance budgeting is about the use of performance information in general – including evaluation and other forms of expenditure analysis  – in the budget, and not just about performance indicators.

What has made the disappointment about the budgetary use of performance indicators even greater has been the tendency in quite a few countries to include as "program" indicators a whole lot of indicators which are not very useful for budget decision-makers because they are more about internal ministry management. For budgetary purposes, it is primarily the top-level indicators – outcome and output indicators – which are really useful, not a mass of input and process indicators. It is, for example, of little use of the budget preparation to attach to programs indicators such as the number of meetings held, the number policy documents developed, the number of positions filled (or vacant) etc. Yet this is precisely what many countries have done in developing so-called performance budgeting systems.

The other key thing is that even when you've developed good performance information which really tells you something about the effectiveness and efficiency of programs, you can't assume that this information will automatically be properly used on budget preparation. You have to create the channels and routines by which this information is fed into the budget preparation process and actually used. In my view, one of the most important "missing links" here is spending review -- that is, a routine process for examining baseline expenditure as part of the budget process. It is in a spending review process that the performance information generated by performance budgeting can be most effectively used. It is good that at the present time so many countries are focusing new attention on developing spending review processes. Because in the past, budgeting has all too often just focused on the consideration of new spending proposals – under which circumstances it is hardly surprising that performance budgeting systems have not appear to have made too much impact.

Marc Robinson

  • *
  • Posts: 18
Re: Conversation with Marc Robinson on performance budgeting
« Reply #4 on: September 10, 2012, 16:09:23 GMT »
Let me also please comment on a couple of the other interesting points which have been made.

Firstly, Glenn is right in saying that there is much confusion about the relationship between program budgeting and performance budgeting. I think the best way of looking at this is to set performance budgeting as the general concept, and program budgeting as one form of performance budgeting. Performance budgeting is, generally defined, the systematic use of performance information in the budget to improve the effectiveness and efficiency. Program budgeting uses one particular approach – a budget classification based on outcomes and outputs – to make use of performance information in the budget. But there are, as I've made clear in my writings, a number of other approaches to performance budgeting.

Secondly, the question of unit costs in performance budgeting: The idea that budgeting can be based on unit costs is one of the other forms of performance budgeting I've just referred to. The idea, roughly speaking, is that you can budget by firstly identifying all the types of outputs – that is, the specific services provided to the public – which ministries deliver, and then multiplying the unit cost of each of these types of outputs by the quantity which you plan to deliver during the budget year. For example, if you want to vaccinate 140,000 people this year, and vaccinations cost $20 each, then you give the Health Ministry $2.8 million for this output.  The problem with this approach is that it only works for a (quite narrow) sub-set of the services (outputs) which government delivers to the community, and that there are many types of services which can't be budgeted for in this way because they have no stable unit cost. I like to give here the example of criminal investigations by the police – the cost of one murder investigation can differ radically from the cost of another investigation because of case complexity, and there is no stable predictable average or unit cost for murder investigations. So while unit costs budgeting is a good performance budgeting model in some specific sectors – such as public hospital funding under the so-called "diagnostic related group" funding system – it does not provide a model for performance budgeting across the whole of government. I think it is unfortunate that consultants with a poor idea of how the public sector really works have all too often peddled the idea of unit costs as the best and universally-applicable model of performance budgeting.

iancdavies

  • Posts: 1
Re: Conversation with Marc Robinson on performance budgeting
« Reply #5 on: September 11, 2012, 14:23:55 GMT »
Dear Marc,

A number of multilateral organisations such as ILO, FAO, Council of Europe, etc. have developed "results based budgeting and/or management" approaches for their own governance and management. They typically use logic modeling techniques, e.g. program logics, logical frameworks, results hierarchies, etc., to identify all manner of results, e.g. outputs, outcomes, impacts, etc. and corresponding indicators. In doing so they accept, most often without realising it fully, a causal performance construct as a basis for valuing their work.

Problems arise when these organisations and/or their activities are primarily normative and political, e.g. the ILO is about working towards towards social justice, promoting social dialogue, establishing norms and standards, building capacity, etc. The dominant performance budgeting and/or management paradigm is causal in nature and appears poorly suited to capturing, valuing and communicating meaningfully the worth or value of the entreprise. The risks are twofold:

- first that, if you only do what you can measure with the typical armamentarium of causal techniques you risk mission displacement, i.e. you end up doing something else than what you were mandated for; and

- second, because you are not doing a convincing job of "reporting on results" you risk seeing your funding cut. The latter has happened now to a couple of institutions through the MOPAN initiative and the UK government's (aka DFID) decision to cut or reduce funding significantly as "results are not being shown".

Finally, if there's one thing I learned during my years as a civil servant actually working with budgets, e.g. I was advisor to the President of the Treasury Board of Canada, is that a budget particularly in government is first and foremost a political statement, and it is negotiated (some would say horse traded) at the margin.

Question # 5
Any ideas or suggestions as to how government departments, multilaterals and the like, can value what they do in ways that are more consistent with the nature of their business and their raison d'κtre? Are there some basic principles and/or guidelines to help use wisely and appropriately use measurement and valuing techniques, to provide sound and evidence informed advice to management and governance functions? Many thanks for your insights.
« Last Edit: September 11, 2012, 15:53:06 GMT by Napodano »

Gord Evans

  • *
  • Posts: 25
Re: Conversation with Marc Robinson on performance budgeting
« Reply #6 on: September 11, 2012, 15:00:22 GMT »
Marc.  I just skimmed through the attached manual on budgeting, so congratulations on preparing a highly readable and thorough presentation of the PFM Board's favourite topic (strengthening public expenditure management through sound budgetary practices).  Like many of the best efforts to explain how government's determine where to spend their money, it presents the budget as a clear process with distinct steps and time frames orchestrated by the Ministry of Finance. 

Although it certainly acknowledges that deviations from the norm are common, I think there is one gap that is not fully addressed: the constant interaction between the budget and the government's ongoing decision-making system.  From my experience in government (initially in Canada but since in many countries), de facto budget decisions are made every week during meetings with the Prime Minister and in Cabinet Committee and Cabinet meetings, but they are not acknowledged as such - instead, they come forward as draft laws (40-300 per year depending on the country), draft regulations (200-2000), and policy papers (0-150 per year), many if not most of which have fiscal implications. 

Although there is generally a "fiscal implications" requirement included in such documents, these are of notoriously poor quality and under-vetted by the Ministry of Finance except in a few "best practice" countries.  By the time the budget "process" rolls around again, it proceeds as a mad scramble by MoF to figure out how to fund not just the Prime Minister's new pet projects, but the whole slew of now-approved legal commitments which ministries parade forth as "givens" for funding. 

Question # 6
I don't really have any brilliant suggestions at to how one would present the chaotic world of political deal-making and decision-making in a budget manual/training curriculum, but did you give any thought to this issue as it is what tends to happen in most governments? 
« Last Edit: September 11, 2012, 15:49:26 GMT by Napodano »

petagny

  • *****
  • Posts: 348
Re: Conversation with Marc Robinson on performance budgeting
« Reply #7 on: September 11, 2012, 15:26:15 GMT »
Hi Marc,

Question # 7
You will have seen Napodano's reference to the recent article by Dirk Kraan on the IMF PFM Blog. He draws attention to a number of potential difficulties with performance budgeting, but I would be particularly interested in your thoughts on the incentive problems he highlights, namely the issue of perverse incentives and asymmetric information between budget holders and the finance ministry. His comments reminded me of John Kay's book 'The Truth About Markets' in which he is very negative about New Public Management, particularly the New Zealand version where there has been a heavy reliance on 'contractualising' performance. He labels the incentive problem 'incentive (in)compatibility'. Here's a flavour of John Kay's argument:

'Whenever competitive mechanisms are not available, or not used, there are potential problems of incentive compatibility. The common response is to set targets, and reward or punish by reference to the targets. This is the system that worked so badly in the Soviet Union. It badly for two main reasons. One was that the centre did not have sufficient information to set targets effectively. The other was that the targets could only imperfectly reflect the centre's real objectives. These problems arise whenever target setting is tried, in public sector activities such as health and education, in the regulation of utilities or in encouraging executives to maximize shareholder value. The same fundamental difficulty arises in each case. If the target setters had enough information to set targets appropriately, they, not the people on the ground, would be the effective managers of the business. So managers aim to meet targets, not the objectives of the targets. 'Plan bargaining', strategic negotiations between the various parties, and the proliferation of more and inconsistent targets cause cynicism and demoralization.'

« Last Edit: September 11, 2012, 15:49:40 GMT by Napodano »

Marc Robinson

  • *
  • Posts: 18
Re: Conversation with Marc Robinson on performance budgeting
« Reply #8 on: September 11, 2012, 17:27:41 GMT »
Performance measurement difficulties and performance budgeting

Ian has raised some very important issues about how to link budgeting to performance when performance is difficult to measure. I think this gets back to the basic point that performance budgeting is not just about budgeting based on performance indicators. It is certainly not just about what some – based on a private sector model – referred to as the use of "performance metrics". There are clearly many parts of government – in particular national government – where outcomes in particular are very difficult to measure. The outcomes achieved by the foreign affairs ministry is a classic example, as is the effectiveness of the armed forces during times of peace (obviously maintaining an army deters aggression from others, but how does one measure deterrence?). So pretending that budget funding can be simply linked to "demonstrated performance" in the sense of clear and unambiguous measured improvements of performance is absurd. But this is not what sensible performance budgeting is about.

What performance budgeting, properly conceived, is saying is that budgeting should be as evidence-based as possible. In other words, we should do our best to assess the efficiency and effectiveness of expenditures, and integrate that information systematically into budget preparation. In this context, the program logic framework itself is extremely useful, because a key starting point is simply being clear about what we are trying to achieve with public expenditure.

I go back to the point I made yesterday, that we should not think of performance budgeting so much in terms of inputting performance indicators directly into budget preparation, but rather in terms of using good performance indicators as the basis for much improved expenditure analysis (outcome evaluation, process evaluations, policy analysis etc), and then using this expenditure analysis in budget preparation.
« Last Edit: September 11, 2012, 18:18:23 GMT by Napodano »

Marc Robinson

  • *
  • Posts: 18
Re: Conversation with Marc Robinson on performance budgeting
« Reply #9 on: September 11, 2012, 17:38:10 GMT »
OUT-OF-CYCLE NEW SPENDING PROPOSALS

Gordon Evans raises the point about governments' tendency to make new spending decisions "on the fly" – in other words, for many decisions about new spending initiatives to be made not in the annual budget preparation process, but at some other stage during the year. He is absolutely right in saying that this is a really major problem for rational budgeting. If new spending proposals are decided on an ad hoc basis at any time, the possibility of comparing alternative proposals in deciding which offer the best value to society within an affordable overall  aggregate expenditure limit is lost. For this reason, a very important theme of budget preparation reform is the importance of absolutely minimising the presentation of new spending proposals outside the regular annual budget preparation process. It will, of course, never be feasible (or even desirable) to entirely ban the presentation of new spending proposals during the budget execution phase of the budget cycle. Unexpected developments which require expenditure response inevitably do arise, and it is not always possible to deal with these entirely through the use of inbuilt budget flexibility (e.g. reserves under the control of the Minister of finance). However, government should develop clear criteria which must be met prior to the consideration of any new spending proposal outside the ordinary budget preparation cycle – including urgency, and inability to foresee the policy problem. The relevant central agencies – either the Ministry of finance, or the president/Prime Minister's office – should then be given a mandate of vetting all "out of cycle" new spending proposals to assess whether they meet these criteria or not. If they do not, they should be rejected. In practice, this is a problem which some countries have dealt with quite effectively – for example, Australia and the United Kingdom – whereas other countries basically permit almost total freedom for spending ministries to put forward out of cycle proposals.

Marc Robinson

  • *
  • Posts: 18
Re: Conversation with Marc Robinson on performance budgeting
« Reply #10 on: September 11, 2012, 17:58:28 GMT »
TARGETS AND PERFORMANCE BUDGETING

Petagny's thoughtful comments raise the familiar issue of the relationship between target-setting and performance budgeting. The first thing to say here is that performance budgeting does not necessarily involve the setting of budget-linked performance targets. It is an empirical question as to how far it makes sense to use performance targets of this type. This is important to emphasize because in program budgeting models which one sees around the world, it is really common to find an assumption that for every program performance indicator, one necessarily must have a performance target. For example, if one uses the new infections rate as an outcome measure for an infectious diseases program, it is supposedly essential to have a target for reductions in the new infections rate over the next 12 months. But this is entirely wrong. There are some performance indicators for which it makes sense to set targets, and other indicators which – although informative about performance – are entirely unsuited to target setting. Outcome indicators are rarely suited to the setting of annual performance targets because it typically takes years to achieve intended outcomes. In some cases, it is not even appropriate to set four or five-year targets for certain outcomes, because government has very limited control over the outcomes concerned – I'm reminded here of the, in my view inappropriate, targets set by the former British Labour government for the reduction of rates of childhood obesity and rates of teenage pregnancy.

Does this mean that target-setting is damaging and misguided? The most vibrant debate on this topic in recent years was conducted in Britain under the Labour government because of the extensive use of budget-linked performance targets under that government's so-called "public service agreement" (PSA) system. John Kay's comments are representative of the major school of opinion which condemned target-setting as the resurgence of Soviet-style economic planning, and which dusted off all of the amusing stories of the perverse effects of Soviet targets (usually drawn from Alex Nove's classic works). These critics also drew on the usual agency theory analysis of the consequences of information asymmetry in a principal-agent relationship. Okay, there is no doubt that they had a point, and that target setting can indeed induce perverse effects. However, it is not sufficient to just argue from economic theory – based, by the way, on naive "homo economicus" assumptions that everyone is simply trying to rip off the system to maxim personal advantage – and the economic history of communism. Nor is it sufficient to recount a few anecdotes about alleged adverse effects of Labour's target-setting. What was really needed was more empirical analysis of the impact of target setting. Unfortunately, there has been too little of this. However, such analysis as has been done suggests that the whole story of perverse effects has been grossly exaggerated, and that target setting did not do anything like the amount of damage that the critics alleged. (My manual refers to some of this literature, but is also discussed in my chapter with Elisabeth Paul in my 2007 book). And if you look at the track record of improvements in measured performance against the PSA targets in the UK, it is actually quite impressive. In my view, target setting under the Blair/Brown government in the UK was, on balance and as far as we can judge based on imperfect information, a success. The critics got it wrong in part because they ignored important effects such as the "public service motivation" of government employees who in so many cases really care about the services they are providing, or the outcomes that they are achieving, and do not necessarily respond to target setting by simply exploiting every loophole offered by imperfect performance measurement.

Having said all of this, I nevertheless advise governments to be cautious and quite limited in the use of centrally-imposed performance targets. Firstly, this is because– as noted above – not all performance indicators are suited to target setting. Secondly, because it makes no sense at all for the government to impose upon its ministries more performance targets then it can actually monitor and act upon.
« Last Edit: September 11, 2012, 18:00:39 GMT by Marc Robinson »

John Short

  • *****
  • Posts: 432
Re: Conversation with Marc Robinson on performance budgeting
« Reply #11 on: September 13, 2012, 08:43:34 GMT »
Marc,

Thank you for your excellent contribution to the PFMBoard.

In his Guidance of Note on Sequencing PFM Reforms Jack Diamond has produced a paper, the aim of which in his words “is to assist in the successful design and implementation of public financial management (PFM) reforms by addressing the issue of appropriate sequencing”. The note is targeted at planners and decision makers in government and donor agencies.  It document summarizes work that began in September 2010, to review the rapidly expanding literature on sequencing public financial management (PFM) reforms in response to a request from OECD DAC for a Guidance Note (GN) to assist donors when sequencing PFM reforms.

The paper is downloadable from http://www.pefa.org/sites/pefa.org/files/guidance-note-on-sequencing-pfm-reforms.pdf  (and attached below)

The Note includes an extremely clear path showing the sequencing PFM reform with the attached chart: Overall Sequencing Strategy for PFM Development

Apart from mentioning the technical issues in delivering the various stages of the sequencing strategy, the paper also notes the systemic risks that may be encountered.  To quote:
"The technical specification of reform actions required to improve the PFM system, (i.e. what is needed), must be consistent both with what is possible (implementable) and what is wanted (politically supported). Bringing all three dimensions into line is the role of change management, a crucial factor in reform success.  The viability of any reform program should be determined by a systematic analysis of the risk and opportunities existing in the specific country environment, and those arising from the nature of the reform actions being contemplated.  External non-technical factors should be recognized as critical for sequencing and must be accommodated in any viable reform program. These non-technical factors can be analyzed by dividing them into three tiers:
1.   top-level conditioning factors, that determine the overall reform environment;
2.   PFM system institutional factors that determine how well reforms will be received and implemented; and
3.   at the lowest level, organizational factors and HR factors that often impose constraints on the extent of reform.
Each level should be analyzed to assess the risk and opportunities they pose for reform.  Based on this analysis, wherever possible action should be taken to reduce risk of reform failure and to enhance support for reform.
Choice of the type of reform action has an important impact on its likelihood of success. A review of the literature identifies five dimensions as influencing the risk of failure:
1.   the number of institutions involved;
2.   the time required for implementation;
3.   scope of reform actions;
4.   the degree of behavior change implied; and
5.   the degree of visibility in the final results.
All are highly correlated: reforms wide in scope usually involve more institutions and greater time for completion, they tend to involve a greater degree of behavior change and often their results are only visible, if at all, with a considerable time lag. Also a trade-off should be recognized: the higher is the risk level of different reforms typically the higher is their potential reform impact.”

I apologise for a long introduction, but in the context of my questions it will save participants in the Conversation from going to the original though it is a paper well worth reading.

While I completely agree with desirability of moving right down the sequencing road and gaining from the benefits of programme/performance budgeting, the sceptic in me suggests that the chart should be titled “The Road to PFM Nirvana” as synonyms for Nirvana include paradise and heaven but also illusion and fantasy!  The road to Nirvana is littered with many good intentions.

Question # 8

My question is twofold:
In your experience from your country work have the costs associated with the systemic risks outweighed the benefits from introducing fully fledged programme/budgets so it may be better to stop at the MTBP stage on the grounds that the pursuit of the excellent is the enemy of the good? (The stage the budget projections change from an information- to a decision-making tool, so next year's estimates have some policy status and would reflect cabinet decision-making on aggregate and line ministry spending ceilings over the medium-term in the Guidance Note.)
and
What time frame has been required to mitigate these risks to reap the benefits when the MTEF stage as described in the chart has been reached? (The MTBF becomes strictly linked to policy and anchored in sector MTEFs and it is presented on a more detailed level, with full integration of recurrent and investment costs of programs projected in the out years. The fiscal strategy may also include non-financial performance ("results based") information linked to program budgets in the Guidance Note).
I realise that not all countries’ experience have been the same, but some ballpark range would be useful!  In my experience the time scale of donor funded MTEF projects is vastly under what is needed and as a result are wrongly judged as they are almost programmed to fail from the start.
« Last Edit: September 13, 2012, 12:18:38 GMT by Napodano »

Marc Robinson

  • *
  • Posts: 18
Re: Conversation with Marc Robinson on performance budgeting
« Reply #12 on: September 13, 2012, 14:17:32 GMT »
SEQUENCING AND PERFORMANCE BUDGETING

John Short's thoughtful comments have raised the crucial issues of: when a country is ready for performance budgeting, and how should performance budgeting implementations be sequenced relative to other reforms such as an MTEF. He asks about when do "systemic risks outweighed the benefits from introducing fully fledged programme/budgets so it may be better to stop at the MTBP stage"?

Addressing the first part of John's question, we have to do ask ourselves first what we mean by "fully fledged" program budgeting. In my view, to work effectively, a program budgeting system involves three elements – a program classification of expenditure which is used as the basis of legal appropriation of expenditure; the development of a useful body of program performance information (performance indicators, but not only performance indicators); and the use of this information to improve budget decisions about where to allocate limited funds. This constitutes the "bare bones" of a meaningful program budgeting system. It doesn't include any fancy add-ons, such as lots of program performance targets, or unit cost calculations of program funding requirements, let alone activity-based costing, accruals and all the rest. These are at best optional add-ons (and some cases, only to be used selectively or not at all).

While there is absolutely no doubt that there are many developing countries which are not yet ready for the implementation of such a "bare bones" version of program budgeting, it is also clear that such a program budgeting system is not so vastly complicated that only very advanced countries could expect to make it work. This is the reason why I have always rejected the proposition that performance budgeting is generally inappropriate for developing countries.

However, it is very true that we must always keep in mind that there are PFM fundamentals which are more important and more fundamental than performance budgeting, and which should be in place before anyone thinks of implementing performance budgeting. For example, the Ministry of Finance and government have to be able to enforce budget spending limits on ministries, and avoid undisciplined spending overruns or buildups of unpaid invoices. And the budget should not be on a fiscally unsustainable trajectory. One could point to a number of other fundamentals of this sort which should be given priority over performance budgeting in technical assistance to developing countries. I am often quite shocked at some of the countries which bilateral donor organisations seem to think should be introducing performance budgeting now – e.g. the Congo and Zimbabwe.

But the other question which John raises is the sequencing of performance budgeting verses an MTEF. I think it's easy to lose sight of how technically difficult an MTEF can be, and how few developing countries have successfully implemented one. Even the most fundamental elements of medium-term budgeting – good medium-term "unchanged policy" expenditure and revenue forecasts – are technically difficult to develop. But going beyond that to fixed medium-term ministry spending ceilings requires a vastly higher order of technical capacity and budget system development. It is one thing for countries like the UK and France to make commitments to spending ministries about the budgets that they will receive in two or three years time. It is a totally different thing for developing countries to make a commitment this sort, and I am generally speaking very sceptical about the wisdom of fixed multi-year ministry spending ceilings in these countries. Without the ability to make accurate unchanged policy expenditure forecasts, the risks of getting these medium-term fixed ceilings wrong – too low or too high – are enormous. Even more seriously, if the country does not have well-developed mechanisms for expenditure prioritisation, then making medium-term commitments to spending ministries can just make expenditure even more rigid, and make it even harder to reallocate spending away from the low priority to high priority areas.

Overall, it seems to me that implementing a basic program budgeting system is actually less technically forbidding than the development of a MTEF with fixed medium-term Ministry expenditure ceilings.

FitzFord

  • *
  • Posts: 154
Re: Conversation with Marc Robinson on performance budgeting
« Reply #13 on: September 13, 2012, 17:59:51 GMT »
Marc,

The subtle distinctions you make between performance budgeting and performance measurement and program budgeting are important and useful. A couple of years ago I did a study for the World Bank specifically related to Aid programs (where some donors were arguing for resource allocation to recipient countries based on performance), and in this study I reviewed the use of performance indicators in the OECD countries - much of the material came from studies done by the EU. A quick summary of the story is that no OECD country used performance indicators as the basis for resource allocation across ministries or sectors. This despite:72% of the countries include non-financial performance data in thier budget documentation; in 44% of the countries, these data are available for more than 3/4 of the programs; in 71% of the countries, performance data included performance targets although there was a wide variation in program coverage; and, in 65% of the countries, these results are included in the main budget documents and/or the annual financial document. Only 18% of countries reported that they specifically link expenditures to all or most of their output or outcome targets (ie Performance Budgeting) and mostly in health and education. There was an acknowledgement that are good reasons for the reluctance to extend the use of outcome performance data into resource allocation decisions for (predominantly) technical, managerial and political considerations. However, perhaps the most important (but not explicitly acknowledged) rationale for this hesitancy was that this could be inherently counterproductive to the underlying objective of a program. The conclusion from experience was apparently that the best use was as a management tool  to engage with relevant stakeholders. Less than 10% of Cabinets and legislative committtes use the available measures in their decision making.

I think your comments on how and when to situate and apply these measures (which are expensive instruments as you know) in a public sector management system.


Fitz.

Marc Robinson

  • *
  • Posts: 18
Re: Conversation with Marc Robinson on performance budgeting
« Reply #14 on: September 13, 2012, 19:59:29 GMT »
INDICATOR/BUDGET LINKS ONCE AGAIN

Fitz makes a good point. Again, one should not generally expect direct links between performance indicators and budgets. For example, in the case of outcome indicators, if the outcome indicators are getting worse, it does not necessarily make sense to cut budgets -- if, say, we are talking about the school system and literacy and numeracy indicators are getting worse, budget cuts are unlikely to be the appropriate response. So the connection between indicators and budget decisions should be more sophisticated than that. In relation to effectiveness (outcomes), the connection should instead be something like the following:

Step 1: Do the outcome indicators look bad?
Step 2: If so, one should try to establish -- e.g. using outcome evaluation -- whether the program really is ineffective, or whether the indicators are misleading (e.g. because of the impact of "contextual factors")
Step 3: If the program is ineffective, does it appear that it could be fixed (e.g. by redesign, or even via more funding)? Or does it look irredeemably ineffective?
Step 4: Only if it is irredeemably effective would one necessarily wish to abolish the program.

So again, the whole obsession with whether performance indicators are directly impacting on budget decisions is misplaced. Indicators are very important, but as the starting point for the analysis of effectiveness and efficiency, not as conclusive stand-alone guidance on funding levels.

(Note, however, the exception to this in the case of sectoral performance based funding systems -- such as output-based hospital funding -- in which funding is directly linked to (output) indicators.)


 

RSS | Mobile

The PFM Board © 2010
Powered by SMF