PFM Board

Start here => Interesting experiences => Topic started by: Martin Johnson on February 06, 2013, 14:25:58 GMT

Title: Some thoughts on Matthew Andrew's 'Limits to PFM Reform in Development'
Post by: Martin Johnson on February 06, 2013, 14:25:58 GMT
First of all, a public apology to Matt who sent me the proofs for his book back in December. I read the book immediately and sent Matt comments within a few days. After promising to post some review-type blogs based on those comments I have already let too much time pass. So, sorry for that Matt and ... for anyone who is interested, here are the first of my thoughts on what I suspect will become an important and (hopefully) paradigm-shifting body of work.

First of all, congratulations are in order for the veritable tour de force that Matt has produced. He has amassed a huge amount of evidence and organised it extremely well to tell a persuasive and important story that has relevance to all members of the development community and to many not in the development community, not the least of which is the international taxpayer who continues to fund those interventions that are making meaningful inroads to stubborn problems and the many more interventions that seem to be having so little impact. If the call to arms at the end of the book is heeded among funding institutions, developed country governments and partners in improving governance, Matt’s book may well have an impact beyond the academic community and in the lives of people who deserve better from the governments that often do not serve them as well as they should. Many of the themes and conclusions in the book chime with my own experience which I hope to convey in the posts that follow.

Title: Re: Some thoughts on Matthew Andrew's 'Limits to PFM Reform in Development'
Post by: Martin Johnson on February 06, 2013, 14:31:27 GMT
Matt’s book draws attention, among other things, to the widespread use of governance and related reforms that are ‘best-practice’ in design but are short-term and seem to ignore context and the myriad layers of minor and major context-specific issues that must be negotiated before technical solutions have even a chance of being adapted properly, never mind implemented at the front line by people that matter (however insignificant or distant those people may appear to project designers or project implementers). He is not alone in being frustrated by observing the widespread use of this approach and it is no surprise that he presents so much evidence of success in terms of ‘signals’ and so (relatively) little impact from that apparent success. The time is ripe for a weighty tilt against this paradigm. The evidence that Matt has assembled is certainly weighty. It is also persuasive and does have the potential to begin shifting the paradigm.

A fair amount of my own practical experience chimes with much of the analysis and evidence in Matt’s book. This includes coming into contact with projects that aim to support governance reforms but have unrealistic or inappropriate log-frame design, both in terms of nonsensical periods of time over which fully-formed reforms are expected to be designed and delivered and in terms of the contextual “constraints” that are simply overlooked or assumed away. In one case I had an unusual opportunity to periodically review a major PFM reform over a ten year period. The anticipated achievements in its original design (covering a period of just 2.5 years – a period typical of many development partner-supported governance reforms) were unrealistic to the point of being wholly unachievable. The support would have resulted in abject failure and closure after (or even before) year 2.5 had it not been re-scoped shortly after it started – an interesting story in itself, but one for another time and another place. Even after re-scoping it remained over ambitious and was on the verge of closure on several occasions because of an apparent failure to deliver results (including when very good results were being delivered!). The story of that reform is one where what was delivered after ten years was more or less what was possible within that period in the highly difficult change management environment of the country concerned (where what Matt refers to as ‘cultural cognative influences’ are extremely strongly embedded). Interestingly enough, whilst a number of different development partner staff members came close to closing the project at various points in its lifetime because of ‘lack of results’, it managed to survive (sometimes narrowly) in part because it was subject to a regular and independent review process that explained to the development partner on most of these occasions that ‘context’ was being addressed and that progress was commensurate with context, even if it was not commensurate with the project’s over-ambitious design. From my own experience, therefore, I have come to appreciate one of Matt’s conclusions with regard to ‘best practice’ and the short term nature of many governance reforms - the need for regular reviews to address context.
Title: Re: Some thoughts on Matthew Andrew's 'Limits to PFM Reform in Development'
Post by: Martin Johnson on February 24, 2013, 08:28:54 GMT
As noted in my previous post on Matt’s book, inappropriate design of development partner support can substantially undermine the effectiveness of that support (and the likelihood of success in the reform itself), particularly where this involves the heady mix of ‘best practice’ outputs and highly ambitious timeframes. I am sure that many observers and practitioners have seen similar examples of such paradox in design. I have little doubt that an analysis of development partner support portfolios will tend to suggest that the typical menu of best practice reforms have tended not to achieve the desired results, as Matt’s evidence shows only too well. One conclusion is that ‘the menu’ should not necessarily be offered to all. This may very well be true. But it is also likely to be the case in some instances that there is not necessarily anything wrong with ‘the menu’ but that there is something wrong in the order of service and the speed with which the head of house expects her diners to be in and out.
Title: Re: Some thoughts on Matthew Andrew's 'Limits to PFM Reform in Development'
Post by: Martin Johnson on February 24, 2013, 08:29:57 GMT
Matt’s analysis of reforms as short-term signals is compelling. But the other side to this story, of course, is that there is a perverse incentive among development partner staff to demonstrate successful engagements with client governments that deliver something that looks like successful reform (and perverse incentives for implementing agents to deliver what terms of reference require of them, irrespective of whether they comprise short-term signals alone or something more than that). Short-term signals, of course, look something like successful reform ... over the short-term at least. Which is often just enough time for an ambitious and apparently successful development official to move on and ascend further up the slippery pole (and more than enough time for consultants to be paid!). By way of contrast, taking time to properly identify problems, underlying causes and the appropriate change agents at each of the relevant locations required and finding contextually appropriate solutions probably takes more time and requires more (apparent) risk than a typical development partner organisation (and individuals therein) is prepared to take. Moreover, the ‘development industry’ thrives on paradigms (structural adjustment, PRSPs, governance, PFM, etc.). Such an approach also requires a deeper skill set within agencies ... as well as among practitioners. Whilst I agree with Matt’s analysis, perhaps the task of reforming ‘ourselves’ to be capable of delivering pegs of the appropriate size and shape for the holes that need filling is as daunting as properly identifying the holes to be filled and crafting the pegs.
Title: Re: Some thoughts on Matthew Andrew's 'Limits to PFM Reform in Development'
Post by: Martin Johnson on February 25, 2013, 08:41:15 GMT
The section of Matt’s book that deals with ‘institutional logics’ and the influence of these on reform possibilities and outcomes articulates very nicely what one observes all the time, with or without realising it. A very practical symptom of a ‘dominant institutional logic’ in a country I am familiar with was brought to my attention with a contemporary story of recruitment of (relatively junior) government officers who, it seems, are obliged to make an informal payment equivalent to several times the annual salary they can expect to obtain from the post they are applying for. The existence of this kind of informal payment, of course, goes a very long way to explain persistent and all pervading petty rent seeking within public services where such situations occur. The incumbents expect to pay ‘tax’ (capitalised) for the employment opportunity. It is expected and understood that they will extract rent from their daily duties (a form of tax-farming I suppose). Similarly, I was once at a social occasion when a former Accountant General of the country I was working in was asked out of the blue by a fellow guest what the ‘cost’ of a police station currently was. The answer was $100,000. What I was most surprised at was not the figure (though it did give me food for thought) but the response that came without any hesitation. It was at that point I realised that everyone who has to put up with such situations in their daily lives (i.e. recipients of public services) knows the informal rules of the game in such countries. The difference between formal and informal rules is also well illustrated by my experience of leading fiduciary risk assessments in various countries. When I have asked national colleagues to prepare text on the corruption risk component of such assessments, they have preferred to limit their presentation to an analysis of Transparency International and other perceptions-led assessments, despite a clear knowledge about particular and well-known corrupt practices in those countries.
Title: Re: Some thoughts on Matthew Andrew's 'Limits to PFM Reform in Development'
Post by: Martin Johnson on February 25, 2013, 08:52:26 GMT
The section of Matt’s book that deals with over-specification and oversimplification of reform is a relief to read from a practitioner’s point of view. This is a problem I have been keenly aware of for some time and have felt frustrated about, particularly when observing practitioners (myself or others) having to grapple with supporting implementation of reforms that assume far too much, in terms of existing capacity (to use an oversimplified term) (and assume far too little in terms of time), to jump to the technical solutions that have been put on the table. (or, as Matt puts it, reforms that ‘introduce rules of the game ... require a lot of content, but then do not provide such’.) I think it important to have this issue articulated and supported so clearly with the kind of evidence that Matt has assembled and organised.
Title: Re: Some thoughts on Matthew Andrew's 'Limits to PFM Reform in Development'
Post by: Martin Johnson on February 26, 2013, 08:58:28 GMT
Matt’s concept of ‘institutional entrepreneurs’ is clearly highly relevant to the development of appropriate approaches to PFM and governance reform (and good practice in these areas more generally) as is Matt’s concept and discussion of those at the centre of ‘social networks’ who, by virtue of their position in these networks, are less able or willing to perceive requirements for change, plus Matt’s discussion of the ‘distributed agents’ that need to be engaged from the very beginning to ensure that a good idea in principle is, in practice, a workable good idea that has half a chance of becoming an integral component of a new institutional arrangement. I imagine some readers may find the language and semantics a bit heavy where Matt deals with these concepts, but it is highly relevant stuff. As with other parts of the book, it touches on areas where I have felt some frustration in approaches that I have observed. One good example of this that I have observed is where the approach of 'external agents' was to write a budget preparation manual (over a very short period) and design a budget preparation IT system (also over a very short period) with little opportunity for testing of the business processes through what Matt refers to as ‘muddling’ or otherwise before locking the technical solution in. Approaches such as this can often minimise consultation with the ‘classic champion’ (Budget Director) and result in very little (if any) effective interaction with the ‘distributed agents’ that are ultimately the individuals who will make the system work (or not) in practice. Interestingly enough, in the example referred to here, the ‘distributed agents’ had been dutifully using the system – which was delivering an exclusively IT solution to a planning and budgeting problem.
Title: Re: Some thoughts on Matthew Andrew's 'Limits to PFM Reform in Development'
Post by: Napodano on August 18, 2013, 18:37:26 GMT
Podcast: Matt Andrews on the limits of institutional reform

Title: Re: Some thoughts on Matthew Andrew's 'Limits to PFM Reform in Development'
Post by: Napodano on September 28, 2013, 15:33:43 GMT
“Purposive muddling” and the limits of institutional reform

'PDIA stands for “problem-driven iterative adaptation” – though I prefer Andrews’ alternate formulation of “purposive muddling” for its simplicity. Still, it’s worth breaking down his acronym.
The idea of being problem-driven means that, rather than focusing on the solution being implemented, reformers should focus on the problem being addressed and then seek small incremental steps that move in the right direction. It’s worth highlighting one critical question that some other reviews of this book have glossed over: whose problems do the driving? The book brings some nuance to the development shibboleth of “ownership” with the concept of multi-agent leadership. Clearly, reforms will struggle if they target problems that are only seen as such by outside actors; yet having an internal champion may also be insufficient. Andrews makes the case (in the book and elsewhere) that a range of lower-profile actors throughout the reforming institution must be involved in the problem definition and search for solutions.'
Title: Re: Some thoughts on Matthew Andrew's 'Limits to PFM Reform in Development'
Post by: Napodano on October 31, 2013, 07:37:08 GMT
Another good paper from Matt on 'problem-driven iterative adaptation' theory compared to ‘solution- and leader-driven change’ one.

'Explaining positive deviance in public sector reforms in development' October 2013, UNU-WIDER.

'Public sector reforms are commonplace in developing countries. Much of the literature about these reforms reflects on their failures. This paper asks about the successes and investigates which of two competing theories best explain why some reforms exhibit such positive deviance. These theories are called ‘solution- and leader-driven change’ and ‘problem-driven iterative adaptation’. They are used to analyse data emerging from a case survey involving thirty cases from Princeton University’s Innovations for Successful Society programme. The bulk of evidence from this study supports a problem-driven iterative adaptation explanation, but there is reason to believe that solution- and leader-driven change hypotheses also have value. It seems that problem-driven iterative adaptation and solution- and leader-driven change are two viable paths through which positive deviance can emerge; although problemdriven iterative adaptation seems to provide the wider path for more positive deviance.'
Title: Re: Some thoughts on Matthew Andrew's 'Limits to PFM Reform in Development'
Post by: Napodano on September 25, 2016, 08:29:38 GMT
In these blog entries Matt starts sharing lessons from PDIA experiments over the past five years

PDIA stands for “problem-driven iterative adaptation"
Title: Re: Some thoughts on Matthew Andrew's 'Limits to PFM Reform in Development'
Post by: Napodano on August 26, 2017, 14:15:19 GMT
There is the unique opportunity to sign up for a Free online 15 wk course on PDIA (Problem Driven Iterative Adaptation). It starts 3 Sept

'This is a 15-week course for practitioners who are in the weeds of development and actually want to learn how to do PDIA. In this course you will have the opportunity to work on your nominated problem, as a team, using our tools. The course will include video lectures, required reading, assignments, reflection exercises, peer interaction as well as group work. We estimate that the weekly effort will be between 5-8 hours. We will use the recently published “Building State Capability: Evidence, Analysis, Action” book as the core text. In June 2017, 30 groups across 15 countries, successfully completed this course'. More at