Author Topic: Conversation with Matt Andrews on limits to externally influenced PFM reform  (Read 40937 times)

Martin Johnson

  • *
  • Posts: 76
Dear PFM Boarders,

For our third fire-side conversation online we have the honour to have with us Matthew Andrews. He is an  Associate Professor of Public Policy at Harvard's Kennedy School, and commentator on governance in development. He manages his blog on Governance reform in international development at

If you are a registered member of the Board, you have the possibility to pose a question to Matthew online. The focus of the conversation with him is 'limits to externally influenced PFM reform and options for overcoming these'. As practitioners in transition and developing countries we all have our success and sorrow stories to share. We now have an opportunity to ask Matthew what his views are on externally influenced PFM reform, limits to the successful implementation of such reform and him possible ways to make our work more effective and sustainable.

The time allotted for questions is  Monday 16th to Wednesday 25th January 2012. After this period the interview will be closed and remain in the archive for future reference.

Martin Johnson

  • *
  • Posts: 76
To get the ball rolling, I would like to ask Matt the following simple question:

Question 1:
What is it that you mean by "limits to externally influenced PFM reform"?
« Last Edit: January 16, 2012, 10:50:02 GMT by Napodano »


  • *
  • Posts: 18
Hi Martin, Mauro and PFM Board readers. First, let me say thanks for inviting me to share. It is great to be here.

Answer to question 1

I would like to start by saying that my work differentiates between endogenous reform and what I call 'externally influenced' reform. I think it is an important differentiation.
Most of the research on institutional change in developed countries looks at reforms that emerge within countries, and are defined, funded, implemented and evaluated from within the context itself. These are endogenous reforms. Most reforms in developing and transitional countries are not so endogenous. (This is also the case in some developed countries, especially given the role of the European Union and in the current crisis in places like Greece). These reforms are motivated and/or financed and/or defined and/or implemented and/or evaluated by outsiders. They are thus 'externally influenced'.

I assume many readers on the PFM board are external agents working on these externally influenced reforms.

I argue that externally influenced reforms are different. They (typically) do not emerge as responses to shared responses as most endogenous reforms do. Solutions are not found through internal 'muddling through' as is the case with most endogenous reform solutions. They are not hatched through ever-expanding networks of internal actors as most endogenous reforms are. Instead, externally influenced reforms are often (but not always) introduced--at least in part--as 'signals' for external agents. They are often introduced as off-the-shelf solutions (typically best practices, to provide the best signal), through very narrow sets of champions who have direct engagement with the oustiders bringing the reforms in. Think of the Budget Director, Head of Treaury, etc.

Now, in my work I have found that these reforms are quite limited--because of these characteristics. I look at the limits in PFM in particular, through case studies and through the PEFA data--which provides a nuanced perspective that is unusual for indicators. With both of these sources of data, and working over a number of years, I find that countries with PFM reforms tend to all have limited results in the same areas:
- First, they make budgets better than they implement them.
- Second, they look better from a legislative perspective than they are, in terms of actual implementation.
- Third, they look great when one views the work of concentrated groups in central departments (like budget, treasury) but all falls apart when we move to the users in line ministries, districts, provinces, etc.

I call these three limits the downstream, de facto, and distributed or deconcentrated agent problems. After externally influenced reforms, upstream dimensions of PFM systems (in budget preparation) typically do better than downstream dimensions. After reforms, de jure dimensions of PFM systems typically look much better than de facto dimensions. After reforms, concentrated agents do great, but deconcentrated or distributed agents don't.

I would guess that many readers have seen exactly these results in places where they have worked. While these reforms deliver some gains, and in some places enough to act as meaningful 'signals' for 4 to 6 years, the reform also face routine limits--which are stubborn and repeat themselves over multiple reform experiences.

I could carry on discussing this all day, and provide some ideas on why these limits are peculiar to externally influenced reforms, exactly why the limits prove so stubborn, and how they might be overcome. But let me hand this back to you for the time being to see how others respond.

Useful links:
2.  (go to Day 2)
« Last Edit: January 16, 2012, 15:08:20 GMT by Napodano »

Martin Johnson

  • *
  • Posts: 76
Matt, thanks for this very interesting introduction to your ideas in this area. Given the experience that I have had in a number of countries and that I have shared with some of my fellow Board members in some of those countries, one of the first questions that comes to my mind is

Question 2

what can we, as external agents, do differently to help in creating demand for reform and to help in widening the constituency for reform, both of which would be required for a determination to 'muddle through' and/or 'do what it takes' to be established? Some of us saw reform move far and (after a lengthy gestation period) fast in Albania, for example, but only when senior managers actively drove the reform and demanded that management per se worked. For a while there was a determination to achieve. The demand was created but the constituency was not as wide as we would have liked. I am not quite sure how much of what was achieved continues to stick although I understand there has been some weakening of what was institutionalised during that period. I am sure you and others have many similar examples ... but what can we do, in effect, to facilitate the creation of the conditions for 'endogenous' reforms through our role as external agents?
« Last Edit: January 16, 2012, 18:41:33 GMT by Napodano »


  • *****
  • Posts: 666
Hi, Matt;

My question (Question 3) is:

who really are these external agents? I believe that donor representatives in countries are the external agents in the first instance.

When I am called into the country with defined ToRs, especially on a short term mission, I have very little scope to 'muddle through'. Indeed I try whatever it takes to involve as many counterparts as possible and instill a sense of 'mission' and a 'drive to results' but these are phases that, in my opinion, should be dealth with early on through dialogue between donors and Government. 

In longer missions, as mentioned by Martin, I agree that the TA team may have a chance to influence directions. So am I advocating long-term missions over short-term ones (shooting myself in the foot?). I am not sure because long term missions can become a real waste of money, if they choose to play safe and not 'muddle through'.

Final note: Once I tried to stop short a contract because the local counterpart was not interested at all in the work, but the donor representative was just not prepared to take the bureaucratic hassle.
« Last Edit: January 16, 2012, 20:12:51 GMT by Napodano »

John Short

  • *****
  • Posts: 506


"Good governance can only come from inside; it cannot be imposed from outside” President Zenawi of Ethiopia quoted in "Comrades or Culprits? Donor Engagement and Budget Transparency" by Paolo de Renzio, International Budget Partnership and University of Oxford, and Diego Angemi (  In Isomorphism and the Limits to African Public Financial Management Reform you presented an in-depth analysis of PFM reforms using the cross country PEFA indicator scores as the basis for the analysis.

Two questions (1) would you agree with President Zenawi or would you modify the quote; and (2) are you confident in the PEFA scoring – methodology and actual application – to be able to use them in cross country analysis?  (The paper had me searching for my dictionary quite a few times and was challenging and thought provoking – in a very positive sense!)


« Last Edit: January 17, 2012, 13:27:52 GMT by John Short »


  • *****
  • Posts: 201
    • REPIM
Question 5

Are we really looking at what stage of development a country is at before we can countenance worthwhile PFM reform?  You mention endogenous reforms really only occur in developed countries.  Certainly it appears to me in my work that there is often little incentive for politicians to embrace a fully transparent PFM system - less so whilst the aggrandising of wealth in the country is as a result of corrupt or semi-illegal practices.  Once a ruling class has captured wealth then we see the move towards greater rule of law, transparency (including PFM) etc. as actors transform their wealth into hotels, gas stations, airlines etc which all need protection under rule of law.
« Last Edit: January 17, 2012, 13:49:49 GMT by Napodano »


  • *
  • Posts: 13
Question #6

Hi Matt,

This is really a great topic and one we at the PEFA Secretariat are very interested in, not least because of the use of the PEFA data set for the analysis. Your pioneering work in that respect has been very inspiring.

My question is how you come to the conclusion that progress on endogenous reforms is more successful than progress on externally influenced reforms. This may be divided into some sub-questions.

Firstly, is there actually hard evidence to support that conclusion? (PEFA data does not cover the developed/OECD countries to an extent that allows comparison with developing countries).

Secondly, would one not expect the same patterns of reform progress to emerge in developed countries? (after all, most reforms require that legislation is in place before it can be implemented, so 'de jure' will typically be ahead of 'de facto' anywhere; and 'actor deconcentration' must be a constraint to progress in any country compared to reforms needing a few reform actors only), or could it be that externally influenced reform programs in developing countries are simply overambitious - especially compared to implementation capacity - whilst they may actually be driving reforms forward as fast as - or even faster than - endogenous reforms in countries at the same stage of development (or even as fast as reforms in developed countries). 

And finally a comment. We have at the PEFA Secretariat tried to build on you methodology of analyzing PEFA data during 2011 but using dynamic data from repeat PEFA assessments. This led to the Monitoring Report MR10 which is on our website, as well as a preliminary analysis of trends in PFM systems in LICs and MICs, presented at the ABFM Annual Conference in October 2011 (attached). This analysis indicated e.g. that progress in upstream and downstream PFM system elements appears to be at the same level even if downstream starts from a lower rating level, whereas the difference between actor concentration and deconcentration becomes even clearer when dynamic data is used. As the Secretariat is not a research institution, we posed in the paper a number of questions for researchers to dig into and hope that someone (like yourself) will be interested in taking over the torch and lead us into new knowledge on the basis of dynamic data.
« Last Edit: January 17, 2012, 20:05:59 GMT by Napodano »


  • *
  • Posts: 18
You guys don't wait with the tough questions. Let me get a start with some answers, though I hope others weigh in as well.

Answer to Question 3: Who are the external agents?

Mauro asks, "Who really are these external agents?" Great question. In my opinion more penetrating than it looks at first glance. Also more humbling--at least that's what I find. I ask this question routinely in class and it spawns some really uncomfortable conversations. So let me try an answer from a perspective I find useful to draw on--network theory and its applications to institutional entrepreneurship.
This theory suggests that we all occupy multiple social networks, and are embedded in each to differing degrees. In some we are located at the center of the network: Think of being at the center of a circle of agents like  ‘A’ is in the dark circle below, or ‘B’ in the red circle [NB: Click twice on the picture below to enlarge it; you need to register to the Board to see it]. Here we are deeply internal, invested in the network, having power in that network, with all its resources, influence, etc. In others we are at the periphery: Think of being at the edges of the same circle of agents, like ‘B’ is in the dark circle.  There are some networks that we do not inhabit at all: like ‘A’ is not engaged in the red circle to any degree. ‘D’ is not engaged in the black circle at all.
What does all this have to do with PFM reform? Well, think of the large dark circle as the PFM Board network—the international PFM reform network, where everything PFM is discussed, enjoyed, devoured, etc. And think of the red circle as the PFM politic and practice community in a given country. Maybe Georgia, or Zambia, Honduras, Cambodia or Jordan.  There are some PFM Board members who are totally external to the country’s PFM community (think of agent ‘A’).  Some members of good standing in the PFM Board network (like ‘C’) are also members in the country’s PFM community. They are peripheral agents in this network, however, maybe outside consultants or IMF or World Bank PFM specialists working on the country. Then there are central agents in the country (like ‘B’) who are peripheral agents in the PFM Board—part of the network but this is not where they get their power and influence and identity. Think the long-time, all powerful Budget Director, or the politically powerful Minister of Finance.
It is easy to see that ‘A’ is external to the country’s PFM network. She does not engage directly in this network at all. But I would say that ‘C’ is a borderline external agent in the country PFM community as well; the kind of agent many readers of this blog find themselves being. These are the types of agents that I would call ‘external’—and you could draw a straight line between them to see the degrees of externalization agents could also enjoy.

So, there is not one type of external agent but rather various types. Some are consultant outsiders who are really on the border of the network or even outside the network (the ‘A’ agents). Maybe this would be the short-term expert coming to take a look at some really technical issue. Then you have the external agents who are partially overlapping (maybe the World Bank team leaders who write the TORs for ‘A’ agents). They are there to build the relationships, get a sense of the network, lay the groundwork, and connect with useful outsiders.  Long term consultants could also be in this category (but they could be ‘A’ agents too—given that many long term consultants I see seem to try hard not to assimilate into local networks).
« Last Edit: January 18, 2012, 06:02:35 GMT by Napodano »


  • *
  • Posts: 18
Answer to Question 2: What do these agents do? And what could they do to help limit reform limits?

In my opinion many reforms are hatched by interactions between the agents I described in my answer to question 3, and the internal actors who overlap with the external PFM community (like B). The different agents play different roles, as above, and all are needed to make reform happen.
Theory on institutional entrepreneurship argues that institutional change requires having (at minimum) someone who is aware that the status quo is not working, someone who has some idea why it is not working, someone who has some idea of alternatives to try as reform, and someone with enough power and influence to get the community to adopt change. Some like to think this is one person. The proverbial champion. Unfortunately, network theory suggests we should expect that those with power (at the center of a network) do not have the awareness of problems. Agents at the periphery enjoy such awareness, but seldom have power.

So, change requires a combined and connected interaction of multiple players across—and between—networks. In the area of PFM, this would be having some power players in countries working with agents who are in the international community, given bridging agents between both worlds. All this to say that we should know what roles we can play as agents on the periphery…or beyond…and we should first of all be pretty humble as a result. As external agents
we cannot really do the change on our own (or even be the main role players).

To answer Martin, then, let me think about what we can do, which is to try and foster endogenous change in the contexts we are working in.
First, this would involve helping build awareness around problems in the context, such that we foster constituencies of internal restless agents, who are aware of the inconsistencies and deficiencies of incumbent structures. There are many issues that we as outsiders see in any country we visit, but we should be aware that issues are not problems and problems are the things that gain attention, cause internal people to examine their situations, and motivate change. So, as outsiders we can help to turn issues into problems. This would involve (a la John Kingdon’s work) using focal events to draw attention to problems, using data to construct stories about problems, and using past experiences to draw attention to problems.

This is best done with groups of internal agents and takes time. But it is the way to ‘hook’ people in. My sense is that this is what you finally got to in Albania, Martin: a place where the inside agents finally saw what their problems were after years of tinkering and ‘got it’. I read about a similar experience in a recent paper about an HMIS reform in Tajikistan’s health sector. External consultants ran into walls of opposition because—they realized at the end—the internal folks did not ‘see’ the same problems they did. The end result was that the internal group was starting a dialog of reflection, where they were questioning incumbent structures that had been in place for 70 years and were in the way of progress (

I would argue strongly against external agents defining problems or—what we usually do—introducing solutions into contexts where internal groups do not see the problems themselves. Where this is the case, I think the chance of getting political buy in to reform is really low.
As Mauro hinted, international organizations don’t hire consultants to go through this kind of ‘problem guiding’ process. If you work on a project preparation team you have a couple of weeks to come up with a set of activities and an end goal for four years out. I argue that this almost always guarantees limits (unless the country is already at a place of problem awareness).

Second, I would recommend that external agents focus less on bringing in specific solutions—MTEF, program budgeting, FMIS (yes, even IPSAS). These things are good, but we all know that they may not fit into contexts and we don’t really know what makes a good fit. Given my work in this area I think external agents are better located to help with broad-based scanning for solutions—given the problems that have attracted attention. This scanning would involve external and internal activities: Looking for a range of good ideas that address the class of problems being addressed, building a theoretical story about each (where they emerged, where they did not, what seemed to make them work, etc). This would then be the basis of a menu and an education campaign for insiders. They must ultimately choose, and I think we all believe that choices are better when there is a well designed menu to choose from. External folks can really help in this process.

For example, I see some great external support for Rwanda’s matching grant program for decentralization. The government got advice on doing matching grants from a variety of players, based on a variety of similar—but different—experiences. The advice was broken down into ideas about laws, supporting activities, etc. Some of the ideas came from inside Rwanda—Imihigo performance contracts, for instance. Ultimately, the country seems to have built a workable system around this assortment of ideas.
This brings me to the difficult part of my second recommendation. The literature on institutional change suggests that viable solutions are almost always hybrids; they are created from elements of different ideas that are both internal and external. In a sense, then, you should never be able to develop an MTEF or program budget that does not have some weird and wonderful element that someone will criticize.

Getting back to Mauro, I worry that international organizations do not provide the time or resources—or have the patience—to allow this kind of scanning. The emphasis is on finding solutions quickly and cost-effectively. Even worse, I think that international organizations—and governments in developing countries—are attracted to pure-form best practice reforms that are like limiting straitjackets. These best practice reforms are the best option when reforms are intended as signals. Consultants are likely to get agreement for their recommendation to develop a program budget, but will struggle if they say ‘we advise a process where Cambodia finds its own approach to strategic budgeting.’ A World Bank project leader will struggle getting a ‘Scan for hybrid’ project through the Board. If she recommends a best practice it will steer through much clearer. And everyone—including governments—will accept best practices over undefined hybrids that need to be found (that sounds like a lot of work). I
Limits to reform result where such external incentives and processes have us bypassing the process of finding and fitting reforms.

Third, external agents can help to build reform communities. We could play the role of convener, or connector. The literature on institutional change finds this kind of role vital. Think about the one who brings disparate groups together for meetings to talk about common problems, or who introduce new parties so that new ideas can diffuse. The idea of groups of institutional entrepreneurs being the facilitators of change is well established, and the idea that change even requires connections beyond these groups—to distributed agents—is also gaining headway.

Limits to reform emerge where these kind of connections are not created, either because time is limited or because the belief in individual ‘champions’ crowds out initiatives to build constituencies beyond such. This is, I believe, one of the main reasons why externally influenced reforms are often limited to change in concentrated groups. No one is connecting more broadly.
« Last Edit: January 18, 2012, 14:38:44 GMT by Napodano »


  • *
  • Posts: 18
Answer to Question 4

Hi John. Long time!

Thanks for the great questions.

First, I would tend to agree with President Zenawi, though you would probably guess from my responses to Martin and Mauro that I do believe external agents have an important role to play in any social change. This is partly because 'good' or 'poor' governance is a relative concept that is defined differently, depending on where one stands in the social network or field being governed. Those who are deeply embedded in any system often do not see the aspects of that system that others may call 'poor governance'. These central agents enjoy power and influence from the system, and have worked out how to govern in the context. The ones who see the need for change are frequently those on the periphery of the field or network, who overlap with other fields. These agents are also often the ones who learn about alternative ways of governing that can become the basis of reforms. The ones showing these various options are often external agents--consultants,  suppliers, etc. Ultimately, I would argue that the story behind any really well governed country involves the interaction of internal and external agents as a result. They just need to be playing their roles appropriately.

Probably my favorite story here is Singapore. Everyone says Lee Kuan Yew was the 'champion' responsible for the island-state's development. I can't argue, but if you read Lee Kuan Yew's autobiography you also learn about the many people who helped this process. Including Albert Winsemius, a Dutch economist who worked for the UNDP and became a llong-time advisor to the government. Winsemius played a great external role to complement the internal reform process. Result = change.

Second, on the PEFA data and my paper on isomorphism. So, I think one needs to be careful with the PEFA data--largely because of reasons Frans and the Secretariat are clear about. I won't get into questions about whether PEFA measures the right things. Beyond this, there are questions about the quality and comparability in assessments across time and countries. There are also questions about whether you can use ordinal data to do statistical analysis. You will see that I try to deal with these questions in the paper on isomorphism (and the earlier piece on African PFM...and a forthcoming Public Administration and Development article).

First, I work with a sample of 31 countries in these papers, even though there are many more assessments in place now. I do this because I worked with three grad studes to go through every single dimension score in all 31 assessments, examining the evidence behind the score and the way the score and evidence related to the PEFA criteria. I found a number of problems through this, which were corrected with the help of my assistants. I would say that the number of problems were higher for earlier PEFA assessments, and were concentrated in specific areas (internal and external audit attracted some attention, for instance, especially in Francophone countries). So: You need to clean the data a lot before using it.

As regards the statistical analysis of ordinal data: You will note that I use a specific statistical technique to deal with this. The technique does not assume equal distances between the different symbols on PEFA dimensions, which normal statistical methods do. In this way, I think it controls for some of the concerns Frans and others rightfully have about using ordinal data. I know he and others have other concerns and I am working on them. It should be noted that ordinal data like that emerging in PEFA assessments are used all the time in research, and can be very valuable.

To me, the PEFA data has actually turned out to be deeper and more valuable than I first assumd it would be. I always think that the quality of data is shown when results make sense in a practical way. In my opinion (and given responses) the analytical work I have done so far, using PEFA data, has yielded results that make sense and take some of our observations a step further than they were before. This, I think, suggests the data is capturing valuable information. Well done to Frans, the secretariat, and the group of people who developed the indicators. At the same time, I worry about the many people who use the data too instrumentally (in practice)...
« Last Edit: January 18, 2012, 06:00:14 GMT by Napodano »


  • *
  • Posts: 18
Answer to Question 5

I think a country’s context is really important to think about when crafting reforms. You mention the idea of 'stage of development' and tie this to the argument that elites behave better when they capture enough of the wealth.

I am not sure what the contextual factors are that matter the most, but I believe there are many of them. I also believe they are commonly overlooked when reforms are designed. I wrote about the idea that good governance looks different in different contexts a while back ( I think there are many contextual factors that determine what countries are ready for: 1. I think the kind of problems countries are dealing with matter a great deal; 2. I think the general capacities matter a great deal; 3. I think the level of economic development and rate of economic growth matter a great deal; 4. I think heritage matters more than we like to admit (colonial, social, cultural, etc.). Add to the mix the structure, strength etc. of elites and there is a lot to consider when even considering motivating reform.

The questions I would ask are: 1. Given that we all know these contextual variables matter, why do we all (and others) routinely overlook them and go ahead in designing our best practice reforms anyway? 2. How would we be able to 'see' these contextual factors, understand their influence on reform possibilities, and shape reforms appropriately? I will weigh in on my own responses to these questions, but would love to hear other ideas as well (including whether others find these useful questions).
« Last Edit: January 18, 2012, 06:03:39 GMT by Napodano »


  • *
  • Posts: 18
Answer to Question 6

Hi Frans. Thanks for the questions.

First, the issue of endogenous versus exogenous change. I don't think it is really possible to put endogenous and exogenous change processes head-to-head. Endogenous change tends to happen over much longer periods, in a much less  structured process ('muddling through' versus specified intervention) and often under the radar. It does not mean that endogenous change always works, and you are right to ask the question. Are the limits I identify simply the constraints to all change?

The empirical evidence I would say is relevant includes stories of successful change and reform in many developed countries. Stories of these reforms tend to tell of the kind of long-term processes I discuss above, where reforms emerge through problem-solving activities that lead to local actors finding and fitting reforms that are usually bricolaged combinations of ideas they see in other countries and their own ideas. I would argue that the path breaking reforms in New Zealand, Australia, Netherlands and others looked like this. I would also say that the more successful developing country reforms look like this too. South Africa's MTEF, Russia's FMIS. I would describe these as internally motivated reforms that drew on external help but were largely endogenous.

Beyond this kind of evidence, let me clarify what I am trying to do in talking about externally influenced reforms. I am not trying to say that these are less successful. Rather, I am trying to say that there are peculiar difficulties with doing this kind of reform that we should be aware of—and that most of the literature on change does not help think these through. The literature on change is mostly about endogenous change (motivated by insiders trying to address problems and make their organizations better). This work tells us that even these reforms face a lot of resistance unless groups of internal agents buy-into the idea of change. Externally motivated reforms face peculiar challenges of getting this buy-in, especially when the reforms are adopted because of some coercive external pressure that makes internal agents potentially more resistant to the initiative (crucial to the idea of isomorphism). This means we should probably expect resistance to be a greater issue with these reforms, and we should think up ways of doing something about it.
As another example, literature tells us that many successful change experiences emerge over long periods in an incremental manner--whereby insiders motivated by the need to address problems take a first step, learn from it, take a next step, learn from it, and ultimately end up with a hybrid that works. The incremental approach is considered important for a number of reasons: small steps take up less political capital and provide for quick wins, can offer a lot of lessons about what works, and allow capacity to develop incrementally in tandem with the change. Externally influenced reforms don't typically look like this. They are pre-designed, with externally defined best practices slated for adoption in rather short periods. So there is a question about how you manage the major political costs of these reforms, or how you do the 'fitting' of best practice that requires real mechanisms (and time) for learning, and how you build up the new capacities to do the new thing.

In short, the limits of externally influenced reforms may be the common in endogenous reforms as well, but I think the characteristics of externally influenced reforms make these limits particularly problematic. The limits of change are exacerbated by the ways of doing external reform.

You asked also if I think the developed world has the same limits as we see in the PEFA data. The idea is that countries pass laws and take time to catch up to these laws, so therefore a de jure-de facto gap is always in place. I think this is an empirical question and would love to hear of folks who have evidence from the PFM world in this regard. But let me give some of my ideas on it, and the evidence I focus on.

First, it seems to me that many developed countries do not start reforms with laws. Laws often emerge in response to changes in the way people are thinking about their jobs, and to catch-up to the new capacities and problems that have already emerged in society. So, I suggest a counter-hypothesis that you may have de facto practice leading de jure laws in many developing countries (especially where laws are slow in changing, like Francophone countries).

The most recent evidence I will point to here is the Global Integrity data on anti-corruption policies. The data shows that developed and developing countries are not that different when it comes to the quality of laws. Most laws look good now. The difference is in the gap between laws and practice. A country like Uganda has laws rated at 99 out of 100, but practice at 47 (I believe)  with a gap of 52! Italy has laws somewhere in the 80s, and implementation in the 70s, for a gap less than 10... Yes, there is a gap, but it is much smaller in developed  countries. This is routinely the case (see my blog post on this

So, I would argue that the gap between form and function is simply lower in developed  countries. And the large gaps in developing countries are a symptom of the incentives and approach to doing externally influenced reform. These reforms are often introduced as 'signals' which means an emphasis on the visible dimensions--like laws--and less emphasis on the unseen dimensions that are required to get things done. This can also happen in developed countries (I think we can see some of this in the recent revelations that countries signaled reform intentions to get into the EU without doing reform) but I would argue that the gap will be lower.

Finally, I really enjoyed the paper you guys did on the repeat PEFA assessments. I would agree that the upstream-downstream gap is the least interesting of the three I talk about. Actually, as your paper points out, one half of the upstream dimensions (related to the multi-year budget) are actually really weak. I think the real action is in the de jure-de facto gap and the concentrated-deconcentrated gap. These gaps turned out to be more important when all three were considered in multi-variate analysis, and I think most readers would agree that this is a real problem in the real world. It is great (at least to me) that the PEFA data helps us see and learn about things we see in the real world. Kudos to you and the secretariat.
« Last Edit: January 18, 2012, 06:10:04 GMT by Napodano »


  • *
  • Posts: 154

Question 7

I would suggest a different approach to some of those proposed in previous comments. I suspect that the circumstances for success depart somewhat from earlier suggestions.  My hypothesis is that success is based on a genuine collaboration between external actors with experience in a variety of circumstances, preferably including in their own countries, as well as knowledge of the results of related studies; and, local interlocutors who understand their own systems and where external experience may be adapted to the specific circumstances of their own countries. This acknowledges that specific solutions are not likely to be directly transferable from other contexts, but there are lessions to be learned from these other experiences that may be adapted to other venues. It is often unlikely that someone outside of the specific complexities of a particular country would be an adequate designer of the details of the necessary systems and interventions that would work in that country's context, without a knowledgable collaborator in the country; on the other hand, someone living within the system may not immediately see the potential strategies and adaptations that would solve their country's particular problems without the interaction with ideas that have been in some form successful elsewhere. Is there anything in your research/analysis that would give credence to this hypothesis?

« Last Edit: January 18, 2012, 06:05:01 GMT by Napodano »


  • *****
  • Posts: 348

Question 8

On the issue of de jure vs. de facto reform, can we perhaps make a distinction between different legal traditions and the relative power of the finance ministry? My impression is that in countries like the UK, Australia and New Zealand reform can go a long way without any legislative backing and (serious) piloting of significant reform initiatives by the finance ministry is relatively easy (and can be done prior to legislative reform). France, however, passed its new organic budget law (the 'LOLF') supporting performance-based budgeting (among other things) in 2001 and then took until 2006 to implement it in full. In formerly centrally planned economies, we find a need for a strong legislative base prior to reform, which is partly a reflection of legal traditions, but may also indicate the relatively weak position of the finance ministries in these countries (which does not have the power to advance reforms without legal backing).


  • *****
  • Posts: 160
Hi Matt,
I'm enjoying this, thanks.

I agree with Petagny and would not wish to detract from his general point, but our system in France is an exception, of course.  Yes the reform is legally backed and the structure was implemented gradually over 5/6 years - we are not there yet though. 

What makes France exceptional is the process that led up to the legislation. It was I think a good example of building a network and perhaps a constituency and above all developing a shared vision. 

It took a long time, but things got moving when a common view was developed across left and right (including (most of) their disparate parts and the bits in between) and as both the National Assembly and the Senate worked closely with the Ministry of Finance with actors moving between the legislature and the executive and with the engagement of a lot of Enarques (now there's a network!).  There is still a need for perfectionnement (muddling through? quel horreur!) but the mannequin (structure) is (I believe) uncontested and it is now a question of couture (haute, bien sur).

I have not really looked for a write up of the case study of the reform-up-to-legislation but the MoF used to have a fair bit of history on its website.
« Last Edit: January 18, 2012, 14:17:45 GMT by Napodano »


  • *
  • Posts: 154

As a follow-up to Petagny: Almost all OECD countries spend substantial amounts on performance measurement, however, none - when I last reviewed this 2 years ago - actually use the results in budget allocation. My suspicion is that France does not to this up to now.



  • *****
  • Posts: 201
    • REPIM

Question 9

On a different note... have you considered expanding your theme to the limits of externally influenced development (in general)?  It seems to me that the arguments you use could easily be applied as such.  There appears to be little evidence for aid being the key factor in promoting development.  Countries that have experienced improvements in key developmental areas appear to have done so outside of the "aid" ambit:  China, India, Brazil, Nicaragua (1979-85), Mozambique (mid 1970s), Costa Rica (1950s) etc.

Appleby Ghost


If I may make just one comment...

Never in my time, or I should hope now, would a civil servant anywhere "muddle through".  One reflects.  One gathers statistics. One consults. One advises. And then, if absolutely necessary (pursuant to the right political decision), one implements at the appropriate juncture, that is to say, in due course and in the fullness of time.  One does not 'muddle through'.

and a question... #10
By a 'network' is one to understand a collective noun for 'inter-departmental committee members'?

« Last Edit: January 18, 2012, 19:57:53 GMT by Appleby Ghost »


  • *
  • Posts: 18
Answer to Question 7

Hi Fitz

I agree with your thoughts. I think that external agents could play a really important role providing outside ideas into developing country contexts. This is often the role I see outsiders playing in the experiences where I see externally influenced reform having a meaningful impact.  I think that the role you suggest is much more complex than we take it as, however. The idea that outsiders can help insiders 'see' their institutional deficiencies requires a bunch of detail: how do they do this? The idea that outsiders can bring external ideas into new contexts and help them fit these ideas to context begs a similar questipon: how do they do this?

The questions do have answers. In my opinion (and given my research) external agents can help local agents 'see' their problems by constructing problems out of issues. Using data to show the extent of money wasted in a budget regime turns the 'issue of efgficiency' into a problem, for instance. External agents can also help local agents make sense of outside ideas if they are translators of those ideas--being able to say where they came from, under what circumstances, with what demands, etc. Beyond providing such knowledge ex ante, these translators should be skilled in facilitating experimentation with hybrid solutions, learn from such, and apply lessons in future experiments until a solution is found.

My concern is that many people working in development have neither the time nor the capacity (nor the understanding) to do these tasks well. Most external agents are 'solutions' people who are introduced to offer a specific solution into a context without fully explaining such or tee-ing the discussion up by helping local people to see why it is needed.

Even when external ideas are well presented, however, this is not enough. Research shows that institutional solutoons that work are seldom pure-form reproductions of one model or example. Instead, they emerge as combinations of attributes of multiple ideas--and some of these are always internal; emanting within the reform context with pre-existing capaicty to work and legitimacy to be accepted. Th question is: How many external consultants, IMF advisors, World Bank team leaders, etc. can help present the ideas of external best practice together with internal ieas, and construct local hybrids that work. This is a great role for outsiders, but not one that I think any education or consulting opportunities teach or facilitate.

Here is a section from my forthcoming book, where I talk about issues of finding and fitting solutions and the type of agents needed for such.
....I am cutting and pasting ext from here on.

One of the emerging lessons from these experiences [of successful institutional reform] is simply that successful institutional adjustments involve the emergence of mixed-form solutions—and not what Elinor Ostrom calls “optimal” institutions (which are embodied in the idea of ‘right rules’ or ‘one-best-way’ or ‘best practice’ reforms espoused by many development 'experts').   The bottom-line is that incremental reforms focused on addressing problems frequently result in combinations of elements that work together to get the job done. In their 2002 article on institutional change, Dacin, Goodstein and Scott present a similar idea in arguing that institutional change solutions often emerge as “bricolage.”  John Campbell examines change as bricolage as well, explaining this as the process whereby actors recombine available institutional principles and practices in ways that yield change. 

Bricolaged solutions only emerge, however, when there are many options to choose from and combine. This raises a second lesson from the two cases, which emphasizes the importance of scanning broadly for reform ideas. This lesson is also apparent in Ostrom’s work, where she describes the process of finding and fitting institutional solutions as follows: “There must be the generation of new alternatives, [and] selection among new and old combinations of attributes that are successful in a particular environment.”  New and old, internal and external ideas can form a menu of alternatives to consider in the change process, but in many senses Ostrom is talking about going beyond this. New alternatives emerge when scanning processes lead to the identification of multiple alternatives with detailed descriptions of their attributes—the laws that make them work, norms that give them credibility, and cultural-cognitive mechanisms that allow them to function. In this way, scanning must provide a menu of alternatives, with the recipes included. Reformers need this detail to see which ideas match the problems being addressed, which can be introduced with extant ingredients, and what new ingredients need to be found.

This leads to additional lessons about the scanners and scanning processes required to facilitate ‘finding and fitting’ of relevant institutional reform content. Reform ideas come from agents engaged in the change process, who act as ‘carriers’. Organizations like the World Bank have been called ‘carriers’ of external ideas, responsible for “promulgating new conceptions, standards, and practices.”  They can only provide for part of the scanning activity necessary in bricolaged situations, however. Internal agents must also be engaged, to ‘carry’ past and present ideas into the discussion about potential reform solutions. Without ‘groups of carriers’ like this, reforms are likely to be narrowly informed, which limits the potential of finding relevant “combinations of attributes” (using Ostrom’s term). 

Carriers need to do more than just propose alternative ideas, however. They must play the role of translator as well, deconstructing ideas so that reformers can determine the fit of such to context. In a practical sense, this means explaining where the idea emanated from, why it was adopted, how it was adopted, and what its core regulative, normative and cultural-cognitive content looks like. This translation is similar to a chef explaining how she cooks items on her menu and is a vital part of the finding and fitting process. It deconstructs interesting reform ideas into the constituent attributes that are ultimately combined as new hybrid solutions. It also helps inform the process of adapting these attributes to local contexts, which Rose notes is often an overlooked part of ‘best practice’ transfer:  “If a relevant programme of another country is selected for adoption, it must still be contextualized, that is, altered to suit the specifics of one's own national setting.”     

In many situations ideas are not translated well through ‘carriers’, sometimes because these agents do not know details behind the ideas they are proposing or because they do not have the time or incentive to explain such. In these cases translation can still happen, however, through the incremental process of trying ideas, experiencing mixed results, and learning from such. In referencing this approach, Elinor Ostrom argues that, “The process of choice … always involves experimentation.”   This, she notes, is because, “It is hard to find the right combination of rules that work in a particular setting” and one has to “try multiple combinations of rules and keep making small adjustments to get the systems working well.”  Experimentation allows reformers to act on ideas that emerge from the scanning process, to get a better ‘translation’ of what makes them work, and to see how they interact with other institutional elements that either pre-exist in the reform context or are being introduced through other reforms.
It is important to note what experimentation did and did not look like in these two examples. First, experimentation was not about performing a scientific experiment where the context is suspended (as in many randomized trials with potential policy solutions). Rather, it is about trying a real intervention in a real context, allowing on-the-ground realities to shape content in the process. Second, this is not about certain success or about proving that a specific intervention works.  It requires a willingness to accept failure of a specific idea, to take some attributes of the idea further than others, and to see lessons learned as the key emerging result. Third, experimentation requires having mechanisms to capture lessons and use such to inform future activities. It is not, therefore, about allowing a thousand flowers to bloom without focus.
« Last Edit: January 21, 2012, 19:54:19 GMT by Napodano »


  • *
  • Posts: 18
Answer to Question 8 (and the discussion about France)

Great comment. Yes, I agree that we should think about the way in which laws are developed in countries, how people see laws and the process of change, what political structures look like, how powerful the Minister of Finance is, etc. These are all contextual factors that are important in determining what kind of change is needed and what kind of change is possible. They are not the full set of contextual factors, however. I would argue that the kind of problem being addressed matters as well (some problems require and facilitate more radical change than other, for instance), as do a set of cultural variables, and a host of variables related to capacities etc. It would be great if the PFM Board could construct a list of the 'cvontextual factors' that impact reform, and start developing stories about these.... There is a lot of experience to capture here because, frankly, we know these things matter but don'[t always know how, why, and under what conditions.

I have developed my own theory about how institutional change happens, and how context matters, and I will add a discussion on it here, for those suffering with late night sleeplenessness.

This is an excerpt from my book, discussing contextual issues and change opportunities.

Complex institutional structures always exist

‘Institutions’ entered the world of development largely because of Douglass North’s historical analytical work linking such to growth. North showed that even the most rudimentary social structures (like small scale villages) have rules of the game, even though these rules differ in size and shape across contexts and through time.  This basic observation is an important starting point for thinking about context. Institutions always exist in all contexts, in many different forms. There are no such things as clean slates on which to write change scripts.

Regulative mechanisms are the most commonly studied form in new institutional economics.  These include laws and shaming practices that are often established, monitored and enforced by third parties. They promise reward or punishment in response to different behavioral choices, constraining behavior through extrinsic means. These regulative devices differ from normative mechanisms emphasized by new institutional sociologists. These include norms and values and communicate what is considered socially acceptable.  They manifest in behaviors one would call ‘acceptable’ or ‘appropriate’ and influence agents intrinsically by evoking feelings of honor or shame with compliant and resistant actions.  A third kind of institutional manifestation is what anthropologists call cultural-cognitive mechanisms. These manifest in ideologies and scripts  that are reflected in symbols of group affiliation and belief like religion, nationality, or language. They create frames through which agents interpret the world, structuring the way information is received, processed and given meaning.  Such frames bias agents to making specific choices, regardless of the extrinsic and intrinsic incentives created by regulative and normative mechanisms.
Many authors refer to individual regulative, normative and cultural-cognitive mechanisms as ‘institutions’. Where World Bank studies discuss ‘Institutional context’, for example, they typically provide lists of laws and other regulations.  In contrast, Richard Scott does not call any one of these mechanisms an ‘institution’ on its own.  Rather, he refers to all three as ‘elements’ of institutional structures, arguing that “Institutions are comprised of regulative, normative and cultural-cognitive elements that, together with associated activities and resources, provide stability and meaning to life.”  A law on its own is not an institution. Neither is a norm, nor is a cultural-cognitive script. An institution emerges as a mix of all three.  It is like an omelet, produced when someone combines eggs, cheese and milk into a bowl, with a whisk, over heat.
Various scholars have embraced Scott’s approach to thinking about institutions,  positing that the three elements (with supporting practices and resources) combine to form ‘rules of the game’. Other variants of such multi-dimensional constructs are evident across the literature as well.  Greenwood and Hinings speak of “interpretive schemes” for instance, which comprise “values, norms, beliefs and rationalizations,” that, “provide both a logic and a propellant” for action.  An even broader set of authors refer to ‘institutional logics’ in describing structures of material practices and rules that emerge from and reinforce entrenched assumptions, values and beliefs.  Such approach has been used to examine a wide set of topics in particular fields of social engagement—“clusters of organizations and occupations whose boundaries, identities and interactions and defined and stabilized by shared institutional logics.”  The role of cultural-cognitive and normative mechanisms is central in such conception of an institution. These mechanisms give durability to the institutional structure and form a foundation for other elements. Evincing such thought, authors have referenced the importance of “widely shared assumptions and values”  underpinning institutional structures and also presented these elements as “master principles”  and “taken-for-granted, resilient social prescriptions.”   

Figure 3.1 illustrates the institutional structure implied in such work. It shows the institution—or logic—as an iceberg. Normative and cultural cognitive mechanisms form a base atop which regulative devices operate. Together, the three elements establish a rule that shapes the way agents think, weigh decisions, and behave. It is important to note that this kind of structure incorporates both informal and formal elements—not one or the other. The iceberg metaphor reflects this, suggesting that a large part of any institutional logic is unseen or below the water line because it is informal—implicit, unwritten and seldom visible.   The inference is that informal elements are not bad or inferior. Informality is rather a fact of institutional structure—and a fundamental one at that.   

Multiple institutional structures always exist

Research further relates the idea of ‘inter-institutional’ contexts that comprise multiple institutional structures.  Studies show that these exist across domains, varying between the marketplace, state, corporation, profession, religious community, and family.  Agents split their lives across these domains and thus continually face multiple logics. For example, doctors in religious hospitals have their perceptions, choices and behavior shaped by rules of the market, state, medical profession and religious order. Institutional structures can vary within domains as well, and in fields and organizations.  Meyer and Hammerschmidt explain that Austrian civil servants are influenced by a traditional public sector logic emphasizing process and a new managerial logic stressing results.  Purdy and Gray find two logics shaping behavior in state dispute resolution offices in the United States.  A judicial logic emphasizes clearing case loads while a public policy logic focuses agents on facilitating better-quality decisions.

In all cases logics comprise multiple elements, having some cognitive basis, normative expression and regulative form. As such, they all resemble ice-bergs like that suggested in Figure 3.1. They combine as institutional contexts that one could compare with a sea of icebergs. Some institutions in this sea may be more prominent than others, and some may be more closely related—even clashing at times. Researchers look for such characteristics when describing institutional contexts. Studies note particularly whether and to what degree some logics dominate their context, how competitive alternative logics are, and if logics seem to co-exist or clash. The Austrian study, for instance, describes the traditional public sector logic as dominant but also sees the new managerial logic as competitive.  Greenwood and Hinings explain that British municipalities conform to either of two competing logics, one favoring “corporate bureaucracy” and the other “professional bureaucracy.”  Purdy and Gray find the judicial logic co-existing with the public policy logic in dispute resolution practices.  In all examples, agents must navigate these contexts given such patterns of dominance, competition and co-existence.

Context shapes change opportunities

One wonders how institutional change happens in places where dominant institutional logics prevail. Explanations of such change vary, but most authors posit that opportunities are created—or limited—by the interplay of incumbent institutional structures or logics. Some approaches describe an equilibrium-seeking dynamic where institutions shift between logics.  Others suggest that change is driven more by political contests that may or may not bring stability, depending on whether there is consensus on a logic or mix of logics. Both perspectives emphasize the importance of pre-existing institutional orders, interruptions to such, and power plays involving agents. Both perspectives also suggest that change is most commonly limited.

Studies commonly examine change in contexts where one structure dominates the institutional order. This structure could be a singular logic or a stable amalgam of co-existing logics. Such structures typically undergo what Bartunek calls first order change, “consistent with already-present schemata.”  In essence, adjustments reinforce the incumbent logic rather than replacing it. This is a key element of path dependence theory, which posits that future institutions typically reflect those of the past. Cognitive and normative foundations of dominant mechanisms become entrenched in new scripts and symbols over time, for example, and embed themselves through repeated behavioral patterns of expanding sets of agents. Regulative mechanisms are fine-tuned to reflect such logic as well. These changes alter power relationships and capability mechanisms, locking-in the dominant logic. This logic can appear irreversible when many agents invest heavily in it over long periods. These investments become like sunk costs and decrease the willingness or ability to switch to alternative institutional structures, even though these may continue to exist.

Unless there is a disruption. This could take the form of an exogenous shock like the recent global economic crisis, or destabilizing endogenous jolts like demographic adjustments. These can be once-off events or gradually emerging disturbances. They create change opportunities if incumbent institutions cannot facilitate the resolution of emerging problems.  In such situations agents are confronted by weaknesses and contradictions in their pre-existing rules.  Faced with failing incumbent rules of the game, agents may look for second order change— replacing an incumbent logic with another alternative. 

Disruptions do not always foster radical change, however. The degree of change depends on the severity of disruption, how much it tests the legitimacy of pre-existing dominant logics, whether viable alternatives exist, and if agents are in place to struggle for and facilitate a transition. Different conditions yield different scenarios, shown in Figure 3.2:
1.   Dramatic shifts in logics—or switches—are possible when faced with disruption, but only where the legitimacy of pre-existing institutional orders is tenuous (a condition one could call weak dominance) and alternative logics are present and competitive, supported by agents capable of challenging the old order. 
2.   Where one or other condition is not met, change will be more limited. It could foster coexistence or hybridization where elements of an alternative institutional logic are introduced alongside elements of the incumbent dominant logic. This can cause uncertainty about rules of the game, supporting and motivating differing choices and behaviors.  Mixed conditions could also facilitate the emergence of a new logic altogether, however, where elements of the dominant incumbent merge with elements of new alternatives.
3.   If conditions are not met one should expect even more limited change. This could involve a limited version of layering, where visible elements of alternative logics are introduced ‘on top of’ less visible norms and cognitive scripts of incumbent structures. This layering could lead to more significant change if new elements are actually implemented and reinforced over time as additional layers of similar change.  Conversion is another option, where the pre-existing logic is preserved but power is moved from one group to another. Incumbent structures are re-directed to serve the interests of different agents and agendas in such situations.
These three scenarios suggest that different conditions affect the degree to which institutional change emerges, even through disruption. In all cases more disruption allows more change. Change is greatest in the Logic Switch scenario given weak dominance, competing alternatives and powerful change agents. Change is more limited in the Coexistence and Hybridization scenarios, given mixed conditions. Change is most limited in the Layering or Conversion scenarios where conditions are not conducive to significant short-term adjustment. As argued, layering could lead to deeper change if reinforced by similar reforms over time, where new layers actually create room for additional change.
Most authors would argue that these last two limited change scenarios are a more likely outcome of institutional adjustment when a dominant logic is in place, even in the face of significant disruption. This is partly because informal dimensions of incumbent logics change slowly, if at all. Mauro Guillen notes such in stating that logics are “deeply rooted in collective understandings and cultural practice, and resilient in the face of changing circumstance.”  As a result, one is likely to find informal elements of incumbent logics holding on even when regulative mechanisms appear to change.

External reformers commonly overlook context 

There is reason to believe that many external reform designers will discount the influence this kind of contextual complexity has on reform, at least in the design phase. The very idea of institutional reform assumes “that institutions can be purposefully created or re-created by rational agents.”  This is an intoxicating assumption for those working in development, who may consider themselves the ‘rational agents’ capable of manipulating the way developing countries work. Such thinking is implicit in the ‘Cargo Cult’ concept discussed by some postcolonial theorists.  This refers to a belief “that development is something that flies in from the outside” to overwhelm and improve the inefficient ways of developing countries.  All that is needed to effect change is the superior outside idea and influence.

Such thinking is reflected in Peter Evans’ description of how reforms work in development:  “International organizations, local policy makers, and private consultants combine to enforce the presumption that the most advanced countries have already discovered the one best institutional blueprint for development and that its applicability transcends national cultures and circumstances.”   Such thought runs through many discussions of institutional reform in development, and the mechanisms used to assess governments. Reforms are typically presented in overwhelmingly positive ways, as initiatives where “the right institutional framework [can] unlock a nation’s wealth potential.”  Constructs like the World Governance Indicators give the same message, promoting a confidence in outside ideas as powerful, generic solutions. This confidence helps to explain chapter one’s finding that reform designers advocate more and more reform when evidence suggests a less than 50-50 chance of success. Contexts in which reform does not work are disparaged as having weak leadership, but not given more attention. The assumption is that superior externally sourced solutions will eventually prevail if local leaders allow them to.

This idea of a Cargo Cult does not hold across all development, however. Many designers of external reforms do see the importance of local context in the change process. The push to introduce contextualized Poverty Reduction Strategy Papers is one indication of this, as is the continued emphasis on country ‘ownership’.  Institutional reform strategies, handbooks and evaluations produced by organizations like the World Bank also routinely cite the important influence context has on reform.
Contextual factors are seldom detailed, defined, or deciphered in such references, however.  This is partly because of an inability to see and map context. There is something to be said for the fact that a large part of any institutional logic is informal and difficult to see. One cannot expect external reform designers to identify informal institutional elements falling below the iceberg’s water-line given the limited engagement they enjoy in target countries. Reform projects in organizations like the IMF and World Bank are designed with limited budgets in periods ranging from six to eighteen months, often by ‘experts’ with limited country experience who fly in and out of the reforming context every six months. Their government counterparts are often in narrow technical agencies at the head of economic agencies like the Ministry of Finance. It is questionable how much these agents can speak to the country’s political and cultural contexts, partly because these contexts usually stretch beyond their reach and also because many institutional elements are invisible even to those affected by them. This is particularly the case for most dominant structures, which are taken-for-granted and hence enjoy low visibility.  Furthermore, reform engagements typically focus on the solutions and not the problems, with pressure to finalize program documents for specific, ambitious, and linear reform programs. This constrains discussion about contextual problems or the cultural roots of such. As Peter Evans says, “Arenas that are less accessible and less transparent … are ignored, almost of necessity.” 

Getting serious about context

There are many stories where external reform designers overlook contextual factors that ultimately limited the space for change. Consider Andrew Wilder’s account of recurring failure in Pakistani civil service reform.  Citing a former finance secretary, he notes that external agencies working in Pakistan treat civil service reform as “a technical exercise” in which “problems are reduced to boxes and then solutions are found to fit into the boxes.” The finance secretary concludes that, “The political and cultural contexts are lost in these exercises” which is a problem because of the “inherently political nature of civil service reform.” Consider also Indonesia’s insolvency law, which suffered premature failure because it was never “socialized”  into the context.  Bemoaning such results, Melbourne University’s Roman Tomasic notes that “multilateral agency expectations and timeframes may not always be in accord with local realities.” 

In these and other situations reforms ran into trouble when they encountered a cluttered institutional context comprising multiple logics—iceberg-like structures combining regulative, normative and cultural-cognitive mechanisms. Contexts with strongly dominant logics and weak alternatives failed to accommodate the intended institutional change, even though there was disruption that many may have thought always facilitates adjustments. The problem was that contexts actually provided limited space for change. The conditions were much like those in the left column of Table 3.1, which shows ‘small hole conditions’ that limit the size of a potential reform peg. When external agents overlook contextual conditions, they assume that all situations have large holes, shown to the right, and accommodate a high degree of change. Unfortunately, this is often not the case, and small hole conditions undermine reform attempts.

There are various ways in which reforms would be improved if context was better considered in design. One could imagine that the selection of where and when to do reform would be sharpened, for example. Fewer institutional reforms would be initiated in contexts with small-hole conditions—where governments and societies may not be ready for change. A greater sense of context could also help inform expectations of reform, and help guide processes of implementation and oversight. Reform designers aware of the importance of context could factor in regular assessments of the factors named in Table 3.1, for instance, monitoring context and not just progress with technical content in reforms. Finally, one could imagine a sense of context assisting in defining reform content itself. Reforms initiated in large-hole settings would be fashioned to support viable alternative logics that have emerged in such contexts, for instance. Reforms initiated in small-hole conditions could be shaped to increase the size of the reform space itself—introducing alternative ideas, gathering potential agents of change, or helping to interpret disruptions as sources of change.

Indonesian insolvency reforms discussed in the introduction might have focused on strengthening informal dispute resolution mechanisms instead of creating more formal laws, for instance.  Informal mechanisms were ostensibly a more viable and accepted mechanism given the context, and their medium-term use could have bought time for government to properly develop a more legitimate court system. Similarly, Malawi’s anticorruption activities could have stressed the role of agencies promoting the new anticorruption logic (churches, for instance) rather than betting on change in the traditional conservator of the old corruption logic (the executive). External agencies could have required that Argentina’s legislature actively debate the causes of fiscal profligacy instead of just rubber stamping reforms proposed by the executive.

An awareness of context could also help external reform designers ensure that their interventions do not undermine change opportunities. A sizeable literature argues that the international aid community impacts the politics of development when it engages in institutional reforms.   Some argue, for instance, that loans from international organizations can contribute to and reinforce dominant logics of soft budget constraints.  Others suggest that external funding for reforms can strengthen a logic of corruption by adding to sources of cheap rent.  These perspectives are controversial. They need not be, however. If one considers the theory of institutional change behind Table 3.1 it is obvious that external engagements can shift a situation from potentially large-hole to small-hole conditions. Loans can stabilize an otherwise disrupted context, for instance, reinforce a dominant logic that would instead be tenuous, or support agents who are unlikely to change the status quo. A mix of these effects was arguably evident in Argentina and Malawi, where donors tempered the severity of disruptions and ultimately facilitated layering or conversion instead of more far-reaching institutional change.

Such argument suggests that external reform designers should better account for context than they have done in the past. Think, perhaps, of project preparation processes that allow one to assess the size of the hole in which reform pegs are planned to fit. This, arguably, is the role political economy analysis is slated to play in project design. Such analysis is surely a good thing, and it is positive to see increasing numbers of reforms informed by perspectives on politics. There are some high quality analyses emerging in this tradition,  but many emphasize contemporary politics only, ignoring the historically embedded logics that may be driving behavior. There is also a tendency to focus on political figures or parties, and not the systemic structural mechanisms in which such actors are embedded. One wonders how political economy studies could actually comment on such mechanisms in even the best case scenario, given their hidden, taken-for-granted nature. Studies can only speak about what is seen, so should be expected to overlook ‘below the water’ content in this institutional icebergs driving behavior. 

The value of external contextual assessments is also questionable because of inevitable external bias in such, which can reinforce an ignorance of the real context. In discussing this point, Kate Kenny notes that external agencies are increasingly recognizing the importance of considering context when designing reforms. She points to a widely held belief that “despite the many differences” it is fully possible for external reform designers to come to know and understand “the Other.”   Often, she notes, this understanding is slated to be “achieved through the project methodology.” Methods involving categorization, quantification and sense-making are always tinged with bias, however. Kenny speaks of a bias towards having ‘binomial’ representations in development. There are experts (in the aid agencies) and non-experts (in the countries), for instance, and good politicians (who support external reforms) and problematic politicians (who do not). Kenny argues that analyses fostering such biased perspectives “ensure that organization members [in external entities] remain blind to their inherent assumptions.” 

Given such problems, this book argues that written studies by outsiders are probably not the best entry point for concerns about context. Chapter seven suggests a different course of action. Reforms should focus on the disruptive problems that emerge in particular countries, and designs should allow contextual complexities to reveal themselves while addressing these problems. This requires an approach to institutional reform that acknowledges the endogenous nature of such and seeks to develop and implement designs actively, through a problem solving process, in the context, with local agents occupying the driving seat. This is proposed as the only way in which contextual factors can really be effectively accounted for in institutional reforms.

Even when context is accounted for, however, externally influenced reforms could yield limited results—especially if they still emphasize ‘reform as signals’. The next chapter argues that such reforms do not only overlook context. They also over-specify externally sourced practices as generic solutions, but over-simplify what these require to work.

« Last Edit: January 21, 2012, 20:07:26 GMT by Napodano »


  • *
  • Posts: 18
Comments on France and on performance management in Europe and beyond

The French example interests me a lot. The most interesting thing is how so many french speaking African countries introduced PEFA-endorsed reforms (like multi-year budgeting) before France. Benin, Guinea, Niger, etc. had adopted this and other reforms before France. Obvioulsy 'adoption' did not mean much in these places. And 'adoption' of many reforms does not mean that much, including performance management reform. I like Don Moynihan's book on performance management in the USA, where he looks at the reasons agencies adopt performance info and the use of such. He paints a picture of many external pressures--of coercion and mimesis--just as I see in development. Under such conditions performance information has been developed but not used in many places, so yes I agree with the comment about it.

France may look different, however, although I have no first hand experience to speak of. I do know that some countries have adopted aspects of performance management over time, as the context provided opportunities, and in functional ways. Renate Meyer and Gerhard Hammerschmidt wrote a great set of articles on this in respct of Austria, for instance. The story in such cases may be that some agents find use for performance info and introduce and use it, even if the rest of the administration does not.

« Last Edit: January 21, 2012, 19:50:42 GMT by Napodano »


  • *
  • Posts: 18
Answer to Question 9

Yes, I have adopted this as a way of looking at development more generally. And yes I also agree that most countries progress when there is a strong endogenous influence. But I also hold to the idea that external influence matters a lot. This is because developing countries grow to be part of the global economy (so the need to engage externally grows when the countries progress) and also because there are lots of great ideas externally (which gets to the role that external sources can play--bringing new ideas in, helping locvals make sense of their problems, etc.)
« Last Edit: January 21, 2012, 19:54:59 GMT by Napodano »


  • *
  • Posts: 18
Answer to Question 10, and HA's comments on 'muddling through'

When I talk about a network I am referring to the idea of an interconnected social group. The PFM Board is an example. the connections between agents in this group could be direct or indirect, via degrees of separation, informal (through personal relationships), or formal (through committees or organizational hierarchies). So it is not the same as an inter-organizational committee although such committee would be part of a network. The members of that committee would bring second degree connections to their staffs, bosses, and political leadership, making the entire network much larger. Theorists find that agents play differnet roles depending on their different connections... some easy reading on this is Malcolm Gladwell's "The Tipping Point"

In terms of muddling through. I do not mean muddled, and I don't mean that officials pursue things without focus or sense. I am rather relating to Charles Lindblom's idea of incremental change, whereby processes of reflection, experimentation, learning, adaptation and combination of new and old ideas lead to the gradual emergence of new institutional strcutures and policies.  This relates very well to the description of French reform... and the approach you say adminstrators take to change--thoughtful, incremental, step by step...

Here is a simple description of my thinking:

Successful reform experiences that take time and involve multiple steps that build on each other remind one of well heeled theoretical arguments about how policy and institutional solutions often emerge; as a puzzle, over time, given the accumulation of many individual pieces.  Modern versions of such perspective are commonly called incrementalism or gradualism, and attributed primarily to Charles Lindblom.  In what is also termed ‘muddling through’, the approach holds that groups typically ‘find’ institutional solutions through a series of small, incremental steps. These steps are relatively cheap and have the prospect of early success, or quick wins. Both of these are important in contexts where change faces opposition, which is usually the case with institutional change in development. The small steps also help flush out contextual challenges, including those that emerge in response to the interventions themselves. This approach is especially relevant in uncertain and complex contexts where reformers are unsure of what the problems and solutions are. A gradual step-by-step reform that incorporates active opportunities to reflect on experience is particularly appropriate in such situations. Elinor Ostrom finds this in many of her complex research contexts, where the combination of ‘muddling through’ and “efforts to learn and the capacity to adapt … [have] contribute[d] to the emergence of effective” institutional solutions.   
« Last Edit: January 21, 2012, 19:56:17 GMT by Napodano »


  • *
  • Posts: 18
More on question 8

The question of strong and weak ministers of finance etc. is relevant to all of us, but the issues go beyond this. They are pretty evident in the USA where there is no conventional minister of finance and the budget comes out at least officially through the Congress. So, how does reform work here? Recent articles on the subject look at the problems of executive budgeting in the USA and the way in which these kinds of issues impact reform possibilities. I have observed that the issues of separation of powers, limited government etc. that lie behind the US model creat an illogical set of tensions that make reform really hard to do. This is me trying t get to the deep factors influencing and shaping opportunities for change.

See (also attached below)
« Last Edit: January 22, 2012, 07:51:44 GMT by Napodano »


  • *****
  • Posts: 160
In your reply to Question 8 you insert a very interesting extract from your book which will be keeping a lot of members busy for some time.  I think we'll all be looking forward to chapter 7 (where you will suggest a course of action?).  In that extract you refer to local agents in the driving seat.  I can't help recalling, from about 10 years ago, a Budget Director in Africa  commenting on the EFAs' continuing stress on the importance of 'locals' (they didn't specify them as 'agents' in those days) being in the driving seat by noting that 'locals will always be in the driving seat: the relevant issue is whether they will be wearing a cap or not'.

I hope we boarders don't lose sight of your request for us to come up with examples of how we have addressed contextual factors. 

I also wonder how one can measure/assess changes in the attention span of external actors.


  • *
  • Posts: 10
Hi Matt,

This is an interesting subject, and I have really enjoyed it, even reading it during the night. As part of the local staff at the time of changing the budget process in Albania I want to add a comment.

External vs. internal/local: Working for about ten years at the Ministry of Finance of Albania, many times I have experienced the situation when high officials at the MoF, including Ministers, have used the expression “we are doing it because it is required from our international advisers”.  What I mean by this is the 'predominant perception' that dictates reforms. Credibility is an issue here; it’s difficult for local officials to undertake a reform without referring it to a good example or to the advice of some clever people working in international institutions, even it is well designed. It should be praised from somebody else.

The project of changing budget process in Albania is an interesting case: First, it started from external influence, including DFID, WB, and IMF. It took many years to be accepted (de facto) form local staff that together with external advisors achieved a successful end. In fact, it is not the end, but the beginning of new area of budgeting. Moving to an output oriented budgeting has affected the manner how the budget is discussed even in the Assembly/Parliament.  The discussions on the performance of the budget are done in terms of outputs rather than in terms of money. Most of the MPs discussed the performance of budget 2010 not in terms of money, but in terms of achievements (outputs). This is good achievement, and thanks to external advisors.

The question (Question 11) that remains is if this new situation is a new norm or just external influence inertia? Is there a need for an external push?


« Last Edit: January 24, 2012, 08:03:49 GMT by Napodano »

Glen Wright

  • *
  • Posts: 66
This is an interesting discussion and one that I have followed the past few days with interest. 

I tell you what has irked me about some of this externally influenced PFM reforms that I have witnessed and unfortunately participated in over the past two decades in the transition countries of CEE, later in Africa and Asia.  External donors have played something of a double game on this.  I have seen instances where the WB goes in and does a PFM, closes the project, and within a few months, the EC, DFID, USAID, or some other donor comes along and starts up the same program with the same components, methods, etc.  The donors sort of take turns in these countries doing these programs.  Sometimes they build on what has been done, but in many cases they try to start all over and throw out all that has been done.  The poor donor recipient doesn't know what to do with this revolving door of donors. 

My question (Question 12) is: has this been your experience or observation?

A second observation is seems there has been more attention by the external donors on the substance of their assistance, with these Paris Agreement, etc, and not enough on how the donors should coordinate their approaches and methodologies.  Question 13: Is this your observation as well?

« Last Edit: January 24, 2012, 13:45:50 GMT by Napodano »


  • *****
  • Posts: 348
Question 13

I'm not wishing to question the importance of the dimensions of reform Matt has drawn attention (which seem spot on), but should we also not under-estimate the importance of luck and patience in the reform equation? Being in the right place, at the right time and with an attractive menu of ideas (and recipes) seems to have been important in the Albanian case that Sybi describes. Patience is also critical: DFID were supporting PFM reform in Albania for around 10 years in total. These also seems to have been a lot of these factors in the Ethiopian case (see attached article by Stephen Peterson).

You win some and you (more often) lose some, but when you win the pay-out is big! (Not an original thought I'm afraid, but I forget who made it first  - in rather more elegant terms, I'm sure).

At some levels at least, the kind of arguments Matt seems to be making are beginning to gain some traction in the donor organisations (even if not fully aligned with Matt's thinking). Here's a quote from a post yesterday by Richard Allen and Francesco Grigoli on the IMF PFM Blog referring to some research they have been doing for the World Bank on enhancing the capability of central finance agencies:

'...the study supports recent thinking that “best fit,” “good enough governance” and second-best solutions, as opposed to “best practice” solutions, provide a better approach to reform, and that there is a need to take account of the institutional environment as well as technical conditions. Technical solutions should be proposed only after a detailed review of institutions has been carried out. Solutions should be tailored to the specific political and institutional environment of the country concerned.'

The question is whether these approaches will have an impact on front-line aid delivery, where disbursement rates are still the most watched performance measure and where more in-depth analysis of context might seem like just more grit in the works.
« Last Edit: January 24, 2012, 16:08:27 GMT by Napodano »


  • *
  • Posts: 18
Response to Stone (Simon?)...

I am glad the extract is not too heavy. It is an example of me trying to create a frame through which one can make sense of the way context matters, and how reforms can best address contextual issues. I have added the conclusion from Chaptr 7 below, so you can get a sense of the way I think reforms can be improved. Essentially, I think the best way of dealing with contextual complexity is by drawing attention to problems that become windows that internal folks use to examine their situations, see the weaknesses of incumbent mechanisms, and look for alternatives.

I liked your story about the Budget Director looking for his hat... In my opinion he is right in saying that 'he is always in the driving seat' but this doesn't mean the car he is in is the one that is doing reform. I have never seen a government that actively says it is not for reform, or a donor-driven project that lacks the 'signature' of a Minister to suggest it is 'owned' -- but I have seen many of these reforms fail because they ultimately were not driven by insiders or owned locally.

This is partly because contextual factors limit the engagement of locals in externally motivated change processes, partly because the external changes are not locally developed and translated, and partly because the folks that do own reforms locally are often in narrow groups that have limited connection with what I call then deconcentrated or distributed agents that ultimately have to implement In essence, effective reform requires a budget director with a bus driver's hat...and a busload of PFM folks from all over government

Here is the conclusion of chapter 7...on 'problem led' reforms as the way of flushing out contextual constraints:

Institutional reforms can be improved through problem-led learning

These examples contribute to this chapter’s basic argument: Externally influenced institutional reforms are more successful when they foster problem-led learning. This happens when interventions exhibit a ‘problem focus’ and ‘flexibility’ in design and implementation. These characteristics help to ensure that projects fit in complex contexts. They help construct issues as problems that garner attention and become vehicles through which the need for change can be seen and addressed. These problems provoke a reflection on contextual constraints that is necessary to instigate change. They also guide efforts to revise or abandon problematic incumbents. This de-institutionalization creates space for new institutions. Finally, problems motivate a search, within the context, for viable alternatives to replace faulty incumbents—a contextually fitted approach to pre-institutionalization that ensures initial ‘solutions’ fit political and capacity realities.

Readers may question this argument, positing that institutional change is facilitated as effectively by positive opportunities—when reform is presented as an innovative way of doing better. Roy Greenwood and colleagues examined a change process that began with such motivation, to Canada’s accounting profession.  They found that the failure to articulate reform in the language of problems led to indifference to change. When change was articulated as a response to a generalized threat faced by the accounting profession, however, it became more accepted and even urgently and broadly demanded. These findings echo with various examples in the sample of 44 health projects and analyzed in this chapter. Many ‘better’ institutions—laws, procedures, standards of care, and ways of thinking—were met with indifference. They offered opportunities for improvement, but would have been better framed as ways of overcoming problems and avoiding failure. A problem focus provides such frame. 

Employing a problem focus in reforms is like ensuring a carpenter keeps his eye on the hole he is trying to fill, noting both opportunities and constraints it poses for peg design as well as the importance of filling the hole as a final objective. It is not good enough to create a poorly fitted peg that does not solve the problem. Project flexibility allows the same carpenter to learn progressively about what peg to create and how to fit it, ensuring effective functionality regardless of form. When combined in a reform setting, the two characteristics can accommodate significant institutional change. Table 7.2 contrasts such approach with the more common solution-driven isomorphism discussed in chapters four and six. Reading from top to bottom, the right hand column shows basic ways in which problem-led learning can draw attention to disruption, contribute to the weakening of incumbents, and motivate the search for alternatives. Combined with the way problems draw groups of agents together, these effects all foster what chapter three called ‘large hole conditions’ conducive to a high degree of institutional change. Isomorphic approaches, in contrast, foster ‘small hole conditions’. They emphasize ‘solution mimicry’ instead of problem solving, provide little impetus for reflection on incumbents, and de-motivate the search for in-context alternatives. As argued in chapters five and six, agents engaged in such change are often located in narrow cohorts at the organization’s boundary, allowing just enough external influence to ensure the organization looks legitimate without really having to change (as per strategic decoupling). Change is limited to the visible dimensions as a result, and reforms are limited to making organizations look better than they are.

The discussion behind Table 7.2 informs four ways in which external agents might improve the way they do institutional reform in developing countries, and overcome constraints related to context. First, they should promote a problem focus in everything they do, using data and the like to help local actors see festering issues as problems. Second, they should create opportunities for local actors to reflect on their problems, without pressure for specific solutions. Third, they should support local initiatives to question dominant incumbents, and to create an urgency about adjusting faulty incumbents. Fourth, they should help local actors see that problems can be solved, and put in place flexible pathways towards finding alternatives to status-quo mechanisms. Finally, they foster groupings of internal actors through all of these actors, helping convene and connect agents who may not ordinarily engage with each other but hold different pieces of the change puzzle.

This chapter used examples from health sector projects to illustrate that externally influenced reforms can foster such problem-led learning. The relative success of projects pursued using this approach speaks to its value. One should note, however, that many of the cases cited as successful reforms in this discussion may not have led to significant institutional changes in the project period. Getting 7 percent of Burundi’s sexually active population to use condoms may not constitute a deep change for many readers. Such examples are evidence of a growing awareness that change is needed, however, and reflect challenges to incumbent structures and attempts to bolster alternative structures. These are the early steps in institutional change. Realistically, these steps may by the most one should expect to achieve from external reforms working through three, four or five year projects—especially in difficult contexts that often prove un-ready for more expansive change.

The hope of many externally influenced reforms is surely that additional steps will build on these kinds of gains, endogenously or with further outside support, and lead over time to expanded possibilities for change. This is like facilitating change in small steps, as repeated acts of layered change (shown as step 1 in Figure 7.2, a revised version of Figure 3.2 discussed in chapter three). These steps are the feasible starting points in contexts where incumbent institutional mechanisms dominate, alternatives have yet to be identified and capacitated, and there are only narrow groups of agents engaged in change. If these steps are built upon they may ultimately lead to greater change opportunities, as the creation of new hybrids and maybe even complete shifts in logics. These steps are shown as ‘2’ and ‘3’ in the figure.
 Figure 7.2 is intended to show that change is iterative, and involves creating more contextual opportunity over time—and then acting on such opportunity. Much like a carpenter may work to make a hole larger so that it can fit a larger peg. It is important to note that problem-led learning is central to this prolonged change process and not just the early steps. As an example, Tulio Falleti’s study of health sector reform in Brazil indicates that an accumulation of responses to problems yielded significant change over decades.   Reforms transpired and deepened over this time as agents drew attention to emerging problems associated with the weak national health system. These problems were dynamic, changing shape and severity as the country decentralized and service demands grew. The overall reform has involved decades of responses to such problems, leading ultimately to far reaching change in the size, shape, and orientation of Brazil’s health sector.
Such process typifies endogenous institutional change in many contexts. The introductory examples of Norwegian and Swedish public financial management reforms are good examples. So is the history of change in the United States civil service system, which has involved many incremental interventions provoked by problems.  Patronage was considered an important aspect of such system until the problems of low-merit bureaucrats and the assassination of President James A. Garfield led to change in 1882, for instance. The assassination was a ‘focusing act’ that drew attention to patronage as problem, not just an issue. Nearly sixty years later, concerns over the political activity of civil servants also flared into a problem that facilitated the Hatch Act.  Perceived problems with civil service activities again emerged in the 1970s, leading to far-reaching reforms at the end of that decade as well.
These examples are intended to provide a contrast with the experience in many developing countries. Governments in these contexts commonly face pressures to introduce a range of externally defined ‘best practice’ reforms in short periods across health sectors, public financial management domains, civil service regimes, and beyond. Bolivia stands out as an example, where externally influenced reforms decentralized the government, professionalized the civil service, and introduced performance management in the period between 1988 and 2002.  The country had been a centralized military dictatorship before this, with entrenched patronage. These systems ultimately proved resistant to much of the proposed reform and Bolivia was criticized for failing in its commitment to reform.  In retrospect it seems that a better explanation was that many Bolivians did not see their incumbent systems as problems and hence did not choose to change these systems.  Repeated efforts to introduce these deep and extensive reforms routinely fail, and fail to build on opportunities for change that do in fact exist.

The example is meant to show that externally influenced institutional reforms often assume that change can happen in deep ways, across multiple domains, over short periods of time, on the strength of interventions that introduce externally identified best practices. This assumption does not reflect typical patterns of institutional change, however, or the way in which contextual realities create opportunities for adjustment. Opportunities arise as issues turn into problems, often over long periods, provoking reflection on deficiencies in incumbent structures, de-institutionalization of these, and a search for new alternatives. This process takes time, is typically incremental, and cannot be downplayed. It is naïve to think that reforms demanded and designed from the outside can short-circuit such process and force through reforms across governments in short periods.   Externally influenced reforms can, however, play a role in facilitating genuine change processes, or contributing at important points when such processes are already under-way, through interventions with a ‘problem’ focus and ‘flexibility’. Such interventions should be underpinned by patience and humble expectations, however, and a commitment to empower endogenous institutional change mechanisms as the ultimate goal.
« Last Edit: January 25, 2012, 08:33:34 GMT by Napodano »


RSS | Mobile

© 2002-2022
Powered by SMF