I made the following comments on the proposed new indicator PI-PIM. I submitted a couple of days after the deadline for consultation, so I'm hoping that the PEFA Secretariat have some flexibility.
Positive Comments
It's good to see that there is reference to a 'national guidelines' with reference to economic analysis. This should mean that feasibility studies done for donors don't count unless they follow national guidelines. A national methodology should not be expected to differ greatly from standard donor practices, but there should at least be a consistent set of national parameter values, discount rate, value of time, etc. used for all projects, rather than the free-for-all that generally exists at present.
It's also good that the focus is on major projects and that a definition of 'major' is given.
'Economic analysis' is defined broadly to cover a variety of methods - not just CBA - which I think is helpful. Also important is the inclusion of PPP projects.
Recognition of the need for independent validation of appraisal findings is also important, but the text is not clear. 'For the analysis to have objectivity, it would have to be conducted independently (or at arms’ length) of the project’s sponsoring agency, i.e. validated by an entity other than the concerned MDA': does this mean the analysis has to be performed by an independent entity or validated by an independent entity, or both?
Possible Areas for Improvement
Rigorous appraisal should normally cover more than the 'economic case'. The UK, for example, also requires consideration of the strategic, financial, commercial and management cases at three points in the project cycle (identification, appraisal and pre-contract signature) prior to implementation. The financial case is mentioned in the PEFA narrative as being a component of the economic analysis: true up to a point, but this might not necessarily mean that affordability and exchequer impact are captured.
I would say that the most important deficiency is the absence of a dimension dealing with project identification/concept development, when a decision to proceed to feasibility and appraisal is made on the basis of strategic/policy relevance, strength of the intervention logic and affordability. The indicator deliberately leaves aside policy relevance stating that this is already captured under PI-12. This is indeed the case under the existing PEFA framework, but the proposed update seems to exclude the critical dimension (iv) of PI-12.
What is meant by 'capital investment project'? I think a definition of capital investment is required, presumably expenditure on fixed assets (tangible and intangible) mentioned in PI-PAM, but it would be good to state this.
Since PEFA is about performance, there needs to be some verification that processes are effective and not 'going through the motions'. The reject rate is a critical variable for assessing the effectiveness of the independent validation process: if all projects get through, then there's something wrong. Similarly, if the monitoring process never results in problem projects being stopped (or at least subject to major re-design) then there's also something wrong. These things might have been mentioned in relation to assessing these dimensions.
I'm not sure if it's essential for Government to maintain a database of all projects (the implication of dimension iii). How many advanced country central financial agencies do this? The UK has just woken up to the idea of having a database of major projects (the Government Major Project Portfolio), which seems like a very good idea, but is it necessary to have a database of all projects? It seems like a recipe for information overload in a large country. It also seems at odds with the decentralised philosophy of performance-oriented budgeting.
The World Bank has identified eight key features of good practice in public investment management. It is not possible to capture all of these in a single indicator. All the same, I am not sure that it is correct to define these as 'elements of the broader public sector context' as is done in footnote 12. Monitoring and evaluation, for example, seem to me to be important parts of the broader PFM system.
