Making sure that there is demand for the services to be provided by the facility and that the costs of meeting this demand are not exorbitant are good starting points. If the project proposers are unable to provide a reasonable estimate of the number of users and their likely consumption of the public service(s) to be offered, then this is already a significant warning sign. If an estimate can be provided, this should be supported by evidence from other similar facilities (if available).
Once a demand estimate is available, then a capital cost per user and per unit of output can be estimated. This is not an ideal measure, but can still give an indication of value for money by bench-marking against other similar projects. For example, if the capital cost per user or unit of output is double recent experience, then this is already an interesting piece of information for making a judgement on feasibility. This obviously gets easier as a database of information is built up.
Of course, looking at discounted life-cycle costs (capital and current) would be better, but this is cost-effectiveness analysis (CEA) and might stretch capacities. Assuming CEA is a step too far, absence of estimates of operating and maintenance costs and of explanations of how these will be covered are obvious signs that there may be a problem with the viability of the project.
Multi-criteria analysis (MCA) can be useful in systematizing subjective estimates of benefits and arriving at a decision about about value for money. It's probably best suited to assessing similar projects against each other though, e.g., rural road projects or water supply projects. Projects can be required to achieve minimum scores to have a positive appraisal or MCA can be used to rank projects. This is obviously somewhat arbitrary, but may be better than nothing. Care needs to be taken to ensure that scoring is done by disinterested parties (easier said than done!) or else every project ends up meeting the threshold. I've seen proposals for using MCA-based appraisal systems that cut across sectors and type of project, but I'm not convinced by these: amongst other weaknesses, they tend to favour multi-purpose projects that tick all the boxes whereas in reality these are usually the most problematic (the best projects have a limited number of purposes - one being best!).
Remember also that where similar projects are being replicated, there is no need to do CBA/CEA for each project: it's OK just to do the analysis of a 'model' project to be sure that the programme is viable. Benchmarks can also be established for model cases and used to justify investment, e.g., the traffic level at which it becomes viable to upgrade a gravel road to asphalt, or the minimum size of settlement for which a waste-water treatment plant can be justified. This approach is helpful where there are some skills in CBA, but these are in short supply.
I attach one of my recent attempts to define screening and selection criteria. In a low capacity environment, it may not be possible to go beyond the preliminary screening criteria in appraising the project. The difference between preliminary screening and appraisal would then rest more on the quality and accuracy of the information. Better information at appraisal stage would allow a second round of filtering out poor projects (to use John's analogy).
