In my search for international comparisons of infrastructure costs (see recent post under Public Investment Management), I came across this useful resource: the International Benchmarking Network for Water and Sanitation Utilities (IBNET).
http://www.ib-net.org/Their database does not give direct information on the cost of infrastructure, but it does give a whole range of performance information on water and sanitation utilities by country and by utility. Quite handy and easy to use.