简洁大方ppt模板.ppt

20112011年年1111月月1精选pptMonitoring and Evaluating ODAThis presentation will cover lWhy monitor?lWhy evaluate?lMonitoring in the World Bank,other multi-lateral development banks,bilateral donorslEvaluation in the World Bank,other multi-laterals,bilateral donors2精选pptWhat is monitoring?Definition of monitoring:a continuous process that uses systematic collection of data about progress in an on-going development project,for use by managers and other stake-holders3精选pptWhy monitor?To improve implementation of on-going projects;must provide practical informationTo assess whether development objectives likely to be metTo identify systematic issues and risks across projects,by country,by sector,and overallTo provide input for ex post evaluation4精选pptWhat is Evaluation?Definition of(ex post)evaluation:assessment of a completed project,program,or policy,its design,implementation,and results.5精选pptWhy evaluate and who cares?Why?Two reasons:To learn from experienceTo provide accountabilityWho uses(or should use)evaluations:Boards of Directors(MDBs)and Ministers and Directors General for oversight;ODA managers for policies and strategies;ODA staff for program and project designs,ways of doing business;Client countries,external partners,and the public for policies,aligning programs,research6精选pptM&E in contextlWorld Bank commits/disburses$20-23 billion/year;lApproves 280 loans and credits/year;active loan portfolio:1,400 in 120 countrieslBank also provides grants and other non-lending services(diagnostic work)lBank has total administrative budget of$2 billionMost other MDBs commit$1 6 billion/year Bilateral ODA:$200 million(Greece,Luxembourg,New Zealand,Portugal)-$25 billion/year(USA)Bilateral agencies have administrative budgets of$11 million(Luxembourg)to$700 million(Japan)7精选pptMonitoring in the World BanklEvery on-going lending operation is monitored by visits from Bank staff about twice a year;lImplementation status and results(ISR)report issued twice each year:lProgress on implementationlProgress in meeting development objectiveslProblem projects identifiedlPortfolio at risk reports separate methodology updated monthlylOther quality control activities include quality at entry reviews;quality of supervision8精选pptEvaluation:the topicslSelf evaluationlIndependent evaluation:why it matterslTools:projects;sectors;countries;etc.lCoverage:how many evaluationslResources/budgetlMain challenge for evaluationlDisclosurelOther Evaluation Issues9精选pptSelf-Evaluation in the BankSelf evaluation in the Bank:lEvery project has an implementation completion report,done by staff of responsible unit(about 280/year)lEvery new country assistance strategy now has completion report covering previous country assistance strategy,done by staff of responsible unit(about 20/year)10精选pptSelf-evaluation in other donorslMulti-lateral institutions:almost all MDBs have requirement for self-evaluation,although in some MDBs(Asian DB,EBRD,EIB),the evaluation department is involvedlBilateral donors:some require for projects over given value,others dont;SDC(Swiss):external evaluators do self-evaluation;GTZ(German)has innovative eVAL(electronic)system.11精选pptIndependent evaluationlWhat is independence?The extent to which the evaluation unit:Reports directly to a Board or other governing unit;Has staff,budget,and functions independent from management;not involved with management of ODABehaves independently by issuing objective,impartial,and balanced reports12精选pptIndependent evaluationIn the World Bank,independent evaluation,since 1973,has looked like this:Board of DirectorsInstitutional IntegrityPresidentPaul WolfowitzIndependent Evaluation GroupInternal AuditingSVP&Chief EconMD OperationsCFOSVP&Gen CounselInspection PanelMD Operations13精选pptIndependent evaluationIn bilateral donors,independent evaluation has evolved to look like this:Ministry of Foreign Affairs Evaluation UnitDirector General Aid agencyOperational unitOperational unitOperational unit14精选pptIndependent evaluationlWhy independence matters:lAllows for objective,unbiased evaluationslEvaluation findings are not tailored to managements wisheslIndependence lends credibility to findingslIndependence difficult to achieve and maintain15精选pptEvaluation Tools:World BanklProject evaluationslImplementation Completion Report(ICR)reviewslProject performance assessment reviews(PPARs)lCountry evaluationslCAS Completion Report(CASCR)ReviewslCountry assistance evaluations(CAEs)lSector,thematic,global evaluationslProcess evaluationslCorporate evaluations(ARDE,AROE)16精选pptEvaluation Tools:interactionsSynergies among different evaluation tools can provide valuable input Global(Global programs)Evaluation tools in the World Bank Corporate(ARDE)Sector(finance,education)Thematic(gender,decentralization)Process (PRSP)Country Assistance Evaluation CASCR Review CASCR ReviewICR Review(validation of self-evaluation)Project Performance Assessment ReviewCountry Assistance EvaluationICR Review(validation of self-evaluation)Project Performance Assessment ReviewProject levelCountry levelSector,thematic,policy levelGlobal and corporate level17精选pptEvaluation Coverage:World Bank18精选pptEvaluation coveragelIn most MDBs,full range of evaluations:project,country,sector,thematic evaluationslIn most bilaterals,few project evaluations;some country level;most focus on themes or programslMost bilaterals produces 3-10 evaluations/yearlDFID produces 20-25 reports/yr;lPotential weakness of country and thematic evaluations without underpinnings of project evaluations to use as input19精选pptResources for evaluation:MDBslResources vary,but as percent of total administrative budget,fairly similar:20精选pptResources for evaluation:bilateralslBilateral donor agencies are more likely to hire external consultants for virtually all evaluations(UK,Norway,Denmark,Switzerland,Australia)lBased on numbers from a few bilaterals(DFID,NORAD,SDC),evaluation budget is 2%of total administrative budget;figures of 0.5 2%have been cited by others.lSmaller donors(Greece,New Zealand)are still setting up their evaluation units21精选pptMain challenge for evaluationlHow to make evaluations usefullMost managers,staff focused on future,not on pastlEvaluation must/should be:lTimelylOperationally relevant:avoid generalizations,platitudeslEasily accessible:avoid long reportslConstructive:good practice should be rewardedlWidely disseminated and disclosed22精选pptDisclosure of evaluation reportslWorld Bank:public disclosure of all:lPPARslCAEslSector,thematic,global evaluations lProcess evaluationslCorporate evaluations lEBRD,ADB:selective public disclosure:evaluations involve private sectorlBilaterals:majority disclose,but varies:UK,Norway full disclosure;Germany selective disclosure23精选pptDisclosurelFor World Bank:reports and information on evaluation tools,approaches,on website:lhttp:/worldbank.org and scroll down to right-had side of home page,or:lhttp:/www.worldbank.org/ieglFor other donors:check their websites24精选pptOther issues in evaluationlEvaluation approachlUse of performance indicatorslSources of informationlRatingslProcesseslPartnershipslOther challenges25精选pptEvaluation ApproachWithin context of results-based management,Bank uses objectives-based approach,focus on outcomes and impactBank loans;grants;diagnostic work;dialogue Physical outputs;privatization;new laws;agency created;training completed;sub-loans disbursed.More efficient operations;better fiscal management;employment creation;better access to services;InputsOutputsExogenous factorsExogenous factorsOutcomeImpactLonger-term objective,like poverty reduction,better educational,health outcomes26精选pptEvaluation ApproachBank assesses:Outcome(relevance,efficacy,efficiency)lUse of performance indicators for outputs,outcomes,impactlRisk to DevelopmentlBank and Borrower Performance27精选pptSources of information for evaluationBaseline data in project appraisal documents(sometimes)Monitoring reports should have information on performance indicatorsData bases for project area;sector;country at beginning of project,end of projectOther country-level indeces(IMF financial statistics,Doing Business surveys;World Bank Indicators of corruption surveys;UNDP human development indicators)SurveysInterviews,focus groups28精选pptUse of RatingslWorld Bank uses ratings on project outcomes and outcomes of country assistance.No ratings on sector,thematic,other evaluations lOther MDBs use ratings for project outcomes;not for country assistancelBilaterals:some use ratings,only for projects;some do not use any ratings29精选pptEvaluation Approach:ratingslProject Ratings in World Bank:lOutcomes:6 point scale(HS-HU)lRisks to Development:4 point scale(H-N)lBank and Borrower performance:4 point scale(HS to HU)lPros and cons of ratinglPro:provides data and accountabilitylCon:takes focus away from lesson learning and richness of evaluation30精选pptProcessesIndependence does not mean isolationMDBs and bilateral donors incorporate comments from operational managers and staff and from stakeholders in client countries into evaluation reports:lBanks initiating papers for country,thematic,global evaluations sent to management for comment and to Board of Directors for information;lIMFs initiating paper for all evaluations posted on IMF website and public comment invited;wide consultations throughout process.31精选pptPartnerships in evaluationlCountry and sector evaluations sometimes involve partnerships with other MDBs and/or bilateral donors(joint or parallel evaluations)lOpportunistic,not systematic,partnershipslPros and cons of evaluation partnershipslPros:wider coverage,other viewpointslCons:more costly and more time required;may not agree on outcomes;may affect timeliness of final report32精选pptOther challengeslData difficult to obtain,or different sources have conflicting datalOften difficult or impossible to have a control group(especially in sector and country evaluations)lBeneficiary surveys can be costly,low response ratelProblem of attribution(cause-effect)difficult or impossible to establish33精选ppt