U.S. Department of Energy: Energy Efficiency and Renewable Energy

Energy AnalysisFeatured Content

Site Navigation ↓

Program Evaluation: Planning and Conducting Evaluations

This area of the website takes you through step-by-step processes for planning and conducting the In-progress Peer Reviews and Impact Evaluations described on this website. In general, for each type of evaluation, this "how-to" section will help you to:

Remember, evaluations help managers determine if timely adjustments are needed in program design or implementation to improve the rate, or quality, of achievement relative to the committed resources. Evaluations also provide programs with the evidence they need to demonstrate impacts/ results.

Begin by selecting the type of evaluation activity in which you are interested.

Web pages on "how-to" guidance for Planning and Conducting a Merit Review and a Stage Gate Review will be added at a future date.

Planning and Conducting In-Progress Peer Reviews

Objective review and advice from peers—peer review—is one of the standard mechanisms for effective management of highly complex and/or technically challenging projects and programs. It is widely used in industry, government, and academia. Peer review is a powerful and effective tool for enhancing the relevance, effectiveness, and productivity of the Office of Energy Efficiency and Renewable Energy's (EERE) research, development, and demonstration (RD2) activities because it taps the experiences and insights of experts in the field.

The EERE Peer Review GuidePDF defines in-progress peer review as:

A rigorous, formal, and documented evaluation process using objective criteria and qualified and independent reviewers to make a judgment of the technical/ scientific/business merit, the actual or anticipated results, and the productivity and management effectiveness of programs and/or projects.

The primary purpose of the EERE Peer Reviewer Guide (Guide) is to provide managers and staff with guidance in establishing formal in-progress peer review that provides intellectually fair expert evaluation of EERE Research, Development, demonstration and Deployment (RD3) and supporting business administration programs, both retrospective and prospective.

The Guide describes in detail a process for planning and conducting a peer review for an individual project, a project portfolio, subprogram, or entire program.

Below are links to key information from the Peer Review Guide, for the following areas:

Back to Top

Planning and Conducting a General Program Evaluation

No matter what type of impact evaluation you are going to undertake, there are certain general steps that can be followed to ensure your evaluation produces valuable and useful information. If you devote sufficient foresight and planning to the evaluation process you will enhance the value of information garnered from an evaluation. The EERE Guide for Managing General Program Evaluation Studies provides a wealth of information to walk you through the process. This section of the website gives you key points from the Guide, which covers the 14 steps within the process outlined below:

Define Purpose

Step 1: Decide the Evaluation Objectives
Step 2: Determine the Resources Available
Step 3: Determine the Timeline for Completing the Evaluation

Define Program Logic

Step 4: Construct or Confirm a Program Logic Model

Determine Questions and Methods

Step 5: Specify the Questions the Evaluation Must Answer
Step 6: Develop a Research Design
6a: Determine the Type(s) of Research Design Needed to Answer the Evaluation Questions
6b: Select Data to Be Collected and Develop a Data Collection Plan
6c: Select the Analytical Methods for Answering the Evaluation Questions
Step 7: Identify the Information that Will Go into the Evaluation Report
Step 8: Establish the Quality-Assurance Review Process

Statement of Work 

Step 9: Develop the Statement of Work and Select an Evaluation Contractor

Monitoring Implementation

Step 10: Develop an Evaluation Plan and Conduct the Evaluation Study
Step 11: Monitor the Evaluation Contractor During the Study

Use the Results

Step 12: Determine Distribution of Final Report and Results and Distribute
Step 13: Make, or Monitor the Making of, Decisions about the Program Based on the Evaluation Results
Step 14: Establish/Update Program Records for Use in Future Evaluations

Back to Top

Planning and Conducting Benefit-Cost Evaluations of Public R&D Programs

Below are the steps in an EERE approach to retrospective benefit-cost evaluation that includes assessing energy, economic, environmental, energy security, and knowledge diffusion impacts—all as part of the benefit-cost assessment.

It should be emphasized that depicting the evaluation approach as a series of sequential steps does not mean that the process is either linear or formulaic. Conducting a successful study requires an evaluator experienced in both the art and science of benefit-cost analysis, able to capture complex and multiple direct and indirect effects of public investment using creativity and a variety of techniques within the organization of a step-wise approach, and aware of the dynamic nature of the process.

The EERE Guide for Conducting Benefit-Cost Evaluation of Realized Impacts of Public R&D Programs covers the eight steps within the process outlined below.

Step 1: Begin an evaluation

Step 2: Estimate energy and economic benefits

Step 3: Estimate environmental benefits

Step 4: Estimate security benefits

Step 5: Estimate knowledge benefits

Step 6: Calculate economic performance metrics

Step 7: Perform sensitivity analysis

Step 8: Report results

Back to Top

Back to Top | View Full Site
Energy Analysis Home | EERE Home
Webmaster | Search
Content Last Updated: 06/11/2013