Program Evaluation: Strategic Planning

One way to have useful and cost-effective evaluations is to strategically plan evaluation activity so it supports management activity cycles for planning, budgeting, analysis, program implementation, and benefits reporting and communication. In this way, evaluations generate information to feed into critical decision processes and continuous improvement. Planning evaluations strategically allows the program to phase in a full range of evaluation activities over a period of years, and to spread out the cost of evaluation over multiple years. A Strategic Evaluation Plan could be a living document that can be modified as the program evolves and changes over time.

Taking a strategic view of evaluation means that you need to:

  1. Establish your program's information needs to, for example, support decision making (see Informed Decisions), and
  2. Develop an overall strategy and lay out a plan for collecting, analyzing, and using evaluation data.

First, establish your program's decision information needs and questions

Since the idea is to have all evaluative information working together, you might start by taking stock of your decision needs and questions that need to be addressed to obtain information necessary to support those decisions. The illustrative table below is one way to begin to take stock of these considerations. Complete the second and third columns and decide what program areas require management attention and need to be evaluated (third column). For your program, fill in the cells (with details on known decision points, timing needs, expected key audiences, type of evaluation, etc.) for the program areas for which you will need evaluation information to inform critical decisions.

Management activity cycles

Decisions to be informed (examples)

Evaluation questions

Program area 1

Program area 2

Program area 3

Planning

       

Budgeting

       

Analysis

       

Implementing

       

Benefits reporting and
other communication

       

 

Second, develop an overall evaluation strategy

Ideally an EERE Office should have an overall evaluation strategy and supporting multi-year evaluation plan that addresses each critical program element, including a schedule for the planned evaluation activities and the resources set aside for them. A mix of evaluation types may be necessary. For instance, the decision information needs for R&D programs might require several different types of evaluation be performed over a multi-year period, going beyond peer reviews alone.

After you have clarified the decisions to be informed and established your program's evaluation questions and information needs, and identified program areas to evaluate, what should be your evaluation strategy? Here are some areas to consider.

1. Target evaluation activities to your information gaps

  • Concentrate evaluation activities on your program strategy where there are gaps in information needed to inform decisions. Developing program logic models can be very useful here. These help capture goals in a concise way and how these goals will be achieved, thus highlighting key performance measures and evaluation questions.
  • Prioritize needs, including considerations of who needs to know what, when, and in what level of detail and format.

2. Plan for high quality, cost-effective evaluations

  • A multi-year strategy is needed. Evaluation studies often take six months or more to complete and a schedule helps ensure that all program elements are assessed on a time scale that will provide information timely for decision points.
  • Avoid the trap of unnecessarily limiting or constraining the scope of an evaluation study to meet a compressed time schedule. For example, in some cases, results from an evaluation may become available every other budget cycle if an in-depth scope of investigation is called for.
  • Put in place standard procedures for gathering and validating as much data as possible as a routine part of program record-keeping.
  • Combine efforts where you can, for example do customer satisfaction surveys for all program customers at one time.
  • Set aside budget resources for evaluation, including an amount for studies that cannot be predicted. In government agencies, evaluation activity typically comprises 1 to up to 10 percent of a program's budget.
  • Establish a procedure to ensure the independence of the evaluation process.
  • Have a quality assurance process in place that calls for external review of the Study Evaluation Plan and the Draft Report and ensures data quality and consistency (especially when evaluation data comes from multiple sources).