When to Evaluate
There is no hard and fast rule on when to conduct a program evaluation. There are EERE requirements for peer reviews and technical reviews, and DOE and OMB requirements for quarterly and annual performance assessments. Deciding what to review and when to review it should be done on a multiyear basis, although unforeseen circumstances may also require review on an ad hoc basis. Generally speaking, all program activities should have some evaluative activity in addition to peer review and technical review and performance measurement and monitoring, at least every 2 to 3 years.
General evaluation studies (e.g., retrospective impact and cost-benefit assessments, process evaluations, and market assessments) can inform decisions at all stages of the program life cycle from planning new activities to following up a few years after that has been completed. The type of general evaluation study done differs throughout the life cycle of the program, covering information needs of each major management area: planning, budgeting, implementing, and assessing.
Ideally, an overall program-wide evaluation strategy should be developed to map out planned evaluation activity over a multi-year time frame so information produced,
- Is timely, without being so frequent that not enough time passes to act on recommendations from the evaluation
- In sync with the program's management and decision processes, and
- Complements other program data collection activities.
In addition to the timing of evaluations dependent on management activities along the program life cycle, here are some circumstances that suggest you need a general evaluation study:
- Activities or performance goals and reports that cover large portions of the program's budget or are highly visible to major stakeholders for other reasons;
- New programs, particularly pilots that you want to understand if and how they work because you want to replicate them if successful;
- Programs or activities that appear to be in trouble, or are experiencing better than expected success.