The Proper Use of Stipulations in M&V
June 1, 2002
The majority of Super Energy Savings Performance Contracts (ESPCs) projects have used measurement and verification (M&V) Option A for at least one energy conservation measure (ECM). This article discusses how M&V methods affect certainty and apportion risk and how to make Option A methods work with, not against, the savings guarantee.
Option A in Context
Risks and Responsibilities
Equipment-Performance Risk—Typically Assumed by the ESCO
Required M&V Activities
What the "Smart Money" Says about Stipulations
Before You Stipulate...
Much of the information in this article was taken from FEMP's new Detailed Guidelines for M&V Option A, which provides recommended practices for using the Option A methods described in FEMP's M&V Guidelines for Federal Energy Projects, Version�2.2. The Option A guidelines bridge the differences between FEMP's M&V Guidelines and the latest revision of the Measurement & Verification Protocol (IPMVP 2000), also discussed in this article.*
Agencies generally use the least expensive M&V option that provides sufficient certainty that savings guarantees are met. Option A, which allows stipulation rather than continuous measurement of some factors, includes some of the simplest and least expensive M&V methods. Overall costs for M&V in Super ESPC projects through FY 2001 have averaged 2.9 percent of guaranteed savings.
Why not use the M&V method that will provide the most accurate results possible?
M&V methods generally increase accuracy and certainty in proportion to their cost. At some point the incremental reductions in savings uncertainty are no longer justified by the increased costs. It makes good sense to do just the level of M&V needed, because M&V costs reduce the amount of savings available to purchase ECMs. Option A methods are generally less expensive than those that use more measurement or complex analysis.
What's the difference between "measured" and "stipulated" factors?
ESPC savings are determined by comparing energy use before and after the installation of ECMs. Savings cannot actually be measured, because they are defined in relation to a baseline that exists only on paper (after project installation) and represents the energy cost that would have occurred if no new ECMs had been installed. Energy usage is the product of factors such as energy demand, motor loading, operating hours, and weather conditions.
"Measured" factors are quantified by metering or monitoring of individual components, systems, or buildings. Measurements can be taken continuously, for hours, days, or weeks, or for moments to obtain data "snapshots." Data from these measurements are used to calculate savings using engineering calculations or models, regression or other analysis algorithms, or computer models.
To stipulate a parameter is to hold its value constant regardless of what the actual value is during the contract term. Option A methods allow some values to be "stipulated" instead of "measured" if they can be estimated with a reasonable degree of certainty and their contribution to overall uncertainty is small. Stipulated values must be based on reliable, traceable, and documented sources of information. Direct stipulation of energy savings is no longer allowed.
A stipulation in an ESPC M&V plan is an agreement between the energy service company (ESCO) and Federal agency to accept a defined value or functional form of a specific factor to be used in determining the baseline and/or post-installation energy consumption, which will be used to calculate the guaranteed savings. If related requirements are met (i.e., satisfactory commissioning results and annual verification of equipment performance and that maintenance is being done), the guarantee is considered to be met.
Measurement and Stipulation as Technical Terms
In fact, short-term measurements or data gathering of some kind is always among the sources of information used to arrive at stipulated values, but this is not equivalent to required measurements to verify savings. Similarly, not every aspect of every contributing factor is directly measured to verify savings. Stipulations are used in every M&V method without being called out as such in the M&V plan. For example, standard, accepted engineering practice used in M&V commonly includes the use of assumptions based on science and experience.
For the purpose of discussing M&V plans and methods, stipulations are distinguished as values that the ESCO and agency agree to hold constant during the contract term. Once set, the stipulated value (or functional form) is always used in calculating savings. Conversely, the value of measured factors is repeatedly determined during the contract term and the actual values are used in calculations to verify savings.
New Definitions in IPMVP 2000
The 2000 version of IPMVP uses new definitions of Option A and stipulation, which makes them different from those in the FEMP M&V Guidelines. Under IPMVP 2000, Option A is now called "partially measured retrofit isolation," and compliance with IPMVP requires measuring at least one parameter. FEMP guidelines still allow verification without measurement in some cases. Also IPMVP 2000 defines a stipulated factor to be estimated or assumed but not measured, while FEMP's guidelines include measurements as a possible source of information for stipulations. The Option A guidelines bridge the differences by using the IPMVP definition of stipulation and by showing which Option A methods no longer comply with IPMVP.
What parameters are commonly stipulated?
Lighting operating hours, lighting fixture power, constant-volume fan power and schedule, and boiler efficiencies are commonly stipulated after equipment performance and schedules have been characterized. More complex parameters that are sometimes stipulated are chiller performance curves, variable frequency drive speed-demand curves, and equipment load frequency distributions.
Under what conditions does it make sense to use stipulated values?
The following are some indicators that stipulation may be appropriate:
- The ECM
- has a high probability of delivering expected savings.
- contributes a small percentage to overall project savings.
- contributes a small percentage to overall project uncertainty.
- The agency
- is willing to accept some uncertainty.
- has experience with similar ECMs.
- The cost of monitoring is not justified by the value of increased accuracy.
- Monitoring serves no other purpose (such as performance monitoring or diagnostics).
- The ESCO has no control over the factor at issue (such as operating hours).
How should uncertainty influence how stipulations are used?
Overall savings uncertainty, and how much individual parameters contribute to overall uncertainty, are key considerations in whether to use stipulations. Savings uncertainty can be assessed by identifying the factors that affect savings and estimating the potential impact of each.
Rules of Thumb for Considering Certainty
These "rules of thumb" are keyed to the figure below:
- The most certain, predictable parameters can be estimated and stipulated without significantly increasing uncertainty.
- Stipulating parameters that represent a very small degree of uncertainty and a small part of overall savings will not significantly increase uncertainty.
- Parameters that represent a higher percentage of project savings and uncertainty should be measured.
- If estimated savings are high but uncertainty is low, the budget will support measurements, which could be used for monitoring equipment performance as well as for M&V.
- If estimated savings are small and uncertainty is high, stipulation would only increase uncertainty, and consideration of whether the ECM is worthwhile might be warranted.
Whether to stipulate parameters that can vary over time because of weather, performance degradation, occupant behavior, or other factors depends on who will assume the "uncertainty risk."
M&V Without Stipulations Leaves Risk with the ESCO
"Risk" in the M&V context refers to the uncertainty that expected savings will be realized. Assumption of risk implies acceptance of the potential monetary consequences. If no stipulated values are used and savings are verified based on measurements, then all risk resides with the ESCO, which must show that the guaranteed savings are realized and compensate the agency for any shortfall, regardless of contributing factors.
Stipulations Shift Some Risk to the Agency
Using stipulations means that the ESCO and agency agree to use a set value for a parameter throughout the term of the contract, regardless of the actual behavior of that parameter. Stipulations shift some risk to the agency, because they mean that the ESCO gets credit for savings determined using set values that may vary in reality. The largest risk is that stipulated values may be incorrect and overstate the potential savings. In that case the agency has agreed to pay for unrealized savings. However, savings may also be greater than anticipated.
Why take the risk of stipulations at all?
Appropriately used stipulations can simplify M&V and reduce costs. They can also be used to precisely craft the guarantee and apportion risks and responsibilities. Both ESCOs and agencies are reluctant to assume responsibility for factors they cannot control, and stipulations are often used to match control and responsibility. Risks in energy projects can be categorized as relating to either equipment performance or usage.
Risk related to usage stems from uncertainty in operational factors. For example, savings fluctuate depending on weather, how many hours equipment is used, or how much energy is consumed for heating or cooling. Since ESCOs have no control over such factors, they are usually reluctant to assume usage risk.
The agency generally assumes financial responsibility for operating hours and load by either allowing baseline adjustments based on measurements, or by stipulating usage parameters. Where measurement is not practical, stipulations can be based on long-term historical trends, such as typical meteorological year weather data, occupancy rates, or production levels.
What are the potential outcomes of stipulating?
Stipulating operating hours often makes sense because the ESCO has no control over this factor. The potential consequences are small if the stipulated value can be estimated to a reasonable degree of certainty and represents an appropriately small proportion of overall project uncertainty. The risk can be reduced by basing the stipulated value on reliable information, including measurements before ECM installation.
In the case of operating hours, risk is mitigated by the fact that a variance from stipulated usage is offset by the correlated variance in utility bills. For example, if operating hours are lower than anticipated, both nominal savings and the utility bill are also less than anticipated (assuming that contracted energy rates for calculating savings match actual rates).
|Example Lighting Project|
If operating hours were 20 percent lower than stipulated for a lighting retrofit that saved 20 percent of lighting energy use, the agency's costs associated with this ECM—taking into consideration costs for energy bills and fixed ESCO payments—would be about 5 percent higher than if no retrofit had been done. A 5 percent increase in lighting system energy costs would typically represent about 2 percent of the site's overall budget for electricity (assuming lighting represents 40 percent of building electrical load).
If operating hours were 20 percent higher than stipulated for the same hypothetical project, utility costs and savings would both be higher, and energy and ESCO costs associated with the lighting system would be about 3 percent less than if no retrofit had been done (again, assuming stable energy costs).
Performance risk is the uncertainty associated with characterizing a specified level of equipment performance. The ESCO is ultimately responsible for selection, application, design, installation, and performance of the equipment and typically assumes responsibility for achieving savings related to equipment performance. To validate performance, the ESCO must (at least) demonstrate that the equipment is operating as intended and has the potential to deliver the guaranteed savings. This almost always requires measurements. The ESCO also must achieve specified standards of service (temperature, humidity, and lighting levels, etc.).
If performance parameters are stipulated rather than measured, then the agency is assuming the risk of unrealized savings. For example, if equipment efficiencies are stipulated, the ESCO has no motivation to ensure that optimal efficiencies are maintained, because nominal savings will be calculated using the stipulated efficiency value. Actual savings, however, will be unknown.
Using stipulations reduces, but does not eliminate, the need for other M&V activities. All Super ESPCs require defined energy-usage baselines, savings estimates, guaranteed savings relative to baselines, and procedures to verify performance and savings. Required M&V activities include the M&V plans, commissioning, and annual M&V reports.
M&V plans must show how performance of each ECM will be demonstrated, including calculations, assumptions, and sources of stipulated values. The ESCO's M&V plan must outline and schedule procedures to be performed during the contract term. The plan must specify periodic activities that will verify the ECM's continuing potential to deliver guaranteed savings and that performance standards are maintained.
Where stipulations are used,
- the source of information and how it will be applied must be shown,
- their impact on savings uncertainty should be discussed, and
- their use (instead of measurements) should be justified.
Commissioning. After installation is completed, the ESCO must demonstrate the potential of the ECMs to perform as specified. The post-installation report should include M&V data resulting from commissioning and estimated first-year savings. The agency should not accept the project before it reviews this report and is assured that the ECMs were installed properly, are operating as expected, and show the potential to deliver guaranteed savings.
Annual M&V reports are required in ESPC projects. ESCOs are expected to submit annual reports that document savings in accordance with agreed-on procedures, show how they compare to guaranteed savings, "true-up" savings relative to the guaranteed amounts if necessary, and document other required activities.
One of the purposes of M&V is to reduce risk to an acceptable level, which is a subjective judgment based on the agency's priorities and preferences. The optimum level of M&V is that which minimizes both uncertainty and cost. Using stipulated values can be a practical, cost-effective way to minimize M&V costs. Stipulations do not erode savings guarantees when they are supported by reliable, documented information and analysis, and corroborated by verification of sustained performance.
Using stipulated values for determining savings does shift some risk to the agency, and the agency should thoroughly understand the risks before accepting them. However, stipulations used appropriately do not increase uncertainty significantly and do not jeopardize the savings guarantee, the agency's ability to pay for the project, or the value of the project to the government.
Check out the new Detailed Guidelines for FEMP M&V Option A, a companion to FEMP's M&V Guidelines, both available at www.eere.energy.gov/femp/financing/espc/measguide.html.
The "Responsibility Matrix," in the Super ESPC and FEMP's M&V Guidelines, is a list of risks and responsibilities to consider when negotiating projects and M&V plans. "Fine-Tuning for Best-Value Super ESPCs Using the Responsibility Matrix" discusses the Matrix and is available on FEMP's web site at www.eere.energy.gov/femp/financing/espc/implementing.html.
* The FEMP M&V Guidelines are an application of IPMVP to Federal energy projects. IPMVP was written by and for technical, procurement, and financial personnel in the Federal government and the private sector to establish a framework for verifying performance in financed energy projects.