U.S. Department of Energy - Energy Efficiency and Renewable Energy

Building Technologies Office – Information Resources

Text-Alternative Version: CALiPER Round 7 Testing Results and SSL Product Life Issues

Below is the text-alternative version of the CALiPER Round 7 Testing Results and SSL Product Life Issues webcast.

Rosemarie Bartlett: We're very happy to have as our speakers today Bob Lingard, Heidi Steward, and Eric Richman of the Pacific Northwest National Laboratory.  Our first speaker is Bob Lingard, an LC who supports the Solid-State Lighting Program and manages the production of the print- and Web-based educational materials.  Welcome to all the speakers.  Bob, please begin.

Robert Lingard: Hello, and welcome to today's webcast. My name is Robert Lingard, and I'm a research scientist with Pacific Northwest National Laboratory supporting the U.S. DOE's Solid-State Lighting Program.  Today's webcast will address CALiPER Round 7 Testing Results and SSL Product Life Issues.

As many of you are aware, SSL products using LEDs are proliferating rapidly in the lighting marketplace.  They're low energy consumption, potential long life, and compact form make them an attractive alternative to traditional light sources in some general lighting applications.  Given the rapid (inaudible) of SSL products, the consumer faces an increasing variety of LED luminaires and replacement lamps, along with a wide range of product claims, and a relative lack of information with which to accurately evaluate LED lighting products, as well as to compare them with traditional technologies.  Now to fill this LED lighting data gap, if you will, the DOE initiated the commercially available LED product evaluation and reporting program, more commonly known as CALiPER in 2006. Working hand-in-hand with CALiPER, the DOE also facilitates ongoing dialogue in collaboration with key standard setting organizations and offers technical assistance in the development of new standards.

Our first presenter today is Miss Heidi Steward. She has supported the CALiPER program since 2006 and will provide us highlights and details from Round 7 of CALiPER Testing. She is lighting certified by NCQLP. Our second presenter, Mr. Eric Richman, works with DOE and PNNL's Standards Development Program for SSL and will speak to emerging product life issues. 

With that, I'll turn the floor to my colleague Heidi Steward.

Heidi Steward: Thank you, Bob. I'm Heidi Steward, a lighting certified research engineer from Pacific Northwest National Lab. I worked on the CALiPER Program since the fall of 2006, as Bob mentioned.  I got in just after the pilot round and have been enjoining it ever since.  These are results from Round 7.  Testing for Round 7 was conducted from September 2008 to January of 2009.  In this round, we tested 29 products using both spectroradiometry and goniophotometry with absolute photometry, following the IESNA LM-79-08 testing method.  We also measured surface temperatures taken at the hottest accessible points on the luminaires.  The final individual reports were released in March and are available on the CALiPER website that's noted at the end of my presentation.  All of the products that were tested this round were purchased in late 2008. 

CALiPER scope is general illumination in the solid-state lighting area.  As a general policy, we only test units of products that are commercially available and have been purchased through retail channels. In some cases, sample products are accepted for testing either because there's no mechanism for purchasing small quantities of a product or because other DOE SSL programs request CALiPER testing of fixture samples. Our detailed CALiPER Reports always state how a product was obtained through purchase or as a sample.  We only issue CALiPER reports on products that are considered to be commercialized, either currently available or very soon to be available for purchase on the open market. Here's a collage of a few of the almost 200 products that we've tested in the residential, commercial, and industrial arenas.  We've tested a wide variety of products from desk lamps, T8 replacement lamps, streetlights, candelabras, garden spots, and MR16s.  You can see an interesting array of heat sinks especially on the directional replacement lamps. There are also at least two RGB luminaires here, though most we see are not. 

There are a lot of solid-state lighting products with lots of promises in their advertising, but it's hard to know which promises will pan out.  How do we know which are good and which aren't?  Are they in the realm of what we've seen recently?  Are we seeing a real breakthrough?  Remember the first few years of compact fluorescents?  A very thorough report, Compact Fluorescent Lighting Lessons Learned, is available on the DOE SSL website which describes it well.  Products didn't perform as promised; consumers became jaded; and even after the original problems were cleaned up, consumers still weren't willing to purchase CFLs.  It took years to recover.  We're trying to avoid some of the mistakes made then.  The CALiPER Program works closely with other DOE commercialization support efforts to meet testing needs for SSL products that are gaining market recognition.  In Round 7, testing a combined effort between the CALiPER Program and the DOE GATEWAY Program allowed testing of a number of outdoor streetlights.  Other near-term CALiPER efforts will focus on supporting ENERGY STAR for SSL, next generation lighting, and the SSL Quality Advocates Program.  We test real products purchased through retail channels using secret shoppers and test to industry standards.  We publish reports showing manufacturer claims and our test results.  We provide objective high quality performance information and try to discourage low quality products.  Currently we test approximately 30 products per quarter, not including other side studies. 

Looking at all of the CALiPER test reports, we see clear progress. This shows all of the market available products we've tested since 2006.  You can see the trend showing the average efficacy almost doubling since our start in 2006.  Remember incandescents run between six and ten lumens per watt. This round, the product with the lowest efficacy at nine lumens per watt was about the same as an incandescent. It was a 4 inch recessed can.  The highest was over 70 lumens per watt, seven times the efficacy of most incandescents.  This was a streetlight.  Obviously there's great potential in this area.  For each application category in this round of testing, there are clearly more SSL products that are approaching or even matching the light output levels, distribution, and color quality of similar luminaires that use traditional sources.  We anticipate this steady increase in performance continuing. Another good note is the percentage of products for which manufacturers provide accurate performance claims is increasing. Unfortunately, approximately half of the products testing still have inaccurate or misleading product literature.

Round 7 of the testing focused on three application areas: outdoor lighting, downlights, and replacement lamps.  One series of tests included eight streetlights, five SSL luminaires, one typical high pressure sodium cobrahead fixture, and two fixtures using fluorescent induction lamps.  For outdoor lighting, CALiPER also tested three bollards from the same product line lamped with solid-state, compact fluorescent, and metal halide sources.  The series of tests on nine downlights, eight using SSL and one using CFL, included a wide range of luminaires that could potentially be used for downlighting, including track lights, a surface-mounted luminaire, recessed downlights, volumetric-recessed lighting, and a 2-by-2 foot flat panel fixture. The replacement lamp category included nine different SSL products: MR16s; some larger directional lamps, PAR20, PAR30, and PAR38; and A-lamps.  Our side-by-side comparisons were the bollards with three light sources and the volumetric resource - - recessed lighting with two light sources.

We did a series of streetlights. In these pictures, you can see most of our streetlights.  We had five solid-state lighting in different shapes, rounded in the center bottom, angled center top, and shoeboxes we have in the upper right or upper left and lower right corners.  We had one high pressure sodium, which isn't shown here, and two induction lamps.  If you look at the lower left, you can see one without its lamp.  We also did a series of three bollards shown in the middle left with the same manufacturer, the same model, and different light sources. 

An outdoor bollard was tested using three different light sources: 45 Watt solid-state, 42 Watt compact fluorescent, and 70 Watt quartz pulse start metal halide.  These were described by the manufacturer as having similar outputs.  The bollard was designed for the light source to be mounted horizontally in the top of the bollard, and the SSL version uses the same design with LEDs mounted in the top of the bollard.  We have photographs here of each of the light sources on the chart.  The SSL version of the product can be purchased in different distributions, Types 1, 3, or 5 with a variety of wattages. The CFL and metal halide versions have a type of - - have Type 5 distribution but an optional house-side shield may be installed to illuminate a Type 3 distribution. This was not truly successful for either the metal halide or the compact fluorescent. They both tested as Type 2 rather than Type 3.  Currently, some market available SSL products have luminaire efficacy levels of 50 to 60 lumens per watt, so the measured performance of 13 lumens per watt of this version of the bollard is disappointing compared to the current state of the SSL technology. We also tested the benchmarks without the house-side shields to see how much light loss was caused by the shields.  Even with the house-side shields, the solid-state version beat both the metal halide and compact fluorescent in efficacy. The metal halide, however, had quite a bit more output without the shield.   The color of the solid-state bollard was much preferred in an unscientific poll of the office staff. The color temperatures of the three versions are similar, and the CRI values for the solid-state and CFL versions were nearly identical. The metal halide version has a CRI that is somewhat lower, but both the metal halide and compact fluorescent had DUVs to put them put out of the range of ANSI-defined white at their respective CCTs leading to greenish bluish color as viewed by the office staff. 

Streetlights are difficult to purchase one at a time; we tried.  So we worked with the GATEWAY Program and their partners that are testing streetlights in situ to get samples for tests.  Many of these products are currently being tested by municipalities across the United States.  We had five solid-state streetlights.  We only show four here because two were the same product with different temperature LEDs; two fluorescent induction, the same fixture with two different source types, and one high pressure sodium.  The solid-state streetlights tested in Round 7 vary in wattage from 55 to 95 watts and a lumen output between 1028 and 3400 lumens.  The luminaire efficacy ranged widely from 19 to 71 lumens per watt.  The high pressure sodium benchmark delivered more lumens, 6540 lumens, than the streetlights. The HPS benchmark delivered more lumens, 6540 lumens, than the SSL streetlights, but also drew more power with an efficacy of 56 lumens per watt, somewhat higher than the average efficacy for the five SSL samples.  On average, the fluorescent induction benchmark streetlights delivered slightly more total lumens than the SSL samples with slightly higher efficacy.  The color rendering for all of the solid-state streetlights were about 70. The induction streetlights were only slightly higher in the high 70s.  The high pressure sodium was a miserable 21. This was not unexpected.  The solid-state streetlights were generally cool from 3100 to 6200K, except for one at 14628K, well above the 6500K on the NC scale considered to be on the blue end of white.  This one had the lowest efficacy end output.  The induction streetlights were about 4000K, and the high pressure sodium was very warm at 2000K.  Clearly, the SSL products are available in a range of distribution types and light outputs.  They can vary considerably in luminaire efficacy.  The best is better than both the high pressure sodium and fluorescent induction fixtures.  The SSL products had vastly better CRI than the high pressure sodium and slightly lower CRI on average than the induction products.  In terms of color appearance, the SSL products tend to be cool in appearance unlike the very warm high pressure sodium source. In one case, an SSL product had a CCT beyond what is typically available for conventional white light products and outside of ANSI-defined tolerances for white light.  Implications regarding comparative long-term performance have not been CALiPER tested at this time. 

We tested a series of nine downlight products, including one side-by-side comparison between CFL and SSL on the same model.  We tested three recessed cans, one was a retrofit lamp for a five- to six-inch can, one 7.5 inch square surface mount, and one thin 2 foot square panel; two types of track lights, and the side-by-side test of a 1 foot square downlight.  When comparing such a variety products with such a range of power levels and light distributions, it would not be appropriate to compare output levels or center beam candle power across the entire set of samples, but we can look at their efficacies and color qualities. 

These are results from our tests, NLPIP reports, and manufactured catalogs.  Note that the 2-by-2 thin panel is not includes in this graph. You can see the clouds grouping the incandescent and halogen, the compact fluorescent cloud, and the solid-state cloud.  The large squares with the Xs are from this Round.  For any given output level, the solid-state downlights meet or exceed luminaire efficacy of the incandescent and halogens.  Most of the Round 7 products meet or exceed the luminaire efficacy levels of compact fluorescent downlight benchmarks as well.  Although there's a large range of light output from the Round 7 products, they all meet or exceed minimal levels for the compact fluorescent or the incandescent and halogen benchmarks.  Except for one product, all of the downlights that were tested use warm-white sources with color temperatures ranging from about 2700K to 3250K.  The other product was only available in a 5000K halogen.  All nine products have CRI values between 70 and 84, except for one RGB source for which CRI is not a reasonable indicator of color quality.  All nine products light was within the ANSI-defined target levels for white light, for color temperatures, and Duv tolerances. 

Here's our second side-by-side comparison.  This was done with a 1 foot-by-1 foot square downlight. They look exactly the same outside, but one is solid-state and the other uses the compact fluorescent light source.  You can see photographs here on the graph of each light source.  They have the same color characteristics and similar light distribution.  As you can see, the solid-state downlight is slightly higher in efficacy and lumens, but it's initial cost is two times higher that of the CFL.  We haven't studied the long-term costs. The solid-state downlights' life depends on how well the luminaire was designed. Heat sinking is usually the major issue.  If it's well done, the lifecycle costs could make up that initial cost difference.

We only looked at nine replacement lamps in this Round: four MR16s, one has a GU10 base and runs on a 120 Volts, the others were bipin 12 Volts; three PAR lamps, plus two omni-directional lamps rounded out our test series here.  If you look at our pictures, I love to look at the heat sinks for the replacement lamps. They're always interesting. 

Let me give you a quick overview on PAR lamps, since we don't have a slide on those PAR lamps.  We had three SSL replacements lamps that were labeled as replacements for PAR lamps.  One was PAR 20 replacement, one PAR 30 replacement, and one PAR 38 replacement. With respect to light output, all three lamps carried performance claims that are overstated, and they do not meet average rated output levels for similar sized incandescent R lamps, reflector CFLs, or halogen PAR lamps.  Despite having highly overstated performance claims and not achieving the average light output of similar traditional products, these lamps are achieving performance levels which may be appropriate for some applications.  All three SSL PAR lamps had efficacies which are two to five times the efficacy of incandescents and halogens. Their light output levels and center beam candle power are approaching the lowest observe levels in manufacture rating for directional incandescent halogen or CFL.  In addition, they were all warm-white products with acceptable Duv and CRI levels; two of the three had a CRI of 93, which is quite good. 

So how well do solid-state MR16s stand up to 20 Watt halogens.  They are often marketed as replacements. The rectangle on the left shows both CALiPER tested and manufacture provided data for an assortment of halogen MR16s. As you can see, efficacy and output vary quite a bit.  The dotted lines show averages for each, about 12 lumens per watt and 260 lumens of output.  All of the solid-state MR16s had higher efficacy than the average MR16s.  The warm solid-state MR16s, the two circled on the left, have efficacy similar to the halogen MR16s but very low output, less than half the average halogen. Their CCT values are close to what is typical for halogen MR16 lamps, but their color quality is poor with CRI values between 65 and 48.  Their Duv values were both outside of ANSI-defined target tolerance values.  The cool solid-state MR16s, circled on the far right, had much higher efficacy levels but are still not achieving light output levels typically as seen in halogens.  The cool color tint MR16s have acceptable Duv but mediocre CRIs, about 50. The best of the cool, the upper right one, is approaching the output levels of lower performing halogens with five times the efficacy but is still way below the average.  If you only needed this level of light output, this could be an appropriate choice.  There can be weight and size issues with the LED MR16s. Sometimes they're built so that they don't fit into the standard.  There can be weight and size issues with the LED MR16s.  Trying to get all those electronics into one little tiny MR16 can lead to interesting shapes and sizes that can affect which luminaires these can actually be placed into.  We also don't have long-term testing done yet on these MR16s, so reliability can be questionable. 

Let's compare incandescent, compact fluorescent, and solid-state replacement lamps.  The red curve to the left shows incandescent. Efficacies run about 10 lumens per watt or lower. The efficacies get better as the wattage goes up.  The green curve on the right shows compact fluorescent.  Efficacies range from about 40 to 65 lumens per watt.  Again, the efficacies get better as the wattage goes up.  SSLs are in blue in the middle, generally about three times higher efficacy than incandescent, lower than CFL.  The majority that we've tested have output below a 25 Watt equivalent, a few about 40 Watt equivalent. Neither of the products tested for this round met their manufacturers' claims. The 14 Watt was claimed to be equivalent to a 75 Watt incandescent. It tested as closer to 40 Watt equivalent. The 7 Watt claimed to equivalent to a 40 to 50 Watt incandescent.  Again, we tested it out as closer to 25 Watt equivalent. This slide does not represent the shape of the light distribution, neither lamp had a distribution like a standard incandescent A-lamp; but depending on the application, their distributions might be preferable.  Again, like with the MR16s, the size and weight of these products varied. The 14 Watt has a very large heat sink and is quite heavy. The 7 Watt that we tested is the one on the far right on the bottom and is shaped very similarly to a standard incandescent A-lamp but weighs quite a bit more.

So here we are Mia Paget's favorite picture.  What an interesting learning curve we all have to ascend.  So here are some pitfalls we might hit along the way.  For color, we've seen so-called white products that look bluish, greenish, even lavender.  The CCT gives an approximation of the color; the Duv tells how close the color is to the blackbody locust.  The closer that Duv is to zero, the more accurate the color will be.  That's why it's an important thing to check. For photometry, absolute and relative photometry are not the same.  Check the photometry reports to be sure they're done to LM-79.  Solid-state products must be tested as a complete luminaire.  It makes no sense to try and test them as relative. It's just not physically possible.  Sometimes luminaire manufacturers who are new to the LED world test in some sort of situ relative manner.  A few times we've seen claims that are too low because of this.  In this round, we saw about 50% of the manufacturers whose claims were incorrect or insufficient for comparison.  Lifetime, we really don't know how long these luminaires will last in situ.  None of the products from this round have had reliability testing done in our program.  We've recently released a study on long-term testing of a small number of products monitoring lumen depreciation and color shift.  The results are widely varied from products completely dead at 400 hours to still at almost 100% after 6000 hours.  As the industry grows and matures, we're seeing manufacturers who are doing everything right - designing luminaires well, reporting their products capabilities accurately, and we're seeing new manufacturers who are just starting to figure it out.  Watch out for that hole in the hill.

If you would like more information, please check out our website.  We have summaries.  We have detailed individual reports.  You can't use them to dis the competition; this is for educational purposes only.  We have benchmarks using both SSL and CFL or incandescent as appropriate.  We have interesting exploratory reports.  This is where the long-term testing results I just mentioned are.  Keep checking back to see what we're doing.  There's lots to read and learn. 

So thank you very much. Here is the link to our website; and I'm Heidi Steward, one of the CALiPER contacts.  That's my spin and off to you, Eric Richman.

Eric Richman: Thank you, Heidi.  As Heidi mentioned in one of her slides, there are issues with SSL product life, and that's one of the things we wanted to talk about today.  So we'd like to look at what LED life is, how it actually manifests itself in the industry and just as importantly how is it going to be measured. This is becoming a critical issue with the LED technology itself. 

As everyone understands, life is important for all light sources.  It's very critical for various reasons, particularly for lighting design.  It affects heavily your technology choice.  It affects heavily your application for those sources; and this is for any source, not just for LEDs.  Energy and cost effective analysis are often driven heavily by lifetimes. It's a critical part of any analysis. In particular for LED technology, life is a really big issue primarily because it was one of the things that LEDs were hyped up on was how long they would last, including that they would last forever, and so it's one of the issues that LED technology has to deal with.  As mentioned, it's also a part of cost effective analysis' and because LEDs are counted as lasting for a very, very long time, this is critically important for cost-effectiveness analysis that's been doing for LEDs.  Again, because of the hype is that it lasts for such a long time, it's an area that has to be considered carefully. Now LED life in comparison to other technologies is not simple or obvious.  LEDs do operate differently than other sources, so that creates differences in how life is considered, and LEDs simply do not have a clear end of life like other lighting sources might have.

So what's the typical life for other lighting sources?  Well for virtually all other lighting sources, it's an operational failure basis that is the light source itself will burn out or have a catastrophic end of life, and these are pretty straightforward in that the lamp itself burns out.  Now typically this is related in the industry as the point where 50% of a sample of lamps fails to operate; and at that point, that's considered the operating life or simply the life of that lighting technology.  Now contrary to this, LEDs don't typically fail in the same fashion. They don't have filaments or (inaudible) necessarily burn out or break in other fashions, so there's a difference here we have to be aware of.  Breakage is another part of operational failure.  Usually the lamp will burnout but sometimes it'll break; and most lamps are created with some type of glass enclosure, which does make them somewhat fragile.  LEDs on the other hand are relatively robust. They're generally small. They don't have large glass envelopes to break necessarily. They're fairly sturdy, so these are some of the differences we need to be aware of for LED life.

So that leads us to the question of what do we do for LED life if they're not going to operate the other sources do? What has surfaced in the industry is the idea of useful light output, or lumen maintenance as it's termed.  Now all light sources as we know do degrade over time.  They'll get dimmer and dimmer due to various different operational characteristics.  But most sources, virtually all other standard sources, do have an operational failure. They will burnout before there's a serious loss of light output, and this is kind of related in the two charts at the bottom.  You'll see every light source has a different degradation point, but those typical sources, again, will burn out before the light loss is so low that you really will notice it.   LEDs, the same; they will continue to degrade. But because they will last potentially a lot longer, they have the potential issue of degrading to a point that's too low for its intended use before it actually fails, so this becomes the critical point for what the life of an LED is.  You don't want to have a light source that's continuing to operate but getting so dim that it no longer performs the way it should. 

So this is where industry is with the life of LEDs, and the industry research and manufacturing have come up with a couple of metrics that are being talked about (inaudible). One is the L metric.  You've probably seen the L70/L50 numbers initiated by the Lighting Research Center as a metric of the percentage of initial output. So in other words, an L70 is a metric of when a lamp has reached 70% of its initial output; it is degraded by approximately 30%.  This number is kind of chosen for a variety of reasons, one being that it's hard for the human eye to really detect light levels around a 30%, or that's where your average eye starts to see a change.  So that has been termed as one that might be good for applications where illuminance is important.  This will be most interior applications where tasks are critical. The other metric, L50, where the lamp has degraded to 50% of its light output, that probably is noticeable, is then considered for maybe less critical needs.  A lot of this would be perhaps outdoor lighting where it's important to have light, but your tasks are not as critical.  There's another metric that's come about initiated by Lumileds, which is the B factor, which is a statistical failure rate or a way to relate when a sample of lamps has reached a certain component failure rate, and this has to do with taking a large grouping of lamps and determining a typical failure rate among that group. That's something that the L metric doesn't do. The L metric just tells you at what light output you're achieving. The B metric will tell you when that is achieved by a large sample. So for example, a B50 metric would be at a point where 50% of the lights in that sample have reached the failure point. Now the failure point could be that they have reached an L70 level or they have broken or for some other reason have had a catastrophic failure.  So when you combine these two metrics, which is something the industry is looking at as a way to relate life for LEDs, an L70/B50, for example, is the life time or time when 50% of that sample has come to the point where it no longer produces 70% of the initial light, and this is something the industry is looking at. You'll see it referenced in different standards and methodologies, but it's not necessarily a full approved metric for all standards at this point, but it's becoming kind of an industry respected way of looking at life for LEDs. 

So since we're talking about life, let's talk about what effects the life of an LED.  There are four major areas: Environment, the weather essentially - hot, cold, humidity.  Material of course will affect the life of any piece of equipment - connections, encapsulants or lenses, the phosphors.  Mechanical and electrical conditions - if the equipment is being vibrated, that'll have an effect on its life.  Voltage and current changes will have an effect.  Then installation architecture, which really refers to any kind of a light source within its housing or its luminaire; and this has to do preliminary with heat sinking, which is, as you'll see, very important for LEDs.

So let's look at each of these conditions.  The environment of course is very important, and here we're talking not just about outdoor weather, although that's important, but also indoor conditions in ceiling spaces or planems*, for example, where lighting equipment might be placed. The issue here, the primary issue is of course heat, and this will… It doesn't matter whether its heat on the inside or heat on the outside, heat is the primary factor in light output, and this light output would of course affect the lifetime of the LED itself.  As you can see from the chart, as the junction temperature gets warmer, the light output gets lower, and this will be true for initial and one point light output as well as it will be for the life of the LED or the light output of the LED.  The juncture temperature that we're talking about is the temperature as close as you can get it to the actual LED chip or module itself, which is really where all the action happens.  This is the critical part of LED light output and therefore its life.  A couple of other environmental factors: cold of course is the opposite of heat, and LEDs really appreciate that.  It's a natural cooling methodology; and that's the whole issue of trying to keep LEDs cool, at least the high powered ones so that they can perform correctly.  Humidity can be an issue mostly in outdoor applications, and this mostly has to do with effects on things external to the actual chip itself for the most part.  Connections and housing materials can be affected by humidity, and that's the same for all types of lighting sources. 

Material stability naturally is going to affect life.  Again this is - - mostly has to do with things external to the chip itself for the most part, and these can apply to all kinds of lighting sources as well.  The first one: Connections, solder joints, and electrical leads. These can be the weakest point of a light source in some cases; and certainly for an LED, these are important.  If any of these break due to weak materials or whatever, that will of course end the life of the source itself. The encapsulant or sometimes it's referred to as the molding compound, you can see from the chart, and perhaps the lens itself, these can degrade or change in color and any of these effects will affect how much light there is coming out of the actual source itself.  If these degrade, for example, the encapsulant, if it degrades enough and causes some kind of leakage, et cetera, this could contribute to failure of the LED module as well.  Phosphors of course exist for all types of lighting, including some LEDs.  Some LEDs are phosphor - - are drivers for phosphors.  If the phosphor gets comprised, these of course could also affect the life.  Material stability is primarily considered for all light sources.  It's a smaller effect than heat, but it is a possibility and something that needs to be considered. 

Mechanical and electrical conditions are also going to affect life and light output. Vibration, again, this is really related to effects on the luminaire housing and auxiliary components.  As we mentioned before, the LED chip or module itself can be pretty robust, pretty sturdy, but how it's connected to the rest of the luminaire can be an issue.  Voltage and current variation, of course when you overdrive or underdrive any kind of electrical component, particularly a light source, that can have an effect on how much light output it provides, which in turn can affect the lifetime of that source itself.  Most voltage/current issues can be taken care of with the driver that can modulate these issues, but it's something, again, that needs to be considered. 

Finally, the installation architecture. This is really all about heat sinking. When you talk about an LED chip or a module itself and if you put a heat sink on it to take care of the heat issues, that's all well and good. You can test that with an effective heat sink. But once you put that into another environment, into a luminaire, that heat sink may be compromised.  The heat sink may be improved as well; but depending on how it's put in, if you over insulate it or don't make the right connections, that can affect how well that heat sink performs. Then if you compromise how the heat sink works, then any testing you did on the module itself is no longer going to be applicable, and this is an important issue as testing for LEDs is try - - we try to do that.  The industry tries to do that on a luminaire level because that's - - the heat sinking is really important, so it's something to definitely keep in mind.  It's one of the big issues with LEDs and how they get applied into luminaires and into building themselves.

So this leads us to the idea of how do you actually measure the performance of LEDS, modules, or luminaires otherwise? There is one standard that exists. It was created through the IES.  It's essentially the measurement of LED life or actually the measurement of LED lumen maintenance, which is really the way we're talking about life in the industry. It does provide good measurement formats.  It provides all the necessary information to come up with repeatable, comparative conditions between products and between samples, and this is very important to have this repeatability. It does cover LED packages, arrays, and modules, but it does not cover complete luminaires.  It's at the module or chip level. Now one important thing about LM-80 is while it does provide information to get good data, it does not divide - - define or provide a method to estimate the actual long life of an LED. You can't… It's not practical to test an LED long enough to find out when it's going to fail; you have to do some kind of an estimation.  But this document itself does not do that, and that's one of the things that the industry is currently working on.  It's also was realized when this document was put together, LM-80, that were issues with that estimation of life, and that's one of the issues that we want to talk about here as well. 

So let's look at life estimation issues.  LM-80 itself talks about 6000 hours of testing, which initially was considered to be a long time and might help relate what the life long after that might be, but that in itself is eight months worth of testing, and that's a long time.  Existing light sources that have been around for a long time have had time to have extensive life testing. They've also got a lot of experience with their application behind them that helps relate what an actual life is, so that's pretty well nailed down; but LEDs are new, relatively brand new in terms of other lighting technologies.  So it's hard to have the data backup necessary to understand how long they last; and because they can potentially can last a long time, the testing could be extremely long, so the struggle is to come up with a good way of estimating the life based on a reasonable amount of testing.  Another issue is that, as I mentioned, when you take an LED luminaire or LED module that may have been tested with a certain heat sink under certain conditions and you put it into a luminaire, that luminaire can change the heat condition, so it's important to measure these LED modules at various temperatures. This is the method that's kind of been vetted through the industry and through standards organizations is that you test the module at various temperatures, then when you put the module into a luminaire, you determine what it's actual operating temperature is going to be in the luminaire and then you will hopefully have one of these multiple temperature tests that you can rely on to tell you how that product is going to perform.  One of the remaining issues is with burn-in or seasoning and all lamps when they get tested have some seasoning period, commonly 100 hours for a lot of light sources. This is important so that any anomalies and when you first fire up a lamp can be taken care of before you actually do the testing.  The issue with LEDs is that within longer periods of time, a thousand hours perhaps, it could be shorter than that/could be longer than that, there's some characteristic bumps or jumps or low points in the actual light output, and these have to be addressed, so you either have to wait long enough till these are taken care of and you can start testing or you have to find a way of incorporating these.  The issue of course is that LEDs are new. There are a lot of different types out there. They don't all have the same characteristic early bumps, if at all, and so the industry needs to figure out a way to take care of these or be able to cease and pass them. 

So these are the main issues.  I did want to say that there is a document being worked on at this point to try and come up with an estimation method the industry can use.  It's going through the IES process currently as a label of TM-21.  It's trying to become a document, like I said, that will provide an estimation method or at least one estimation method. It is being developed to go along with the LM-80 testing or feed off of the LM-80 testing, which is currently at 6000 hours. However, in the process of coming up with TM-21, the realization is that 6000 hours may not be enough. There's some data showing changes near the 6000 hour point which would mean you'd want data after 6000 hours to give a really good estimate of life.  So what the groups working on this TM-21 are looking at or leaning towards is a rather conservative approach for now, particularly if you only have 6000 hours of testing to base your life on. They are exploring a format of looking at multiple models that perform in different ways depending on different or failure or degradation possibilities for LEDs.  These would represent all the different degradation paths; and then with all these different models, perhaps there's a good or at this point conservative way to estimate what the life would be. It's important to understand that different parts of the LED that effect its degradation will have different effects and there are different models that could be built to kind of address that, and that's again what TM-21 is looking at, but it's constructive to look at a couple of examples.

It was originally thought that light output like a lot sources is exponential and a lot of the earlier data show that this was a pretty good fit, but it's also been realized that these early bumps we talked about can really skew the results.  If you look at the top plot, for some fairly consistent smooth around 6000 hours of data when you look at the various models that are used to estimate the life, there are a relatively close grouping. When you get out to long periods of time, this only shows at 50,000 hours, you get beyond that and the L70 point, for example, can be quite large, but it's still a relatively close grouping compared to the bottom chart, which you'll see the first 6000 hours of data there shows some bumps and some anomalies; and when you have that in the first operating part of the LED, your estimation models are going to vary wildly.  In this case, you'll see some have very catastrophic results in terms of its life and some have very long relative life.  So this is what the industry is struggling with right now in terms of coming up with a good estimate for life. 

Bottom line to kind of sum up is that LED life is primarily tied to its anticipated level of light output. That's what the industry has settled on, and that's what we need to be aware of. There is of course more testing needed before standards groups can feel comfortable putting together at least a way to estimate the life. You also want to be mindful of the fact that LEDs don't have an operational failure. There's no point where they're going to all of sudden burnout so you can replace them. It's just something you need to keep in mind in terms of design.  All of this leads to being aware of these issues; and whenever you consider LEDs, try to be somewhat conservative in your approach when assigning a value to them, especially if you're going to use that for any analysis or economic justification, not that they aren't going to last a long time, because they eventually will and some of them do now, it's just something to be mindful of when you're applying the technology. 

That concludes this part of the presentation.

Rosemarie Bartlett: Well thank you very much for such an informative webcast Bob, Heidi, and Eric, and thanks to all of you.  The U.S. Department of Energy appreciates your attendance today.  The slide up on the screen right now gives you some links for more information.  So if you want to make notes of those links, go ahead and do that. We'll leave that slide up here while we answer the questions.

I think Eric's going to start us off.  Eric.

Eric Richman: Yes, we have several questions coming in, a couple of them that I can answer. One asks about infant [sic] mortality:  Are devices that fail early included in the L or B ratings that were talked about in one of my slides?  Yes, they are addressed in the B ratings. The B rating covers anything that fails for whatever reason, including just dying straight off the vine initial failures, that would include that.  The L rating of course just talks about the lumen output value, but the B rating takes care of that infant mortality, and that's why the industry is kind of looking at both of those as a combined metric.

Another question here: If you were trying to get junction temperature, that's sometimes a hard thing to get to. How do you make this equal across the broad?  What industry is looking at, particularly ENERGY STAR and programs like that is they're going to look at requiring a manufacture to identify a temperature measurement point on the LED package or strip or module, whatever the product might be, a point that is going to be accessible.  Then when a luminaire manufacturers produces a luminaire with a package inside of it, it can be tested with that point which is accessible, and that point measurement will relate to the testing for life that's done on the package itself so you can actually on a one-to-one comparison. It's a good question and the industry has thought about that and came up with a TMP point issue. 

Some other questions have come in also for Heidi or Mia. I'll let them take some at this point.

Heidi Steward:  Hi. This is Heidi, and I have a few questions here.  Let's see what I can do.  Our one question: Is lumens per watt calculated on luminaire output or lumens delivered to the task?   We are calculating lumens per watt based on luminaire output.  We are also looking at the distribution of the light, and we provide that information in our reports.  With directional lighting, we also look at center beam candle power as appropriate. 

A second question had to do with whether we're evaluating costs and costs per 1000 lumens.  We do some dollar pre lumen calculations for internal purposes to track market evolutions, but we don't publish specifics on those calculations for a number of reasons.  Results from cost calculations are used by various DOE SSL commercialization support activities and particularly for program planning. 

I have one more.  It says that not all the tests shown in CALiPER Round 7 are available for download on the website.  Why have some of the reports shown here not available?  Actually they are all there; I just looked. There are 29 reports available in Round 7, that's how many reports we produced.  They're available both individually, so you could just download one or two if you wanted, and they're also downloaded as a zip file with all of Round 7 included.  The numbers of CALiPER tests are not necessarily sequential.  We also do testing for other programs, for example, for Next Generation Lighting and the GATEWAY Program.  We number the products as they come in and as we send them out for testing. We don't produce reports for most of the products from other programs unless we're working with them on a special project like we did with some of the streetlights that we tested in this Round with GATEWAY.  On occasion, a test will take longer either in the laboratory or in the reporting process, so it might take a little longer to get out to the final round , but usually we try to get all of them out about the same time for the Round; and this current Round, Round 7, is in fact complete as is out on the Web. 

I believe there's one more question that I have or that we have for Mia.

Mia Paget: Thank you, Heidi.  This is Mia Paget. I'm also in the CALiPER Program with Heidi Steward and I work with Eric Richman.  Heidi sent along one of these questions to me which said, "Can you explain how the CALiPER Program will work with the ENERGY STAR Program, if at all?"  They say, "We run an efficiency program and are really looking for a standard to use in our specifications for products eligible for incentives in efficiency projects."  Well the ENERGY STAR Program is quite different than the CALiPER Program. The CALiPER Program does not qualify products; it doesn't really try to judge them against any criteria; although, we often indicate in our reports how they would probably perform for specific criteria under the ENERGY STAR criteria.  At this time, the ENERGY STAR Program has just started up ENERGY STAR for SSL. It's still fairly young, and they are putting in place a Quality Control Program and they are asking us for help in putting that in place, so we are - - the CALiPER Program is working with ENERGY STAR to help them put in motion their Quality Control Program.  Ultimately, it will not be the CALiPER Program that actually manages the ENERGY STAR quality control; although, they'll be using the same testing methods that we use for the basic photometric testing.  So we are helping them with that, but ultimately they will have their own quality control program in place. 

As far as looking for a program or set of standards that you can use for your efficiency programs, I really do recommend as much as possible using what's available on the ENERGY STAR for SSL Criteria; and their listing, which is online right now on the ENERGY STAR website, that is a listing of products that have - - are now currently eligible for ENERGY STAR under the ENERGY STAR for SSL Criteria.  Although that - - it can be frustrating because that list doesn't include all of the applications that people might want to see at this time, the program is going cautiously forward in order to ensure as much as possible that they provide the ENERGY STAR criteria to products that will meet people's expectations both in light output, color, performance, and in reliability; and because standards are not yet available for some of these characteristics, the program needs to go - - is going very slowly and making sure they get appropriate review from industry on any new applications that are added to it.  So that is coming along; and I think as long as everybody is up to date on what they're doing, you will see that there's a lot of material there that you can use to help you define your programs. 

Eric or Heidi, I'll leave it for you guys for the next questions.

Rosemarie Bartlett: I'm just going to interject very quickly. I noticed that there seem to be several people who have raised their hands in the Q&A pane; and I'm sorry, we don't have the ability to allow you to speak in this webcast.  So if you have a comment or a question that you would like to make please type it into the Q&A pane so that it can come in to the presenters. 

Okay, Eric.

Eric Richman: There are a few more questions here that I can talk about.  One asks about vibration effects on SSL given the fact that it may affect SSL.  How will you test vibration for luminaires compared to the recommended applications as given by the manufacturer?  That's to the point that the vibration really does effect the luminaire more than the SSL product itself.  Testing for vibration is not something that I know of as being done by laboratories necessarily that is done or is required by some specifications for when someone wants to purchase or specify specific lighting product, and those tests, as far as I know, are likely going to be the same kind of test that are done for any other standard luminaire with any other light source, and those truly are done by those wishing to buy the products.  I'm not aware of any general, federal, or laboratory testing along those lines.

Another question on the data on the life estimation issue slide which shows the or talks about the bump, the early bump that might affect life estimation.  That was some data on several products provided by a manufacturer.  In the process of working on TM-21 , the groups have looked at some what I'll call sanitized data that was provided by manufacturers what's identifiable to the manufacturer, it's just samples that have been tested giving us examples of what LEDs operate like so we can try and figure out how to come up with the best estimation method.

There are a few other questions. Let's see:  For LM-80 or TM-21, does the IES standardized testing temperatures, where is that temperature coming from?  I've heard 25 or 55 cellulous.   The… Of course, TM-21 isn't completed yet, but LM-80 does prescribe three different temperature ranges to do the tests on. Two of them are specified, and I'm probably going to get this wrong. I believe there are 55 and 85 or no 55 and 25, and a third one is as specified by the company doing the test. So if you have a particular LED product that you believe will perform typically at a certain temperature or near one, that might be the third one you'd want to pick so that when manufacturers go to look at that data and apply them to putting your product into a luminaire, they'll have close to the right temperature.  Because at some point, they're going to want to extrapolate or interpolate between those values and you want to cover the spread, so to speak, so yours is included. Now where that temperature point comes from, LM-80 specifies that to be within a certain distance of the product as its operating. That's the current specification. 

Another question here about: Is there going to be continued testing of the LED samples until failure, and perhaps that was related more to the CALiPER Program. I know that CALiPER is doing life tests and maybe I can let Heidi or Mia refer to that one.  Also, Heidi and Mia have some additional questions I think they can answer.

Heidi Steward: This is Heidi again. I'm going to let Mia talk about the life tests because that's her baby, but let me get into some of the other questions we've had.  Someone asked about the types or companies of devices that we use to test and how our testing facilities are set up in general.  We use independent testing labs. We have a number of them. They're all listed on the CALiPER website.  Currently we are fully active with four testing labs.  We have a few other testing labs that are in process of getting set up.  All of our testing labs are required to be able to do LM-79 best-based testing.  LM-79 does not require a specific manufacturer of testing devices, so there's a variety of those that are being used. They all have the ability to do goniophotometric testing and they have to have the ability to do sphere testing.  So the testing facilities are set up based on each of their individual companies.  Some of our labs are very small, some of our labs are physically quite large.  Again, you can look at the list and see if one of the labs is nearby.  You could contact them and see if they could be visited. They tend to be fairly tight about not wanting people to come in and look around, but again you could contact them and see.  I would be surprised… For a number of them, at least, I would be surprised if they would be interested in doing tours.  
There was a question about the A-19 SSL lamps, what beam pattern did we see?  I just looked at the two of them and basically they look like clam shells. Of course, there's no distribution above 90 degrees.  They're basically kind of blobby co-signed distribution. They do not look like a standard incandescent A-lamp in shape, which does have light above the 90 degree mark. 

What else?  Someone requests or wanted to know about LED tubes and noted that almost all of the manufacturers use a 5 millimeter LED package.  We have tested and we are currently testing other tubes using other LED types.  I have two in my office right now that I can look at.  I have some samples that are actually being tested right now that are not 5 millimeter, and we're hopeful that that will lead to better fixtures with more light output. 

Anyway, that's what I've got so far. Mia.   

Mia Paget: Thank you, Heidi. I'll just go back real quick to the question that Eric deferred to me about the long-term testing that we're doing on LED luminaires.  So our testing is not being done at the chip level; it's being done at the luminaire level, and we are starting out with about 6000 hours of testing; and then in some cases, not in all cases, we are continuing testing those products, so I think we have some products that have now operated up to 12,000 hours.  We haven't made a decision as of now whether we will continue testing those all the way out to failure or not; but for now, as long as we are getting additional information out of them, we are continuing those tests and probably every 6000 hours repeating absolute testing in a sphere on those products just to correlate the long-term spot testing with absolute photometry to make sure that things are not deviating over time.  So some products so far have failed; but for the most part, the products that are in those tests, even the ones at 12,000 hours, are doing quite well.  It's very interesting for us to see, and we will be periodically issuing reports on that.

I had another question here that Heidi asked me to answer about whether or not we are taking into account scotopic versus photopic lumens in our testing.  We are not doing any specific analysis developing, for example, factors such as scotopic efficacy or photopic efficacy.  We do, however, publish all of the spectral data on the products that we test so that if people do want to do those analyses, they could use that data to do some estimations, recalculate some estimations based on scotopic lumens.  At this time, we're not doing any sort of assessment of that type. 

Heidi, you want to go on or Eric?

Eric Richman: I have a few more here coming in that I can respond to. One is on reliability standards.  Are there any reliability standards for LEDs specifically?  It mentions LM-80s about lumen maintenance only.  A good question. The questioner also mentions such as JEDEC or mill standards.  Yes, those do exist for solid-state devices which could also apply to LEDs.  Those really are the only current standards that exist for those that I am aware of. However, there are a whole host, a long, long list of standards that are being developed within the lighting organizations specifically aimed at LED products. None of them, as I recall, are specifically aimed at vibration type issues, but there is a lot of concern about reliability and so those issues will get addressed but there isn't an IES or an ANSI standard, for example, yet that includes those, but it's definitely on everyone's mind and it is being considered.

Another question.  Please describe what is degrading in the phosphor. Typically for phosphors in LEDs, and it's going to be the same as it is for fluorescents or any other sources, typically it's clouding of the phosphor or darkening of the phosphor usually identify or usually produced by filament degradation, which of course doesn't exist for LEDs, but there are other issues with darkening of the encapsulant or lenses or other components that would affect the phosphor. That's typically the method.  Now as LEDs last a lot longer, there may be other elements of the phosphor that degrade over time. I'm not personally aware of what those are. They may exist, but typically it's going to be other items that affect the phosphor itself. 

Another question on: Do you have typical values for L70 and B50? I'm not exactly sure what the asker is referring to, but L70 and B50 have a specific definition, L70 being 70% of the light output and B50 being when 50% of a sample cease to operate to the conditions required.  What are those like in practice? That really varies a lot as you can see from the CALiPER testing. Some products are doing very well in terms of L70, at least with the preliminary data, and some will not do as well.  In terms of B50, it really depends.  Some products again are doing very well and some may not. Typically for B50 though because LEDs are typically robust and don't certainly fail right away, the B50 is going to be hard to identify what the numbers truly are until we have a lot of long-term testing.

I'll turn it back over to Heidi or Mia.  Do you have any other questions?

Heidi Steward: I can take the one question asking about an agency that certifies testing labs for performing the recommended or proposed tests.  NAVLAP is currently or currently has available the capability of certifying testing labs for LM-79 testing; however, it is a very long process to get certification. This certification will be required to do testing to be available for ENERGY STAR.  Because that process will take so long, ENERGY STAR is allowing the CALiPER pre-qualified labs or the CALiPER qualified labs to be used for LM-79 testing, but I believe that's it. 

Mia, do you have others?

Mia Paget: Sure, I can answer the question.  Somebody asked us: Why does CALiPER report lumens per watt, overall lumens per watt for the luminaire rather than delivered lumens per watt where delivered lumens is an important metric?  Well right now there is not any standardized methodology for calculating what somebody might call delivered lumens.  In some cases, for particular applications, different users, different groups have mechanisms by which they try to provide a metric that they might call delivered lumens, but there's no standardized methodology or standardized metric which we could use across the board. One of the difficulties of the CALiPER Program is to be able to provide information that can be used and understood by quite a variety of different stakeholders that are out there, and that's one of the reasons that CALiPER tries to rely as much as possible on values which are solid and standardized and which don't require us to do a whole lot of explaining details and defending or defining our own types of systems.  So in this case, lumens per watt is a very useful value. It's not the only value, and we hope that people don't focalize only on lumens per watt, but it is one of the more solid values that you can compare across the board as long as you keep in mind that delivered lumens can also be important, central beam candle power can also be important, all sorts of other parameters are important. 

You want to go next, Heidi, regarding the 5 millimeter question that you saw?

Heidi Steward: Sure, I can take the 5 millimeter question.  Many of the 5 millimeter SSL products are - - have high efficacy values as do the power LEDs.  But in general, the 5 millimeter LED products don't deliver as many lumens. If you're looking for more light output, in general the high power LEDs have more; that's kind of an overall answer for that. 

Mia Paget: We have also seen more failures in the 5 millimeter LED products than we have in other ones. That's just an observation of products that come in the door that fail even before testing is conducted or completed, we do tend to see failures in the 5 millimeter products. 

Eric Richman: There's one more question I can address and one more I think Heidi or Mia should talk about.  One for me:  What percent sample size would you recommend for testing a single type of luminaire?  How (inaudible) models, et cetera?  The testing sample size varies greatly depending on who you're talking to, what they're doing it for, what program they're doing it to, et cetera.  Sample size is a very sticky issue and a lot of work has gone into trying to figure what that is.  Just three examples just to show you what the variability is:  If you talk to lamp manufacturers of standard sources, you'll find a lot of them say:  "Well typically we test six units," and then the next question would be: Well does the number six come from?  And you'll get a wide variety of answers, and typically it's just become an industry standard and it's kind of hard. At least I've never gotten a straight answer as to why the number six emerges, but it's probably a combination of you want enough so you feel confident that you're getting consistent results, and you don't want to do so many that you're increasing your cost a lot.  Another value that I've seen for some specifications that are being prepared for LEDs is a sample size of 25.  Again, where that comes from is a combination of factors, but it's a balance between having to test too many and doing enough so you get consistent results. There are other statistical ways of doing it. The federal regulations have methods that say you'll test a certain number with a minimum of a few, maybe four or five or six, until you get consistent results based on some statistical method; and if you don't achieve statistical consistency, you have to keep testing.  A final example is the Federal L Prize that you may have heard of, the initial sample size is 200.  Of course there are large stakes there with the L Prize, so they want to make sure that those are consistent.   So really the answer is it depends.  I myself like the statistical way of looking at the consistency and continuing to test until you get - - you do get a consistent value, but that sometimes is impractical as well.  So the answer truly is it depends on the application. You would have to look at the program you are doing it for and see what their requirements are.

There's another question here on what labs are ready to perform LM-80 testing, and I'm going to leave that to Heidi or Mia because they're closer to the labs that are actually doing the work.

Mia Paget: Yeah, I'll take that question, Eric.  Right now LM-80 testing for ENERGY STAR purposes is primarily being performed by the manufacturers of the LED chips.  In some cases, different LED sources are not made just by manufactures of the chips alone. They're made by integrators or other groups that may not be able to use the data just from the manufacturer of the chip.  So in those cases, they may be looking for outside laboratories to perform the LM-80 testing. The NAVLAP accreditation process that Heidi spoke about, which was just recently put into place, for covering both LM-79 and LM-80 will be something that can be used to get LM-80 accreditation. But that's just recently gone underway and as far as I know, there's no labs that have even really started the process of getting accreditation through NAVLAP for LM-80 testing.  So at this time, there are no qualified laboratories aside from the manufactured labs themselves that are performing LM-80 testing. There are a couple of laboratories that we are aware of that most likely really do have the capabilities to do this style testing and have actually - - they have experience and have been performing the testing for a certain amount of time, and I think that right now the ENERGY STAR Program is working with various laboratories to try to find a mechanism by which to make sure that these labs get accreditation or some form of qualification so that people that need outside or independent labs to do these tests can do them, and that is a process that's underway right now.

Rosemarie Bartlett: Thanks, Mia.  I think we've answered the questions that have come in. There were a couple questions related to the availability of the presentation and a video of the webcast.  So I did just want to let everybody know that if you look at the screen right now, within the next two weeks, so by April 23rd, the webcast video will be available at the Solid-State Lighting website. The handout is already available at the website, and a link to the handout was actually provided in your confirmation emails. 

So with that, I'd like to thank everybody for participating in today's webcast that was brought to you by the U.S. Department of Energy. You may all disconnect.