U.S. Department of Energy - Energy Efficiency and Renewable Energy

Building Technologies Office – Information Resources

Text-Alternative Version: CALiPER Report 20.1 Webinar

Ruth Taylor: Welcome everyone. This is Ruth Taylor with Pacific Northwest National Laboratory. I'd like to welcome you to today's webcast. We're going to be looking at the latest CALiPER study in our subjective evaluation of beam quality, shadow quality, and color quality for LED PAR38 lamps. We're pleased to have Michael Royer presenting today. He's also with Pacific Northwest National Laboratory.

He joined the lab in 2011. We're really happy about that, and he works mostly on CALiPER and all different areas of CALiPER, but he also helps out with GATEWAY and with fact sheets, and all over the SSL program. Before joining PNNL, he earned his Ph.D. in architectural engineering from Penn State University, and he focused on light and health, cycle physics of lighting, and color quality issues and other things. You can take it from here, Michael.

Michael Royer: Thanks, Ruth, and thanks everyone for dialing in. It's always a little bit hard on these webinars since some of this is better presented in person at our demonstration facility using photos and some things throughout here today, so hopefully that all translates pretty well across the screen. So as Ruth mentioned, this will be mostly about the CALiPER report 20.1, which was sort of extending the CALiPER study on PAR38 lamps beyond what we get in LM79 photometric testing. But just to sort of step back, the first thing we'll cover is just briefly look at what we saw in the initial round of PAR38 testing. So this report was released a little over a year ago now. There were 38 LED PAR38 lamps and five benchmarks, and just the regular CALiPER process with LM79 testing, comparing the performance of the LEDs to the benchmark products and looking at how they stack up against the claims that manufacturers are making.

So really, when we look at that, it's sort of "is this category ready for primetime for LEDs" and more and more, we do that, and we see yeah, of course, you're comparing it to halogen lamps that are maybe 15 lumens per watt, and yeah, LEDs are going to look pretty good. They're usually about 50 lumens per watt. But what about all these other aspects that we're not getting out of an LM79 report? So things like beam quality, shadow quality, color quality, that was sort of the focus of CALiPER report 20.1. We took 26 of the lamps that were tested for CALiPER report 20, so sort of a subset, and a couple of those in the intervening time had been upgraded, so we went with the upgraded product. We didn't have to stick with the old one. It was just a few though in that situation. So going beyond those metrics, do LED PAR38s outperform halogen PAR38 lamps? Then there will be a little bit of preview at the end of the presentation for the upcoming reports that aren't out yet. We're sort of still processing the data and writing up those reports. The next one to come out, I'm always hesitant to give dates, but probably looking at early January, would be flicker and dimming. Then following that, a stress testing report, and the final in this series will be a long-term testing report looking at lumen maintenance and chromaticity maintenance.

Again, just to review a little bit about the background data that sort of led us down this path, in 2012, we were testing this whole broad array of LED PAR38 lamps. You can see all the LED versions here as well as the five benchmarks aren't shown here. So we really tried to get a broad swath of the market from small manufacturers to big manufacturers. Generally targeting products that either were – had a lumen output equivalent to a 75-watt halogen lamp or somehow claimed that equivalency, but you sort of, as we're trying to include a broader range of products, sometimes it goes outside that boundary. It's not always this neat, perfectly grouped set of products.

So the basic results of this, and if you're interested in looking at these in more detail, I know they're sort of jam packed, and I go through them quick in a presentation, but these are all in the CALiPER report that's on the DOE SSL webpage. I have a link at the end that you can get to these reports. So we can see all the light blue squares in this slide where the products that were tested by CALiPER for Report 20, sort of the dark blue diamonds were some earlier 2008, 2010 products that CALiPER had tested. There's also ENERGY STAR and Lighting Facts data on here as well, sort of showing a complete range of products in the LED PAR38 space.

And really, what stands out here is well, almost all of them are meeting ENERGY STAR requirements. They're obviously way more efficacious than the halogen benchmarks. They are generally at least as efficacious as the CFL and metal halide benchmarks, and some even going above that. So really, LEDs include the very best options in terms of energy efficiency in the PAR38 category. So this is all good news, and if we looked at this again, we'd probably see that these efficacy numbers were probably five to ten lumens per watt higher even than what this report was showing from 2012.

So moving on, not just looking at energy efficiency, color quality as well, just looking at the metrics here, this is a plot of Color Rendering Index and Correlated Color Temperature. We can see the ENERGY STAR range and almost all the products falling in that range. A very tight cluster of products in the low 80s CRI, and because we targeted that, around 3000K, but we can also see there are some products that we had available that are above 6000K. You know, ENERGY STAR, at least at the time of this publishing, didn't recognize categories that high. Generally, if you're trying to replace a halogen lamp and you weren't aware of what CCT was, you might be dissatisfied with that 6000, 6500K product.

But that doesn't make it necessarily a bad product. Although you could say CRI down to 70 - 75, probably not the best use in an interior application. We can see again the spread from LED Lighting Facts data, lots of products available in that sort of LED sweet spot of low 80 CRI and around 3000K, 2700K. But again, spread throughout the whole range there. Looking also we get luminous intensity distribution data from a photometric report. So we can see here, again, a complicated chart, and there's more explanation in the full report, but the black lines are using the ENERGY STAR Center Beam Candlepower tool for a given beam angle and give it wattage of halogen lamp.

So you can see, for example, the 60-watt halogen lamp with ten degrees would be expected to have a center beam candlepower just above 10,000 candela. And the orange dots on this plot represent sort of a survey of halogen lamps looking through manufacturer catalogues and where they fall at a given beam angle. And arranged wattages, that's the sort of vertical spread and those orange lines. It's the range of wattages. And then the blue dots are the CALiPER-tested LED products. And so generally, I think it's fair to say that the LED lamps can cover most of the range of the halogen lamps. When we get up to the higher wattages, at least what was tested for this round wasn't quite there. That's also just an artifact of what we chose to test.

So if we were really starting to look for some of those higher wattage equivalents today, we might be able to find some of them. Again, I think the main point here is that at a given beam angle, the center beam candlepower is roughly the same to the given wattages, sort of that 75-watt target we were going for. So just going over the conclusions from that report, there were a lot of lamps available that could approximate a 75-watt halogen lamp. Some of what we had ranged from as low as roughly a 40-watt equivalent to as high as a roughly 90-watt equivalent. One product really low efficacy, but in general, 45, 44, up to around 80 lumens per watt, and again, this is 2012 numbers.

And so I would expect that to be a little higher today, and a lot of those products are right above the ENERGY STAR threshold, so 50 to 60 lumens per watt. Luminance intensity distributions ranging from very narrow to very wide. This is sort of one of the important findings of this because earlier LED products had often been criticized for not being able to produce that narrow spot, that eight-degree beam angle, but we found several products actually that did, and did it pretty well, and you know, they claimed that, and we were able to measure it and verify it through CALiPER testing.

Most of the manufacturer claims were pretty good. That's something that CALiPER has historically looked at. The areas we did find were often in underreporting. So the measured efficacy was actually higher than was claimed, although from product to product, there were some that were a little bit off. So moving on to 20.1. As I mentioned, we sort of chose a subset of the products, and we tested for the Report 20 basic edition. We just chose that to make it a little more manageable size for this demonstration event that was here in our facility in Portland. So we had 20 LED lamps and six halogen lamps. We invited members of the local IES chapter to come out and visit and provide feedback on the sort of more difficult to quantify performance areas.

So beam quality and shadow quality, those don't really have well defined and widespread use metrics, especially not shadow quality. Color quality does, but as everyone knows probably, there's sort of the controversy of well, how well is CRI working for this type of situation. So the lamps were divided into four groups. We had a spot group, a narrow flood group, and a flood group. Those were also used for distribution, narrow flood also used for shadow, and then we had a separate group specifically for evaluating color quality.

We had a wireless control system that allowed for group or singular viewing of any of these products, and so we had a protocol we went through where the observers would view all the lamps together, then they'd see them one at a time, and then they'd see them all together again, and I'll look at the feedback for them in just a second. I want to point out now, and I'll probably repeat myself again, but this really wasn't a scientific experiment. We weren't doing things like counterbalancing and a lot of other control measures that would really make this data more appropriate to extrapolate. So really, this was more of a market survey, more of getting feedback on these specific products, and so while we'll sort of make some conclusions based on this data on the product types, how we can sort of group them, it's not explicitly valid to really make very broad generalizations from this data set. So this is a photograph of the lamps installed out at our demonstration facility.

You can see they're all lined up. They're mounted on a truss system using a simple lamp holder. There was a black piece of fabric from behind, so the observers were blinded to the actual lamps. Members of the IES can be manufacturers and other people, so we didn't want anyone to be able to pick out their product and just give it all the highest rating. You can see the wires going back there. I think you can see here these are the wireless control devices that allowed us to switch individual lamps on and off as we desired.

So again, just because this is kind of confusing, we had these different groups. So for the beam quality evaluation, we had three groups. So the observers looked at that group of lamps alone, made all their evaluations, then went onto the next group of lamps. There was a spot group, a narrow flood group, and a flood group. They were broken down that way just to provide an easier way for someone to compare those products. Looking at apples to apples comparison. For this evaluation, they were aimed at the typical office color wall. There was a push pin that they were all aimed at, and we weren't using any kind of, taking measurements to ensure their aim. It's just simply on a ladder, a person doing their best, as you would in many typical installations, to aim the center of that beam at the push pin, which had an identifier that people could recognize each lamp.

Shadow quality, we only did that with the narrow flood group. It was just a way to keep things a little simpler, and we had the variety of products in there that we thought was necessary to gain some information about what's going to produce the best shadow quality, and in this case, they were aimed at a synthetic flower in a glass vase, which was sitting about one foot in front of the wall, cast a shadow onto that white wall. Sort of both these situations might not necessarily be that representative of a real world situation. They're almost more of a worst case situation where if you have retail environments where you're illuminating merchandise, you're probably going to have a polychromatic background, and some of the issues with beam quality might sort of get washed out and lost in all that extra stuff that's going on. But on a white wall, all those I'll call them flaws, not necessarily flaws, they become very apparent. But the idea was that by putting them through this worst case scenario, we can identify these issues, and if we have the capability with LEDs to eliminate these things that are typically – we can design around them, and they just get lost, that's going to make the LEDs generally more desirable so we don't have to worry about any of those limitations.

And finally, the last one I didn't mention. Color quality, one group there, just a color specific group, and we chose those lamps specifically for their CRI and CCT combinations. They're aimed at a color sample board that included an x-ray color checker chart, fabric samples, and some scenic paint samples. So this is a shot of the evaluation in progress. We sort of had two groups, about ten each, as to allow easier maneuvering around in this smaller space. So you can see this is an example when all the lamps were on. The observers are looking at all these together, and they have clipboards, and they can take notes.

So a little bit washed out in the photo, but you can see perhaps in the center of some of these beams, there's a little card that identifies the exact lamp type they were looking at. They could use that in their markings. There's another shot with just one lamp on. So this went down the line, coming on one at a time. And this is just an example, this was the color setup. I don't have an example in here of the shadow setup, but we'll see that in just a moment. So the observers were asked not to talk to each other at all until this was all done and over with. They took about ten minutes per evaluation group to make their observations, so fairly limited time. They weren't allowed to go in front and obviously look at the lamps or use their skin tones or anything like that to make color evaluations.

This is the feedback form. This is an example for one of the groups. So they had a set of instructions, and then they were asked to rank each of the lamps from least desirable to most desirable. After they were done ranking, they were asked to essentially give a rating, so sort of breaking it down whether it's unacceptable, acceptable, or outstanding. And most of the people were internally consistent, sort of as this example shows, whereas the lowest ranked products were unacceptable, then acceptable, and outstanding. A couple of instances where people had trouble doing the ranking, we do our best to interpret exactly what they were trying to get.

Then the final third step of this process was to list the most desirable attributes and the least desirable attributes. So essentially, what was leading to those rankings and ratings. So we had a list of attributes that you can see down at the bottom of the slide there, things like the hardness and softness of the beam edge, the gradient, brightness, pattern, consistency, color pattern, center beam intensity, stray light, and so all these things sort of gave us some additional information about what exactly is causing negative reactions and what are people reacting positively to.

So moving into the results here. So the top part, the sort of colorful chart here, shows a breakdown of the ratings. So the ratings were the outstanding, acceptable, and unacceptable portions of that. At the bottom is statistics on the ranking. So mean ranking, the mode ranking, which is the most frequent, the lowest ranked and highest ranked. So you can see for this group for example, almost all of the products were ranked lowest or the best. So one is the best, and almost all of them were also ranked the lowest. So that really indicated some level of indecision among the full group and some sort of difficulty in rating these products.

We can see the products over to the right, and so you can see – and this will be sort of a repeating trend here, the single emitter products, actually, tended to get rated the highest. So A1 was rated and actually ranked. The mean rank was the highest. You can see in this case as close as a single emitter as we could get for a spot lamp. So sort of a diffuse disk at the front of the lamp there. Then we see a halogen lamp coming in second, and the halogen lamps were also six and eight. So the halogen lamps were indicated by this orange bar at the top. Then multi-emitter LED lamps shown with these four blue dots.

We'd see generally a pretty clear gradation, actually, in the unacceptable ratings, the outstanding or acceptable ratings, a little bit less clear in terms of how we'd show that, and the rankings and ratings generally were roughly in order. We can see some juxtapositions. Looking at the mean rankings here, these two would be flipped based on their order, based on the rating, and also these two here. They're generally pretty consistent between the ranking and the rating. So one of the interesting things to come out of this set was the big difference in the halogen lamps.

So if we take a little bit closer look at those, we can see that they were actually quite different from a technology we think of often as being fairly well standardized and the same. These two products actually have pretty different lenses on them, which likely contributed some to that very different ranking. So A2, for example, had just a slightly diffuse lens, whereas A4 has this dimple pattern on the lens, but essentially more of a circular pattern, and here a different pattern in the lens. If we go back to these rankings and ratings, we see that A2, the one that really didn't have that pattern, produced the highest or most favorable beam quality. And we're going to get through these beam quality evaluations, and I'll show some photographic examples of some of that stuff and what the observers actually saw. So moving onto the narrow flood group, and this group was probably the most clear out of any of them. There were two lamps that were strongly favored, ranked very highly, zero unacceptable ratings, and those were B6 and B2. Again as I previously mentioned, you can see the distinguishing characteristic of these two lamps was that they were single emitter lamps. So they just have the single emitter here as opposed to multiple LED packages disbursed throughout the area of the PAR38 lamp.

We can also see lamps like B1, which actually did its best to mimic the construction of a halogen lamp, actually rated very poorly. As we can see from this and both the previous slide on the spot lamp rankings, halogen lamps don't necessarily rate very well. So that's something to keep in mind, that we're not always trying to emulate halogen technology. It's not necessarily an ideal source technology. LEDs, as this data shows, can actually do quite a bit better.

So moving on now to the flood group. We see pretty clear results here as well, and again, it was the two single emitter lamps. So that's C6 here and C1 here that were rated and ranked very highly. I believe these two were ranked one and two by every single observer in one order or the other. And again, the halogen lamp not ranking extremely high, and the sort of lamp mimicking a halogen lamp in a way rated fairly poorly. So if we sort of conglomerate all of that data, it really again shows the difference between the single emitters, multi-emitters, and some of the other sort of ways to conceive of an LED PAR38 lamp. Single emitters rating and ranking very highly. The sort of other methods, some doing very poorly, some doing very well, and multi-emitter lamps generally somewhere in the middle, and sort of the halogen lamps, again, not rated and ranked very highly. I think this will become a little more clear.

We're going to look at some examples now, and I hope some of this translates pretty well to your screen. This is sort of going from a demonstration event to a photograph to your screen, and so the exact way these are being presented are probably not exactly what was seen by the observers. So a little caveat at the bottom of every one of these that these photographs can be very deceiving. But I hope it just gives you an idea of some of these issues.

So this was lamp A1 in the spot beam quality evaluation group. It was by far rated the highest in that group, and you can see people identified the beam edge and the falloff in gradient as being very favorable attributes. This is where ten or more people identified that attribute on their recording form as something they liked about the lamp. So there were no attributes that ten or more people found unfavorable, and in general, only one or two people noted any of those things as unfavorable.

Now if we look at this one in contrast, this is A7, which was rated fairly lowly within the spot group. There were no favorable attributes, but both the color pattern and stray light outside the beam were considered unfavorable. One thing is this is shown in black and white here because photographing the color was rather difficult, so that's not going to show up, but the stray light, at least on my screen if you look here where the arrow is pointing, we sort of get this halo effect, and this is sort of common to the LED lamps that had unfavorable stray light pattern. You'd get some kind of ring or some kind of artifact outside the main beam, and that was sort of a signal to the observers that maybe this is not favorable – less desirable than some of the others that didn't have this issue.

And again, I'll repeat that maybe in an environment, this wouldn't be that big of a deal, but when we're comparing all of these, and I think when you have the choice of having that effect or not, people reacting to that would choose not to have that stray light. So this was the worst rated LED in the spot group. This is A6, and this one is the only photograph I'm going to show that's in color because I did think this actually captured a little bit of what was going on here. It really shifted strongly towards green as it faded out from the center of the beam, and that was very recognizable to the observers, and a lot of them, I believe more than ten, did note that on their recording form.

So one other example – I'm not going to show every product that was shown. We'll just go through just some examples. This is one of the halogen. This was A8, sort of the worst one in the group, and I don't know how apparent it is to all of you, but you can see these vertical striations emanating from the center of the beam. If you see these patterns here – and they're much more apparent when you're actually looking at the demonstration with your own eyes instead of on a computer screen. Those are very clear both within the beam and extending far out beyond the edge of the beam. And in general, that was a consistent characteristic to almost all of the halogen lamps with any kind of dimpled bend structure. The one that had just more of a slightly diffuse lens, A2, didn't quite suffer this effect as severely, although it was still there a little bit.

Moving onto some examples from the narrow flood group, this one was one of the highly rated ones, the single emitter lamp. People really found the center beam candlepower favorable for this one. You can see there's not really any stray light here. Generally soft edge to the beam, nothing extremely distracting about it. Similar, B2, similar design in the lamp. This time, there just happened to be more people that said the falloff in the gradient was what attracted them to rate this lamp highly.

Pretty similar situation there. Now if we look at some of the ones that were poorly rated LED, this was B3, and if you look so that there's this almost edge within the beam and then there's some more light that goes out beyond that, and so the observers, more than ten of them, noted stray light as being unfavorable attributes of this product. This was the worst rated one, and this was the one I pointed out earlier that's trying to mimic the lens in a physical structure of a halogen lamp. It doesn't translate that well, but you can really see there's a sort of mottled spottiness to the beam and extending out beyond it. You don't get the vertical striations or the radial striations that you would with the halogen lamp. Sort of a similar effect of this uneven luminance pattern projected on the wall.

So moving on from beam quality, we'll go to shadow quality next. And again, this was the narrow flood group, very clear results. The single emitter lamps were heavily favored. That's B2 and B6. In this case, B1 was also rated fairly highly, even though in the beam quality, it was not rated very well. So that's a good indication to us, actually, that people were able to separately evaluate these attributes. Again, the ones sort of as we would have hypothesized that didn't perform as well were the multiple emitter LED lamps, B5 and B3, almost unanimously unacceptable, and the halogen lamps actually rated very poorly as well.

We can go to the next slide here and see the examples of those six products. So you can see the highly ranked ones tended to produce a very crisp shadow. The ones that didn't do so well, the multiple emitter lamps, LED lamps, sort of have a multiple shadow effect, sort of a blurriness to it, and the halogen lamp actually is the most blurry of any of those in the shadow. One of the observers was quick to point out that, well, sometimes I might want a very crisp shadow. Sometimes I might want a very blurry shadow, depending on the type of situation I'm illuminating. Do I have key light and fill light, anything like that? But in this situation, it was sort of clear that the observers felt that crisp shadow was the least objectionable and something they would have the least problem with or something that would be the least undesirable in a typical installation if you were just buying these for a retail application.

Color quality results, and these ones are probably the most difficult to interpret. We look at the ratings, or we look at the rankings, we get fairly different results, and we get a lot that's right in the middle. So in this ordering here, order at the top by the rating. The halogen lamp with 100 CRI was actually – had the least unacceptable ratings, and then a high CRI 3000K lamp and the other 3000K lamps. The two that really stuck out as lower rated were the 2700K LED lamps. And this is really not your typical way you'd ask someone about color quality. We normally think of CRI as being divided between, you evaluate that at a specific CCT. In this case, we presented the situation as something a designer specifier might be faced with when they're choosing their product for their retail environment. So they might be looking at 2700 or 3000K simultaneously and different CRI levels in any of those.

So we had both higher CRI and lower CRI in 3000K and 2700K, and the halogen, it's about 2850K in the middle. These were all either dimmed or screened to produce approximately the same center beam illuminance, so we didn't suffer from hunt effect or other issues with different illuminance levels on the color targets. But again, I'll repeat, not necessarily a scientific and carefully – very rigorously controlled experiment. You can see the spectral power distributions for those lamps. Actually fairly similar amongst all those lamps. The higher CRI versions simply are converting more of slightly more energy to longer wavelengths. So the purple was the highest CRI 3000K lamp, and this red was the highest CRI 2700K lamp, and the black line is the halogen benchmark.

So just to sort of illustrate more of the lack of clarity coming out of this, these are the histograms of responses for all of the products, and you can see I don't think any of them received more than ten ratings, rankings in one category there, one through six, with one being the highest, six being the lowest. We did see in some instances, for example, the 2700K 96 CRI lamp D1 some people really liked it, some people really didn't like it. In general, we found roughly 15 of the people almost always rated the 3000K higher, and five or so people almost always rated the 2700K higher.

So it was really sort of an initial split just looking at CCTs, about what would be higher. But then even if we break down the groups and look at only the 3000K group, the ratings and rankings really didn't follow what CRI would predict. So they actually – the 83 and 86 CRI 3000K lamps, so D3 and D6 in the middle actually had more outstanding ratings than did the 93 CRI 3000K. So again, we're using CRI here, which is purely a measure of fidelity, and so it's just sort of one more piece that shows CRI doesn't necessarily correlate with perceptions of perceived color quality or preference.

So some conclusions. In each of the quality categories, at least one LED lamp was rated more favorably than the benchmark halogen lamps. So this is a good sign that LED lamps, they might not just be higher efficacy, but they can also improve lighting quality, and I think that's a good position to be in for LED lamps because you don't have to sell them as just an energy efficiency upgrade when you can also have a simultaneous improvement in quality which could lead to more widespread adoption. The single emitter LED lamps, very clear results from here were almost unanimously favored for beam quality and shadow quality. Now that's not to say that that's the only design that works or that these lamps will always be better, but it was something very apparent from the results from this specific evaluation.

Poor color consistency within the beam and stray light outside the beam were the most negatively noted attributes, so those were very apparent to the people making the observations. And in general, if we looked at on a group by group basis, the narrow spot lamps were generally viewed as having the best acceptable beam quality than the narrow flood or flood counterparts, although within any of those groups, there was pretty substantial variation. So that sort of indicates that while we've seen some of those narrow spot LED PAR lamps make it to market, maybe there's still a little bit more work in that category, more demanding optical systems, which means it's a little harder to get the desired beam quality.

And with the color quality, again, the least apparent conclusion that could come out of that, CRI didn't work really well at ordering this, although very small sample size and sort of not your typical situation of how you would be evaluating CRI, sort of using it outside of its intended application. So just quickly, I want to move just to preview some of these upcoming reports, a couple more minutes on this, and then we'll move to questions.

This is report 20.2, flicker and dimming. So one of the things we've done with all of these PAR38 lamps, and this is the full set, not just the subset used for the quality evaluation, but looking at flicker at various dimmed loads with various dimmer combinations, here is just some example data. This product actually doesn't have much flicker. Looking at flicker versus the light output, you can see at very low outputs, you start to get a little bit more flicker. Looking at power quality over dimming range, so this is THD and power factor over the range of light output. So we'll have plots like this for all these types of products and sort of some combined analysis on these different trends we're seeing.

Stress testing will be the next one that's coming out. This is a picture of the apparatus. I don't have any data on that yet. I know just anecdotally, we're having a very difficult time getting any failures out of any of the LED products, which is a good thing. We're subjecting them to vibration and heat and humidity and electrical stresses, and they're generally very robust. So this is not a measure of lifetime, sort of a measure of robustness.

So the final part of that that's coming down the road is the long-term testing. So this is all the PAR38 lamps installed in a lumen maintenance test apparatus at PNNL in Richland, WA. You can see the benchmarks in there as well. So far, these have been going for near 6,000 hours, and this was just – yesterday, I just took a quick check in on the data, and this is what we're seeing so far. So this is the averages for the different benchmark groups. We can see the LEDs doing pretty darn well, still near 100 percent output at 6,000 hours. We can see the CFL and ceramic metal halide benchmarks. They've lost roughly 20 percent of their initial output, and then the halogen lamps – well, it's going way down because they've all failed already after 3,000 or 4,000 hours there.

Looking more closely at the LEDs, we can see a fairly tight spread that does pretty well. I believe the ENERGY STAR criteria for a 30,000 hour L70 is 93 percent at 6,000 hours. We can see just about all of those are above that level so far, but there is almost a ten percent difference in the lowest to the highest lumen maintenance at 6,000 hours. So it'll be interesting to see and track these over time, how that spread continues to increase, and then we can also see a couple early failures in this spot. So each of these lines are the average for five samples. So we can see that one failed, you're going to lose roughly 20 percent instantly, and so those two early failures out of about 170 products of various types – whether you considerable that reasonable or not I guess is up to interpretation. But considering that all the halogen lamps have already failed, I think they're doing all right so far for just off the shelf, not specifically engineered – they're not like the L Prize. These are just everyday available LED products.

Looking at color maintenance, this is a plot of CCT. Again, the average for all the LEDs, and when you have them all, the average is going to generally even things out, but we can look compared to, obviously, everyone knows metal halide, not the most stable source. It's jumping around a bit. The CFLs and halogens there, also pretty stable. Looking at DUV, again, just at this early stage, rough look at the data. Just looking at averages for all the products, but generally, the LEDs have shown the least color shift of any of those products. So again, doing pretty well. I don't think I have a plot of it, but looking at any of the individual LEDs, I think they're all at 6,000 hours, the Delta U-Prime V-Prime, less than .004 could meet the ENERGY STAR criterion, which I think was good news because we had been seeing some other products coming back from field installations that had shifted pretty substantially, and those tended to be a little bit older products that had hours of use on them to this point. Some of these products purchased in 2012 are doing a little bit better in that regard. I think that's a bit reassuring.

So that's all I have for right now. I'm happy to answer any questions that are available. I think Ruth is going to moderate that, so thank you for listening.