6 Comparison of benefits and costs of VTO’s R&D investments
This chapter compares VTO’s R&D investments from Chapters 1 and 2 with the economic and environmental health benefits, attributable to those investments, from Chapters 4 and 5. The benefits are attributable to VTO investments by way of their association with the impact of those investments (as established through interviews, by methods described in Chapter 4) on the diffusion of EDVs (presented in Chapter 3). To make an appropriate comparison, the streams of costs and benefits are discounted to a common date: January 1, 1992, the beginning of the stream of investment costs.
We present three impact metrics: net present value (NPV), benefit-to-cost ratio (BCR), and internal rate of return (IRR). We address the robustness of our estimates of these metrics in three ways: by applying both 7 percent and 3 percent discount rates to present-value calculations (for NPV and BCR), by providing estimates using the upper and lower bounds of confidence intervals based on the variability of respondents’ opinions, and by considering a truncated stream of benefits that stops on December 31, 2012, and thus does not include any benefits associated with the remaining useful lives of attributable EDVs beyond that date.
We find that benefits exceed costs (with BCR = 1.73) even under the most conservative scenario: using a 7 percent discount rate, confidence-interval lower bound, and considering only retrospective benefits (those accruing prior to December 31, 2012). A more optimistic and still defensible assessment can be based on the central tendency of respondents’ opinions and consider the full useful lives of attributable EDVs. In this scenario, the IRR is 17.7 percent, and a 7 percent discount rate yields a BCR of 3.63.
The chapter develops these estimates step-by-step, based on the information presented in earlier chapters, and presenting all arithmetic formulas used. The chapter concludes with a discussion of why even the more-optimistic defensible assessment (IRR = 17.7 percent and 7 percent BCR = 3.63) may be considered rather conservative.
VTO’s R&D investments
Column 1 in Table 6.1 is reproduced from column 4 in Table 1.1. VTO R&D investments, in 2012 dollars (2012$), are in column 1 over the years 1992 through 2012. These costs represent VTO’s total R&D investments in all areas of energy storage, including NiMH and Li-ion battery technologies. We emphasize that these investments overstate the relevant costs associated with the economic and energy benefits and environmental health benefits measured and monetized in Chapters 4 and 5, respectively, because investments are included that did not specifically target NiMH and Li-ion technology. Also, the significant lag between R&D investment and returns is ignored when investments through 2012 are counted against the benefits associated with attributable numbers of vehicles sold through 2012, a point revisited at the end of this chapter.
VTO R&D investments in energy storage technology, 1992 through 2012 |
Year |
(1) VTO R&D investments (thousands of 2012$, rounded) |
(2) VTO R&D investments discounted at 7 percent to 1/1/1992 (thousands of 2012$) |
(3) VTO R&D investments discounted at 3 percent to 1/1/1992 (thousands of 2012$) |
1992 |
39,783 |
39,783 |
39,783 |
1993 |
45,557 |
42,577 |
44,230 |
1994 |
51,699 |
45,156 |
48,731 |
1995 |
39,200 |
31,999 |
35,874 |
1996 |
37,145 |
28,338 |
33,003 |
1997 |
34,763 |
24,786 |
29,987 |
1998 |
37,397 |
24,919 |
31,319 |
1999 |
30,272 |
18,852 |
24,614 |
2000 |
30,474 |
17,736 |
24,056 |
2001 |
35,234 |
19,165 |
27,004 |
2002 |
33,375 |
16,966 |
24,834 |
2003 |
30,051 |
14,277 |
21,709 |
2004 |
26,987 |
11,983 |
18,928 |
2005 |
29,832 |
12,379 |
20,314 |
2006 |
31,444 |
12,195 |
20,788 |
2007 |
48,238 |
17,484 |
30,962 |
2008 |
52,128 |
17,658 |
32,484 |
2009 |
77,347 |
24,486 |
46,796 |
2010 |
82,035 |
24,271 |
48,187 |
2011 |
84,778 |
23,442 |
48,348 |
2012 |
93,034 |
24,042 |
51,511 |
Totals |
970,773 |
492,491 |
703,463 |
Note: Investment costs are assumed to have occurred at the beginning of each time period.
Columns 2 and 3 in Table 6.1 show the present value of VTO’s R&D investments (2012$), discounted to January 1, 1992, by 7 percent and 3 percent discount rates, respectively.1 Formally, the discounted totals provided at the bottom of columns 2 and 3 are derived from the undiscounted costs in column 1 by the following formula, where r is the discount rate and Cy is the undiscounted VTO investment in year y:
Economic and environmental health benefits attributable to VTO’s R&D investments
Column 1 in Table 6.2 is reproduced from column 8 in Table 4.11. Columns 2 and 3 show these attributable fuel savings discounted to January 1, 1992, using 7 percent and 3 percent discount rates. Formally, the discounted totals provided at the bottom of columns 2 and 3 are derived from the undiscounted values in column 1 by the following formula, where r is the discount rate, and By is the undiscounted value of attributable fuel savings in year y, and y_stop is either 2012 or 2022 (y+1 is used in the exponent because benefits are assumed to accrue at the end of each year, whereas costs are incurred at the beginning of each year):
Column 1 in Table 6.3 presents the average of the environmental health benefits in columns 3 and 4 in Table 5.9 (recall that these values were generated by COBRA, applying a 7 percent discount rate). Column 2 in Table 6.3 presents values analogous to those in column 1 but generated by applying a 3 percent discount rate in COBRA; applying the lower discount rate to the streams of benefits in COBRA results in values that are 12 percent higher for each year. Column 3 in Table 6.3 discounts the values in column 1 to January 1, 1992, by applying a 7 percent discount rate; column 4 discounts the values in column 2 to January 1, 1992, by applying a 3 percent discount rate. Discounted totals are derived from undiscounted yearly values using the same formula provided above for economic benefits.
Table 6.4 presents the combined economic and environmental health benefits from Tables 6.2 and 6.3, respectively. Column 1 in Table 6.4 adds, for each year and total, column 1 of Table 6.2 to column 1 of Table 6.3; column 2 in Table 6.4 adds column 1 of Table 6.2 to column 2 of Table 6.3.
Economic evaluation metrics
Three economic evaluation metrics were calculated using the data on attributable total benefits in Table 6.4: NPV, BCR, and IRR. Each of these metrics was discussed at a conceptual level in Chapter 1.
Attributable economic and energy benefits, 1999 through 2022 |
Year |
(1) Attributable fuel savings (thousands of 2012$) |
(2) Attributable fuel savings discounted at 7 percent to 1/1/1992 (thousands of 2012$) |
(3) Attributable fuel savings discounted at 3 percent to 1/1/1992 (thousands of 2012$) |
1999 |
2 |
1 |
2 |
2000 |
1,611 |
876 |
1,235 |
2001 |
5,027 |
2,555 |
3,741 |
2002 |
10,773 |
5,118 |
7,783 |
2003 |
21,419 |
9,510 |
15,023 |
2004 |
43,500 |
18,051 |
29,621 |
2005 |
106,175 |
41,176 |
70,194 |
2006 |
187,326 |
67,896 |
120,237 |
2007 |
295,234 |
100,006 |
183,980 |
2008 |
423,230 |
133,984 |
256,061 |
2009 |
356,448 |
105,460 |
209,376 |
2010 |
461,639 |
127,647 |
263,266 |
2011 |
627,142 |
162,065 |
347,233 |
2012 |
736,723 |
177,928 |
396,025 |
2013 |
706,913 |
159,560 |
368,933 |
2014 |
668,594 |
141,038 |
338,771 |
2015 |
613,955 |
121,039 |
302,025 |
2016 |
539,246 |
99,356 |
257,547 |
2017 |
453,881 |
78,156 |
210,462 |
2018 |
355,171 |
57,158 |
159,894 |
2019 |
273,935 |
41,200 |
119,731 |
2020 |
203,423 |
28,594 |
86,322 |
2021 |
143,097 |
18,798 |
58,954 |
2022 |
73,768 |
9,057 |
29,506 |
Undiscounted totals |
|||
Retrospective (1999–2012) |
3,276,249 |
||
Remaining lifespan (2013–2022) |
4,031,983 |
||
Life-cycle (1999–2022) |
7,308,232 |
||
Discounted totals |
|||
Retrospective (1999–2012) |
952,275 |
1,903,777 |
|
Remaining lifespan (2013–2022) |
753,955 |
1,932,145 |
|
Life-cycle (1999–2022) |
1,706,230 |
3,835,922 |
|
Note: For purposes of discounting, economic benefits are assumed to be realized at the end of each time period.
Attributable mean environmental health benefits, 1999 through 2022 |
Note: Environmental health benefits are assumed to be realized at the end of each time period.
Attributable total economic and energy and environmental health benefits, 1999 through 2022 |
Notes
Economic and environmental health benefits are assumed to be realized at the end of each time period.
Column 1 is the sum of column 1 in Table 6.2 and column 1 in Table 6.3.
Column 2 is the sum of column 1 in Table 6.2 and column 2 in Table 6.3.
Column 3 is the sum of column 2 in Table 6.2 and column 3 in Table 6.3.
Column 4 is the sum of column 3 in Table 6.2 and column 4 in Table 6.3.
where, By is the value in 2012 dollars of the sum of economic benefits plus environmental health benefits realized in year y, Cy is the value in 2012 dollars of R&D investments made in year y, and r is the discount rate (analogous to a real interest rate used to reflect the opportunity cost of investment capital). Note that benefits are discounted an extra year, following the EERE guidelines that benefits for a given year are assumed to have been realized at year-end, while costs are assumed to have been incurred at year start (Ruegg and Jordan, 2011). These formulas convert benefits and costs to January 1, 1992, present value.
The IRR is the discount rate r* such that NPV = 0 and BCR = 1:
Table 6.5 presents the evaluation metrics, for both lifecycle (1999–2022) and retrospective (1999–2012) streams of benefits and using both 7 percent and 3 percent discount rates for calculating NPV and BCR. Confidence intervals based on the variability of respondents’ opinions are provided in parentheses after each metric, and additional explanation of their derivation is provided below.
The evaluation metrics in Table 6.5 suggest that VTO’s investments in NiMH and Li-ion battery technologies have been socially valuable, even considering only the benefits through 2012. The most conservative estimates, ignoring benefits associated with the remaining useful life of attributable vehicles as of December 31, 2012, suggest that these investments were worthwhile as long as the IRR of the best alternative use of the funds would have been less than 12.8 percent, an assertion that is easy to defend.
Confidence intervals
Confidence intervals for the evaluation metrics reported in Table 6.5 were calculated based on our estimates of the market adoption of EDVs attributable to VTO as presented in Table 4.5 and Figure 4.6.2 Recall that these estimates were obtained from interview and survey responses to Questions 5 and 7 from the interview guide. The confidence intervals around the metrics in Table 6.5 account for the variability of respondents’ opinions based on 90 percent confidence intervals (plus or minus 1.65 standard deviations from the mean) around the percentage of market adoptions of EDVs attributable to VTO investments for each year, 1999–2012, reported in Table 4.5.
Evaluation metrics: economic and environmental health benefits, retrospective evaluation 1999 through 2012 (confidence intervals in parentheses) |
Note: Internal rates of return were derived using the combined benefits in column (1) of Table 6.4, by discounting these values back to January 1, 1992. This results in a small overestimation of the IRR because the discount rate applied in COBRA is fixed at 7 percent. The overestimation is small: a rough overcorrection still gives an IRR greater than 14 percent for retrospective and greater than 17 percent for lifecycle.
Recall that Table 4.6 translated the mean attributable percentage of market adoption for each year to a number of attributable EDVs, and that subsequent steps (illustrated with tables) translated these numbers of attributable EDVs to the metrics reported in Table 6.5. The confidence intervals for each metric in Table 6.5 were calculated by performing exactly the same steps – using, instead of the mean values in Table 4.5, the mean values plus or minus 1.65 their standard deviations.
Concluding observation
In the authors’ judgment, it is appropriate to consider the benefits associated with the remaining useful life of attributable EDVs on the road on December 31, 2012. We would therefore suggest that the most informative evaluation metrics are the BCR of 3.63 (based on a 7 percent discount rate) and the IRR of 17.7 percent. The confidence intervals on these metrics – (3.08, 4.18) for the BCR and (16.4 percent, 18.8 percent) for the IRR – reflect the variability of respondents’ opinions.
Although these metrics are more optimistic than the strictly retrospective metrics that truncate the stream of benefits at December 31, 2012, one might still argue that they understate the net social benefits associated with VTP’s R&D investments. We have noted before that the investments counted as costs included investments not strictly pertaining to NiMH or Li-ion technology. By limiting the scope of the study to benefits attributable to the accelerated adoption of EDVs embodying NiMH and Li-ion technology, we have implicitly assumed that the value of the VTO’s investments in other technologies since 1992 has been zero, which is of course unlikely. We have also made no attempt to quantify the economic value of knowledge spillovers associated with VTO-funded R&D or other positive externalities, stemming for example from the retention of human capital in scientific fields related to energy storage technology.
A further issue is that we have included as costs all investments up to and including 2012 while including as benefits, even in the lifecycle analysis, only those associated with vehicles put on the road by December 31, 2012. A portion of VTO’s R&D investments support the development of next-generation battery technologies, and as we learned from interviews it may take between 7 and 12 years for these technologies to be incorporated fully into EDVs and made commercially available. An example of this type of R&D would be the development of new anode materials for Li-ion batteries that will, as one respondent described, first appear in consumer electronics and may within ten years be scaled up for automotive application (as opposed to optimization of current cathodes for more immediate automotive application). Asked if the application of new Li-ion chemistries first to consumer electronics, almost as a proving ground prior to automotive application, was typical, this respondent said yes – or, if not typical, then certainly not unusual – citing as an example the use of lithium iron phosphate first for power tools. Many other Li-ion chemistries were first used in cell phones and laptops.
It is true that some VTO R&D investments could be expected to have shorter lags. As an example, funding from the American Recovery and Reinvestment Act (ARRA) was channeled through VTO to support the scale-up of production facilities for technologies that were already in place. In such cases, according to a senior R&D manager familiar with the program, the public investments would have had an observable impact within 2 to 3 years, allowing 18 months to 2 years to set up and 6 months to 1 year to validate the production lines. This 2- to 3-year lag could then reasonably be considered a lower bound.
As an illustrative example of the longer lag more typically associated with R&D investments, VTO supported the development of LG Chem Power pouch cell technology through successive USABC contracts over 10 years:
• DOE began evaluating the technology for EDV application with benchmark testing at Argonne National Laboratory (ANL) in 2002.
• USABC initiated a Technology Assessment Agreement with LG Chem Power in 2004.
• Following successful completion of that agreement, USABC entered into a technology development contract with LG Chem Power in 2006; a second phase development contract began in 2008.
• Following completion of the second phase development contract, GM began production of the Chevy Volt employing the USABC-developed LG Chem pouch cells in late 2011; Ford began production of the Focus EV, also employing the cells, in early 2012.
As this example illustrates, a portion of VTO investments of the past decade have supported technologies that have only recently had an economic impact. There is little doubt that it will be several more years before some technologies that received recent support from VTO’s R&D investments will appear in vehicles on the road, as several other anecdotes make clear.
One respondent described the development of a new process for manufacturing cathodes and anodes that replaces the traditional solvent-based binder with a UV-curable binder. The process reduces curing time from 45 minutes to less than 1 second, replaces ovens the length of football fields with desk-sized UV lamps, and has the potential to reduce by half the cost of manufacturing EDV batteries. In this respondent’s opinion, it is highly unlikely that this project would have been undertaken at all without VTO cost-sharing.
Another respondent described having recently completed a 3-year, $6 million project ($3 million in VTO funding) that developed a process to make a chemical that enables an increase in energy density of Li-ion cells: holding the size of the battery constant this process will result in a 30 percent increase in driving range, or, holding driving range constant the battery could be smaller and lighter.
Notes
1 See Chapter 1 and especially Note 22 for a discussion about the use of the 7 percent and 3 percent discount rates.
2 Regarding sensitivity analysis, OMB Circular A-94 (1992, p. 10) states: “Sensitivity Analysis: Major assumptions should be varied and net present value and other outcomes recomputed to determine how sensitive outcomes are to changes in the assumptions. The assumptions that deserve the most attention will depend on the dominant benefit and cost elements and the areas of greatest uncertainty of the program being analyzed.… In general, sensitivity analysis should be considered for estimates of: (i) benefits and costs; (ii) the discount rate; (iii) the general inflation rate; and (iv) distributional assumptions. Models used in the analysis should be well documented and, where possible, available to facilitate independent review.”