Department of Energy: Advanced Research Projects Agency-Energy Could Improve Its Collection of Information from Applications

Published by the Government Accountability Office on 2012-01-24.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                            United States Government Accountability Office

GAO                         Testimony
                            Before the Subcommittee on
                            Investigations and Oversight, Committee
                            on Science, Space, and Technology, House
                            of Representatives
                            DEPARTMENT OF
For Release on Delivery
Expected at 2:00 p.m. EST
Tuesday, January 24, 2012

                            Advanced Research
                            Projects Agency-Energy
                            Could Improve Its
                            Collection of Information
                            from Applications
                            Statement of Frank Rusco, Director
                            Natural Resources and Environment

Chairman Broun, Ranking Member Tonko, and Members of the

I am pleased to be here today to discuss our work on the Department of
Energy’s (DOE) Advanced Research Projects Agency-Energy (ARPA-E).
As you know, in 2007, the America Creating Opportunities to Meaningfully
Promote Excellence in Technology, Education, and Science (America
COMPETES) Act established ARPA-E within DOE to overcome the long-
term and high-risk technological barriers in the development of energy
technologies. 1 ARPA-E borrows from the model of the Defense Advanced
Research Projects Agency (DARPA), an agency created within the
Department of Defense (DOD) in 1958 to direct and perform advanced
research and development projects.

Since first receiving an appropriation in 2009 in the American Recovery
and Reinvestment Act of 2009, ARPA-E has awarded $521.7 million to
universities, public and private companies, and national laboratories to
fund 181 projects that attempt to make transformational—rather than
incremental––advances to a variety of energy technologies, including
high-energy batteries and renewable fuels. 2 Award winners must meet
cost share requirements, through either in-kind contributions or outside
funding sources. 3

ARPA-E is required by statute to achieve its goals through energy
technology projects that, among other things, accelerate transformational
technological advances in areas that industry by itself is not likely to
undertake because of technical and financial uncertainty. At the same

Pub. L. No. 110-69, § 5012 (2007).
 ARPA-E has released a total of four funding announcements—meaning the
agency was accepting project proposals for a set period of time—in April 2009,
December 2009, March 2010, and April 2011. ARPA-E generally uses
cooperative agreements to make funding awards, which involve the transfer of a
thing of value to the recipient to carry out a public purpose authorized by law.
Cooperative agreements differ from grants because substantial involvement is
expected between ARPA-E and the recipient.
 The cost share requirement for award winners is generally at least 20 percent of
total allowable costs, although under section 988(b)(3) of the Energy Policy Act
of 2005, ARPA-E has reduced the cost share requirement for certain applicants,
such as universities, to 5 percent or 10 percent for all of the funding rounds
except the first. Award winners’ cost share must be provided by a nonfederal

Page 1                                                                GAO-12-407T
time, the Director of ARPA-E is required to ensure, to the maximum
extent practicable, that ARPA-E’s activities are coordinated with, and do
not duplicate the efforts of, programs and laboratories within DOE and
other relevant research agencies.

My testimony today focuses on the key findings and recommendations
from a GAO report on ARPA-E being released today by the
subcommittee. 4 For that work, you asked us to examine (1) ARPA-E’s
use of criteria and other considerations for making awards and the extent
to which applicants identify and explain other private funding information,
(2) the extent to which ARPA-E-type projects could have been funded
through the private sector, and (3) the extent to which ARPA-E
coordinates with other DOE program offices to avoid duplicating efforts.

We conducted this work in accordance with generally accepted
government auditing standards. More detailed information on the scope
and methodology from this work on which this testimony is based can be
found in appendix I of the report.

In summary, the agency uses several selection criteria in making awards
though its requirements for information on private sector funding could be
improved. Also, our review suggests that most ARPA-E projects could not
have been funded solely by the private sector. Finally, ARPA-E officials
have taken steps to coordinate with other DOE offices to avoid

•   ARPA-E uses four selection criteria—the impact of the proposed
    technology relative to the state of the art; the overall scientific and
    technical merit of the proposal; the qualifications, experience, and
    capabilities of the applicant; and the quality of the proposed
    management plan—in awarding funds. ARPA-E’s eight program
    directors, who are generally scientists and engineers, create and
    manage funding programs for the agency and apply these selection
    criteria when reviewing applications. Of the 20 applications we
    reviewed for award selection criteria, all contained supporting
    information addressing the agency’s four criteria. In addition to
    applying its four criteria, ARPA-E gives program directors discretion to

 GAO, Department of Energy: Advanced Research Projects Agency-Energy
Could Benefit from Information on Applicants’ Prior Funding, GAO-12-112
(Washington, D.C.; Jan. 13, 2012).

Page 2                                                           GAO-12-407T
    use additional considerations to award funds to projects, including
    whether ARPA-E applicants received private funding. Identification in
    applications of sources of private funding and the extent to which that
    funding might support the proposed projects can help provide
    program directors with assurance that ARPA-E funds do not overlap
    with private investment. We identified 18 out of 121 award winners
    through ARPA-E’s first three funding rounds that had received some
    prior private sector investment, and ARPA-E took steps to identify and
    understand how this funding was related to proposed projects. During
    the first two funding rounds, ARPA-E required that applicants identify
    relevant private investors if the applicant believed these funds were
    related to the proposed project. When applicants provided little prior
    funding information, ARPA-E’s program directors spent time and
    resources to determine the extent of such funding for proposed
    ARPA-E projects. According to our review of ARPA-E data from the
    first three rounds of funding, the agency reduced requested award
    amounts by 5 percent or more on 31 out of 121 projects, for a total of
    $59 million below total requested award amounts for these rounds. 5
    Beginning with the third funding round, ARPA-E began requiring that
    applicants explain why private investors were not willing to fund
    proposed projects. However, ARPA-E did not provide applicants with
    guidance, such as a sample response, to assist them in completing
    this requirement, and responses were generally limited. Some
    applicants provided general information about prior research but did
    not specifically explain why private investors would not support their
    projects. One applicant included a letter from its venture capital
    investor to explain why the investor was not willing to fund the work
    proposed to ARPA-E, an approach the National Institute of Standards
    and Technology uses as a check in its funding applications for
    advanced research but that ARPA-E currently does not use. Also,
    ARPA-E officials said that they have considered but have not used
    venture capital data to identify applicants with prior private investors.
    Examining such data allowed us to quickly cross-check applicants’
    prior private funding.

•   Our review suggests that most ARPA-E-type projects could not be
    funded solely by private investors. This finding is based on our

 ARPA-E can reduce the proposed project scope to fund only what the program
directors consider to be the transformational part of the project to avoid funding
applied research or development work that would be outside ARPA-E’s program

Page 3                                                                 GAO-12-407T
    interviews with representatives of venture capital firms and our
    analysis of subsequent funding received by contingently selected
    APRA-E applicants—those applicants that met ARPA-E’s selection
    criteria but were not selected for an award. 6 The representatives we
    spoke with from six venture capital firms indicated that they generally
    do not fund the types of projects that ARPA-E looks to fund for three

•   First, venture capital firms generally do not fund projects that rely on
    unproven technological concepts or lack working prototypes
    demonstrating the technology. Data from ARPA-E on award winners
    show that 91 out of 121 ARPA-E projects from the first three funding
    rounds had technological concepts that had not yet been
    demonstrated in a laboratory setting. 7

•   Second, venture capital firm officials told us that they focused closely
    on the timeliness of investment returns, with one firm noting that the
    industry tended to invest in technologies that could be commercialized
    in less than 3 years and that would potentially exhibit exponential
    market growth in approximately 5 to 7 years. However, we found that
    nearly all of the 13 ARPA-E award winners and most of the 22
    contingently selected applicants we spoke with estimated that their
    projects were 3 or more years away from potential
    commercialization. 8

•   Third, venture capital firms may not be comfortable investing in new
    energy technologies, noting the historical lack of successful venture
    capital investments in these types of projects. Venture representatives
    said that venture firms were more comfortable investing in software
    companies or other businesses with higher potential profit margins
    and less costly product development than new energy technologies.

 According to ARPA-E officials with whom we spoke, these applicants would
have been selected for an award had additional funds been available.
 These data showed that most ARPA-E award winners were at or below
technology readiness level (TRL) 3. TRL 3 represents a level where potential
technologies are still unproven.
 On the basis of the initial testing of our questions, we determined that these
estimates may be optimistic, given that respondents are invested in attempting to
bring a technology to market as soon as possible.

Page 4                                                                GAO-12-407T
The 18 award winners we identified as having received prior private
venture capital told us that with the ARPA-E funding, they were generally
able to pursue the development of energy technologies with greater
scientific or technical uncertainty than they had when they were working
with their private funding. About two-thirds of these award winners told us
that the ARPA-E funding has allowed them to develop prototypes or to
prove basic technology concepts on more advanced ideas than their prior
work—6 of these award winners said this was for completely new
research and 7 said it was for major advancements to prior research. A
few of these award winners also told us they were able to work on
projects with outstanding scientific research questions that private
investors would not have funded. Five of these award winners reported
that they would likely have been able to pursue some research similar to
their ARPA-E projects, but it would have taken years longer without
ARPA-E funding. In addition, officials from two public companies we
spoke with that were awarded ARPA-E money told us that although their
companies had internal resources devoted to research and development,
they were not able to internally fund the projects they proposed to ARPA-
E for two following reasons. First, existing product lines placed heavy
demands on their internal research and development budgets, and there
is continuous pressure from existing customers and competitors to
improve existing products. Second, these companies told us that internal
investments had to meet minimum investment return thresholds, and that
ARPA-E-type projects were not able to meet these thresholds. 9 In
addition, we found that few contingently selected applicants found funding
from private investors or public sources. Eighteen of the 22 ARPA-E
contingently selected applicants we interviewed sought funding after
being turned down for ARPA-E funds. Of the 18 that sought funding
elsewhere, 13 submitted project proposals to government sources, such
as other DOE offices, the National Science Foundation, or nonprofit
academic research institutes, and the remaining 5 submitted proposals to
private investors such as venture capital firms. 10 As of September 2011,
we found that 2 out of the 22 contingently selected applicants secured
funding from venture capital firms for work that was very similar to their

 Officials from one company told us that the rate of return on investment required
by its management was at least 20 percent per year.
  One of the 5 contingently selected applicants that sought funding from a private
investor also sought public funding.

Page 5                                                                GAO-12-407T
ARPA-E project proposals. 11 We also found that 4 contingently selected
applicants secured funding from a government or nonprofit source for
their projects. 12 The 4 contingently selected applicants that secured
funding from a government or nonprofit source modified their ARPA-E
proposals to be more focused on basic science research, rather than on
developing a commercial technology.

•    According to ARPA-E officials and documents, agency officials have
     taken steps to coordinate with other DOE offices in advance of
     awarding ARPA-E funds to help avoid duplication of efforts. These
     coordination efforts can be categorized into three areas: (1)
     prefunding coordination, (2) coordination of application reviews, and
     (3) participation in official DOE coordination groups. For prefunding
     coordination, ARPA-E officials told us that program directors engage
     with officials from related DOE offices in advance of announcing the
     availability of ARPA-E funds. For example, ARPA-E officials told us
     that directors use the workshops and other meetings to identify
     research areas that other DOE offices are not working on, and the
     other DOE officials provide insights on funding areas where they are
     not active. For coordination of application reviews, some ARPA-E
     program directors told us that they have recruited officials from other
     DOE offices to review applications submitted to ARPA-E and that
     these officials made up as many as one-third of the reviewers for one
     director. ARPA-E has also used application reviewers from other
     federal agencies, such as the Department of Defense. One program
     director told us that these reviewers have also helped avoid funding
     projects similar to those potentially funded elsewhere. Finally, ARPA-
     E is also a participant in official DOE coordination groups. For
     example, ARPA-E is a participant in DOE’s SunShot Initiative within
     the Solar Energy Technologies Program. The SunShot Initiative is an
     effort to coordinate solar energy research across DOE’s Office of
     Science, four national laboratories, the National Science Foundation,
     and ARPA-E, with the goal of achieving costs of $1 per watt for solar-
     generated electricity. Additionally, the ARPA-E Director created the

  In addition, our review of venture capital funding data for the other 11
contingently selected applicants with whom we did not speak did not show that
any had received venture capital funding since not being awarded ARPA-E
  Three contingently selected applicants that submitted proposals to government
or nonprofit sources were still awaiting responses at the time of our review.

Page 6                                                              GAO-12-407T
     Panel of Senior Technical Advisors (PASTA), which is a group of
     high-level DOE managers that meet periodically to discuss current
     and future DOE research efforts. ARPA-E officials told us that PASTA
     is an attempt to avoid duplicating efforts within DOE. PASTA meeting
     attendees have included officials from DOE’s applied and basic
     science offices. We were not able to directly evaluate the
     effectiveness of ARPA-E’s efforts to coordinate with other DOE
     offices. Nevertheless, on the basis of our interviews with ARPA-E
     award winners and contingently selected applicants, we found that 4
     award winners and 2 contingently selected applicants had received
     prior funding from other DOE offices. 13 According to these award
     winners and contingently selected applicants, the prior funding was
     either for more proven technologies or was focused on more basic or
     foundational research than was the ARPA-E funded project.

On the basis of these findings we recommended that ARPA-E:

•    provide guidance with a sample response to assist applicants in
     providing information on sources of private funding for proposed
     ARPA-E projects,
•    require that applicants provide letters or other forms of documentation
     from private investors that explain why investors are not willing to fund
     the projects proposed to ARPA-E, and
•    use venture capital funding databases to help identify applicants with
     prior private investors and to help check information applicants
     provide on their applications.

ARPA-E commented on a draft of the report being released today and
concurred with these recommendations.

Chairman Broun, Ranking Member Tonko, and Members of the
Subcommittee, this concludes my prepared statement. I would be
pleased to respond to any questions that you may have.

 These award winners included those in our nonprobability sample of 13, as well
as the 18 we identified with VentureDeal data.

Page 7                                                              GAO-12-407T
                  For questions about this statement, please contact Frank Rusco at (202)
GAO Contact and   512-3841 or ruscof@gao.gov. Contact points for our Offices of
Staff             Congressional Relations and Public Affairs may be found on the last page
                  of this statement. Individuals making key contributions to this statement
Acknowledgments   include Tim Minelli, Assistant Director; Karen Keegan; Rob Marek; and
                  Jeanette Soares. Key contributors for the work that this testimony is
                  based on are listed in appendix V of the report being released today.

                  Page 8                                                        GAO-12-407T
This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (www.gao.gov). Each weekday afternoon,
GAO Reports and       GAO posts on its website newly released reports, testimony, and
                      correspondence. To have GAO e-mail you a list of newly posted products,
Testimony             go to www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
To Report Fraud,
Waste, and Abuse in   Website: www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548

                        Please Print on Recycled Paper.