R&D policies aim to improve the effectiveness of government R&D and support R&D investment from the private sector. Countries with strong R&D also stand to benefit from the ability to sell new technologies abroad.

The relationship between investment in R&D and emission reductions is highly uncertain. We therefore do not estimate the potential impacts of R&D on meeting long-term emission targets. Nevertheless, R&D policies can reduce the costs of emission reductions, improve the performance of existing technologies, and unlock new technologies that can ultimately make carbon reduction easier and more cost-effective.

The starting point in thinking through technology development is to understand that different strategies are needed for different stages of a technology’s lifecycle.

The policy-technology learning curve (illustrative).

R&D policies are most effective for technologies early in their lifecycles. Any country that has either public or private R&D efforts aimed at bringing new technologies to market can benefit from R&D support policies.

A country that does not have substantial resources to invest in public R&D efforts (such as by establishing or funding national labs) may be able to entice multinational companies to conduct R&D locally, if the country can provide favorable tax incentives and a sufficiently strong educational system to provide the highly skilled scientists and engineers necessary for R&D operations.

For example, Ireland is known for proactively courting innovation-focused multinational companies by using these mechanisms. Developing countries without the necessary conditions for either public or private R&D efforts are technology importers, and these countries may be better served by focusing on other types of policy, such as performance standards for vehicles and appliances, policies to promote renewable energy development, and financial assistance for building and industry energy efficiency upgrades.

Even within the R&D span of time, conditions are not uniform, and government financial support (whether through grants, loan guarantees, or other mechanisms) is most crucial where gaps exist in other funding opportunities.

This gap most commonly happens between basic research (which is often tackled through government labs and academic institutions) and the scaling up of production for commercial use (which is funded by the private sector). This gap is commonly referred to as the “valley of death,” because promising technologies often fail to cross the gap and move from the laboratory to full-scale demonstration projects or production.

The gap in funding availability between basic research and commercialization is typically referred to as the “Valley of Death.” Source: U.S. Government Accountability Office.

Some R&D policies help at all stages of technological development, such as ensuring companies have access to high-level science, technology, engineering, and mathematics talent. Others, such as grants or loan guarantees, may be most effective when designed to help technologies overcome funding gaps.

Research fundamentally involves speculation and experimentation. Even with perfect policy incentives, many research projects will never become commercial products. Sometimes technical or scientific issues interfere, sometimes the marketplace changes, and sometimes a more innovative approach makes a project obsolete before it can be commercialized.

This reality demands that policymakers and investors tolerate risk and research failures. If you don’t have any failures, then you will not find true successes.

Sometimes government may be rewarded for taking on this risk. For example, the U.S. Department of Energy’s (DOE) loan guarantee program for clean energy achieved a $30 million profit from 2010 to 2013, despite backing several unsuccessful companies.

Create long-term commitments for research success

One of the most challenging issues at the interface of legislation and technology development is the need for a long-term outlook for technology policy. Private sector companies need consistency and reliability before they make big bets. Both government and private sector labs must buy equipment, recruit experts, and build and run careful experiments. Policies that promote R&D therefore must match the long time horizons of technology development, or researchers and government will squander opportunities and waste money.

Confronted with political and budgetary challenges, policymakers tend to fund things a year at a time. But it cannot be overemphasized just how deleterious stop-and-start policy is to serious energy innovation.

For example, since its introduction in 1981, the U.S. R&D tax credit was repeatedly extended for short periods of time and allowed to expire. One CEO of an especially research-driven energy technology company told us that, as a result, it “considers the R&D tax credits just to be a windfall, with no impact on the company’s R&D choices.” In 2015, after 15 piecemeal extensions, the R&D tax credit was finally made permanent.

By ensuring that commitments to R&D last for long periods, government, academia, and private companies will have the confidence to rely on that support when making R&D investments.

Use peer review to help set research priorities

Selecting research projects among many competitors is difficult and complex. When prioritizing different research projects to receive government funding, the government should conduct peer review of the options, involving experts from both within the government and within industries that might benefit from technological progress in the relevant field.

Consulting with experts inside and outside the government can help ensure projects are technically feasible, would be useful to society if accomplished, and have an acceptable risk–reward profile.

For example, the U.S. DOE developed a “Quadrennial Technology Review” that considered the potential for breakthroughs in many areas and overlaid them with national priorities, such as reducing dependence on imported oil. The work engaged 600 experts from the private sector, national labs, and academia. The experts were asked to consider the technologies’ leverage against a list of national policy goals and against three explicit measures of potential:

• Maturity: Technologies that have significant technical headroom yet could be demonstrated at commercial scale within a decade.
• Materiality: Technologies that could have a consequential impact on meeting national energy goals in two decades. “Consequential” is defined as roughly 1% of primary energy.
• Market potential: Technologies that could be expected to be adopted by the relevant markets, understanding that these markets are driven by economics but shaped by public policy.

This process helped DOE identify issues with its past funding methods (such as the need to achieve a better balance between projects with near-, medium-, and long-term impacts) and helped identify where to focus efforts to better achieve national priorities.

Use “stage-gating” to shut down under-performing projects

Research is an inherently risky endeavor, and there is always a possibility that a line of inquiry will produce no results or will produce too few results to justify the necessary investment.

To ensure large amounts of money and staff time are not wasted, it is important to establish stage-gates or milestones that a research project must pass in order to continue to receive funding. A project should be shut down if it fails to achieve these milestones, so the staff and funding allocated to that project can be reallocated to more fruitful endeavors. Although some research failures are inevitable, a strong gating procedure helps ensure that when you fail, you fail early and fail fast, before vast quantities of money have been expended.

Funded projects can generate entrenched interests, making it more challenging to remove funding from an existing project than to fund a new project. Therefore, it is critical that gating include independent experts with a combination of scientific and industrial expertise in the relevant field. With the inclusion of an industry perspective, project funding decisions can be made based on a project’s scientific merits and ultimate commercialization potential, not on political considerations.

For example, DOE’s Industrial Technologies Program uses stage-gates to manage its R&D allocations. An example review team might be composed of:

• A representative of the funding department
• Outside (i.e. industry) technical expertise
• Inside (i.e. government) technical expertise
• Representatives of proposed end users
• Members of the R&D team

Concentrate R&D by type or subject to build critical mass

Providing a small quantity of R&D funding to each of many different institutions is inefficient, because coordination between these institutions and duplication of work will consume an inordinate share of the R&D investment. It is better to concentrate R&D funding on a specific topic into a smaller number of institutions—potentially co-located with each other or with relevant industry players—to reduce coordination challenges, facilitate knowledge sharing, and avoid duplication of work.

One way to accomplish this is to create “innovation hubs” or “centers of excellence,” each of which is composed of academic, private sector, and government researchers, ideally in the same metro area. In addition to avoiding duplication of work, bringing together researchers with different backgrounds unlocks further synergies:

• Researchers feed off each other’s ideas.
• Students gain technical skills through internships and university-industry partnerships, and businesses have access to talent.
• Business interests working side-by-side with academia make technologies’ transitions from lab to market faster and more reliable.
• Early stage investors, such as venture capital, move in and work as a further accelerant.

Make high-quality public sector facilities and expertise available to private firms

In some countries, such as the U.S., the government has invested in the development of extremely expensive, high-quality scientific and engineering research facilities, such as DOE’s National Laboratories.

These facilities are staffed with skilled experts. A private company that wants to conduct R&D to improve the performance of its products might be unable to afford to build its own cutting-edge laboratories and staff them with experienced scientists only to make small improvements to its product line. A research partnership with a national lab allows the company to benefit from high-quality facilities and expertise for a comparatively small payment, enabling it to gain the benefits of research without replicating R&D capabilities.

A national lab can partner with many different companies, improving technical prowess across the economy (as long as careful attention is paid to protecting intellectual property). These partnerships can also provide a source of revenue for the national lab, making it less dependent on taxpayer funding.

For example, India’s Central Power Research Institute has housed R&D facilities for use by government, industry, and utilities alike over the past 50 years. The public–private research facilities have helped India make progress on high-voltage transmission, power system resilience, and other electricity distribution components.

Meanwhile, the U.S. National Renewable Energy Laboratory has built more than a dozen centralized testing facilities—such as the Energy Systems Integration Laboratory, which studies grid modernization.

Similarly, Sandia National Laboratories offer 18 test facilities (such as the Combustion Research Facility and the Mechanical Test and Evaluation Facility) where researchers from private companies, academia, other laboratories, and state and local governments may visit and perform research, or they may contract directly with Sandia to perform their testing.

Protect intellectual property (IP) without stifling innovation

Intellectual property (IP) protections are necessary to protect private firms’ investments in R&D. If patents are not protected, then any firm can use research results in its own products, reducing or eliminating the incentive for firms to engage in R&D in the first place.

However, it is also important to avoid allowing patent and IP protections to stifle innovation. The difficulty inherent in ensuring the novelty and uniqueness of every patent submission has led to two problems.

First is the rise of patent assertion entities (also known as patent trolls), companies that acquire a key patent (often a vague or overly broad patent) without any intent to use the patent in a product or service. Rather, they sue carefully selected companies that could be construed as violating the patent, in hopes of extracting a legal settlement. This requires small companies to agree to unreasonable settlement demands or to spend a larger amount of money defending themselves against a frivolous lawsuit, providing a disincentive to innovate.

Second, in some fields, such as information technology, existing patents are numerous, and the need to use underlying technologies is so universal (e.g., to achieve interoperability with other systems or hardware components) that avoiding infringement while achieving innovation is impossible.

Large technology companies have learned to defend themselves by acquiring thousands of patents and threatening to countersue any company that sues them for patent infringement. Rivals recognize that suing a deep-pocketed company with many patents will probably result in high legal fees and substantial risk to their own product lines and so are discouraged from suing.

However, small companies that do not own thousands of patents don’t have this deterrent capability, so they are vulnerable to lawsuits that may force them to go out of business or to sell their company to one of the technology giants. This is far removed from the original intent and purpose of a patent system.

Designing appropriate IP protections is complex. Some advocacy groups have thought carefully about patents and devised principles that can be used to develop suitable patent protections.

Ensure companies have access to high-level STEM talent

In order for private companies to conduct R&D successfully, they need a ready supply of talented people with skills in science, technology, engineering, and mathematics (STEM). From a public policy perspective, there are two ways that governments can assist.

The first is to establish top-quality education programs focusing on these areas, helping students acquire science and math skills early and providing a route to further develop these skills at the university and graduate school levels. In the U.S., primary and secondary schools are funded primarily by state and local governments, so schools in less wealthy communities receive less money and produce students with poorer scores in science and math (as well as other subjects). Accordingly, in addition to directing sufficient resources to STEM, policies to tackle income inequality and poverty can contribute to improving access to technical education.

Government-sponsored research internships at labs or private firms can help students further develop technical skills. For example, the government of Ireland funds university students and postdocs while they work at internships in the R&D divisions of innovative companies such as IBM.

The second policy mechanism is to ensure that immigration laws enable companies to hire skilled technical talent from other countries. Researchers are highly skilled people who contribute to a country’s economy. In the U.S., groups from across the political spectrum have advocated for streamling in visa and permanent residency procedures, including a proposal to offer automatic residency to graduates of U.S. universities with advanced degrees in the STEM fields. Australia, Canada, and the United Kingdom use point-based or merit-based immigration systems, which give priority to people who possess degrees and work experience in areas of need, which typically include these fields.

The U.S. Advanced Research Projects Agency-Energy

Seeking a highly effective way to fund research into energy technology and bring new technologies from the laboratory to the market, in 2007 the U.S. established a new R&D funding agency: the Advanced Research Projects Agency–Energy (ARPA-E).

The agency’s approach was modeled after that of the Defense Advanced Research Projects Agency, a hugely successful government R&D operation that played a key role in the development of technologies we use every day, including GPS satellites, packet-switched computer networks, and the internet.

ARPA-E focuses on funding research projects that are too early to attract private sector funding (such as venture capital) but that have the potential to rapidly advance and achieve commercialization. Thus, they span the gap between basic research and product development.

Funded projects must be transformational: They must have the potential to “radically improve U.S. economic prosperity, national security, and environmental well-being.” Projects must have specific, proposed applications toward products or processes that could be commercialized, and the agency provides resources for research teams on how to seek commercial funding and proceed down the path of commercialization to follow up their grants. ARPA-E also adopts a nimble funding structure, making funding decisions quickly and relying on program directors who are experts, often from industry, and who “serve for limited terms to ensure a constant infusion of fresh thinking and new perspectives.”

ARPA-E has distributed $1.5 billion in R&D funding to more than 580 projects since funds were first disbursed in 2009. Many recipients have gone on to form new companies and partnerships with other funding entities. Advances have been achieved in grid-scale and flow batteries, electric vehicle systems, power flow and grid operations, power electronics, advanced materials, and more.

Innovation Network Corporation of Japan

In 2009, Japan launched the $1.9 billion Innovation Network Corporation of Japan collaboration between the public and private sectors to achieve advances in energy, infrastructure, and other high-technology sectors. The Japanese government invested 95% of the upfront capital to create the corporation, and 26 private companies made up the final 5% investment.

The Innovation Network Corporation is an investment company. It directs its investments in order to nurture the development of next-generation industry through applied technology, focusing on innovations that will have “social significance.” The company aims to achieve positive economic returns from its investments, so it will not need ongoing government support. Example investments in the energy space include small wind power, laminated lithium ion batteries, smart meters, and semitransparent solar cells.

The Japanese government has offered $8.5 billion in loan guarantees for the corporation’s investments, mitigating the risk in the event that some investments perform poorly. The Innovation Network Corporation has formed partnerships with 10 external organizations, including Japanese universities, government agencies, venture capital, and several research institutes. This allows it to benefit from the knowledge and talent of other organizations when making investment decisions.

Germany’s Fraunhofer-Gesellschaft

The Fraunhofer-Gesellschaft is a network of 69 research institutes throughout Germany. Fraunhofer emphasizes applied research: Most projects last no longer than two years “and focus on immediate, applicable results.” This helps to fill the budget gap between basic research and commercialization (the “valley of death”).

Fraunhofer is the largest research organization in Europe, with a staff of 24,500 and an annual budget of €2.1 billion. 30% of the organization’s budget comes from the public sector, and 70% is derived from contract research done for public or private entities.

Fraunhofer’s institutes are grouped into eight alliances covering specific research areas, such as “materials and components” or “microelectronics.” These groups coordinate research, pool resources, and avoid duplication, serving as an example of an “innovation hub” model (as discussed earlier).

Fraunhofer also plays a role in cultivating technical talent, necessary to ensure that German companies and Fraunhofer itself have access to the scientists and engineers they need for R&D success. Each Fraunhofer institute is partnered with a university, and Fraunhofer employs graduate students and postdocs part-time, helping them acquire industry experience alongside their academic studies. “Graduates typically spend from three to six years at Fraunhofer before moving on to positions in industry or academia.”

Fraunhofer has played a role in ensuring that German manufacturing businesses remain globally competitive, even in the face of low-cost products from Asia. Many German small and medium enterprises are market leaders for their products, which offer higher quality and performance than inexpensive alternatives. As a result, manufacturing accounts for 21% of the German economy, a much larger share than in similarly developed, high-wage countries such as the U.S. (13%) and the United Kingdom (12%). Fraunhofer is an example of how policies to strengthen R&D, designed well and applied consistently, can offer outsized returns to a national economy.

A robust pipeline of new energy technologies is crucial to enable a country to continue to meet energy demand and expand the economy while transitioning to clean energy. Strong R&D support policies are crucial early in the lifecycle of a technology, before it reaches market and other policies (such as performance standards and financial incentives) can take over. In particular, many research projects find it hard to acquire funding to overcome the “valley of death,” the gap between basic research and early commercialization. This presents an opportunity for particularly high-leverage R&D policy support.

Government R&D financial commitments, programs, and incentives should be guaranteed for the long term, to match the timeframe needed to scope a research project, hire staff, expand or retool laboratories, and turn early research into a marketable product.

Peer review, including experts from government and industry, can help identify priorities for research dollars that will achieve practical benefits for the private sector, the environment, and public health.

Stage-gating can be used to ensure research dollars and staff time are not wasted, and concentrating research by subject into “innovation hubs” can improve coordination, reduce administrative burden, and accelerate progress.

Carefully designed IP protections, as well as education and immigration policies that provide an abundance of top technical talent, lay important groundwork for R&D success in both the public and private sectors.

With smart policies to promote R&D, national governments can strengthen their technological prowess and economic position, attract R&D investment, improve energy security, and reduce pollution.