Modern economies wouldn’t be modern without the availability of various forms of energy—electricity, gasoline, diesel fuel, and natural gas. Except for the blending of gasoline with supplements like ethanol, the entire automotive and trucking transportation sector depends on crude oil. Natural gas, meanwhile, once thought to be a low-value fuel and confined to some manufacturing uses and fuel for running electric generators in peak periods, has become the hottest source of energy because it is turning out to be plentiful and is much cleaner to burn than oil or coal.
Even as late as the beginning of the Great Recession and in the early years of the recovery, however, there was deep pessimism among policy makers and the public about the prospect of continued dependence on foreign oil, especially from countries with less than friendly relations with the United States. This pessimism now seems like ancient history. In a few short years, the combination of hydraulic fracturing (fracking) and horizontal drilling has unleashed a massive increase in oil and gas production, by unlocking these fuels from shale formations throughout the continental United States. As a result, the dependence of the United States on foreign oil has dropped sharply and is widely projected to continue declining. Likewise, the price of natural gas, which is set more in domestic than global markets because it is less transportable, has also declined (though like the prices for other commodities, it also continues to fluctuate).
This chapter tells the story of how both oil and natural gas—essential to running a modern economy—came to be regulated and then deregulated, the latter consistent with a long line of advice from many economists. I’m not making the claim that the turnaround in U.S. energy markets is due solely to the economists—clearly, the development and use of new technologies, coupled with a revival of the entrepreneurial spirit in this sector, is mostly responsible. But I am making a modest claim that this turnaround would not have occurred had the prices of oil and gas been regulated during the past two decades, and some, if not much, of the credit for that goes to economists.
I also trace here how the oil and natural gas supply revolutions have affected virtually the entire economy. In this way, the energy revolution, enabled in significant part by sound economic advice, represents another policy-platform technology. Like the deregulation initiatives discussed in the previous chapter, an unregulated energy market has facilitated more rapid growth for existing businesses and contributed to the formation of some new firms.
The stories of how the prices of oil and gas came to be regulated and then deregulated are very different, but they ended up at the same, right place.
The regulation of crude oil prices came relatively late compared to natural gas. Apart from the four years of World War II when oil and virtually everything else was subject to price controls, crude oil prices were largely unregulated (except in Texas, where state authorities limited production in an effort to stabilize prices). This was true until the quadrupling of world oil prices caused by the Arab oil embargo of 1973–1974 prompted the Nixon and then Ford Administrations to limit the prices of “old” domestic oil (oil discovered before the embargo), while allowing newly discovered oil to be bought at world prices. As will become evident in the history of natural gas price regulation below, this split between old and new oil struck a compromise to keep a lid on most oil production in order to contain inflation while preserving incentives for producers to look for new oil.1
The problem is that once the government interferes with the pricing mechanism of the market, there is no telling how things will turn out—much like what happens when taking a loose thread out of a sweater only to discover that the sweater itself starts to become unraveled. A similar process happened with crude oil price controls.
Consider the following series of events. First, the cap on old oil, at about $3 per barrel, encouraged owners of existing wells to cut back production, which meant a cutback in fuels derived from crude oil. Shortages, manifested in long lines of angry motorists at gas stations, were the inevitable result. Second, government officials responded to this problem by introducing a system of odd–even rationing, so that drivers could only fill up on days on which the last digit on their license plates were odd or even (matching the day).
What is surprising, at least to this author, is that this complicated system of price controls and rationing was introduced and maintained by two successive Republican administrations, or those that ordinarily one would assume never to have the government interfere with the market, at least when it comes to setting or limiting prices. Reportedly, Nixon’s chief economist, Herbert Stein, opposed both the general wage and price controls of 1971 and the oil price controls adopted after 1973, to no avail. Even a Republican administration turned out to be more fearful of both inflation and consumer anger at higher gas prices at the pump than they were of consumers upset by rationing. Roughly three decades later, it was another Republican administration, this one led by President George W. Bush, which abandoned free market principles again to rescue the banking system in order to avert a worse calamity. These episodes illustrate that crises cross partisan lines; they make it hard to stick with rules meant for more quiescent times.
The 1970s were a marked decade apparently, because at their end the nation and the world faced another large jump in oil prices, this one precipitated in early 1979 by a cutback in Iran’s oil production and later a ban by the United States government on oil imports from Iran. These steps induced panic buying by oil traders and oil companies, which pushed crude oil prices from around $15 per barrel to $40 per barrel. With controls on domestic oil still in place, and consumers’ experiences with the post-1973 oil price jump and long lines at gas stations that followed fresh in mind, lines began to reappear as many consumers sought to top off their tanks to ensure they always had enough gas to get where they wanted to go. Some states re-imposed the odd–even rationing system that had existed only a few years before to reduce the lines.
But then something surprising happened. Despite another bout of general economy-wide inflationary pressures, largely at the behest of his economic advisers, a Democratic presidential administration began dismantling the remaining system of crude oil price controls in April 1979, which ultimately was completed when President Reagan took office in 1981.
While Carter gave up controls with one hand, he took something from the oil industry with another, by tying oil price decontrol to a windfall profits tax on oil producers (computed as the difference between a base 1979 price and the average market price for crude oil over the year). The purpose of the tax was to recapture the profits owners of old oil would earn from decontrol. Congress agreed and imposed the tax as a quid pro quo for price decontrol in 1980.
The key point in this story is that despite some of the failures in Carter’s energy policy, such as the synthetic fuels program that produced essentially no new fuel, and the controversies over the details of his energy plan, Carter’s economists won the day. They persuaded the president to do something that his Republican predecessors could not bring themselves to do—end crude oil price controls—because controls were discouraging owners of these older wells from producing more oil. To be sure, the positive impact of oil price deregulation on production was limited largely because of the windfall profits tax. Nonetheless, deregulation reestablished the important principle that all crude oil prices, whenever oil is discovered, should be set by the market and not by regulators.
This principle proved to be hugely important in the 2000s when global oil prices again increased sharply. This time policy makers did not impose a date-based regime of oil price controls or a windfall profits tax, which allowed the market to clear without long lines and shortages. In addition, regulatory forbearance gave oil producers market incentives to marry new and existing drilling technologies that eventually led to the unlocking of vast amounts of oil from existing and newly discovered fields and in the process fundamentally changed the way policy makers have viewed the oil and gas markets.
Natural gas, oil’s cleaner cousin, has a much more complicated regulatory history. Since the mid-1800s most segments of the natural gas industry—production, transmission, and local distribution—have been regulated in some manner. That is less true over the last few decades, however, with production prices no longer regulated and left to the free market. How that story unfolded is what I briefly discuss here.2
Natural gas was first used as a heating fuel after methods for extracting it from coal were developed. Since the economics of distribution favored natural monopolies, municipalities began regulating natural gas prices, at the point of delivery to consumers, as a way of protecting them against the power of the gas companies.
By the early 1900s, gas was being shipped between cities over intrastate pipelines. Since the economics favored the construction of only a single pipeline, which by that time typically transported gas from multiple sources (wells in addition to gas produced from coal), states began regulating intrastate rates on the same antimonopoly rationale. Eventually, pipelines were built across state lines, and state regulators attempted to follow them, but were slapped down by the Supreme Court in a series of cases in the 1920s. The Court held, quite understandably, that the Commerce Clause of the Constitution preempted the states from extending price controls on commodities such as gas shipped beyond their borders.
Interstate pipelines remained unregulated until Congress stepped in by enacting the Natural Gas Act (NGA) of 1938, which gave the power to set interstate pipeline rates (based on the cost of transmission) to the Federal Power Commission (the FPC later became the Federal Energy Regulatory Commission or FERC). In addition, Congress directed the FPC not to authorize the construction of new interstate pipelines where one already existed, effectively giving the agency both price and entry control over the pipeline business.
One thing the NGA did not seem to authorize, however, was the regulation of the prices gas producers could charge pipelines. That situation was short-lived because in the early 1940s the Supreme Court decided that gas producers affiliated with pipelines also were covered by the NGA. In 1954, the Court extended its ruling to cover unaffiliated gas producers, defining them as natural gas companies, thus bringing them within the ambit of the NGA’s price control regime.
But how was the FPC going to regulate the prices of thousands of gas production wells? The answer is with great difficulty, if not impossibility. At first, the agency tried, unrealistically, to set prices for each well, based on its cost. This clearly proved to be impractical, so by 1960 the commission tried to set prices in five broad gas-producing regions, based on average costs of wells drilled in each region. That approach, too, took longer than anticipated and so in 1974 the commission adopted a national price ceiling of $0.42 per million cubic feet (mcf), a price that was lower than unregulated intrastate prices but still much higher than was allowed in many parts of the country before.
The below-market level for gas prices encouraged consumption, but discouraged the search for new gas, a recipe for eventual disruption, which a number of economists pointed out. They noted that the fragmented nature of the gas production market made it ill-suited for price regulation, paralleling arguments for deregulating transportation markets that also were a far cry from being natural monopolies, as discussed in the last chapter. The disruption eventually came when oil prices surged in 1973 to 1974 and many industrial and residential heating consumers who tried to switch to cheaper, regulated gas found out they couldn’t.
Gas shortages were aggravated by the side-by-side existence of unregulated intrastate gas and regulated gas sold to interstate pipelines. If you were a producer, to which pipelines would you sell? Over time, more producers understandably picked intrastate pipelines, which made the natural gas market much less national in scope than crude oil, where no such distinctions existed, even when old and new oil were defined as part of the crude oil price controls introduced after the embargo-induced price jump. That price spike, by the way, widened the price differential between oil and gas (calculated at a common energy equivalent, British Thermal Units or BTU equivalents). With prices for natural gas suddenly relatively low, consumer demand for gas also spiked while additional supplies were not forthcoming. A severe gas supply shortage developed in the winter of 1976 to 1977, forcing the FERC to ration gas across the nation just as gasoline was being rationed.
Throughout this period and well before, economists well versed in the economics of energy had been calling for the deregulation of natural gas at the wellhead. With thousands of wells in business, it was impossible to call this industry a natural monopoly. Perhaps the most well-known economist making this argument was Paul MacAvoy, now emeritus professor at Yale’s School of Management (where he was a former dean) and a former faculty member and dean of Rochester’s Business School. MacAvoy also spent two years as a member of President Ford’s Council of Economic Advisers at a critically important and convenient time in the history of natural gas policy, when natural gas supply curtailments or shortages reached a peak during Ford’s tenure.3 In the legal academy, Stephen Breyer of Harvard Law School and later a Supreme Court justice, also was a vigorous critic of natural gas price controls, citing MacAvoy’s research.4
Backed by the intellectual firepower of the economics profession and coupled with the harsh realities of gas shortages, Congress finally began the lengthy process of dismantling production price controls in the National Gas Policy Act of 1978, which was part of the larger National Energy Act that the Carter Administration was pushing in an effort to make the nation less dependent on foreign sources of energy. The NGPA/NEA is yet another example where a Democratic president and a Democratic-controlled Congress did what free market Republicans had long been associated with—letting the market, rather than the government, set prices.
To be sure, the process was a gradual one. Wellhead prices on newly discovered gas were to be phased out over a seven-year period ending in 1985, while old gas would be forever subject to the pre-NGPA price limits. The rationale for the delay was that in an era of higher oil prices, a totally decontrolled gas market would lead to much higher gas prices and aggravate inflation, which was already a major national concern. In addition, the higher prices would hit colder, more Democratic areas of the country harder than the southern states, where less energy was needed to heat homes in the winter. The price limits on old gas, meanwhile, reflected the belief at the time that no additional gas could be recovered from wells that had already been drilled (erroneous as it later turned out with the development of enhanced recovery techniques), so why give producers of this gas a windfall?
The process of unwinding price controls even for new gas at the wellhead proved to have more complications than may have been originally thought. Even though pipelines blended the cost of newly discovered gas with old, regulated gas, average prices increased, inducing users to reduce their demand, relative to what pipelines expected. This mattered greatly, because after the NGPA became law, many pipeline companies believed that unregulated gas prices were going to continue to increase and so they signed long-term “take or pay” contracts with producers in an effort to lock in prices for substantial periods of time. When demand and average prices fell below expectations, a number of pipelines were stuck paying more, on average, for gas than they could sell it for.
Ultimately, all of the major distortions in the gas market were removed when Congress enacted the Natural Gas Wellhead Decontrol Act of 1989, which put all natural gas prices, including the price of old gas, on a path to decontrol by 1993. Since then, all natural gas prices have been determined by market forces, although because of differences in transportation arrangements, and differing local supply and demand conditions, the costs of gas delivered to customers still exhibits some regional variability. Other orders by FERC during the 1980s and 1990s also freed pipelines to deliver gas on a dedicated basis to certain, mainly industrial, users.
So, the lesson in all this history is that economists, such as MacAvoy, who argued that the price of natural gas at the wellhead should have been deregulated long ago were right all along. Had their advice been heeded, then many of the distortions associated with the natural gas market after the 1973 to 1974 oil embargo could have been avoided. Still, it is important that the policy eventually turned out right (except for the continued regulation of old gas), and helped set the stage for the technological revolution in oil and gas discovery that eventually came.
Now, it’s time for full disclosure: My father, a highly intelligent man whose education was cut short by the Depression, was a reluctant independent oil producer, and I sort of grew up in the business. I assure you that this background does not explain my opposition to price controls on oil; readers by now will have understood that, like virtually all economists, I am opposed to such controls in any industry that is not characterized by natural monopoly. Indeed, I spent much of my dinner-table time getting into vociferous arguments with my dad about the wisdom of all the tax breaks from which oil producers (and thus I indirectly) have benefited.
But Dad was wise, and among the many things about his business he told me (much of which went in one ear and out the other at the time because after seeing him struggle anxiously for so many years, I knew early on I never wanted to be in the business) was that eventually enhanced recovery techniques would unlock substantial additional amounts of oil and gas from existing wells, and most importantly our wells. So, he urged that when he passed away (which he did in April 2011, at the age of 99) I should hold on to his oil producing properties for this reason alone, since not only would the requisite technology arrive, but the price of oil would rise sufficiently high, if left unregulated, to make it economically attractive to deploy the technology (Dad ventured nothing about gas prices because he wasn’t a gas producer).
If only both of us had listened to his advice, but we didn’t. As Dad approached the age of 92 his main wells needed substantial additional investment to keep them pumping and I was too busy working elsewhere, so a combination of risk aversion on the part of Dad (understandable at his age) and my inability to manage not only the reinvestment but any subsequent operation of the wells if the investment worked led Dad to sell out. Shortly thereafter, oil prices began to jump sharply, and several years later the fracking revolution was in full bloom, so the buyer of Dad’s wells made out handsomely. Perhaps the only upside to this story for readers I hope—apart from confirming that an oil and gas technology revolution was inevitable and totally foreseeable—is that had we kept the property, I might be doing something very different and not have written this book!
The only thing that neither Dad nor I could have known at the time (this was in 2004) is how quickly the technological revolution in enhanced recovery would come. The revolution, in fact, was like other innovations—a combination of two separate, but related, technologies that combined to allow a quantum leap in results (the proverbial 1 + 1 = 3 synergistic result, rare but occasionally real).
One of the technologies, hydraulic fracturing, or fracking, has been the most controversial, but it is hardly new. My dad was doing it on his wells in the 1960s, and according to my research, it has been in use in some form in the oil industry since the late 1940s. The idea behind it is very simple and is suggested by the name: Highly pressurized liquid (typically water, sand, and some chemicals) is injected in an oil or gas producing well to break up the rock at the bottom so that it releases more oil and gas, which is pumped to the surface.
Fracking has become a much bigger deal in the last decade or so because it has been combined with newer directional drilling technology.5 When my dad was in the oil business in the 1950s through much of the 1970s, oil producers drilled downward in a straight line for oil since that was the only way known for doing it (some producers would drill at a slant, if they had rights to the underground oil or gas on adjacent lands, which some of them did not, but the drilling still would proceed in a straight line). With directional drilling, oil companies (or their contractors) can essentially snake their way down the holes they drill in the ground, using motors and drills attached to pieces of pipe that can be attached in such a way that oil and gas can be found in multiple pockets underground using one wellhead. This eliminates the need to punch multiple holes in the ground, each drilled straight down, which collectively is far more expensive than using directional drilling techniques from a single wellhead. Directional drilling is especially cost effective in offshore locations where oil can be found, often only at very deep levels underground, using very expensive drilling platforms.
If one combines directional drilling with the fracking of so-called tight formations of shale or similar rock, then vast new horizons open up for recovering oil and gas from locations that otherwise never could be reached. When you see the fracking revolution referred to in the popular media, what the writers really mean (knowingly or unknowingly) is the combination of fracking with directional drilling.
The fracking or unconventional drilling revolution has been a major surprise, catching a lot of people, many of them quite knowledgeable, off guard. Until quite recently, there was much talk that U.S. oil production had peaked and was forever headed downward, which meant that the United States was destined to become ever more reliant on foreign-produced oil.
For example, it took only one quick look at a graph of U.S. oil production over time to become convinced that this peak oil thesis, named after geophysicist M. King Hubbert, must be right. Domestic oil production seemed to spike in 1970 at about 3.5 billion barrels a year (about 10 million barrels/day), but declined, more or less steadily, for almost 40 years thereafter. It only bottomed out in 2009, when oil captured from tight formations began to be recovered.6 Because U.S. oil demand was pretty steady during the nearly four decades that domestic oil production was falling, foreign oil increasingly made up the difference.7 The widespread pessimism throughout this period, up to its very end—that this trend would only continue—was thus readily understandable, and drove politicians in both political parties to call repeatedly for comprehensive energy independence policies (though the parties differed on how this was to be achieved). Indeed, the fear that Iraq could play a destabilizing role in the Organization of the Petroleum Exporting Countries (OPEC) was one (albeit not publicly stated) factor that drove the United States to fight in both Gulf Wars (true, there were other factors, such as Iraq’s invasion of Kuwait in 1991 and the belief that Iraq had weapons of mass destruction in 2003, that drove these decisions, but one wonders if the United States would have led the attack in each case if Iraq were not a major oil producer).
The problem with the peak-oil thesis that the fracking revolution has so clearly revealed is that it takes little or no account of the economics of the oil and gas industry, which if one consults any basic introductory economics textbook, are really no different from the economics of any other industry. Peak oil didn’t take account of economics of the supply curve, which posits that as prices increase, firms find new, more expensive ways of bringing things to market, or in energy’s case, more expensive energy technologies. Prices and profits were the powerful market-based motivators for oil and gas companies and drillers to perfect the combination of horizontal drilling and hydraulic fracturing that led to the new oil and gas energy boom. In hoping that he or I would hold on to his wells waiting for the secondary oil recovery revolution to happen (which not only enhanced recoveries from existing wells but encouraged drilling for new, more difficult-to-find oil and gas), Dad was unwittingly thinking like an economist. I wish I could have told him that.
While most commentators have focused on the major impacts of the fracking revolution on the oil and gas industry, the effects on other sectors may be even more important. Oil and gas are inputs, directly or indirectly (for example, as fuel for electricity), in virtually every other sector of the economy. So it stands to reason that the benefits of any major technological change in oil and gas discovery, like innovations elsewhere in the economy, are going to be captured to a significant degree by the consumers of the products and services that depend on the innovation. Indeed, one well-known study by Yale University economist Bill Nordhaus (who is profiled in Chapter 14) documented that just 4 percent of the benefits of the invention of the lightbulb were captured by the inventor, and the other 96 percent by purchasers of lightbulbs.8 Nordhaus argued that this was likely to be a general phenomenon, because in competitive industries, as most industries are, any reductions in cost get passed through entirely or nearly so to consumers.9
I will not be surprised if many readers are shocked, or even in a state of disbelief, over this result. After all, isn’t the reason for our patent, copyright, and trademark system to give strong incentives to inventors and innovative entrepreneurs to come up with and commercialize innovations by granting them some of the monopoly rights to their inventions? What about the fabulous riches earned by the founders of Microsoft, Apple, Google, Intel, and so on? Doesn’t that prove that most of the benefits of innovation accrue to inventors and entrepreneurs?
Actually, the answers are no. Even for firms that have temporary or moderately long-lasting monopolies in the industries defined by their inventions, they only get rich by selling massive quantities of their inventions to the consuming public. Think about Microsoft for a moment, which has sold billions of copies of its operating system and key applications software programs to consumers throughout the world. While these consumers fork over $100 to $500 or more for copies of this software, the benefits to purchasers and society as a whole surely exceed these costs.
Individuals use the software on their PCs, and more recently tablet computers, to organize their daily lives, and in many cases, to work at home, either for the companies that employ them or for their own startups, in ways that simply would not have been possible before the PC revolution (for which companies like Microsoft, the PC manufacturers, Intel, and others are responsible). For business users, or those who buy PCs, tablets, and related software in bulk, the information technology (IT) revolution has fundamentally transformed the way businesses operate. Organizations are flatter, which speeds up decision making, while supply-chain management, customer service, human resource departments, and virtually any other cross-cutting function of the modern corporation operate in ways that corporations in the pre-PC wouldn’t even recognize.
To be sure, the IT industry has some large firms that in their niches have had either large market shares or market power (the ability to influence price as firms in competitive industries cannot), but even the firms in these industries cannot capture all of the gains from their innovations. In the IT sector, this is probably an understatement, since there are intangible ease-of-use benefits that individuals or companies realize from the use of Microsoft or Apple software, or the mobile apps enabled by Apple or Google, that are not necessarily captured in the productivity statistics.
In any event, the discovery of oil and gas, as a distinct activity or industry, is certainly a more competitive activity than the niche IT sectors where some firms still have dominant positions. Many oil companies either have the ability in-house, or can hire it from drillers, to deploy directional drilling and fracking technology to find oil and gas in tight formations, which makes the production end of the oil and gas industry highly competitive. One widely cited study, by IHS Global Insight, a leading economic and energy consulting firm led by, among others, the energy expert Daniel Yergin, confirmed this by calculating in 2013 the total benefits of fracking shale oil and gas. The study’s verdict: In 2012 alone the energy revolution had saved the average household $1,200 per year, which is equivalent to increasing take-home pay by the same amount, or about 2.5 percent.10 For the whole economy, McKinsey has estimated that the oil/shale gas boom will have increased GDP economy-wide by $380 to $690 billion by 2020, representing a 2 to 3.7 percent increase in annual GDP by that year (an estimate in line with IHS estimate for 2012).11
These are enormous figures, since annual productivity growth has been in the neighborhood of 2 percent per year over the last decade, which makes the gains from the shale oil/gas boom alone as much as one to two years’ worth of economy-wide productivity growth.
Industries and sectors of the economy vary considerably by how much energy they use. But clearly, those industries that are more energy intensive, and especially those using natural gas or petroleum products as feedstock for other products, such as the chemicals industry, have especially benefited from the energy revolution.
Lower costs are analogous to platform effects, since they reduce costs and encourage investment by users of innovations elsewhere in the economy. The shale and oil gas revolution also created important geographically based platform effects. North Dakota, one of the most sparsely populated states in the United States, has experienced rapid growth in both population and incomes because of the discovery of oil in tight shale formations. To attract men willing to work in the harsh climate, oil-drilling companies are paying high wages coupled with bare-bones residential quarters. The men use some of the money to buy gas, food, and other items locally, while sending much of the rest home to their families. This multiplier effect may be less innovative than the new apps that software developers create for new operating systems for computers and mobile devices, but the economic effects are similar. The initial innovation—in this case, new drilling technologies—becomes the equivalent of a platform on which other industries and firms either expand or are created.
In addition, eastern states in the Marcellus shale gas formation also have experienced benefits, or really revivals, in many areas that were headed on their way down: portions of Ohio, West Virginia, and Pennsylvania in particular. New York is also in this gas region, but its policy makers have limited drilling out of fear of the environmental impacts of fracking on underground water supplies and the release of methane that sometimes accompanies the fracking process. At this writing, the Environmental Protection Agency has also been looking at the water contamination issue, and there is the possibility that the federal government will issue minimum standards drillers must meet to ensure underground water quality in states where fracking is taking place. Research about the methane problem, at least at wells operated by larger companies, suggests that it has been overstated.12 Few energy experts believe that any new environmental requirements the federal government may set will significantly slow the fracking boom.
In sum, free markets, encouraged by economists, have had much wider benefits than those limited to looking for oil and gas. We can thank a number of economists in the 1960s and 1970s for that, as well as those of recent vintage who stood ready to counsel against any attempts to impose the oil and gas controls of an earlier era in the 1990s and 2000s, when prices soared (but since have come down).
Accordingly, when you think about the U.S. oil and gas boom remember the economists. Of course, there were powerful political forces pulling with them, unlike airline and trucking deregulation. But the intellectual case helped.