CPR Perspective: Cost-Effectiveness Analysis, Risk-Risk Analysis, and Other 'Cost-Benefit Cousins'

A half century ago, political scientist Charles Lindblom famously described the setting and implementationof policy as an incremental and basically pragmatic process of “muddling through.” In contrast, academics and regulatory overseers at the Office of Management and Budget often favor policymaking approaches that aspire to achieve comprehensive rationality. Such synoptic approaches typically fit available empirical data regarding anticipated policy consequences into a formal decision rubric that generates an “optimal” outcome. The most well-known and controversial of such synoptic decisionmaking paradigms is economic cost-benefit analysis, which seeks to translate all expected consequences of policies into monetary equivalent values so that society can decide whether environmental, health, and safety regulations are “worth” their costs. Closely allied with cost-benefit analysis are a cluster of decisionmaking techniques that also seek to assess policies through formal quantitative assessment. They go by such names as “cost-effectiveness analysis,” “risk-risk analysis,” “risk tradeoff analysis,” and “health-health analysis.” At their heart is the idea that tradeoffs are inevitable in environmental, health, and safety planning and that society ought to take account of such tradeoffs directly, in order to ensure that regulations deploy resources in a socially valuable manner.

This is a perfectly sensible sounding proposition. It is also one that, on close inspection, turns out to have more than a few devils lurking in its details.

This Perspective seeks to provide guidance on the role of “cost-benefit cousins” in debates over regulatory reform. Like cost-benefit analysis, these techniques suffer from significant methodological limitations. More importantly, they rest on an inherently conservative idea: Tradeoffs are inevitable, as economists are wont to remind us, but there is nothing inevitable about the manner in which knowledge is organized to create the perception of a need for tradeoffs. When faced with the mantra that tradeoffs are inevitable, the progressive political instinct should always be to reevaluate the underlying empirical and normative assumptions that are driving the appearance of inconsistent goals. With appropriate imagination, the politics of stasis can be replaced by a politics of possibility.

“Cost-effectiveness analysis” as used in this Perspective should be distinguished from the familiar idea that actors should seek to achieve a given goal in the most cost-effective manner possible. For instance, traditional pollution control techniques - such as requirements that firms limit pollution to the level of reduction achieved by the best available control technology - promote cost-effective compliance because firms are typically given flexibility to determine the method that is the cheapest for achieving the mandated pollution reduction level. This traditional approach is not only pragmatic in Lindblom’s sense, it also represents a clear demarcation of authority: Congress determines the overall social goal of pollution minimization by mandating adoption of the best available technology; agencies become responsible for the largely technical judgment of identifying the performance standard attainable by the best available technology; and individual firms are left to marshal their own specific knowledge and innovative potential in order to achieve the mandatory performance standard in the cheapest manner possible. This understanding of cost-effectiveness is not all that controversial.

More recently, however, “cost-effectiveness analysis” has come to stand for a particular type of regulatory review, associated with what are known as “league tables.” Such tables seek to rank and evaluate environmental, health, and safety regulations according to their overall cost-effectiveness, where cost-effectiveness is defined as the dollar cost of the regulation per life saved or per life-year saved by the regulation. The goal of such tables is nominally to help policymakers and their publics better understand the “return on investment” provided by various regulations. A perfectly rational society would devote its scarce resources to regulatory opportunities that provide the biggest lifesaving “bang for the buck.” Armed with the empirical evidence seemingly provided by regulatory league tables, commentators frequently argue that a reordering of public risk regulation priorities could result in the same overall number of lives saved for dramatically less cost or, alternatively, dramatically more lives saved for the same cost. Some commentators go further to argue that the league tables could be used to implement a cost per life saved ceiling. This is simply another name for full-blown cost-benefit analysis, since the same monetary value of life used to conduct cost-benefit analysis would be used to establish the cost per life saved ceiling within the league tables. All of the complications and objections that plague cost-benefit analysis also would plague this use of regulatory league tables.

Some commentators would go further still: They would analyze government regulations not simply according to their cost-effectiveness at preventing losses of life compared to other regulations or to the standard monetary value of life used by agencies when conducting cost-benefit analysis, but also compared to private activities that seem to have the effect of preserving life. Relying on what is known as the “health-health” literature, thesecommentators claim to have identified a dollar amount of government expenditure that necessarily entails the statistical loss of a life, by virtue of its displacement of private expenditure. Proponents of health-health analysis cite evidence finding a significant correlation between income and health, particularly as measured by longevity. In light of this correlation, supporters argue that any regulation will entail at least some adverse health consequences, so long as compliance with the regulation requires a reduction in private income. In other words, since wealthier appears to be healthier, any government regulation that reduces wealth must therefore reduce health. Or so the argument would go if correlation always implies causation. In truth, the argument linking regulatory expenditures with reduced health depends on little more than faith in a series of counterfactual assumptions. Other empirical work suggests that causality might run in the other direction – i.e., that healthier is wealthier – or that both wealth and health might be importantly determined by other variables, including the very kinds of positive life conditions that regulation often seeks to provide.

Health-health analysis is related to “risk-risk analysis,” which seeks to evaluate policies with the same synoptic lens as cost-benefit analysis, but without necessarily translating all policy impacts into dollar values. Starting with the premise that risks often are created as well as reduced by regulation, analysts seek to identify the overall profile of “risk tradeoffs” posed by environmental, health, and safety regulation. Risk tradeoffs take a variety of forms. Some safety regulations are said to cause unintended consequences that may, on net, increase risk: Mandatory seat belt laws lead individuals to drive faster, childproof safety caps lull parents into being less diligent about hiding medicines, operation of air pollution control technology increases greenhouse gas emissions associated with electricity production. Similarly, some regulations cause harm by delaying the adoption of a product or technology that might be beneficial to human health or the environment: Premarket testing deprives patients of access to potentially efficacious drugs, restrictions on genetically modified crops might cause more rainforest to be converted to agricultural land.

To the extent that such risk tradeoffs are empirically robust, they should be attended to. They should not, however, be used as an all-purpose deterrent to government regulation. Nor, as Richard Revesz and Michael Livermore have stressed in their important book, Retaking Rationality: How Cost-Benefit Analysis Can Better Protect the Environment and Our Health, should it be forgotten that regulations often create ancillary or unintended benefits in addition to risks. To focus only on one side of the secondary impacts of regulation is to stack the deck inappropriately against government action. Reducing greenhouse gas emissions, for instance, would also create enormous health and safety benefits through the ancillary reduction of other air pollutants.

The failure to attend to ancillary benefits of regulation is symptomatic of the broader problem posed by league tables and other cost-benefit cousins. By seeking to distill what can be known and understood about the impact of regulations to simple tables or calculations, analysts inevitably misstate and distort. This lesson was demonstrated by former CPR Member Scholar Lisa Heinzerling in a landmark critique of league tables, Regulatory Costs of Mythic Proportions. As Heinzerling demonstrated, the leading league tables used in regulatory reform debates had been constructed in ways that were downright manipulative: Regulations were included that had never been adopted, authors adjusted scientific data in ways that were ad hoc and undefended, discount rates were applied to future values without adequate explanation, and so on.

Despite this devastating critique, the influence of league tables has not waned. One explanation for this persistence is that the promise of comprehensive rationality is deeply appealing. Scientific knowledge is frustratingly incomplete, direct value conflicts are uncomfortable, and political compromise is unseemly – thus, the prospect of rising above the mud through formal, constrained decision analysis seems attractive when compared to yet more “muddling through.” But when this prospect is repeatedly shown to be illusory – when critics show again and again that the mud seeps through the cracks of decision analysis – then alternative explanations for the persistence should be examined.

Perhaps the most important such alternative sees cost-benefit analysis and its cousins not merely as abstract decisionmaking techniques, but as techniques that are promoted and deployed in a specific political institutional context. In the United States, formal regulatory analysis of any stripe cannot be separated from the history of presidential power that such analysis has helped to consolidate. No matter which party occupies the White House, formal regulatory analysis of agency action helps to ensure that the President maintains a strong grip over the goals and accomplishments of the Environmental Protection Agency, the Occupational Health and Safety Administration, and other agencies charged with protecting public health and the environment. In fact, the rationale behind league tables – that they help enable allocation of scarce public resources to the most cost-effective lifesaving opportunities available – seems to presume a unitary executive capable of shifting budgets and targeting investments across the full range of ways in which government can seek to enhance well-being. Agencies cannot range in this way, nor would we necessarily want them to: Agencies are limited entities created and tasked by Congress to accomplish specific goals. Often, the impetus behind league tables and related regulatory reform efforts seems to be a desire to cut Congress out of the national conversation on how best to achieve a just, sustainable, and prosperous society.

No one would suggest that agencies should ignore relevant empirical information regarding the impacts of proposed policies. Reluctance to endorse formalized regulatory impact analysis is not driven by an aversion to data, but by a belief that important moral and political issues often are buried in the construction of formal analytical methods. Within cost-benefit analysis, the choice of willingness-to-pay as a basic value metric serves to powerfully reinforce the status quo distribution of wealth and resources in society; similarly, the decision to discount future values at some nontrivial rate indirectly resolves questions of intergenerational justice in a way that, when surfaced for unprejudiced consideration, is at best dubious. Within cost-effectiveness analysis and related cost-benefit cousins, the important conservative conceptual move is to present policy choices as if they only involve optimizing scarce resources in the face of inevitable tradeoffs. Dig beneath these stylized decision frames, however, and one will find a host of vital questions: What outcomes should be used to determine comparative regulatory effectiveness? Should lives saved be counted equally or discounted by the age of the individual saved? If years of life saved becomes the important metric, should the “value” of those years be further adjusted to reflect their “quality”?

One might think that the very project of life is learning how to live it well, that is, with quality. But in fact a highly technical literature exists within health economics devoted to developing “quality-adjusted life year” metrics in order to enable comparison of competing treatment priorities. Such metrics are another important cost-benefit cousin attracting notice by those who wish to discipline and rationalize agency decisionmaking through regulatory impact analysis. Quality-adjusted life years could provide an even finer grained metric to evaluate the relative cost-effectiveness of environmental, health, and safety regulations than lives or life years. Whether they could do so in a way that is sensitive to the nuances of particular contexts and that continues to utilize environmental, health, and safety law to express social solidarity is an entirely different question.

The dilemma seems to be that no formal method of regulatory analysis has been presented that successfully avoids the moral and political questions raised by environmental, health, and safety law. To the extent that such questions become implicitly resolved by the regulatory impact analysis method – often in direct contravention of the expressed wishes of Congress – our national conversation suffers. Moreover, agencies tasked with implementing those results of our national conversation that have been codified into environmental, health, and safety law already face a daunting array of resource constraints and other barriers to effective action (See CPR Perspective on Regulatory Underkill). To add further layers of analysis and paperwork requirements might only make sense after the lion’s share of important environmental, health, and safety goals have been accomplished, at which point the finely-tuned regulatory “investment” strategy promoted by reformers would no longer impose such high opportunity costs on agency time and budget. Unfortunately, we have not yet reached that luxurious moment.

• Lisa Heinzerling, Regulatory Costs of Mythic Proportions, 107 Yale L. J. 1981, 2042-68 (1998).  (Questia Excerpt)
• Douglas A. Kysar, It Might Have Been: Risk, Precaution, and Opportunity Costs, 22 J. Land Use & Envtl. L. 1 (2006).  (SSRN Abstract)
• Richard W. Parker, Grading the Government, 70 U. Chi. L. Rev. 1345 (2003).  (SSRN Abstract)
• Richard L. Revesz & Michael A. Livermore, Retaking Rationality: How Cost-Benefit Analysis Can Better Protect the Environment and Our Health (2008).  (SSRN Abstract)