No, the GAO Didn't Say EPA's Cost-Benefit Analyses are Bad—But Here's What We Should Take Away from Their Report

If you’re an antiregulatory, anti-environment member of Congress, such as Sen. David Vitter (R-LA) or Darrell Issa (R-CA), how do you get the Government Accountability Office (GAO) to issue a report that criticizes the cost-benefit analyses that the Environmental Protection Agency (EPA) has performed on some of its recent rules?  That’s easy—you simply ask for one.  Then, when the GAO issues the report, like it did a few weeks back, you can begin issuing press releases filled with invective and righteous indignation.  The report’s findings, you can assert, are smoking-gun evidence that the EPA is running amok, issuing burdensome rules that are harming small businesses and families.  And just like that, you’ve conjured the latest antiregulatory, anti-EPA scandal du jour out of thin air.

Vitter and Issa have followed this playbook to a T and will no doubt continue trying to spin political gold out of this meaningless hay as part of the Republican’s broader strategy of using antiregulatory rhetoric to undermine the work of the Obama Administration while simultaneously boosting their electoral prospects in the fast approaching mid-term elections.  “Rather than using a fair and open rulemaking process, EPA pushed through regulations using sloppy analysis without sufficiently informing Congress or the public of the economic impact,” Issa predictably huffed following the report’s release.

Even for a manufactured controversy, though, this one is a complete Nothing-Burger.   With a side of Yawn-Fries. Washed down with a “Who Cares”-Milkshake.   Vitter and Issa ordered the GAO to review the EPA’s recent cost-benefit analyses and identify faults.  Because the GAO must do what members of Congress tell them to do, the GAO attempted to comply as best as they could.  What they came back with were some of the most picayune nitpicks that were ever nitpicked.  For example, the GAO found that the Executive Summary for many of the EPA’s cost-benefit analyses could be improved if the agency included such things as clearer statements of the problem the regulation will solve or a summary of the analyses’ results.  (In nearly all cases, this information was available in the body of the cost-benefit analysis or in the rule’s preamble.)  Note that this criticism relates to the analysis’s style, rather than its substance, and in no way calls into question the results of the analysis or, indeed, the quality of the underlying rule.

When the GAO report did pass judgment on the substance of the EPA’s cost-benefit analyses, the criticisms were meek at best.  For example, the GAO noted that the EPA did not always perform full quantitative analyses of the alternative policy options that the agency considered.  Circular A-4, a White House Office of Management Budget (OMB) guidance document that outlines best practices for conducting cost-benefit analyses, does recommend that agencies perform such analyses on their larger rules but, as the GAO report noted, also leaves it up to agencies to exercise their judgment whether to do so in light of practical considerations, such as limited resources and data.

Similarly, the GAO also observed that the EPA did not always monetize key benefits for their rules.  In addition, the GAO was concerned that the EPA was relying on old studies to assess the employment impacts of its rules.  Again, as the GAO report recognized, Circular A-4 anticipates that agencies cannot monetize all benefits due to lack of data and resources.  As for the employment impact analyses, Circular A-4 provides no guidance for how this task is to be performed at all.  Tellingly, the GAO could provide no specific advice for how the EPA could improve its employment impact analyses, since it recognized that the studies the agency was using were the best that are currently available and that the EPA is currently working to develop new tools to inform these analyses.

When understood in context, the GAO criticisms of the EPA are revealed to be quite modest—if indeed they can be framed as criticisms at all.  In conducting its review, the GAO recognized that the EPA faces real barriers in how the agency performed its cost-benefit analyses and that these barriers are far beyond the agency’s control.  The GAO also acknowledged that cost-benefit analysis is far from an exact science and that Circular A-4 directs agencies to exercise their judgment in the amount of detail or thoroughness they achieve in their analyses given the practical resource and data constraints they face.  If anything, the GAO’s recommendations to the EPA can best be read as parroting Circular A-4’s advice to agencies that they should seek to achieve a proper balance between these competing demands of thoroughness and practical constraints when conducting their analyses.  As the GAO found no evidence that the EPA isn’t already working to achieve this proper balance, it’s hard to find much in the way of a strong critique of the agency’s performance in conducting cost-benefit analyses.

 Just because the GAO report didn’t find what Vitter and Issa said it did doesn’t mean that it offers no useful information, however.  Based on my reading, the report imparts two important lessons.

First, it clearly illustrates the dangers that would result from efforts by conservatives to enact legislation, such as the Regulatory Accountability Act or the House Unfunded Mandates Information and Transparency Act, that would make cost-benefit analysis a judicially reviewable legal requirement for agency rules.  After all, even when agencies do a pretty good job on these analyses, it’s still possible to find problems with them.  That’s because, as noted above, cost-benefit analysis is hardly an exact science—despite what its proponents claim.  One of the biggest impediments to performing a cost-benefit analysis is that the necessary data are not always available to make meaningful conclusions about a rule’s potential impacts.  Sometimes, it’s because the data cannot possibly exist, such as a coherent monetary “value” of saving a human life.  In some cases, the data are prohibitively expensive to obtain, likely wouldn’t impact the analyses’ overall results, or both.  In other words, performing cost-benefit analysis—and especially what is included and what is left out—requires the exercise of judgment on the part of agencies, as both Circular A-4 and the GAO report acknowledge.  If cost-benefit analysis was a judicially reviewable legal requirement, as conservatives are pushing for, then businesses that don’t like the EPA’s regulations could challenge them by attacking how the agency exercised its judgment in performing the underlying cost-benefit analysis.

At best, legal challenges to the EPA’s rules would descend into irrelevant and unhelpful squabbles over the minutiae of the cost-benefit analysis, while more important issues—such as whether or not the rule is adequately protecting people and the environment—would get ignored.  At worst, these legal challenges would provide activist conservative judges with virtual carte blanche to strike down rules they disagree with.

Second, the GAO report’s conclusions unwittingly highlight the essential indeterminacy of cost-benefit analysis—and its essential uselessness as an analytical tool.  Take, for example, this statement: “Without enhancements to its review process targeted at improving adherence to Circular A-4, EPA cannot ensure that its cost-benefit analyses provide the public with a clear understanding of its decision making.”  (See page 28.)  This statement suggests that improvements to the EPA’s internal management processes governing the conduct of cost-benefit analysis are a necessary (though perhaps not a sufficient) condition for helping the public to understand why its rule turned out the way it did.  This statement is demonstrably false, since such enhancements are not necessary for achieving this result, and indeed may run counter to its achievement.  In practice, cost-benefit analysis does not clarify agency decision-making; rather, it obscures it behind technical economic formulas and theories that are well beyond the ken of most citizens.  To make matters worse, this economic analysis is almost invariably irrelevant to or even prohibited by the various statutes under which the EPA’s rules are promulgated.  As such, more or this analysis or “improvements” to it will do nothing to help regular people understand why the EPA has designed its regulations in a particular way.

Or take this statement: “However, when EPA does not monetize key benefits and costs, the cost-benefit analysis may be limited in their usefulness for helping decision makers and the public understand economic trade-offs among different regulatory alternatives.”  (See pages 28-29.)  Again, this statement asserts that more monetization of costs and benefits is a necessary condition for helping people understand whether a rule does more good than harm.  And again, this statement is demonstrably false, since monetization is actually detrimental to promoting this kind of understanding.  Telling an average person that preventing a death is worth only $10.8 million or that preserving a child’s IQ point is only worth $1,100 doesn’t help them evaluate a particular regulation.  Rather, it serves only to confuse.  Understandably, they’ll want to know how you came up with those numbers, and the explanation that you provide (wage premiums, willingness-to-pay surveys) is more likely to horrify than elucidate.  More to the point, the average person will want to know whether a particular regulation represents our best efforts to protect lives and IQ points.  Monetization cannot answer that question now, just as more monetization cannot do that in the future.

I wouldn’t expect the GAO to weigh in on such a politically charged question as “should the EPA being doing cost-benefit analysis at all?”  Based on the evidence outlined in its recent report, though, the GAO could build a strong case that the answer should be a resounding “no.”

Let’s hope other policymakers are paying attention to these more important lessons of the recent GAO report, and not falling victim to Vitter’s and Issa’s misrepresentations about the report’s findings.