A Real, Not Faux, Transparency Proposal for Regulatory Science

Originally published on The Regulatory Review. Reprinted with permission.

In a previous essay, we critiqued the U.S. Environmental Protection Agency’s (EPA) recently proposed transparency rule, arguing that the proposal conflicts with best scientific practices and would further erode the EPA’s ability to do its job. According to supporters, the central goal of the proposed rule is to increase the transparency of regulatory science. Unfortunately, the proposal does not begin to deliver. No matter how many times the word “transparency” is repeated to characterize the proposal, its effects would reverse progress. It also gives appointees like former EPA Administrator Scott Pruitt and his successors unrestricted and unreviewable authority to reach politically motivated decisions that exclude high quality research.

Of all the problems that plague EPA today, ensuring scientific transparency is not all that difficult. A real transparency proposal, as opposed to the Pruitt EPA’s faux proposal, would closely track the standards developed by the scientific community and include the following five features.

1. Rebuttable Presumption. A commitment to transparency must not compromise the agency’s ability to use the very best information possible.

The faux transparency proposal would exclude research — regardless of quality — if the “dose response data and models” are not “publicly available in a manner sufficient for independent validation.” The proposed rule does contain a section that lists several discretionary exceptions. But the best reading of the rule reveals that these exceptions are limited to legal considerations; they arise only when the Administrator determines that the disclosure of data is infeasible because of “privacy, confidentiality, confidential business information, and…national and homeland security.” Of course, the preservation of study subjects’ privacy and confidentiality is vital, and the law already protects confidential business information. But the faux proposal incorporates a flawed procedure for making such decisions that could be subject to abuse because it confers exclusive discretion on the EPA Administrator to grant or deny the exceptions.

In contrast to the faux proposal, a real transparency proposal would create a rebuttable presumption that more closely tracks existing, established scientific practices.

First, such a presumption would allow for scientific justifications, not just legal ones, to make maximal use of the best available evidence. The decision whether to include a study in an analysis or model is case-specific: A study’s inclusion depends on the quality and rigor of the study under question in relation to the other evidence available to inform the policy question. Under a real transparency proposal, then, when research does not meet transparency requirements, it would be excluded unless a proponent of the research offers arguments that overcome the presumption. For example, if a scientific expert involved in the regulatory process believes that decision-making will be improved by including the research despite the fact that the research does not satisfy the transparency criteria, he or she would have the opportunity to provide an explanation for why the research should not be presumptively excluded. Once a reasonable set of mandatory disclosures about scientific research is established, these exceptions for research that does not comply should become relatively rare. We make a start at proposing a set of disclosures below, although we stress that consultation with scientific experts is essential before any such standard is established.

Second, technical experts would judge the sufficiency of each rebuttal, as opposed to a political administrator as is the case under the current EPA’s faux proposal. For example, an EPA-staffed internal office could be assigned the job of assessing rebuttals, with a standing Federal Advisory Committee Act (FACA) committee presiding over any appeals or disagreements. The FACA panel would need to consist of experts in the relevant disciplines who are able to critically evaluate the arguments. Ideally, then, the panel would include several scientific journal editors as well as experts in trade secret and privacy law. Membership on the panel should, of course, follow the time-tested conflict of interest and related policies used by EPA prior to 2017, rather than Pruitt’s recent directive barring scientists with EPA grants from serving on science advisory boards. These reviewing bodies presumably would also enjoy special access to confidential information through nondisclosure agreements.

2. Broad Conception of Scientific Transparency. In contrast to the crabbed conception of transparency as applying only to “data” in the current, faux proposal, a real proposal embraces the conceptions of transparency developed by the premier scientific journals. Transparent science, accordingly, includes at least the following principles:

• a conflict of interest disclosure statement if the study was privately sponsored, as well as the underlying contract governing that research, in order to ensure that researchers’ independence to determine study design and report results was preserved;
• a clear statement of the methods for data collection and analysis used in the study to allow for scrutiny and even replication of the study; and
• all of the underlying data, presumably in digital form (e.g., not original specimens, etc.).

3. Prospective Application. In contrast to the faux proposal, real transparency requirements would not apply retrospectively to research already completed. Researchers who have already published or completed covered studies may fail to satisfy the requirements in the faux proposal through no fault of their own, and they should be given fair notice of new requirements.

Indeed, retrospective application is one of the characteristics of the EPA proposal that has convinced many observers that the motives of Pruitt and other political officials, including U.S. Representative Lamar Smith (R-Texas), were less concerned about scientific integrity and more concerned about knocking out — or, at the very least, engendering a big fight over — past studies that they and their constituencies do not like. Their targets include the Harvard Six Cities Study mentioned in our earlier essay.

4. Coverage of All Research. Unpublished research submitted by industries in support of a license or permit is at least as susceptible to bias and error as prominent academic research published in the top journals. Yet the faux transparency proposal — which applies only to an ill-defined set of “pivotal” research — would likely only reach the latter and leave most industry research untouched.

A real commitment to transparency should apply the same standards to all scientific research and analyses used by the agency, particularly research that is not published and that has escaped rigorous peer review.

5. Disclosure of Exclusions. When information is excluded from consideration in the agency’s analysis, the public should be able to learn about those exclusions. A real transparency proposal would require that a list of all excluded research be shared with the public as the decisions are made. Such disclosure could be accomplished by listing excluded — or presumptively excluded — information on a dedicated website in the course of a rulemaking or agency decision. The faux proposal skips this vital step. Under the faux approach, the EPA need only identify the research it uses; the EPA’s equally important decision about which research it excludes can be completely hidden from view.

A real transparency proposal, moreover, would be applied to all technical analyses prepared by the agency. For example, former Administrator Pruitt’s recent proposal to change the way costs and benefits are evaluated would benefit greatly from a rebuttable presumption that all underlying data and analytical methodologies be disclosed.

Of course, before publishing a real proposal — to enhance transparency or anything else for that matter — the agency should document the problem it is attempting to solve. And the agency should make sure that whatever resources are required to implement its new reform are available and justified in light of the nature of the problem. The faux proposal skips these two critical steps as well.

In an ideal world, regulatory analysts and participants would be debating real transparency proposals in a genuine effort to improve regulatory processes. Instead, most of the current debate focuses on opposing faux “Trojan Horse” proposals that pretend to advance the public interest and scientific integrity, while surreptitiously promoting the self-interest of well-funded constituencies. If people of good faith wish to pursue the issue of transparency in regulatory science honestly, we have suggested a place to start.