Font size:
The Journal of the American Medical Association (JAMA) devoted a recent issue to the study of “peer review,” an accepted safeguard of scientific integrity. Peer review is an evaluation system upon which careers depend, and by which billions of research dollars are allocated. But it is also a practice prone to error, sabotage, censorship and bias, according to the latest studies. These findings argue for a more open marketplace of research ideas not controlled by those who control peer review processes.
Peer review is employed when research papers are submitted for publication in professional journals like JAMA, which function as powerful and exclusive sentinels of science scholarship. Journal editors commission experts in the appropriate field to scrutinize submissions for methodological and analytical flaws. Results of the review drive publication decisions.
Despite 200 years of use, peer review itself has largely escaped the very scrutiny the practice is supposed to engender among researchers. Until the late 1980s, in fact, few empirical studies of the merits and shortcomings of peer review had been undertaken. Responding to the void, JAMA editors have sponsored an international “Congress” every four years since 1989 to investigate the credibility of the peer review process. The June edition of JAMA features the latest results.
Inasmuch as empirical research entails testing ideas, peer review essentially governs scientific progress. The results of a review can sanctify or doom a particular course of research. And when science intersects with public policy-as is increasingly the case-peer review (or the lack of it, or the type of it) can be manipulated as a political device. Obviously, then, the credibility of peer review carries serious ramifications.
In surveying the latest research, however, JAMA editors concluded that the time-honored practice does not make perfect. As stated in the Journal’s June editorial: “Once again, in this issue of The Journal, we publish studies that fail to show any dramatic effect, let alone improvement, brought about by editorial peer review.… Indeed, if the entire peer-review system did not exist but were now to be proposed as a new invention, it would be hard to convince editors looking at the evidence to go through the trouble and expense.”
Indeed, there is no shortage of instances in which peer reviewers have overlooked major errors, and hindered or even sabotaged scientific breakthroughs. And while a great many reviewers remain scrupulously honest and well-intentioned, it is not difficult to understand how competing incentives can undermine the peer review process.
The federal budget allocates some $20 billion annually for basic research (and an additional $42 billion for non-military applied R&D). Competition for funding is fierce. And so specialized has modern science become that qualified reviewers are likely to compete directly for funding and distinction with the very authors they are called upon to judge. While competition proves useful in producing results, such rivalry, with billions of dollars at stake, also may provoke plagiarism or other fraud. Especially susceptible is the biomedical arena, in which potentially lucrative patents are in play.
Academic journals are no less impervious to economic pressures. As reported by Wager and Jefferson, 85 percent of the submissions rejected for publication in the New England Journal of Medicine and the Journal of Clinical Investigation were published elsewhere-many without revisions. Meanwhile, only 23 percent of the press releases touting journal contents included any mention of research limitations, according to Steven Woloshin and Lisa Schwartz.
Exacerbating matters is the custom of withholding a reviewer’s identity from authors as well as the reading public. Envisioned as a way to ease reviewer inhibitions, the practice instead diminishes accountability.
Current peer review practice also may punish truly novel work. Scientists whose theories challenge the status quo are a threat to those whose careers are rooted in the paradigm of the day. New ideas can jeopardize special interest groups and the funding they receive to pursue traditional approaches. Peer review thus becomes a means by which to censor rivals. Meanwhile, potentially valuable lines of inquiry go ignored by those who fear unconventional research won’t fly with the establishment. But as Wager and Jefferson point out, “Truly imaginative scientists have no peers.”
The “soft” sciences, being more closely aligned with political and social activism, are particularly prone to peer review manipulation. Research that defies conventional wisdom often is criticized for bypassing the peer review process. In actuality, as the JAMA research underscores, a fair reading of divergent work would be difficult to obtain from those wedded to conventional wisdom.
As much as we might wish to regard scientists as paragons of virtue they are hardly immune to the very same biases present in most every other profession. Studies have revealed that reviewers are influenced - unwittingly or otherwise - by an author’s racial or ethnic characteristics as well as affiliations and even mother tongue.
It is likely no accident that peer review has come under investigation just as technology offers new alternatives for the information exchange at the heart of science. The Internet now provides the means for a much broader and far more dynamic market of ideas, serving mass audiences in real time. As noted by peer review researcher Maciej Henneberg, “Throughout history, most scientists published their views without formal review and peers published their criticisms openly.”
With new communications tools at our disposal, the scientific community and public at-large need not settle for the insularity of the conventional peer review process. We must allow truth to emerge with or without the approval of those in power.
###Diane S. Katz is director of science, environment and technology policy for The Mackinac Center for Public Policy, a Midland, Michigan-based research and education institute.
The Mackinac Center for Public Policy is a nonprofit research and educational institute that advances the principles of free markets and limited government. Through our research and education programs, we challenge government overreach and advocate for a free-market approach to public policy that frees people to realize their potential and dreams.
Please consider contributing to our work to advance a freer and more prosperous state.
