The dissemination of science follows the conventional route of rigorous peer-review followed by publication in an accredited scientific journal. This process has been the standard foundation from which the general public can trust that the science is, at the very least, valid and honest. Of course this system is not without its flaws. Scientific papers of questionable authority, dishonest methodology, or simply flawed design frequently make it through the gates of peer-review. Politically charged papers possess strong biases and many high impact journals favor sexy or controversial topics.
Beyond the conventional route of peer-review, there exist a vast accumulation of gray literature – conference reports, technical notes, institutional papers, various articles written for specific entities that enter into general circulation without the filter of peer-review. Much of gray literature is valid, robust science, but much of it is not. The challenge is that sometimes gray literature is the only science available.
Especially in fisheries, where policy and management must be made on the temporal scale of a fishing season, gray literature is heavily relied upon. Peer-review can take upwards of a year from submission to publication, at which point stock assessments and management proposals may already be out of date. for conservation issues that are time-sensitive, gray literature may be the only option. Even scientific papers in certain marine fisheries rely heavily on gray literature. Some journals have been found to cite up to 94% gray literature.
A perfect example of this recently came across my inbox. In A preliminary investigation of smooth dogfish (Mustelus canis) at-sea processing techniques, a policy-driven report funded by the Pew Charitable Trust, the authors attempt to answer a practical question – “Does smooth dogfish meat spoil quickly if not processed immediately after landing?” Though seemingly mundane, this question has major implications for shark conservation. Many fishermen believe that, post-mortem, smooth dogfish meat spoils more rapidly than other shark meat, and that the catch must be bled and finned before nitrogenous waste builds up in the tissue and ruins the catch. Smooth dogfish are finned at sea, before the boat reaches port. Because a national ban on all finning would allegedly prevent smooth dogfish fishermen from processing their catch before it spoils, the Shark Conservation Act is currently pigeonholed in the Senate. As it stands, this one piece of gray literature is the only report that counters political claims that a ban on finning will hurt smooth dogfish fishermen.
So what are scientists and policymakers to do? Shark conservation is a pressing issue and every month counts. Without a national ban on shark finning, the United States lacks the political authority to influence other nations to adopt a fin ban. On the flip side, if a national fin ban really does negatively impact one of the few shark fisheries that might actually have a chance of becoming sustainable, then the law does not accurately reflect the current state of conservation. Waiting for solid peer-review before any action is taken may result in still more shark population collapses as the remaining fin fisheries continue, unopposed. Time is of the essence and the pace of peer-review is such that many fisheries papers emerge only as epitaphs. On the flip side, making policy decisions based on preliminary, incomplete data is equally rash and could result in a complete reversal of the ban as more data becomes available, compromising a national ban.
When gray literature is the only option, it cannot be ignored. Although limited, this one report does begin to point us towards an answer. It is absolutely necessary to recognize that this is a policy-driven piece with a specific goal and specific biases. While formal peer-review may not be an option, the vast network of ocean science bloggers provides one of several opportunities for vigorous, public discussion among scientist from relevant fields and the chance for interested parties to engage and ask questions.
This is not to say that gray literature should be elevated to the same status as peer-review, but rather that it should not be completely ignored. The precautionary principle dictates that if if an action has a suspected risk of causing harm to the public or to the environment, in the absence of scientific consensus that the action or policy is harmful, the burden of proof that it is not harmful falls on those taking the action. Here we see the first indication that finning dogfish at sea is not necessary, shifting the burden of proof on those claiming that finning dogfish at sea is necessary.
But of course, the final word can never be gray literature. This report is a preliminary study, designed to launch a research project to determine through rigorous, controlled observation and experimentation. But, in the absence of evidence to the contrary, and provided there are not serious questions about the quality of the gray literature, precaution should be taken. Any initial judgment based on gray literature must be made with the caveat that policy will be re-examined post-peer-review.
When time is of the essence and peer-review is not available, gray literature cannot be ignored. But when gray literature is the only option for informing management decisions it must be subject to rigorous and public scrutiny, and any policy that emerges from it must be subject to the caveat that it will be re-addressed as more data becomes available.
~Southern Fried Scientist
UPDATE: David weighs in with some background on urea and osmoregulation in sharks.
UPDATE 2: See here for Chuck’s analysis of the gray paper at Ya Like Dags.
Flor Lacanilao (1997). Continuing problems with gray literature Environmental Biology of Fishes (49), 1-5