Peer Review Reviewed

As long as I have managed to log onto blogger I thought I ought to throw in a link to this article about the flaws of peer review. http://www.the-scientist.com/2006/2/1/26/1/. I am, as you know, skeptical of the quality of work that passes as scientific these days and I find that this article fits comfortingly into my way looking at these things.

I really do think that the system of peer review is seriously flawed. I think one of the central ways in which it is flawed is a result of the efforts that universities have made to quantify or make measurable their 'productivity'. The things that universities are intended to produce, like useful knowledge an educated populace and a thriving culture are very easy to observe subjectively but very difficult to observe 'scientifically'. The way that many universities have attempted to solve this problem is by making peer reviewed articles the ultimate measure of accomplishment. The result of this focus is that academic workers are forced to literally spam the journals with endless waves of content in order to attempt to drive up their stats. This flood of papers, most of them of little interest or significance, breaks down the whole system. People do not have the time to read the articles being published because there are so many. Reviewers cannot be overly critical lest their own careers be jeopardized by overly stringent reviewers (a sort of grade inflation one might say). There are so many papers to be reviewed and the people who are supposed to be reviewing them are so busy trying rattle off papers themselves that it is not at all surprising that so much junk flows through the system. Nor is it surprising that it is so difficult for those with truly worthy ideas to be heard amongst all the noise (the whole point of journals in the first place).

What is perhaps more troubling is how this system reflects on the university system as a whole. Just examining the way the journals ideally function raises some serious questions. Researchers, funded by the public, produce papers, supposedly in the interest of public knowledge, which are then reviewed by other researchers, also funded by the public, so that they can then become the property of journals that then charge the public institutions at which these researchers work for access to the very papers they paid to create. Astonishingly enough, it is the very universities, who appear to be exploited by this process, that lend the journals the legitimacy they then turn around and pay for. What particularly bothers me is that, not only are the universities paying journals for doing virtually nothing but they are compromising their fundamental purpose by allowing the products of their researchers to be locked away from the eyes of the public whom they purport to serve.

In the interest of brevity I will close my comments here by praising the people at MIT for their success in recognizing what their institution's fundamental purpose is. The founders of our universities, whose goal it was to promote the growth and dissemination of knowledge and culture, would have embraced the internet as the ideal tool for the job. If the modern leaders of our universities paid anything more than lip service to these ideals they would provide every bit of information that they possibly could to the public through the internet. This is what MIT is doing and I cannot praise them highly enough for their efforts.

Appropriately, I think, I will sign off with a link to a lecture about this very subject on the MITWorld Distributed Intelligence server (I came across it (it is one among several that I have listened to there on related topics) while searching for a different lecture that I thought was particularly relevant to my topic but which I have thus far failed to locate).

http://mitworld.mit.edu/video/72/