Reviewing Peer Review

Abstract

Peer review is widely accepted as the hallmark of scholarly, scientific publication. It helps journal editors determine whether research conclusions are justified and “new,” gives them an idea of the potential or actual significance of a work¹ and adds a “human judgment” element to the academic process while lightening their workload.² Ideally, this results in a “decision that is constructive, transparent, timely and fair,”¹ and that will enhance the final writing product.³

Journals use peer review because it serves as a quality control filter for scholarly information.⁴ More papers are churned out than can be printed, and the peer review process can weed out fraud and eliminate “bad” science, pseudoscience and harmful science, thereby upholding ethical standards.^2,4 Peer review also serves as a mechanism for improving manuscripts; it promotes originality, academic rigor and improves the critical thinking and writing skills of authors, reviewers and editors.³ It reduces bias and improves the quality of published articles.³ The peer review process bestows a collegial stamp of approval on a manuscript and bestows an aura of “quality.”⁴

However, as Dominy and Bhatt⁴ point out, there are famous papers that were published and did NOT get peer reviewed (including Watson & Crick’s 1951 letter on the structure of DNA in nature, Abdus Salam’s 1968 paper on weak and electromagnetic interactions that led to the Nobel Prize, and Alan Sokal’s 1996 hoax now known as the Sokal Affair). There are also famous papers that were published and passed peer review, but later proved to be fraudulent (including Jan Hendrik Schon’s 15 papers from 1998-2001 in Science and Nature and Igor and Grichka Bogdanov’s 1999 and 2002 theoretical physics papers “believed by many to be jargon-rich nonsense”).⁴ Perhaps most telling are the famous papers that got rejected and later turned out to be seminal works (including Krebs and Johnson’s “1937 paper on the role of citric acid on metabolism” … “rejected by Nature as being of ‘insufficient importance’ – “now known as the Krebs Cycle” and “recognized with a Nobel prize in 1953).⁴

      Peer review is clearly not infallible; but its benefits still far outweigh its flaws. This greatly depends on the quality of reviewers and on the system of peer review. Our journal utilizes a double-blinded multi-stage review system that allows reviewers to judge, and authors to respond and revise, manuscripts. This system presumes that reviewers are experts in their chosen field, and are able to provide an “unbiased opinion on the quality, timeliness and relevance of the submitted manuscript.”⁵ Reviewers have four responsibilities – to the editor and journal, to his/her specialty or subspeciality, to patients and study subjects, and to the author.⁵ 

        The responsibility to the editor and journal means reviewers are responsible for protecting the reputation of the journal, as well as the integrity of their specialty/subspecialty, their area of expertise and profession. Reviewers, therefore, should “make sure rubbish does not get published.”⁵ Reviewers are also responsible for protecting the welfare of subjects, both human and animal. Finally, reviewers should ensure a fair treatment of the authors’ manuscript, remembering that all manuscripts are the private property of authors and highly privileged communications.⁵ They should never publicly discuss the authors’ work, or steal their ideas before publication. Public discussion includes soliciting opinions on the manuscript from others, including postings on internet discussion groups. Stealing ideas can range from plagiarism to intellectual property and patent theft. Such misconduct is of a higher order, because of the authority of the reviewer. Inadvertent breach of confidentiality, while unintended, has the same consequences for the author. To further ensure confidentiality, reviewers should destroy copies of the reviewed manuscript after completing the review, to prevent such copies from falling into the wrong hands.

      Reviewers should try to make the manuscript better, providing helpful suggestions for improvement, even if the manuscript is rejected.⁵ Needless to say, they should be familiar with the journal and its requirements, including its instructions to authors, types of papers published, journal style, and standards of the journal. They should also understand the basics of the peer-review process. Such understanding can be facilitated through formal training, complemented by actual review experience.

      In general, a manuscript review consists of comments to the editor, and comments to authors. The confidential comments to the editor should include a conflict of interest disclosure of any real or potential matters that may result in a biased review.³ If in doubt, it is better to inform the editor.⁵ Confidential comments to the editor are “not forwarded to the authors, and may include a ‘bottom line’ summary, hunches, ethical concerns.”³ A suggested disposition (accept, minor revisions, major revisions, reject) is part of these comments. The comments to authors start with a “summary of key findings, validity and value to readers.”³ These are followed by general comments on “relevance to mission, internal validity, external validity, level of evidence and ethical conduct”³ as well as major strengths and weaknesses⁵ The review may then give specific comments by section (title, abstract and keywords, introduction, methods, results, discussion and conclusion, references)⁵  or by specific page, paragraph and line number.³ A concluding paragraph summarizes “key positive and negative comments without any statement of recommended disposition.”³ The actual structure and contents of the review will also vary depending on type of scientific article reviewed.

      Various organizations, including the Philippine Council for Health Research and Development and the Philippine Society of Otolaryngology Head and Neck Surgery organize workshops on Medical Writing for authors, as well as workshops on Peer Review for reviewers. Reviewing for local and international journals further enhances the knowledge, skills and attitudes of the reviewer. As with our other roles as clinicians, scientists, leaders, and researchers, that of educator, mentor and peer-reviewer needs adequate training and experience. We invite our reviewers to make good use of such opportunities to acquire knowledge, hone their skills and develop appropriate attitudes that will enable them to take on the great privilege and responsibility of reviewing the unpublished work of others.