Refereeing

In 2006, the editor of the British Journal of Educational Technology emailed his panel of over 150 referees to ask them if they were happy with the procedures used to referee articles submitted to his journal. In this case, the referees choose papers that they would like to referee from an electronic menu. They make their selection from the editor's list, which gives them the names of the authors and the titles of the submitted publications. The referees review the paper and send their reports via the editor to the author, unsigned. This process is called a 'single-blind review'. In a 'double-blind review', the names of the authors and their institutions are deleted from the manuscripts, and the referees do not sign their reports. (Two other possibilities exist: 'open review' — where both the authors and the referees names are known to each other, and another (rare) form of single-blind review — where the referee's name is known to the authors, but their names are not known to the referee.)

There were some two dozen replies to the editor's question. Most of these supported the editor's approach, some others were more neutral and raised additional concerns, and only three supported double-blind refereeing. These responses suggest that there was no serious opposition to the notion that referees should get to see the name(s) of the authors in advance, but that they should not give their names when refereeing the papers — a rather self-indulgent position.

But what does the research say about the strengths and limitations of these different procedures? There have been several studies of blind reviewing detailing its advantages and disadvantages. Weller (2001) provides a useful, if now somewhat dated, summary. It appears that:

1 referees can usually detect correctly the name of a deleted author about forty per cent of the time;

2 there appears to be little difference in the rejection rates of papers refereed blind or not;

3 the language used in open referees' reports differs little from that used in blind ones; and

4 there is little evidence for any gender bias in the refereeing of journal articles.

'Peer review' is the general name given to this process, where articles are submitted to a journal editor and are then sent out to two, three or more colleagues for review. These colleagues make recommendations, and the editor then decides whether or not to accept or reject the paper. Rejection rates vary across and within disciplines. Thus, eighty per cent of submissions in the social sciences are typically rejected, whereas this figure is typically twenty-five per cent in the sciences.

Normally, when asked to referee a paper, the referees have to:

1 fill out an evaluation form;

2 make an overall recommendation;

3 provide comments for the author(s) supporting their judgements; and

4 provide comments for the editor(s) supporting their judgements.

Actions 1, 2 and 3 are usually done anonymously, and copies of 2 and 3 are sent by the editor to the author(s), together with the decision letter.

0 0

Post a comment