The role of the reviewer is to assist the Program Committee (PC) in assessing the quality of proposals submitted. Reviews do not wholly determine which proposals will be accepted or rejected, but rather provide expert information which the PC uses in making its decisions. The second major role for reviewers is to provide helpful, constructive feedback to authors, which can strengthen both the quality of the conference and the collegiality and intellectual rigor of the digital humanities.
A good review will suggest concrete ways in which the proposal may be strengthened. This feedback is important whether you are recommending that the paper or poster be rejected or accepted. If the former, it will enable the author to submit a stronger proposal next year (and may encourage and benefit a new member or a young scholar); if the latter, it will result in a stronger paper being presented. In either case, constructive criticism projects collegiality and an interest in others’ work.
Whatever you may think privately of the proposal or project, and whatever you may know (or think you know) about the author, it is essential that you be uncompromisingly professional and courteous in reviewing all submissions. Rudeness of any sort is destructive to the morale of the community and is absolutely unacceptable in a review. Comments which are purely negative should be addressed solely to the program committee.
To a certain extent, given the small size of the DH conference community, the identity of paper authors has often been evident to an astute reviewer, and discretion has always been required in handling that knowledge appropriately. Since ADHO has discontinued the anonymization of papers during the review process (moving to single-blind review), this discretion has become even more necessary.
The identity of an author will necessarily be a factor in evaluating the proposal, but it may operate in complex ways. A famous name should not be taken as de facto evidence of a strong proposal, and a weak proposal by such an author should be given the same critique as a weak proposal by an unknown author. However, there may also be cases where an author’s known expertise may strengthen the value of the proposal: for instance, a representative of a standards workgroup might have more credibility in discussing the standard than someone with no involvement in the effort. Reviewers should feel free to be candid in their comments to the Program Committee in cases where they feel the identity of the author plays a significant role in the assessment of the paper.
Continuing the new approach applied for the DH conference this year, you will be allowed to read comments by others who have been assigned to assess the same submission. These peer reviews will only be visible after you have submitted your own review, and all will remain anonymous. Reviewers are expected to assess each contribution independently, on its own merits, and are asked not to address other reviewers’ comments directly. If, however, you are prompted to think more deeply and augment the text of your own review after reading peer assessments, you are welcome to do so up to the review deadline, and are expected to justify any scoring change in the text of your review.