The University of Tokyo

Survey on Peer Reviewers’ Inline Citation Review Practices

This document explains the details of the “Survey on Peer Reviewers’ Inline Citation Review Practices.” Please be assured that you will not face any disadvantage if you choose not to participate in this study. If there is anything that is unclear, please do not hesitate to ask the person in charge.

1. Research Overview

Research Title

Survey on Peer Reviewers’ Inline Citation Review Practices

Principal Investigator

• Koji Yatani, Professor,
  Department of Information and Communication Engineering,
  Graduate School of Information Science and Technology,
  The University of Tokyo

Research Member

• Xinrui Fang, 3rd-year Doctoral Student,
  Interactive Intelligent Systems Laboratory (IIS Lab),
  Graduate School of Interdisciplinary Information Studies,
  The University of Tokyo

Research Purpose

The purpose of this study is to clarify how reviewers verify citations used within a paper during the peer review process and to identify which aspects of this verification process are perceived as problematic. Peer review is a vital mechanism where experts verify whether the content of a research paper is correct and reliable. However, in recent years, the increasing number of papers has placed a significant burden on researchers performing reviews. In particular, the task of verifying whether citations supporting a paper’s claims are appropriate is labor-intensive, yet the actual nature of this work remains largely unclear. This study will conduct a survey targeting researchers with prior peer review experience. The survey will ask questions regarding the frequency of citation verification, evaluation criteria, perceived burden, and identified problems. By analyzing the collected data, we aim to clarify common perspectives and difficulties among reviewers, which will help in creating future systems to support peer review and literature verification.

Research Method

We will conduct an online survey for researchers with peer review experience. Prospective participants will receive research explanations and the questionnaire link via email; they will answer the survey after providing consent. The survey can be completed on your own computer at any time and location, taking approximately 30 minutes. It consists of questions regarding review behavior, methods for evaluating inline citations, and burdens or challenges faced during verification. The survey will be conducted using forms.app, and no audio or video recording will take place. Participants will receive a 1,500 JPY Gift Card as compensation. The survey will be conducted in English.

2. Voluntariness of Participation and Right to Withdraw

Participants will be presented with an information sheet and consent form within the survey. Consent to participate in the study will be obtained by asking participants to check the “I agree” checkbox.

Participation in this study is voluntary, and participants may withdraw their consent at any time before submitting the survey. Participants can withdraw their consent by choosing not to submit their responses.

Please note that once the survey responses have been anonymized and processed in a way that no individual can be identified, it may no longer be possible to delete the data even if consent is withdrawn.

3. Protection of Personal Information

We will exercise maximum caution and effort to protect personal information and respect privacy so that you do not suffer any disadvantage. Personal information such as email addresses will be anonymized (without creating a correspondence table) before being used for research. Your personal information will be stored strictly and responsibly.

4. Publication of Research Results

Research results will be published at academic conferences, in academic journals, and on databases in a manner that ensures individuals cannot be identified.

5. Benefits and Disadvantages to Participants

The likelihood of this research providing immediate beneficial information to you or society is currently considered low. However, the results are expected to serve as important fundamental findings contributing to the future development of Human-Computer Interaction research.

6. Policy on Handling Materials and Information

After the compensation has been sent, the correspondence table will be discarded to separate the data from the individual (anonymization). The data will then be used for research and analysis in an unidentifiable format. Furthermore, the data will be stored at the Yatani Laboratory, Department of Creative Informatics, Graduate School of Information Science and Technology, The University of Tokyo, for at least 10 years after the publication of the research results.

7. Cost Burden

There are no costs for you to participate in this study. Participants will receive a 1,500 JPY Gift Card.

8. Others

This research has been reviewed by the Ethics Committee of the Graduate School of Information Science and Technology, The University of Tokyo, and is conducted with the permission of the Dean of the Graduate School. Costs for this research are covered by the University of Tokyo’s operating expense grants. If you have any opinions or questions, please feel free to reach out to the contact below.

Contact Information