Challenges of Using ATI

Time-consuming and labour-intensive: Using ATI can be time-consuming throughout research stages:

• Beginning: Researchers must plan the documentation of decision-making processes that shape the research, which may add additional workload.

• During: The ability to link datasets and add evidence as the research progresses is beneficial, but it still requires additional effort. However, it encourages researchers to incorporate openness and transparency into every research project.

• Retrospectively: While researchers can have a clearer understanding of what needs to be annotated, it can be more time-consuming if the research has already been conducted.

Relevance and appropriateness: Annotations should be relevant and appropriate without distracting the reader from the main text. Being selective about the number of annotations to include is crucial to maintain the quality and focus of the research.

Duplication of efforts: Annotations could mean duplication of efforts as comparable goals can be achieved through footnotes or endnotes.

Standardising documentation in collaborative research: Requires every member of the research team to document their methodological and analytical processes thoroughly and may entail a standardisation of the documentation process.

Lack of universal platform support: Currently the implementation is more straightforward with Cambridge Core publications and QDR. Nevertheless, annotations are highly flexible, making it relatively simple to incorporate them into journals that currently lack this feature. To do this, the researcher must inform the journal at the start of the publishing process, that they are utilising annotations in their articles. Further guidance can be found in section “Publishing with ATI” of this document. Instances of successful implementation can be observed in the following examples:

• UC Davies Law Review journal, was very excited to implement this feature and work with QDR, demonstrating the potential for successful collaborations. The author, Melissa Hamilton discusses ATI in the first footnote and first annotation.

• For journals with less compatible publication platforms, alternative solutions can be effective. For example, with Lindsay Mayka’s piece, the journal Comparative Politics, allowed QDR to host a copy of the paper in their site, which is linked from the first note. This journal was happy to collaborate with QDR, demonstrating that even journals lacking internal annotation support can still facilitate the use of ATI by allowing alternative hosting arrangements.

• Web annotations are compatible with most journal pages, but they require the Chrome/Edge extension to be installed. For example, in this article. It is important to note that these annotations, like those from QDR’s first pilot project, were added after publication and therefore, are not mentioned in the text. 

• Annotations can be transformed to compatible formats, such as XML, using standard libraries in most programming languages. For example, in Melissa Hamilton's project, annotations are saved as JSON, which is the most robust format for data exchange on the web. By clicking the preview (eye icon) button for the Annotations.json file, annotations can be viewed in an HTML rendering. QDR is open to considering additional formats if they prove useful or commonly used.

Lack of clarity in publication processes: There is inconsistency regarding whether annotations should be peer-reviewed or implemented retrospectively. Cambridge Core publications take both approaches. The 'include in peer review' model can be compared to supplementary files and appendices in journals, which typically are peer reviewed. Conversely, the 'do not include in peer review' option is similar to how most journals handle data and code. Due to the lack of guidelines, authors should decide when to share annotations. If authors believe annotations will aid in the review process, they should include annotations in this process. However, if annotations could cause confusion or add unnecessary scrutiny, they should submit them after the peer review process. 

Dependence on specific software: Dependency on Hypothesis software adds uncertainty regarding the long-term stability of this format. However, annotations are in a standardised format following the open web annotations standard endorsed by the W3C (World Wide Web Consortium). While there are not many other fully compatible implementations beyond Hypothesis, this standardisation facilitates easier transfer. Hypothesis is also open source, allowing QDR for the possibility of running an in-house version of the service if necessary. QDR confirmed that Hypothesis has been acquired by Ithaca, the provider of JSTOR and Portico, which has a strong preservation record for digital content, increasing the reliability of the maintenance of the software.