Cross-linguistic annotation transfer with Classical texts
Laura Soffiantini (laura dot soffiantini at kuleuven dot be)
Embedded Files
In this page, I describe step-by-step how to run a Python script to automatically transfer annotations from the English translation of an ancient classical text to the original text.
What? Semantic annotation consists of the mark-up of words or pieces of text which have a certain meaning. For instance, we can annotate all the persons (PER) or locations (LOC) mentioned in a text, as in the example below.
I shall attempt instead to recount to you a discourse concerning love that I once heard from a woman of Mantinea [LOC], Diotima [PER].
Why? The semantic annotation can be useful to search for information in a text, link the text to an external knowledge base (e.g., Wikipedia) or perform other kinds of analyses.
How? Semantic annotation is traditionally a manual task. However, new technologies can help us to automatically identify significant pieces of texts.
In this tutorial, we will re-use existing annotated texts in translation from ToposText and we will project the annotations to the original text. To achieve this, we will leverage automated sentence alignment and token alignment algorithms.
What follows is part of my current FWO PhD research project 'Greek Spaces in Roman Times' hosted by KU Leuven (Belgium). The technical description of the pipeline and the evaluation of the results and error rate can be found in my paper 'Cross-Linguistic Annotation Transfer in Geo-Parsing Experiments with Classical Texts: The Case of Pliny the Elder's Natural History' (2024).
The code was adjusted for ancient Greek texts from PerseusDL by Rachel Milo (PhD candidate, University of Crete). 🙏
In recent years, the advent of machine learning models has ushered in a new era of technologies for the automated annotation of unstructured texts. This shift is exemplified by the development of various pre-trained Named Entity Recognition (NER) systems.
In the context of historical data, a number of additional factors come into play. One of the most significant is the often-noted underdevelopment of NLP systems tailored for languages beyond the modern ones with significant exceptions (cf. Beersmans et al. 2023).
In this demonstration, I show how cross-lingual annotation transfer can be applied from a translation in a modern language back to the original text in the context of low-/medium- resource languages, such as Latin.
Let's start!
3. Download the GitHub repo
3. Download the GitHub repo
Access the GitHub repository of the project here.
Download the repo from '<> Code', 'Download ZIP' (see image).
Unzip the folder of the repo.
4. Have a look inside the folder!
4. Have a look inside the folder!
The 'annotation_transfer-main' folder you downloaded from GitHub contains:
The 'annotation_projection_lat.ipynb' file, which contains the code to run for Latin texts.
The 'annotation_projection_grc.ipynb' file, which contains the code to run for Greek texts.
A folder 'utils' which contains other code.
The 'requirements.txt' file, which contains a list of Python libraries that will be automatically installed.
A 'README.md' .file.
5. Launch JupiterNotebook
5. Launch JupiterNotebook
Open Anaconda and launch JupiterNotebook.
A new page will be automatically opened in your browser.
6. Open annotation_projection.ipynb
6. Open annotation_projection.ipynb
Access the Download folder, open the annotation_transfer-main folder, and open the annotation_projection.ipynb file choosing between the _lat and _grc files depending on the text you will annotate.
7. Have a look at the content
7. Have a look at the content
The Python code is written in cells (each cell has a [ ]: symbol on the left).
Your turn!
Before running the kernel, you need to indicate which text you want to process.
The code takes as input the annotated English translation of ancient works that can be found in the ToposText repository.
Navigate through the Annotated Texts in ToposText and choose your text! Then, copy the url.
For instance, Apuleius'Metamorphoses has url: https://topostext.org/work/523.
The first cell of the code contains:
url_topostext = ' ..... '
Copy inside the 'quotation marks' the url of the ToposText page of your English text:
url_topostext = 'https://topostext.org/work/523'
Then, you also need the Latin or ancient Greek text.
The code takes as input the Latin texts that can be found in the GitHub repo of the LascivaRoma project or the ancient Greek texts that can be found in the GitHub repo of the Perseus Digital Library project.
Look for your Latin text in the lemmatized/xml folder of the LascivaRoma repo using the text's urn, which can be found in Perseus or in the ToposText page.
Alternatively, look for your Ancient Greek text in the canonical-greekLit/data folder of the PerseusDL repo using the text's urn, which can be found in Perseus or in the ToposText page.
For instance, Apuleius'Metamorphoses has urn: https://catalog.perseus.org/catalog/urn:cts:latinLit:phi1212.phi002.
Search for your urn using find (on the left).
Select the file.
Click on 'View raw'.
Copy the url of the new page.
The example below refers to the annotation_projection_lat.ipynb file.
The second cell of the code contains:
url_lascivaroma = ' ...... '
Copy inside the 'quotation marks' the url of the LascivaRoma page of your Latin text:
url_lascivaroma = 'https://raw.githubusercontent.com/lascivaroma/latin-lemmatized-texts/refs/heads/main/lemmatized/xml/urn%3Acts%3AlatinLit%3Aphi1212.phi002.perseus-lat1.xml'
Finally, the third cell contains:
work = ' ...... '
Write inside the 'quotation marks' the name of the work you are going to annotate. This will facilitate the creation of files and folder with a stable name. Eg., since I will annotate Apuleius'Metamorphoses:
work = 'Apul_Metamorphoses'
Yuppie! 🎉 You are almost there!
Now, you need to run the code, that it is to make the code work!
To do it, select 'Kernel', 'Restart Kernel and Run All Cells'.
And now ... wait! The code will take a while to complete the entire process.
At the end, you will find in the 'annotation_transfer-main' folder new files: the new annotated ancient text will be in the folder named after the work you are annotating.
How does the new file look like?
It is a tabular .csv file containing different columns.
The column 'token' contains each separated word of the ancient work.
The 'flair_ner' column contains the automated annotation in BIO format.
The 'topostext' column contains the ToposText link for the annotated entity.
The 'wikidata' column contains the Wikidata link for the annotated entity.
Thank you for using and testing my code! For any feedback/requests contact us at our email address!
Page updated
Google Sites
Report abuse