Pythia

Available

 * Colab: https://colab.research.google.com/drive/16RfCpZLm0M6bf3eGIA7VUPclFdW8P8pZ
 * Wikipedia page: https://en.wikipedia.org/wiki/Pythia_(machine_learning)
 * DOI: https://www.aclweb.org/anthology/D19-1668/

Authors

 * Thea Sommerschield, University of Oxford
 * Yannis Assael, DeepMind
 * Jonathan Prag, University of Oxford

Description
PYTHIA: a deep learning model for the automatic restoration of Greek inscriptions is the first ancient text restoration model that recovers missing characters from a damaged text input using deep neural networks. Bringing together the disciplines of ancient history and deep learning, this research offers a fully automated aid to the epigraphic restoration of fragmentary inscriptions, providing ancient historians with multiple textual restorations, as well as the confidence level for each hypothesis.

Pythia takes a sequence of damaged text as input, and is trained to predict character sequences comprising hypothesised restorations of ancient Greek inscriptions. The architecture works at both the character- and word-level, thereby effectively handling long-term context information, and dealing efficiently with incomplete word representations. This makes it applicable to all disciplines dealing with ancient texts (philology, papyrology, codicology) and applies to any language (ancient or modern).

To train the model, we wrote a non-trivial pipeline to convert PHI, the largest digital corpus of ancient Greek inscriptions, to machine actionable text, which we call PHI-ML. On PHI-ML, Pythia's predictions achieve a 30.1% character error rate, compared to the 57.3% of human epigraphists. Moreover, in 73.5% of cases the ground-truth sequence was among the Top-20 hypotheses of Pythia, which effectively demonstrates the impact of such an assistive method on the field of digital epigraphy, and sets the state-of-the-art in ancient text restoration.

To aid further research in the field we created a freely accessible online interactive Python notebook, where researchers can query one of our models to get text restorations and visualise the attention weights. Follow the instructions on the Colab notebook to restore texts for your own personal research.

Furthermore, the PHI-ML dataset and processing pipeline have been published on GitHub, so that any researcher may regenerate PHI-ML and train new models offline. Follow the instructions below under the section "Pythia offline".

Pythia offline
The following snippets provide references for regenerating PHI-ML and training new models offline.

Dependencies
pip install -r requirements.txt && \ python -m nltk.downloader punkt

PHI-ML dataset generation
python -c 'import pythia.data.phi_download; pythia.data.phi_download.main'
 * 1) Download PHI (this will take a while)

python -c 'import pythia.data.phi_process; pythia.data.phi_process.main'
 * 1) Process and generate PHI-ML

Preprocessed PHI-ML uploaded by @Holger.Danske800: link

Training
python -c 'import pythia.train; pythia.train.main'

Evaluation
python -c 'import pythia.test; pythia.test.main' --load_checkpoint="your_model_path/"

Docker execution
./build.sh ./run.sh  python -c 'import pythia.train; pythia.train.main'

Media
Our research has been featured in blog posts by DeepMind and the University of Oxford. It has also received press coverage by The Times, NewScientist, Financial Times, La Repubblica, Ekathimerini, TechCrunch and others.

The project has been presented at EMNLP - IJCNLP 2019 in Hong Kong, at the British School at Rome, at the Oxford Epigraphy Workshop, at the OIKOS National Research School in Classical Studies, during lectures at Venice Ca' Foscari and Rome La Sapienza.

The project poster is published on the Epigraphy.info website (on occasion of the V Workshop at KU Leuven) and can be accessed here.

Pythia is a partner project of epigraphy.info and EADH.

Presentations

 * Thea Sommerschield (2020), "PYTHIA: a deep neural network model for the automatic restoration of ancient Greek inscriptions", Digital Classicist London Seminar. Available: https://www.youtube.com/watch?v=nKSfzHYmLtQ&t=26s
 * Alek Keersmaekers, Marton Ribary, Thea Sommerschield (2020), "Introduction to Computational Linguistics", SunoikisisDC. Available: https://www.youtube.com/watch?v=zjkyZUpvhAQ

Contact

 * thea.sommerschield@classics.ox.ac.uk