PyEpiDoc is a Python (>=3.9) library for parsing and interacting with TEI XML EpiDoc files. It has been tested on Python 3.9 on Linux (Ubuntu). It should work on later Python versions.

PyEpiDoc has been designed for use, in the first instance, with the I.Sicily corpus. For information on the encoding of I.Sicily texts in TEI EpiDoc, see the I.Sicily GitHub wiki.

NB: PyEpiDoc is currently under active development.

To install PyEpiDoc along with its dependencies (lxml):

1. Clone or download the repository.

2. Navigate into the cloned / downloaded repository.

3. From within the cloned repository, install at the user level with:

   pip install . --user
   

If you are using a venv virtual environment:

1. Make sure the virtual environment has been activated, e.g. on Linux:

   source env/bin/activate

2. Install with pip:

   pip install .

pip uninstall pyepidoc

To install PyEpiDoc along with its dependencies (lxml) and development dependencies (pytest, mypy), e.g. in a virtual environment:

1. Clone or download the repository;

2. Navigate into the cloned / downloaded repository.

3. From within the cloned repository, install with:

   pip install .[dev]
   

Jupyter notebooks are included in the repository under notebooks/ to provide example usage:

• getting_started.ipynb
• abbreviations.ipynb
• setting_ids.ipynb

For instructions on installing Jupyter notebook, see https://docs.jupyter.org/en/latest/install/notebook-classic.html. Alternatively, see also https://jupyter.org/install.

Once Jupyter notebook is installed, to run getting_started.ipynb, type:

jupyter notebook getting_started.ipynb

Given a tokenized EpiDoc file ISic000001.xml in an examples/ folder in the current working directory.

from pyepidoc import EpiDoc

doc = EpiDoc("examples/ISic000001_tokenized.xml")

print(doc.edition_text)

tokens = doc.tokens
print(' '.join([str(token) for token in tokens]))

Given an untokenized EpiDoc file ISic000032_untokenized.xml in an examples folder in the current working directory:

from pyepidoc import EpiDoc

# Load the EpiDoc file
doc = EpiDoc("examples/ISic000032_untokenized.xml")

# Tokenize the edition with default settings
doc.tokenize()

# Print list of tokens
print('Tokens: ', doc.tokens_list_str)

# Save the results to a new XML file
doc.to_xml_file("examples/ISic000032_tokenized.xml")

Given a corpus of EpiDoc XML files in a folder corpus/ in the current working directory, the following code filters the corpus and writes a text file containing the ids of all Latin funerary inscriptions from Catania / Catina:

from pyepidoc import EpiDocCorpus
from pyepidoc.epidoc.enums import TextClass
from pyepidoc.file.funcs import str_to_file

# Load the corpus
corpus = EpiDocCorpus('corpus')

# Filter the corpus to find the funerary inscriptions
funerary_corpus = corpus.filter_by_textclass([TextClass.Funerary.value])

# Within the funerary corpus, find all the Latin inscriptions from Catania / Catina:
catina_funerary_corpus = (
    funerary_corpus
        .filter_by_orig_place(['Catina'])
        .filter_by_languages(['la'])
)

# Output the of this set of documents to a file ```catina_funerary_ids_la.txt``` 
# in the current working directory.
catina_funerary_ids = '\n'.join(catina_funerary_corpus.ids)
str_to_file(catina_funerary_ids, 'catina_funerary_ids_la.txt')

There are two ways to validate an EpiDoc XML file:

1. Validate on load, e.g.:

from pyepidoc import EpiDoc

doc = EpiDoc('examples/ISic000001_tokenized.xml', validate_on_load=True)

• This validates according to the RelaxNG schema tei-epidoc.rng in the pyepidoc root directory.
• By default validate_on_load is set to False.

2. Validate against a custom RelaxNG schema:

from pyepidoc import EpiDoc
doc = EpiDoc('examples/ISic000001_tokenized.xml')

doc.validate_by_relaxng(fp='path/to/relaxngschema.rng')

The PyEpiDoc package has four subpackages:

• xml containing modules with base classes for XML handling;
• epidoc containing modules for handling EpiDoc specific XML handling, e.g. <ab>, <w> etc.;
• analysis containing modules for analysing EpiDoc files and corpora, e.g. of abbreviations;
• shared containing modules and classes for use generally in the project.

Probably the most useful subpackage in the first instance will be epidoc, and in particular epidoc.py and corpus.py, which, via the classes EpiDoc and EpiDocCorpus, provide APIs to EpiDoc files and corpora respectively.

The treatment of a given token by the tokenizer may be affected by one or more of the following:

• Status in pyepidoc/epidoc/epidoctypes.py
• Presence in pyepidoc/constants.py in SubsumableRels

The token will be subsumed into a neighbouring <w> token if it is not separated by whitespace if:

• it is listed in as a dep of e.g. <w> in SubsumableRels

The token will be subsumed into a neighbouring <w> token regardless of the presence of intervening whitespace if:

• it is listed in as a dep of e.g. <w> in SubsumableRels and
• it is a member of AlwaysSubsumableType in epidoctypes.py

with pytest installed (the dev installation will do this for you):

2. To run all the tests, in the project root directory, type:

   pytest
   

If pytest is not available to the currently active version of Python, it may be necessary to specify the Python executable with pytest installed, e.g.:

```
python3.10 -m pytest
```

To check the integrity of the type annotations, with mypy installed (the dev installation will do this for you):

mypy src/pyepidoc

If mypy is not available to the currently active version of Python, it may be necessary to specify the Python executable with mypy installed, e.g.:

```
python3.10 -m mypy src/pyepidoc
```

XML comments should now be handled correctly, and reproduced in new files.

PyEpiDoc depends on lxml (BSD). Development dependencies are mypy (MIT) and pytest (MIT). Licenses for these dependencies are included in the LICENSES directory.

The software for PyEpiDoc was written by Robert Crellin as part of the Crossreads project at the Faculty of Classics, University of Oxford, and is licensed under the MIT license (see LICENSES/LICENSE-pyepidoc). This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 885040, “Crossreads”).

Example and test .xml files, contained in the examples/, example_corpus/ and tests/ subfolders, as well as in the source code, are either directly form, or derived from, the I.Sicily corpus, which are made available under the CC-BY-4.0 licence (see LICENSES/LICENSE-texts).

For further details and acknowledgements on the generation of ISicily token IDs (pyepidoc/epidoc/ids), see https://github.com/rsdc2/ISicID.