As a Pangolin looks for bugs and catches them, the goal of this library is ot help developers finding bugs in their neural networks and newly-created models.

pangolinn is available on PyPi and can be installed by running:

pip install pangolinn

Alternatively, the latest development version can be installed with:

git clone https://github.com/hlt-mt/pangolinn.git
cd pangolinn
pip install -e .

If you want to contribute to the project, you can install the additional development dependencies by using the dev specifier:

pip install -e .[dev]

To test your model/module you need to:

• create a pangolinn wrapper that builds it and determines how to use it;
• create a test suite that inherits from the pangolinn tester you want to use and set the attribute module_wrapper_class to the name of the wrapper class of your module.

For complete examples, please refer to the UTs in this repository, e.g. Transformer decoder causality test.

The repository currently contains the following test suites:

• Encoder padding tester: checks that the presence of padding does not alter the results.
• Causality tester: checks that a module fulfils the causal property, i.e. it does not look at future elements of the sequence (e.g., as autoregressive decoders have to do).

Our goal is to provide a comprehensive test suit for neural networks, therefore contributions from interested researchers and developers are extremely appreciated.

You can either create a feature request to propose new tests or a pull request to contribute to our repository with your own tests.

pangolinn is licensed under Apache Version 2.0.

If using this repository, please cite:

@inproceedings{papi-et-al-2024-when,
  title={{When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLP}},
  author={Papi, Sara and Gaido, Marco and Pilzer, Andrea and Negri, Matteo},
  booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
  address = "Bangkok, Thailand",
  year={2024}
}