Skip to content

Complete set of English dialect transformation rules and evaluation code

License

Notifications You must be signed in to change notification settings

SALT-NLP/multi-value

Repository files navigation

Multi-VALUE: The VernAcular Language Understanding Evaluation benchmark

Setup

Prerequisites:

  1. Create a virtual environment
conda create --name value python=3.7.13
conda activate value
  1. Install requirements:
pip install -r requirements.txt
  1. Install spaCy English pipeline and nltk wordnet
bash downloads.sh
  1. Confirm that your setup is correct by running the unittest
python -m unittest tests.py

Build Multi-VALUE CoQA (optional)

  1. Pull data
bash pull_coqa.sh
  1. Run for each dialect
python -m src.build_coqa_value --dialect aave &
python -m src.build_coqa_value --dialect appalachian &
python -m src.build_coqa_value --dialect chicano &
python -m src.build_coqa_value --dialect indian &
python -m src.build_coqa_value --dialect multi &
python -m src.build_coqa_value --dialect singapore &

About

Complete set of English dialect transformation rules and evaluation code

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published