XCrowd: Combining Explainability and Crowdsourcing to Diagnose Models in Relation Extraction

Data format

The annotations are stored in TSV format. We include both annotations from each worker and those aggregated by the majority vote.

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
Load annotations example.ipynb		Load annotations example.ipynb
README.md		README.md
aggregated_annotations_conll_dev.tsv		aggregated_annotations_conll_dev.tsv
aggregated_annotations_tacred_dev.tsv		aggregated_annotations_tacred_dev.tsv
aggregated_annotations_tacred_test.tsv		aggregated_annotations_tacred_test.tsv
instructions.pdf		instructions.pdf
raw_annotations_conll_dev.tsv		raw_annotations_conll_dev.tsv
raw_annotations_tacred_dev.tsv		raw_annotations_tacred_dev.tsv
raw_annotations_tacred_test.tsv		raw_annotations_tacred_test.tsv