💬SPEECH: Structured Prediction with Energy-Based Event-Centric Hyperspheres
🍎 The project is an official implementation for SPEECH model and a repository for OntoEvent-Doc dataset, which has firstly been proposed in the paper 💬SPEECH: Structured Prediction with Energy-Based Event-Centric Hyperspheres accepted by ACL 2023 main conference.
🖥️ We also release the poster and slides for better understanding of this paper.
🤗 The implementations are based on Huggingface's Transformers and also referred to OntoED & DeepKE.
🤗 The baseline implementations are reproduced with codes referred to MAVEN's baselines or with official implementation.
SPEECH is proposed to address event-centric structured prediction with energy-based hyperspheres.
SPEECH models complex dependency among event structured components with energy-based modeling, and represents event classes with simple but effective hyperspheres.
The structure of data and code is as follows:
SPEECH
├── README.md
├── ACL2023@Poster_Speech.pdf
├── ACL2023@Slides_Speech.pdf
├── requirements.txt # for package requirements
├── data_utils.py # for data processing
├── speech.py # main model (bert serves as the backbone)
├── speech_distilbert.py # main model (distilbert serves as the backbone)
├── speech_roberta.py # toy model (roberta serves as the backbone, not adopted in the paper and just for reference)
├── run_speech.py # for model running
├── run_speech.sh # bash file for model running
└── Datasets # data
├── MAVEN_ERE
│ ├── train.jsonl # for training
│ ├── test.jsonl # for testing
│ └── valid.jsonl # for validation
├── OntoEvent-Doc
│ ├── event_dict_label_data.json # containing all event type labels
│ ├── event_dict_on_doc_train.json # for training
│ ├── event_dict_on_doc_test.json # for testing
│ └── event_dict_on_doc_valid.json # for validation
└── README.md
-
python==3.9.12
-
torch==1.13.0
-
transformers==4.25.1
-
scikit-learn==1.2.2
-
torchmetrics==0.9.3
-
sentencepiece==0.1.97
1. Project Preparation:
Download this project and unzip the dataset. You can directly download the archive, or run git clone https://github.com/zjunlp/SPEECH.git
in your teminal.
cd [LOCAL_PROJECT_PATH]
git clone [email protected]:zjunlp/SPEECH.git
2. Data Preparation:
Unzip MAVEN_ERE and OntoEvent-Doc datasets stored at ./Datasets
.
cd Datasets/
unzip MAVEN_ERE
unzip OntoEvent-Doc
cd ..
3. Running Preparation:
Install all required packages.
Adjust the parameters in run_speech.sh
bash file.
pip install -r requirements.txt
vim run_speech.sh
# input the parameters, save and quit
Hint:
- Please refer to
main()
function inrun_speech.py
file for detail meanings of each parameters. - Pay attention to
--ere_task_type
parameter candidates:- "doc_all" is for "All Joint" experiments in the paper
- "doc_joint" is for each ERE subtask "+joint" experiments in the paper
- "doc_temporal"/"doc_causal/"doc_sub" is for each ERE subtask experiments only
- Note that the loss ratio λ1, λ2, λ3, for trigger classification, event classification and event-relation extraction depends on different tasks, please ensure a correct setting of these ratios, referring to line 56-61 in
speech.py
andspeech_distilbert.py
file for details. We also present the loss ratio setting in Appendix B in our paper.
4. Running Model:
Run ./run_speech.sh
for training, validation, and testing.
./run_speech.sh
# Or you can run run_speech.py with manual parameter input in the terminal.
python run_speech.py --para...
Hint:
- A folder of model checkpoints will be saved at the path you input (
--output_dir
) in the bash filerun_speech.sh
or the command line in the terminal. - We also release the checkpoints for direct testing (Dismiss
--do_train
in the parameter input)
We briefly introduce the datasets in Section 4.1 and Appendix A in our paper.
MAVEN_ERE is proposed in a paper and released in GitHub.
OntoEvent-Doc, formatted in document level, is derived from OntoEvent which is formatted in sentence level.
The statistics of MAVEN-ERE and OntoEvent-Doc are shown below, and the detailed data schema can be referred to [./Datasets/README.md
].
Dataset | #Document | #Mention | #Temporal | #Causal | #Subevent |
---|---|---|---|---|---|
MAVEN-ERE | 4,480 | 112,276 | 1,216,217 | 57,992 | 15,841 |
OntoEvent-Doc | 4,115 | 60,546 | 5,914 | 14,155 | / |
The data schema of MAVEN-ERE can be referred to their GitHub. Experiments on MAVEN-ERE in our paper involve:
- 6 temporal relations: BEFORE, OVERLAP, CONTAINS, SIMULTANEOUS, BEGINS-ON, ENDS-ON
- 2 causal relations: CAUSE, PRECONDITION
- 1 subevent relation: subevent_relations
Experiments on OntoEvent-Doc in our paper involve:
- 3 temporal relations: BEFORE, AFTER, EQUAL
- 2 causal relations: CAUSE, CAUSEDBY
We also add a NA relation to signify no relation between the event mention pair for the two datasets.
🍒 The OntoEvent-Doc dataset is stored in json format. Each document (specialized with a doc_id, e.g., 95dd35ce7dd6d377c963447eef47c66c) in OntoEvent-Doc datasets contains a list of "events" and a dictionary of "relations", where the data format is as below:
[a doc_id]:
{
"events": [
{
'doc_id': '...',
'doc_title': 'XXX',
'sent_id': ,
'event_mention': '......',
'event_mention_tokens': ['.', '.', '.', '.', '.', '.'],
'trigger': '...',
'trigger_pos': [, ],
'event_type': ''
},
{
'doc_id': '...',
'doc_title': 'XXX',
'sent_id': ,
'event_mention': '......',
'event_mention_tokens': ['.', '.', '.', '.', '.', '.'],
'trigger': '...',
'trigger_pos': [, ],
'event_type': ''
},
...
],
"relations": { // each event-relation contains a list of 'sent_id' pairs.
"COSUPER": [[,], [,], [,]],
"SUBSUPER": [],
"SUPERSUB": [],
"CAUSE": [[,], [,]],
"BEFORE": [[,], [,]],
"AFTER": [[,], [,]],
"CAUSEDBY": [[,], [,]],
"EQUAL": [[,], [,]]
}
}
📋 Thank you very much for your interest in our work. If you use or extend our work, please cite the following paper:
@inproceedings{ACL2023_SPEECH,
author = {Shumin Deng and
Shengyu Mao and
Ningyu Zhang and
Bryan Hooi},
title = {SPEECH: Structured Prediction with Energy-Based Event-Centric Hyperspheres},
booktitle = {{ACL} {(1)}},
publisher = {Association for Computational Linguistics},
pages = {351--363},
year = {2023},
url = {https://aclanthology.org/2023.acl-long.21/}
}