forked from CoyoteLeo/labeling
-
Notifications
You must be signed in to change notification settings - Fork 0
/
instruction.html
211 lines (204 loc) · 12 KB
/
instruction.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
<!doctype html>
<html lang="en">
<head>
<title>Annotation Instructions</title>
<meta name="robots" content="noindex, ">
<meta name="googlebot" content="noindex, ">
<meta http-equiv="Pragma" content="no-cache">
<meta charset="utf-8">
</head>
<body>
<div class="ui container">
<script type="text/javascript" src="https://code.jquery.com/jquery-latest.min.js"></script>
<script type="text/javascript" src="https://cdn.jsdelivr.net/mark.js/8.6.0/mark.min.js"></script>
<link href="https://cdn.jsdelivr.net/npm/[email protected]/dist/css/bootstrap.min.css" rel="stylesheet"
integrity="sha384-eOJMYsd53ii+scO/bJGFsiCZc+5NDVN2yr8+0RDqr0Ql0h+rP48ckxlpbzKgwra6" crossorigin="anonymous">
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/js/bootstrap.bundle.min.js"
integrity="sha384-JEW9xMcG8R+pH31jmWH6WWP0WintQrMb4s7ZOdauHnUtxwoG2vI5DkLtS3qm9Ekf"
crossorigin="anonymous"></script>
<style>
h3 {
margin-top: 30px;
}
h5 {
margin-top: 10px;
margin-bottom: 0;
}
.example_sent {
font-style: italic;
color: #808080;
}
table {
table-layout: fixed;
}
td,
th {
text-align: center;
vertical-align: middle;
}
</style>
<div style="text-align: center; margin:30px auto 30px auto">
<div style="text-align: left;">
<div>
<h3>Background</h3>
There are two articles followed by multiple pairs of short paragraphs (most fewer than 3
sentences) from the articles. The two articles are labeled as "Recalled Paragraph" and "Retold Paragraph" with events from the two found below in bolded text.
The goal is to label the pair of paragraphs as either "Match" or "Conflict" depending on whether or not the meanings of the two are consistent.<br />
</div>
<div>
<h3>Annotation</h3>
<h5>Steps</h5>
<ol>
<li>Read the original articles.</li>
<li>Read the paragraphs.</li>
<li>Connect the events with the correct label based on the information from the articles and
paragraphs.<br>
PS: Just drag the dot near the source event to the dot near the target event and the
connection will be built automatically.
PS: If you want to remove the connection, you can click the "X" button near any end of the
connection, it'll remove all connections about that dot.
</li>
<li>After going through all pairs using the "Next" button, click "Submit."</li>
</ol>
<h5>WHAT is an Event</h5>
<div style="margin-left: 50px;">
An event consists of a subject, a predicate, and an object. Below is an example,<br>
<img src="event.jpg" width="700px" style="border: 5px red solid;"><br>
The above image tells us that there are two events belonging to the bolded sentence at the top.
</div>
<h5>Relation Label</h5>
<table class="table table-borderless">
<tr>
<td style="text-align: right;width: 120px;font-weight: bold;vertical-align: top;">Match:
</td>
<td style="text-align: justify;">
The event from both the Recalled Paragraph and the Retold Paragraph describe the same thing, and
there are no conflicts between each other (i.e. they can use different words, but they
express the same meaning). The following are some examples:
<table class="table">
<thead>
<tr>
<th>Recalled event</th>
<th>Retold event</th>
<th style="width: 120px;">Relation</th>
<th style="text-align: justify;">Reason</th>
</tr>
</thead>
<tbody>
<tr>
<td>Our family, consisting of, 8 - year old son.</td>
<td>we, have, an 8 year old</td>
<td>Match</td>
<td style="text-align: justify;">different description, but same meaning
</td>
</tr>
<tr>
<td>We, proceeded to, our hotel.</td>
<td>We, reaching, our hotel on the Canadian side</td>
<td>Match</td>
<td style="text-align: justify;">the second is more detailed, but both of
them have same meaning.</td>
</tr>
<tr>
<td>1. he, asked, me<br>
2. he, asked, marry him
</td>
<td>he, propose</td>
<td>Match</td>
<td style="text-align: justify;">Each recalled/retold event may matched to
multiple retold/recalled
events.</td>
</tr>
</tbody>
</table>
<div style="color: red;">
PS:<br>he, asked, me
<ul>
<li>he → subject</li>
<li>asked → predciate</li>
<li>me → object</li>
</ul>
</div>
</td>
</tr>
<tr>
<td style="text-align: right;width: 120px;font-weight: bold;vertical-align: top;">Conflict:
</td>
<td style="text-align: justify;">
In contrast to "Match," <span style=font-weight:bold;">"Conflict"</span> means the events
describe the same thing, but the meaning is inconsistent. Here are some examples:
<table class="table">
<thead>
<tr>
<th>Recalled event</th>
<th>Retold event</th>
<th style="width: 120px;">Relation</th>
<th style="text-align: justify;">Reason</th>
</tr>
</thead>
<tbody>
<tr>
<td>No one, plays, sports</td>
<td>you, do, sports</td>
<td>Conflict</td>
<td style="text-align: justify;">Same event, but their meanings conflict.</td>
</tr>
<tr>
<td>I, walk to, school</td>
<td>I, drive to, school</td>
<td>Conflict</td>
<td style="text-align: justify;">Same event, but their meanings conflict.</td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr>
<td style="text-align: right;width: 120px;font-weight: bold;vertical-align: top;">(No
relation):
</td>
<td style="text-align: justify;">
If there are no relations between the events, you can leave it empty and continue on to the next pair.<br>
In other words, you don't have to link the dots between the recalled and retold events.
<table class="table">
<thead>
<tr>
<th>Recalled event</th>
<th>Retold event</th>
<th style="width: 120px;">Relation</th>
<th style="text-align: justify;">Reason</th>
</tr>
</thead>
<tbody>
<tr>
<td>we, drive to, our hotel on the Canadian side </td>
<td>we, exploring, many activities
</td>
<td>(No relation)</td>
<td style="text-align: justify;"></td>
</tr>
<tr>
<td>We, go to, Toronto</td>
<td>We, Driving back</td>
<td>(No relation)</td>
<td style="text-align: justify;">Although the events seem to either match or conflict, the original sentences are,<br>
- recalled: Our next stop was Toronto.<br>
- retold: We ended our visit and driving back made me a little
nervous.<br><br>
In the original sentences/paragraphs, they describe different things, so
there are no relations between these events.
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</table>
Example result of one pair:
<img src="example.png" width="100%" style="border: 5px red solid;">
</div>
</div>
</div>
</div>
</body>
</html>