Model structure redundancy

[grig-guz]

Hi,

The span width embedding over here:

coref/independent.py

Line 379 in bd04f2e

span_width_emb = tf.get_variable("span_width_prior_embeddings", [self.config["max_span_width"], self.config["feature_size"]], initializer=tf.truncated_normal_initializer(stddev=0.02)) # [W, emb]

is pretty much equivalent to the span embedding over there, since the width embedding is concatenated to other span embeddings and then passed through a linear layer:

coref/independent.py

Line 362 in bd04f2e

span_width_emb = tf.gather(tf.get_variable("span_width_embeddings", [self.config["max_span_width"], self.config["feature_size"]], initializer=tf.truncated_normal_initializer(stddev=0.02)), span_width_index) # [k, emb]

I am trying to reimplement your model in Pytorch, so I was just wondering if there is any rationale for using two sets of span width embeddings?

Thank you.

[Fantabulous-J]

Hi @grig-guz! I have also implemented this model using Pytorch but always have a performance gap of around 1.2 F1 scores with the official results reported on paper. How does your implementation go on? Maybe we could share some ideas and experiences with each other.

[grig-guz]

Hi @Fantabulous-J, sure. I've got around 74 F1 on the dev set with Spanbert-Base, didn't run on the test set yet. My email is on my github page, you can write me there.