How to use the FlairScorer to predict given *one* broken word?

[kkew3]

Hello. Thanks for the great project.

I have a handful of broken words with hyphen removed, and I want to predict if the hyphen in between should be recovered. I'm wondering how to achieve this with FlairScorer. Input example: ('state', 'of-the-art').

From the code:

dehyphen/dehyphen/dehyphen.py

Lines 51 to 57 in 92306a6

	# 2. some compound-word (keep hyphen), remove whitespace
	option2 = last_word.strip() + next_word
	# 3. remove hyphen, most likely to happen
	option3 = last_word.strip()[:-1] + next_word
	scores = self.score((option1, option2, option3))

I imagine the following usage:

from dehyphen import FlairScorer

class DehyphenModel:
    def __init__(self, lang: str):
        self.scorer = FlairScorer(lang=lang)

    def predict(self, last_word, next_word):
        options = [
            last_word + next_word,  # no hyphen in between
            last_word + '-' + next_word,  # keep the hyphen
        ]
        score_0, score_1 = self.scorer.score(options)
        # If option 1 is larger, return 1 (keep hyphen); else return 0
        return (score_1 > score_0) * 1

# Test cases
dehyphen_model = DehyphenModel('en')
print((
    dehyphen_model.predict('p', 'value'),
    dehyphen_model.predict('state', 'of-the-art'),
    dehyphen_model.predict('hel', 'lo'),
    dehyphen_model.predict('require', 'ment'),
))

Output: (0, 0, 1, 1), all wrong!

Therefore, I suspect my usage is incorrect.

Could you please tell me how to achieve this function? Thank you so much!