Create a mechanism for model installation

[dpalmasan]

Currently TRUNAJOD models live in a folder called models, and they are loaded using a snippet as follows:

from TRUNAJOD import surface_proxies
from TRUNAJOD.entity_grid import EntityGrid
from TRUNAJOD.lexico_semantic_norms import LexicoSemanticNorm
import pickle
import spacy
import tarfile


class ModelLoader(object):
    """Class to load model."""
    def __init__(self, model_file):
        tar = tarfile.open(model_file, "r:gz")
        self.crea_frequency = {}
        self.infinitive_map = {}
        self.lemmatizer = {}
        self.spanish_lexicosemantic_norms = {}
        self.stopwords = {}
        self.wordnet_noun_synsets = {}
        self.wordnet_verb_synsets = {}

        for member in tar.getmembers():
            f = tar.extractfile(member)
            if "crea_frequency" in member.name:
                self.crea_frequency = pickle.loads(f.read())
            if "infinitive_map" in member.name:
                self.infinitive_map = pickle.loads(f.read())
            if "lemmatizer" in member.name:
                self.lemmatizer = pickle.loads(f.read())
            if "spanish_lexicosemantic_norms" in member.name:
                self.spanish_lexicosemantic_norms = pickle.loads(f.read())
            if "stopwords" in member.name:
                self.stopwords = pickle.loads(f.read())
            if "wordnet_noun_synsets" in member.name:
                self.wordnet_noun_synsets = pickle.loads(f.read())
            if "wordnet_verb_synsets" in member.name:
                self.wordnet_verb_synsets = pickle.loads(f.read())


# Load TRUNAJOD models
model = ModelLoader("trunajod_models_v0.1.tar.gz")

However, other similar projects (e.g. spacy), have models in a different repo, and they are installed via python tooling (e.g. cmd line options). This ticket is to track the effort of implementing such mechanism.

[dpalmasan]

Some decisions on this, probably it is best having other repo with the models and do something like spaCy does for its spacy models, for example:

• https://github.com/explosion/spaCy/blob/master/spacy/__init__.py
• https://github.com/explosion/spaCy/blob/master/spacy/cli/_util.py
• https://github.com/explosion/spaCy/blob/master/spacy/cli/_util.py#L64

Probably a small cli should be implemented with the download model option.