Neural Model converting Image to Latex.
As the picture shows, given an input image, a CNN and RNN encoder is applied to extract visual features firstly. And then the encoded features are used by an RNN decoder with attention mechanism to produce final formulas.
pip3 install -r requirements.txt
We provide a small preprocessed dataset (./sample_data/) to check the pipeline. To train and evaluate model in this sample dataset:
python3 train.pyDownload the prebuilt dataset from Harvard and use their preprocessing scripts found here.
After that, to train and evaluate model on full dataset, you can just pass the full dataset path to train.py:
python3 train.py --data_path=FULL_DATASET_PATH
What You Get Is What You See:A Visual Markup Decompiler