SPINN, or Stack-Augmented Parser-Interpreter Neural Network, is a recursive neural network that utilizes syntactic parse information for natural language understanding.

SPINN was originally described by: Bowman, S.R., Gauthier, J., Rastogi A., Gupta, R., Manning, C.D., & Potts, C. (2016). A Fast Unified Model for Parsing and Sentence Understanding. https://arxiv.org/abs/1603.06021

Our implementation is based on @jekbradbury's PyTorch implementation at: https://github.com/jekbradbury/examples/blob/spinn/snli/spinn.py,

which was released under the BSD 3-Clause License at: https://github.com/jekbradbury/examples/blob/spinn/LICENSE

Other eager execution examples can be found under tensorflow/contrib/eager/python/examples.

• data.py: Pipeline for loading and preprocessing the SNLI data and GloVe word embedding, written using the tf.data API.
• spinn.py: Model definition and training routines. This example illustrates how one might perform the following actions with eager execution enabled:
  • defining a model consisting of a dynamic computation graph,
  • assigning operations to the CPU or GPU dependending on device availability,
  • training the model using the data from the tf.data-based pipeline,
  • obtaining metrics such as mean accuracy during training,
  • saving and loading checkpoints,
  • writing summaries for monitoring and visualization in TensorBoard.

• Make sure you have installed TensorFlow release 1.5 or higher. Alternatively, you can use the latest tf-nightly or tf-nightly-gpu pip package to access the eager execution feature.

• Download and extract the raw SNLI data and GloVe embedding vectors. For example:

  curl -fSsL https://nlp.stanford.edu/projects/snli/snli_1.0.zip --create-dirs -o /tmp/spinn-data/snli/snli_1.0.zip
  unzip -d /tmp/spinn-data/snli /tmp/spinn-data/snli/snli_1.0.zip
  curl -fSsL https://nlp.stanford.edu/data/glove.42B.300d.zip --create-dirs -o /tmp/spinn-data/glove/glove.42B.300d.zip
  unzip -d /tmp/spinn-data/glove /tmp/spinn-data/glove/glove.42B.300d.zip

• Train model. E.g.,

  python spinn.py --data_root /tmp/spinn-data --logdir /tmp/spinn-logs

  During training, model checkpoints and TensorBoard summaries will be written periodically to the directory specified with the --logdir flag. The training script will reload a saved checkpoint from the directory if it can find one there.

  To view the summaries with TensorBoard:

  tensorboard --logdir /tmp/spinn-logs

• After training, you may use the model to perform inference on input data in the SNLI data format. The premise and hypotheses sentences are specified with the command-line flags --inference_premise and --inference_hypothesis, respectively. Each sentence should include the words, as well as parentheses representing a binary parsing of the sentence. The words and parentheses should all be separated by spaces. For instance,

  python spinn.py --data_root /tmp/spinn-data --logdir /tmp/spinn-logs \
      --inference_premise '( ( The dog ) ( ( is running ) . ) )' \
      --inference_hypothesis '( ( The dog ) ( moves . ) )'

  which will generate an output like the following, due to the semantic consistency of the two sentences.

  Inference logits:
    entailment:     1.101249 (winner)
    contradiction:  -2.374171
    neutral:        -0.296733
  

  By contrast, the following sentence pair:

  python spinn.py --data_root /tmp/spinn-data --logdir /tmp/spinn-logs \
      --inference_premise '( ( The dog ) ( ( is running ) . ) )' \
      --inference_hypothesis '( ( The dog ) ( rests . ) )'

  will give you an output like the following, due to the semantic contradiction of the two sentences.

  Inference logits:
    entailment:     -1.070098
    contradiction:  2.798695 (winner)
    neutral:        -1.402287