Skip to content
/ NATS Public

Neural Abstractive Text Summarization with Sequence-to-Sequence Models

License

Notifications You must be signed in to change notification settings

tshi04/NATS

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

NATS toolkit

image image image image image

Our new implementation is available at https://github.com/tshi04/LeafNATS. Please check it.

  • Check python2.7 version of NATS from here.
  • This repository is a pytorch implementation of seq2seq models for the following survey:

Tian Shi, Yaser Keneshloo, Naren Ramakrishnan, Chandan K. Reddy

Requirements and Installation

  • Python 3.5.2
  • glob
  • argparse
  • shutil
  • pytorch 1.0

Use following scripts to

Dataset

In this survey, we run an extensive set of experiments with NATS on the following datasets. Here, we provide the link to CNN/Daily Mail dataset and data processing codes for Newsroom and Bytecup2018 datasets.

In the dataset, <s> and </s> is used to separate sentences. <sec> is used to separate summaries and articles. We did not use the json format because it takes more space and be difficult to transfer.

Usuage

  • Training: python main.py

  • Validate: python main.py --task validate

  • Test: python main.py --task beam

  • Rouge: python main.py --task rouge

Features

The NATS is equipped with following features:

  • Attention based seq2seq framework. Encoder and decoder can be LSTM or GRU. The attention scores can be calculated with three different alignment methods.

  • Pointer-generator network.

  • Intra-temporal attention mechanism and intra-decoder attention mechanism.

  • Coverage mechanism.

  • Weight sharing mechanism. Weight sharing mechanism can boost the performance with significantly less parameters.

  • Beam search algorithm. We implemented an efficient beam search algorithm that can also handle cases when batch_size>1.

  • Unknown words replacement. This meta-algorithm can be used along with any attention based seq2seq model. The OOV words in summaries are manually replaced with words in source articles using attention weights.

Experimental results can be found in our survey paper.

Problems and Todos

  • Some models failed during the training after several epochs. For example, on CNN/Daily Mail dataset,
concat + temporal
concat + temporal + attn_decoder
  • We have tried to optimize memory usage, but we are still not quite happy with it.
  • Merge the LSTM and GRU decoders.

About

Neural Abstractive Text Summarization with Sequence-to-Sequence Models

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published