Skip to content

An LSTM in PyTorch with best practices (weight dropout, forget bias, etc.) built-in. Fully compatible with PyTorch LSTM.

License

Notifications You must be signed in to change notification settings

keitakurita/Better_LSTM_PyTorch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Better LSTM PyTorch

An LSTM that incorporates best practices, designed to be fully compatible with the PyTorch LSTM API. Implements the following best practices: - Weight dropout - Variational dropout in input and output layers - Forget bias initialization to 1

These best practices are based on the following papers: A Theoretically Grounded Application of Dropout in Recurrent Neural Networks Regularizing and Optimizing LSTM Language Models An Empirical Exploration of Recurrent Network Architectures <https://proceedings.mlr.press/v37/jozefowicz15.pdf>

This code is heavily based on the code from this repository: most of the credit for this work goes to the authors. (All I have done is update the code for PyTorch version 1.0 and repackage it).

Installation

Install via pip.

$ pip install .

Requires PyTorch version 1.0 or higher.

Usage

>>> from better_lstm import LSTM
>>> lstm = LSTM(100, 20, dropoutw=0.2)

About

An LSTM in PyTorch with best practices (weight dropout, forget bias, etc.) built-in. Fully compatible with PyTorch LSTM.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages