burn-transformers

Burn Transformers is an early-stage work-in-progress port of the Huggingface Transformers library from Python to Rust. It endeavors to provide many pretrained models to perform tasks on different modalities such as text, vision, and audio.

🔥 Transformers provides APIs to quickly download and use those pretrained models on a given input, taking advantage of the great tools Huggingface has provided as part of Candle - their minimalist ML framework for Rust - and fine-tune them on your own datasets using the excellent Burn deep learning library in Rust.

⚠️ Alpha Disclaimer NOTE: This library is in early development, and the API may shift rapidly as it evolves. Be advised that this is not yet recommended for Production use.

Install Rust and Cargo by following this guide.

Add this to your Cargo.toml:

    [dependencies]
    bert-transformers = { git = "https://github.com/bkonkle/burn-transformers" }

Once bert-burn is published to crates.io, this project will be able to be published there as well.

To set up a development environment to build this project, first install some helpful tools.

For helpful linting rools, install Clippy

Run it with cargo:

cargo clippy --fix

If you're using VS Code, configure the rust-analyzer plugin to use it (in settings.json):

{
    "rust-analyzer.checkOnSave.command": "clippy"
}

Install pre-commit to automatically set up Git hook scripts.

In Ubuntu, the package to install is pre-commit:

sudo apt install pre-commit

On Mac with Homebrew, the package is also pre-commit:

brew install pre-commit

The cargo-spellcheck utility depends on libclang.

In Ubuntu, the package to install is libclang-dev:

sudo apt install libclang-dev

To use build scripts from the Makefile.toml, install Cargo Make:

cargo install cargo-make

Run "setup" to install some tooling dependencies:

cargo make setup

First, install the outdated command for cargo:

cargo install cargo-outdated

Then, update and check for any major dependency changes:

cargo update
cargo outdated

Licensed under the MIT license (LICENSE or http:https://opensource.org/licenses/MIT).