Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Load LLMs in FP16 for Faster Inference #10

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open

Load LLMs in FP16 for Faster Inference #10

wants to merge 10 commits into from

Commits on Jul 5, 2023

  1. Load LLMs in FP16 for faster inference

    @Kyle1668
    Kyle1668 committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    acbb67f View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2023

  1. Add error handling for when there is an exception in the pple code

    @Kyle1668
    Kyle1668 committed Jul 15, 2023
    Configuration menu
    Copy the full SHA
    85b45e8 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2023

  1. Begin making code parralel

    @Kyle1668
    Kyle1668 committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    5630341 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2023

  1. Checkpoint

    @Kyle1668
    Kyle1668 committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    f4b02c0 View commit details
    Browse the repository at this point in the history

  2. Created configureble batch sizes

    @Kyle1668
    Kyle1668 committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    83f2c2a View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2023

  1. Fix batch size issue

    @Kyle1668
    Kyle1668 committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    96053f7 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2023

  1. Updated files to handle new schema

    @Kyle1668
    Kyle1668 committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    a22fa4c View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2023

  1. Introduce parallel

    @Kyle1668
    Kyle1668 committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    f56cf7a View commit details
    Browse the repository at this point in the history

  2. Begin focusing just on ppl

    @Kyle1668
    Kyle1668 committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    feda73f View commit details
    Browse the repository at this point in the history

  3. Have multiple options

    @Kyle1668
    Kyle1668 committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    f6da96b View commit details
    Browse the repository at this point in the history