Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Samples #40

Open
wants to merge 8 commits into
base: inference
Choose a base branch
from
Open

Samples #40

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e0d6a2b
Fix the error when passing load_4bit=True to load_pretrained_model
deece May 24, 2024
b449a0c
Add the sample script from the docs
deece May 24, 2024
b71b9bf
Don't pin bits and bytes, this old version won't cache 4-bit conversions
deece May 24, 2024
6cb8267
No need to pin gradio version
deece May 24, 2024
2cfab9c
Pinning torch versions prevents installation on Python 3.12
deece May 24, 2024
c43cbad
Ignore venv subdir (for Python virtual environments)
deece May 25, 2024
0dc4977
Don't clobber kwargs in load_pretrained_model
deece May 25, 2024
2194392
Add a sample Gradio UI
deece May 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Don't clobber kwargs in load_pretrained_model 
Signed-off-by: Alastair D'Silva <[email protected]>
  • Loading branch information
@deece
deece committed May 26, 2024
commit 0dc4977882d1ac1e8715ef6dea85d3f7bd5d5731
2 changes: 1 addition & 1 deletion llava/model/builder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@


def load_pretrained_model(model_path, model_base, model_name, load_8bit=False, load_4bit=False, device_map="auto", attn_implementation="flash_attention_2", customized_config=None, **kwargs):
kwargs = {"device_map": device_map}
kwargs.update({"device_map": device_map})

if load_8bit:
kwargs["load_in_8bit"] = True
Expand Down