Skip to content

Commit

Permalink
all docs
Browse files Browse the repository at this point in the history
  • Loading branch information
namoray committed Apr 10, 2024
1 parent 3ac3d07 commit 377562c
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 2 deletions.
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,13 @@
# runpod-debug
# runpod-debug


Try to run the load_model.py script. After the initial download of stuff (which is fine), take note of the loading time of the model.

On good hardware, this takes ~5s to load the whole model. On bad hardware (is my theory), it takes 60 seconds. Why?

```bash
python -m venv venv
. venv/bin/activate
pip install -r requirements.txt
python load_model.py
```
3 changes: 2 additions & 1 deletion load_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,6 @@
path, torch_dtype=torch.bfloat16
).to(device)

print("Time to load (should be very small amount of seconds after the initial download):", time.time() - time1)
print("Time to load: ", time.time() - time1)
print("If this was your first time running, run it again please as the download tainted it, or:")
print("Look in your console logs. If you see more than ~2s per step to load the model, there is an issue")

0 comments on commit 377562c

Please sign in to comment.