Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Check for install requirements before downloading models #778

Closed
0xSage opened this issue Jun 25, 2024 · 2 comments
Closed

bug: Check for install requirements before downloading models #778

0xSage opened this issue Jun 25, 2024 · 2 comments
Assignees
Labels
P1: important Important feature / fix type: bug Something isn't working

Comments

@0xSage
Copy link
Contributor

0xSage commented Jun 25, 2024

  1. Users will (often) just do cortex run
  2. Model starts downloading
  3. Takes forever 10mn+
  4. Users is missing dependences like MPI, NVCC, etc.
  5. Many errors. Huge explosion. No way to troubleshoot.
  6. BUG: the model also doesn't seem to get saved in /models
  7. Users have to install deps and redownload.
  8. This is enough for most people to abandon cortex.

Ideal flow.

  1. If users are using trt/onnx models for the first time, check for deps
  2. If missing, guide users through the respective init process...

Question:

  1. Is there a way to check for deps efficiently?
  2. i.e. after users configure everything, maybe theres a flag or something.
  3. not sure - up for discussion
@0xSage 0xSage added P1: important Important feature / fix type: bug Something isn't working labels Jun 25, 2024
@louis-jan
Copy link
Contributor

That's a critical issue, I'm trying to address this. Thank you @0xSage

@louis-jan
Copy link
Contributor

This is addressed, MPI dependency is bundled with the engine by @vansangpfiev . Cuda version check is added by @marknguyen1302. There is a gate to check software & hardware compatibility before downloading model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1: important Important feature / fix type: bug Something isn't working
Projects
Archived in project
Development

No branches or pull requests

3 participants