Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: better cortex serve CLI experience #755

Closed
Van-QA opened this issue Jun 24, 2024 · 3 comments
Closed

feat: better cortex serve CLI experience #755

Van-QA opened this issue Jun 24, 2024 · 3 comments
Assignees
Labels
P1: important Important feature / fix type: bug Something isn't working

Comments

@Van-QA
Copy link
Contributor

Van-QA commented Jun 24, 2024

Description:
Running cortex serve returns a 404 error but the error message is not helpful.

Reproduction Steps:

  1. Open a terminal.
  2. Run cortex serve.
  3. Observe the error message

Expected Behavior:
The error message should provide more information on why the command failed.

Environment:
Github codespace
4 cores 16gb RAM

https://discord.com/channels/1107178041848909847/1204617960615055401/1253826168520445996
image

@Van-QA Van-QA added P1: important Important feature / fix type: bug Something isn't working labels Jun 24, 2024
@Van-QA Van-QA added this to the Cortex with CLI milestone Jun 24, 2024
@0xSage
Copy link
Contributor

0xSage commented Jun 24, 2024

I think the problem is that users expect models to be started along with server.

Here's a rough spec for expected behavior:
https://www.notion.so/jan-ai/cortex-serve-2f5293507b094d2abcdf1c0aa809d370?pvs=4

@0xSage 0xSage changed the title bug: Confusing Error Message with Cortex Serve feat: better cortex serve CLI experience Jun 24, 2024
@louis-jan
Copy link
Contributor

louis-jan commented Jul 3, 2024

Updated: There is Playground URL printed out, also for the error information. Regarding model auto start, it will be confused since we support loading multiple models. It's better to support model param instead. Adding...

image image

@louis-jan
Copy link
Contributor

Regarding whether cortex serve should auto-start models, this is not relevant since it's more about smart routing. We allow users to pull and load multiple models at the same time. It's better to do this with certain requests instead of global serve

e.g.
#803

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1: important Important feature / fix type: bug Something isn't working
Projects
Archived in project
Development

No branches or pull requests

3 participants