S3 Adapter Loading Failing

[llama-shepard]

System Info

ghcr.io/predibase/lorax:87412e1

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

Getting Same Error even after providing fix on #234

Docker Logs:
2024-02-11T09:09:19.286855Z INFO lorax_router::loader: router/src/loader.rs:150: adapter s3:s3:https://adapters/gsm8k downloaded
2024-02-11T09:09:19.286896Z INFO lorax_router::queue: router/src/queue.rs:135: set adapter s3:s3:https://adapters/gsm8k status to Downloaded
2024-02-11T09:09:19.293490Z ERROR lorax_client: router/client/src/lib.rs:34: Server error: Can't find 'adapter_config.json' at '/data/models--s3:----adapters--gsm8k/snapshots'
2024-02-11T09:09:19.293526Z INFO lorax_router::loader: router/src/loader.rs:206: FAILED loading adapter s3:s3:https://adapters/gsm8k
2024-02-11T09:09:19.293550Z INFO lorax_router::queue: router/src/queue.rs:135: set adapter s3:s3:https://adapters/gsm8k status to Errored
2024-02-11T09:09:19.293624Z INFO lorax_router::loader: router/src/loader.rs:276: terminating adapter s3:s3:https://adapters/gsm8k loader

Expected behavior

S3 Adapter Loading should work properly

[llama-shepard]

@tgaddair

In /predibase/lorax/blob/main/server/lorax_server/utils/sources/s3.py, Can I change the

bucket, model_id = _get_bucket_and_model_id(model_id)
self.model_id = model_id

into

bucket, _ = _get_bucket_and_model_id(model_id)
self.model_id = model_id

This will use "/data/models--s3:----{bucket}--{model_id}/snapshots" convention instead

[tgaddair]

Thanks for reporting the issue @llama-shepard, and apologies for the errors here. @magdyksaleh is planning to take a look into the S3 codepath this week to fix this and related issues.

[DhruvaBansal00]

Not sure if this helps debug things, but the model is getting downloaded at "/data/models---{model_id}/snapshots" but lorax is trying to read the weights from "/data/models--s3:---{bucket}--{model_id}/snapshots"

[DhruvaBansal00]

#234 is supposed to fix this, but I am still seeing this behavior. Does the latest image have the patch from #234?

[llama-shepard]

#234 is supposed to fix this, but I am still seeing this behavior. Does the latest image have the patch from #234?

I have used ghcr.io/predibase/lorax:87412e1 from CD run of #234

But still getting the same problem. Waiting for fix from @magdyksaleh

[DhruvaBansal00]

#247 should fix this imo. Building the image locally and testing things out.

[DhruvaBansal00]

Slightly unrelated, but I am also concerned that all deployments are failing because of dependency issues: https://github.com/predibase/lorax/actions/workflows/build.yaml. The latest dockerfile doesn't have some of the most recent changes.

[DhruvaBansal00]

Building locally with rust updated to 1.74

[jeffreyftang]

Should have been fixed by #246.