first commit #205
first commit #205
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
| While the nitty-gritty technical details of scaling are handled by Kubernetes, we have the ability to tweak it based on the type of pipeline. To do this, it’s useful to think about the following questions: | ||
|
|
||
|
|
||
|
|
There was a problem hiding this comment.
And 1 question: What's the role of model deployment services here? Does it make sense to mention them? E.g.: If I go with Sagemaker, will sagemaker handle scaling model requests? That incurs a cost but maybe it's worth it?
@izzbizz @ArzelaAscoIi
There was a problem hiding this comment.
Yes. We might want to be a bit careful with not create confusion for the reader, but what do you think about adding a comment about: "Considering an additional model hosting services like sagemaker (or hf inference) might be helpful to spearately scale model inference"
There was a problem hiding this comment.
not 100% sure about the wording
There was a problem hiding this comment.
How about:
The nitty-gritty technical details of scaling are handled by our orchestration tool. Additionally, model hosting services like SageMaker or Hugging Face Inference can be helpful to scale model inference separately. Aside from these automated solutions, we have the ability to tweak the scaling of our pipelines ourselves.
| While the nitty-gritty technical details of scaling are handled by Kubernetes, we have the ability to tweak it based on the type of pipeline. To do this, it’s useful to think about the following questions: | ||
|
|
||
|
|
||
|
|
There was a problem hiding this comment.
Yes. We might want to be a bit careful with not create confusion for the reader, but what do you think about adding a comment about: "Considering an additional model hosting services like sagemaker (or hf inference) might be helpful to spearately scale model inference"
| While the nitty-gritty technical details of scaling are handled by Kubernetes, we have the ability to tweak it based on the type of pipeline. To do this, it’s useful to think about the following questions: | ||
|
|
||
|
|
||
|
|
There was a problem hiding this comment.
not 100% sure about the wording
No description provided.