Batch Inference using Ray and vLLM

[ratnopamc]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

What is the outcome that you are trying to reach?

Current examples mostly showcase online inference.
We need to add example of batch inference on Ray with vLLM.

Describe the solution you would like

Add example of batch inference using RayJob under the JARK-stack blueprint (for GPUs).
Refer to Ray Documentation for an example.

Describe alternatives you have considered

Additional context

[thangalv]

Interested in working on this issue

[ratnopamc]

Thanks @thangalv, assigned to you. Once you're done testing your changes, please raise a PR.