Added Volcano as CNCF Sandbox.

[k82cn]

This is the proposal to add Volcano to the CNCF.

Name of Project: Volcano
Description:
Volcano is a batch system built on Kubernetes for the above requirements. It provides a suite of mechanisms that are commonly required by many classes of batch & elastic workload including: machine learning/deep learning, bioinformatics/genomics and other "big data" applications. These types of applications typically run on generalized domain frameworks like TensorFlow, Spark, PyTorch, MPI, etc, which Volcano integrates with.

[wpeng102]

+1

[tizhou86]

+1

[lucperkins]

Everyone, please no more “+1”s here. They do not provide useful information and do not make a difference in the TOC’s decisions.

[quinton-hoole]

I am familiar with this project and believe it would be a good addition to the CNCF sandbox to promote industry collaboration towards shared implementations of common extensions to Kubernetes to better support batch AI/ML, big data and other similar batch workloads.

[caniszczyk]

@quinton-hoole have they considered LF AI? There's a scope question here potentially and there may be other organizations that are a better fit: https://landscape.lfai.foundation

[k82cn]

@quinton-hoole have they considered LF AI? There's a scope question here potentially and there may be other organizations that are a better fit: https://landscape.lfai.foundation

The Volcano is a common extensions to Kubernetes for batch workload (A Kubernetes Native Batch System), and its target is to "help batch workload/application to be cloud native"; the batch workload includes not only AI, BigData, but also HPC, e.g. Gene, and others. So, CNCF seems matching our target better :)

[quinton-hoole]

I agree Klaus.

…

On Tue, Nov 12, 2019, 18:35 Klaus Ma ***@***.***> wrote: @quinton-hoole <https://github.com/quinton-hoole> have they considered LF AI? There's a scope question here potentially and there may be other organizations that are a better fit: https://landscape.lfai.foundation The Volcano is a common extensions to Kubernetes for batch workload (A Kubernetes Native Batch System), and its target is to "help batch workload/application to be cloud native"; the batch workload includes not only AI, BigData, but also HPC, e.g. Gene, and others. So, CNCF seems matching our target better :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#318?email_source=notifications&email_token=AKNAA6B53KN4E6J2ZNXW3D3QTNRX3A5CNFSM4JJQVDY2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOED4VHQI#issuecomment-553210817>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKNAA6C3IOINBLL7K3BA7QDQTNRX3ANCNFSM4JJQVDYQ> .

[k82cn]

hi team, is there anything I can help for next step? :)

[caniszczyk]

@k82cn my advice would be to schedule a presentation to one of the CNCF SIGs, I think in this case, App Delivery may be best (e.g., https://github.com/cncf/sig-app-delivery) as I can't think of another SIG that would be better, @quinton-hoole?

[k82cn]

schedule a presentation to one of the CNCF SIGs

Is that the new process? :)

I think in this case, App Delivery may be best (e.g., https://github.com/cncf/sig-app-delivery) as I can't think of another SIG that would be better

hm..., IMO, SIG Runtime seems better for Volcano according to the chart of them; is SIG Runtime ready to do that?

[caniszczyk]

yes we are requiring all projects to present to a SIG first

SIG Runtime could be great but it's not formed yet, I believe we are almost there cc: @amye

[quinton-hoole]

@caniszczyk Yes, I discussed with @amye today, and she's busy tying up some loose ends to bring SIG Runtime to a TOC vote.

[k82cn]

We reviewed it and our recommendation is to accept to Sandbox. The next step is getting sponsors in the TOC. Here are the recommendation and presentation record :)

/cc @raravena80 @amye

[lizrice]

I've just been reviewing the presentation and recommendation. It looks good but I have a couple of questions:

• Are there any similar projects that you're aware of?
• In the presentation there was a good list of companies evaluating their use of Volcano, have any more of them reached a conclusion and decided whether to adopt Volcano since this presentation happened?

[raravena80]

I think maybe @k82cn can better answer these questions but I'd like to pitch in for

Are there any similar projects that you're aware of?

There is Cyclone, K8s native but it seems to focus more on AI.
There is also Flyte, K8s native but it focuses on Machine Learning.

There are also a few similar projects but they are not K8s native. For example, Airflow, Luigi, and Oozie.

Then Peloton is a mixed batch, stateless and stateful jobs scheduler but it mainly works under Mesos.

An interesting feature of Volcano is that it allows for DRF allocation in K8s, which I think no other project implements. Something that the Big Data folks running something like Spark or Flink would remember coming from the Mesos world being the default.

[k82cn]

Are there any similar projects that you're aware of?

I think @raravena80 cover most of the similar projects. Beside DRF allocation, Volcano also provides other features, e.g. Gang, Queue, to run kubeflow, spark and so on better on k8s.

In the presentation there was a good list of companies evaluating their use of Volcano, have any more of them reached a conclusion and decided whether to adopt Volcano since this presentation happened?

Some of them decided to use Volcano + Kubeflow or Spark for their ML/BigData platform according to offline discussion; but it's up to the adopters to decide when update the status to avoid any media communication concerns.

[haosdent]

+1024

[Jeffwan]

I would love to vote for Volcano to join CNCF Sandbox projects.

In ug-machine-learning, volcano is widely adopted by most of the distributed training operator like tf-operator, pytorch-operator, mpi-operator, etc. As Kubernetes default scheduler is not well designed for batch jobs, a lots of features user need like gang-scheduling, binpack are missing. Users from traditional platform plan to move to Kubernetes have to leverage secondary scheduler solutions. Volcano is the most popular one and it solve most of the common problems user have and becomes the essential component to run big data and machine learning workloads on Kubernetes.

[lizrice]

it's up to the adopters to decide when update the status to avoid any media communication concerns

@k82cn is there an Adopters file that they are updating status in? (We don't have a particular gate on usage for Sandbox acceptance, but evidence of adoption could be a persuasive argument to help attract TOC Sponsors.)

[k82cn]

@lizrice , discuss with some adopters about latest status; for now, 3 adopters updated status, e.g. iQiyi (Staging, pre-production), Xiaohongshu.com (Production), HuaweiCloud (Production). It'll take time to get more adopters to update the status; do you think that's enough for a sandbox application?

[k82cn]

BTW, there're also two KubeCon presentations from different companies on use case of Volcano:

• KubeCon 2019 NA: Improving Performance of Deep Learning Workloads With Volcano - Ti Zhou, Baidu Inc
• KubeCon 2020 EU: Improving GPU Utilization with Kubeflow and Volcano at Tencent - Lei Xue, Tencent & Da Ma, Huawei

[lizrice]

I am happy to sponsor Volcano for Sandbox. (Could have sworn I already added a comment to that effect yesterday - maybe I failed to hit the green button!)

Thank you SIG Runtime for your review, and @k82cn for the additional answers

[sheng-liang]

I am happy to sponsor as well.

[k82cn]

@lizrice , @sheng-liang thanks very much !

[happy2048]

I find sig-scheduling may be doing something similar with the new scheduler framework. Is there any relationship or difference ?
framework：https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/20180409-scheduling-framework.md
binpack：https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/20190311-resource_bin_packing_priority_function.md
coscheduling：kubernetes/enhancements#1463

[k82cn]

I find sig-scheduling may be doing something similar with the new scheduler framework. Is there any relationship or difference ?

1. Volcano is a batch system instead of a scheduler; it includes job controller, admission controllers, scheduler and device plugins for batch workload on k8s. That's why k8s steering committees also suggested to donate to CNCF.
2. When Bobby and I built scheduling framework as sig-scheduling co-chair, we found there're still some gaps to support job level scheduling, e.g. fair-scheduling (cross time), queue, reservation/backfill and so on. So we prefer to ask Volcano to unblock batch workload on boarding.
3. In additional, the purpose of scheduling framework is to help user to build customized scheduler; so we may leverage it with batch algorithms as scheduler, and work with job controller, admission and device plugins for batch workload on k8s.

[alena1108]

I am happy to sponsor Volcano for Sandbox. Support for running batch workloads on Kubernetes is in high demand by Big Data/Machine Learning projects. And having Volcano in the Sandbox can foster community collaboration and knowledge sharing in the Cloud Native space.

@k82cn thank you for providing all the extra information! Also great to see GPU topology/share support on 2020 roadmap.

[k82cn]

@alena1108 , thanks very much for your sponsoring :)

[raravena80]

@amye @caniszczyk Volcano now has 3 TOC sponsors for Sandbox. Any next steps for the project team? Thanks.

[amye]

We'll work with them on transferring assets over, after that's complete I will merge this in.

[caniszczyk]

Thanks @amye can you start on the onboarding checklist here:

• Create maintainer list + added to aggregated list
• Website: logo updates
• Domain: transfer domain to CNCF and under LF ITx
• Trademarks: transfer any assets over to the LF if they exist
• Devstats: add to devstats
• Marketing: cncf/artwork repo, slide decks
• Documentation: maintainer docs, project tracker
• Update CNCF Landscape
• Events updates: CFP + Registration + CFP Area
• ServiceDesk: confirm maintainers have read https://www.cncf.io/services-for-projects/
• Welcome Email Sent

[caniszczyk]

hey @k82cn can you confirm you and the maintainers have read https://www.cncf.io/services-for-projects/ :)

[k82cn]

hey @k82cn can you confirm you and the maintainers have read https://www.cncf.io/services-for-projects/ :)

Confirmed. I and the maintainers already read https://www.cncf.io/services-for-projects/ .

[caniszczyk]

Thanks @k82cn, @amye will send out the project welcome email and schedule the monthly project sync with CNCF staff, thanks!