EMPOWERING SCIENTIFIC WORKFLOW RESEARCH

        

WfCommons is a collaborative framework empowering scientific workflow research and development. Our tools analyze workflow execution instances, generate synthetic yet realistic workflows, define benchmark specifications, and pioneer new techniques to optimize execution on complex distributed infrastructures.

• WfFormat: A universal JSON format, ensuring seamless integration across simulators and frameworks supported by WfCommons

• WfInstances: Curated collection of open-access production workflow executions in the standardized WfFormat

• WfChef: Automated creation of synthetic workflow generators by analyzing real workflows to uncover task patterns and performance characteristics

• WfGen: Automated generation of realistic synthetic workflows using recipes to create diverse instances for simulation or benchmark generation

• WfBench: Generate realistic workflow benchmarks with diverse performance characteristics and task dependencies for current workflow systems

• WfSim: Facilitate simulation for developing and evaluating scheduling algorithms and computing platforms, overcoming real-world limitations

When citing WfCommons, please use the following paper. You should also actually read that paper, as it provides a recent and general overview on the framework.

@article{wfcommons,
    title = {{WfCommons: A Framework for Enabling Scientific Workflow Research and Development}},
    author = {Coleman, Taina and Casanova, Henri and Pottier, Loic and Kaushik, Manav and Deelman, Ewa and Ferreira da Silva, Rafael},
    journal = {Future Generation Computer Systems},
    volume = {128},
    number = {},
    pages = {16--27},
    doi = {10.1016/j.future.2021.09.043},
    year = {2022},
}