The Dagster platform

Dagster is a next-generation open source orchestration platform for the development, production, and observation of data assets.

Productivity platform
Frictionless end-to-end development workflow for data teams. Easily build, test, deploy, run, and iterate on data pipelines.
Robust orchestration engine
Modern, flexible architecture built to be fault-tolerant.
Unified control plane
The critical “single pane of glass” for your data team: observe, optimize, and debug even the most complex data workflows.
Ready to get started?
Get even more value from Dagster with robust, elastic infrastructure and powerful new capabilities in Dagster+.
Read the docs
Try Dagster+

Dagster as a productivity platform

With Dagster, you can focus on running tasks, or you can identify the key assets you need to create using a declarative approach. Embrace CI/CD best practices from the get-go: build reusable components, spot data quality issues, and flag bugs early.

Software-Defined Assets (SDAs) are a core concept in the Dagster framework. Working with SDAs is not mandatory, but they add a whole new dimension to the orchestration layer. SDAs allow you to:

  • • Manage complexity in your data environment.
  • • Write reusable, low-maintenance code.
  • • Gain greater control and insights across your pipelines and projects.

Make your team more productive and your code easier to maintain

Shared, reusable, configurable data processing components.
Modularized testing and familiar programming model.
Elegant, Pythonic APIs.
Flexible, programmable first class sensors that can be individually monitored.
Robust, fault-tolerant scheduling gives you all the options you need.
A thriving community and full-time engineering support.

Build a maintainable and performant data platform

Deliver high-quality data pipelines rapidly without compromising performance or manageability.
No more interdependencies or crashes
Eliminate lockstep upgrades and irreconcilable dependency conflicts with per-job package dependencies and Python version.
Build quality checks in at every step
Dagster typing ensures high-quality data at each step.
Separate orchestration logic and business logic
Dagster lets you focus on the business logic, not the data hand-offs.

Dagster as a robust orchestration engine

Put your pipelines into production with a robust multi-tenant, multi-tool engine that scales technically and organizationally.

Dagster’s server model eliminates common points of failure

Isolated codebases and environmental requirements, and new processes on each run make Dagster more robust and less prone to failures.

Real-time/event-based, fault-tolerant run scheduling

Dagster’s flexible architecture (K8s, ECS, Docker, etc.) opens up your infrastructure options.
Read the docs

Dagster as a unified control plane

The ‘single pane of glass’ data teams love to use

Rein in the chaos and maintain control over your data as the complexity scales. Centralize your metadata in one tool with built-in observability, diagnostics, cataloging, and lineage. Spot any issues and identify performance improvement opportunities.

Run Timeline

Run Timeline provides a full instance overview with real-time status in one elegant UI.

Launchpad

Easily configure and launch runs with Dagster’s Launchpad and its powerful editor. Start runs as defined or easily experiment with their configurations and observe the result.
Read the docs.

Asset Catalog

Automatically maintain an audit of all assets with the built-in Asset Catalog. Once built and defined, any stakeholder can start, inspect, and manage their own runs.
Read the docs.

Detect issues and act with surgical precision

Asset Graph

Get details on each asset: Freshness, status, schema, metadata, and dependencies displayed in one consolidated view. Model and organize thousands of assets, giving you plenty of room to grow.

Partitions and Backfills

Manage complexity with Partitions and Backfills. Audit historical runs at a granular level.
Read the docs on partitions | Read the docs on backfills

Runs

Observe and assess individual runs and the state of assets generated. Query logs for detailed diagnostics, discover the most time-consuming tasks via a Gantt chart, re-execute subsets of a run, and more.

Automate and integrate your work

Browse all integrations

Automate your efforts with scriptable alerting, asset-level status reports, and Dagster’s fully extensible alerting framework.

Integrate your infrastructure and monitoring tools with the most versatile and extensible platform.

Ready to get started?
Get even more value from Dagster with robust, elastic infrastructure and powerful new capabilities in Dagster+.
Read the docs
Try Dagster+