Dagster Integration:
Using AWS Athena with Dagster

About this integration

This integration allows you to connect to AWS Athena, a serverless interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Using this integration, you can issue queries to Athena, fetch results, and handle query execution states within your Dagster pipelines.

Installation

pip install dagster-aws

Examples

from dagster import Definitions, asset
from dagster_aws.athena import AthenaClientResource


@asset
def example_athena_asset(athena: AthenaClientResource):
    return athena.get_client().execute_query("SELECT 1", fetch_results=True)


defs = Definitions(
    assets=[example_athena_asset], resources={"athena": AthenaClientResource()}
)

About AWS Athena

AWS Athena is a serverless, interactive query service that allows you to analyze data directly in Amazon S3 using standard SQL. Athena is easy to use; simply point to your data in Amazon S3, define the schema, and start querying using standard SQL. Most results are delivered within seconds. With Athena, there are no infrastructure setups, and you pay only for the queries you run. It scales automatically—executing queries in parallel—so results are fast, even with large datasets and complex queries.