Add AWS CDK support

[vlesierse]

This pull request is intended to gather feedback for supporting AWS CDK with Aspire. It is build upon the work @normj is doing for general AWS with CloudFormation support (#1905 Add AWS CloudFormation Provisioning and SDK Configuration).

AWS CDK helps developers to write Infrastructure as Code using imperative programming languages like (C#, JavaScript/TypeScript, Python, Java, Go). In contrast with using the AWS SDK, CDK synthesized the code to CloudFormation which describes the desired state of the infrastructure and uses CloudFormation to deploy the resources.

Building CDK Stacks

Building CDK stacks with resources in Aspire should feel very familiar.

var builder = DistributedApplication.CreateBuilder(args);

var awssdkConfig = builder.AddAWSSDKConfig().WithProfile("default").WithRegion(RegionEndpoint.EUWest1);

var stack = builder.AddAWSCDKStack("stack").WithReference(awssdkConfig);
var bucket = stack.AddS3Bucket("bucket");

builder.AddProject<Projects.WebApp>("webapp")
    .WithEnvironment("AWS__Resources__BucketName", bucket, b => b.BucketName);

builder.Build().Run();

This library comes with convenient methods for creating common resources like S3 Buckets, DynamoDB Tables, SQS Queues, SNS Topics, Kinesis Stream and Cognito User Pools. All these construct are reference to projects as environment variables or configuration sections. It is also possible to build custom resources in stacks.

var builder = DistributedApplication.CreateBuilder(args);

var awssdkConfig = builder.AddAWSSDKConfig().WithProfile("default").WithRegion(RegionEndpoint.EUWest1);

var stack = builder.AddAWSCDKStack("stack").WithReference(awssdkConfig);
var custom = stack.AddConstruct("custom", scope => new CustomConstruct(scope, "custom"))
    .AddOutput("MyOutput", c => c.OutputId);

builder.AddProject<Projects.WebApp>("webapp")
    .WithReference(custom);

builder.Build().Run();

Using existing CDK Stacks

This PR supports CDK stacks written with C#. A developer can still write their infrastructure using a CDK project and use the CDK CLI to deploy this to their AWS account. One of the challenges developers usually have if to leverage AWS services like SQS & DynamoDB in their local development environment. Tools to simulate these services locally exists, but are usually lacking features. Aspire allows the developer to deploy their CDK stacks and reference them to their application projects. Aspire will provision the resources and hookup the projects with the output values from the stack.

AWS CDK Project

Using CDK with .NET requires you to create a console application. This application contains Stacks which contains AWS resources from S3 Buckets, DynamoDB Tables, Lambda Functions to entire Kubernetes Clusters.

public class WebAppStackProps : StackProps;

public class WebAppStack : Stack
{
    public ITable Table { get; }

    public WebAppStack(Construct scope, string id, WebAppStackProps props)
        : base(scope, id, props)
    {
        Table = new Table(this, "Table", new TableProps
        {
            PartitionKey = new Attribute { Name = "id", Type = AttributeType.STRING },
            BillingMode = BillingMode.PAY_PER_REQUEST
        });
        var imageAsset = new DockerImageAsset(this, "ImageAssets", new DockerImageAssetProps {
            Directory = ".",
            File = "WebApp/Dockerfile"
        });
        var cluster = new Cluster(this, "Cluster");
        var service = new ApplicationLoadBalancedFargateService(this, "Service",
                new ApplicationLoadBalancedFargateServiceProps
                {
                    Cluster = cluster,
                    TaskImageOptions = new ApplicationLoadBalancedTaskImageOptions
                    {
                        Image = ContainerImage.FromDockerImageAsset(imageAsset),
                        ContainerPort = 8080
                    },
                    RuntimePlatform = new RuntimePlatform { CpuArchitecture = CpuArchitecture.ARM64 }
                });
        Table.GrantReadWriteData(service.TaskDefinition.TaskRole);
    }
}

A stack can create CfnOutput output resources to expose attribute values of resources to the outside or expose them through an interface/property. This allows other stacks to reference resources for their configurations.

In the example below you can see the CDK project that allows you to deploy the stacks using the CDK cli (cdk deploy)

using Amazon.CDK;
using WebApp.CDK;

var app = new App();

var dependencies = new WebAppDependenciesStack(app, "WebAppDependenciesStack", new WebAppDependenciesStackProps());
_ = new WebAppApplicationStack(app, "WebAppApplicationStack", new WebAppApplicationStackProps(dependencies.Table));

app.Synth();

Aspire Stack Resource

In order to make Aspire to work with CDK we introduce a StackResource which allows you to create a stack using a StackBuilderDelegate. The reason to create the stack delayed through a delegate is because Aspire is going to take ownership of the stack and provisioning and needs to additional resources for the outputs. Outputs can be defined using a StackOutputAnnotation which will allows you to point to the resource attribute in stack. When the stack get referenced by the ProjectResource/IResourceWithEnviroment it will take the output values and inject them as environment variables, just like the CloudFormationResource does.

var builder = DistributedApplication.CreateBuilder(args);

var awssdkConfig = builder.AddAWSSDKConfig()
    .WithProfile("default")
    .WithRegion(RegionEndpoint.EUWest1);

var stack = builder.AddAWSCDKStack(
    "AspireWebAppStack",
    context => new WebAppStack(context.Application, context.Name, new WebAppStackProps()))
    .AddOutput("TableName", stack => stack.Table.TableName)
    .WithReference(awssdkConfig);

builder.AddProject<Projects.WebApp>("webapp").WithReference(stack);

builder.Build().Run();

Notes

• Using Aspire, the developer doesn't need to install the AWS CDK CLI in order to provision and use CDK resources in their project. However CDK makes use of JSII which is a proxy to the TypeScript code which is used to create CDK. For this reason the Node runtime needs to be installed.
• Currently Aspire isn't able to deploy asset resources like Container Images, custom resources or S3 objects. An error will be shown when stacks require asset deployments.

Microsoft Reviewers: Open in CodeFlow

[ReubenBond]

I like where this is heading! Have you explored the idea of synthesizing the stack from pieces of the Aspire app model? I believe that would be the ideal user experience, since it unifies the two worlds (Aspire app model & CDK stack)

[vlesierse]

@ReubenBond thank you for the feedback! Yes, I started with experimenting of building the stack within Aspire in this branch. My first focus will be to rebase this PR on top of what @normj is merging into the Aspire main branch and add the AddConstruct method to the StackResource builder. It is sort of already supported for the outputs. It basically adds a CfnOutput construct to the stack.

Currently I'm working on the asset deploying (S3, Docker Images) as it is needed to complete the functionality of CDK deployments.

I like the idea to provide some out-of-the-box resources like a Bucket, DynamoDB Table or SQS Queue it we can easily do this with a couple of extensions methods that add the constructs to the stack underneath.

[vlesierse]

Ok, after some busy weeks, I have picked up this PR again. First step is a rebase onto the work of @normj with the CloudFormation support. I have a made a couple of changes their to improve re-usability of the CloudFormation provisioner.
Next step is to get the asset provisioning working and the ability to build stacks out of single constructs.

[vlesierse]

Building a CDK Stack has been made easier by using Construct resources. I still need to annotate the resource states correctly, but in general it works like this...

var stack = builder.AddAWSCDKStack("Stack").WithReference(awsConfig);
var table = stack.AddConstruct("Table", scope => new Table(scope, "Table", new TableProps
{
    PartitionKey = new Attribute { Name = "id", Type = AttributeType.STRING },
    BillingMode = BillingMode.PAY_PER_REQUEST,
    RemovalPolicy = RemovalPolicy.DESTROY
})).WithOutput("TableName", c => c.TableName);

builder.AddProject<Projects.WebApp>("webapp")
    .WithReference(table)
    .WithReference(awsConfig);

[vlesierse]

Next steps could be to compose list of most used resources and make convenient methods for these resources like;

var stack = builder.AddAWSCDKStack("Stack").WithReference(awsConfig);
var table = stack.AddDynamoDBTable("Table");
var queue = stack.AddSQSQueue("Queue");

builder.AddProject<Projects.WebApp>("webapp")
    .WithReference(table)
    .WithReference(queue)
    .WithReference(awsConfig);

[davidfowl]

I suggest looking at the azure package layering as it should be very similar to what AWS needs to do.

[davidfowl]

Seems redundant to pass the same config twice

cc @normj

[normj]

Seems redundant to pass the same config twice

cc @normj

Are you suggesting that if a project is referencing a CloudFormation / CDK construct that the project should infer the AWS SDK Config from the CloudFormation / CDK construct and pass that configuration to the projects? There could be cases later on for AWS SDK config to have a different configuration when configuration is extended for things like retry settings and timeouts but I can see allowing that fallback mechanism if a project doesn't have an explicit SDK reference.

[davidfowl]

but I can see allowing that fallback mechanism if a project doesn't have an explicit SDK reference.

Yep. Exactly this.

The stack has the config in the fallback case.

[vlesierse]

Getting closer to building AWS CDK stacks.

// Create a Distributed Application builder and enable AWS CDK, needed to host the App construct and attach the stack(s)
var builder = DistributedApplication.CreateBuilder(args).WithAWSCDK();

// Setup a configuration for the AWS .NET SDK.
var awsConfig = builder.AddAWSSDKConfig()
    .WithProfile("default")
    .WithRegion(RegionEndpoint.EUWest1);
// Create Stack
var stack = builder.AddStack("Stack").WithReference(awsConfig);
// Add DynamoDB Table to stack
var table = stack
  .AddDynamoDBTable("Table", new TableProps
  {
      PartitionKey = new Attribute { Name = "id", Type = AttributeType.STRING },
      BillingMode = BillingMode.PAY_PER_REQUEST,
      RemovalPolicy = RemovalPolicy.DESTROY
  })
  .AddGlobalSecondaryIndex(new GlobalSecondaryIndexProps {
      IndexName = "OwnerIndex",
      PartitionKey = new Attribute { Name = "owner", Type = AttributeType.STRING },
      SortKey = new Attribute { Name = "ownerSK", Type = AttributeType.STRING },
      ProjectionType = ProjectionType.ALL
  });
// Create project
builder.AddProject<Projects.WebApp>("webapp")
    .WithReference(table); // Reference DynamoDB Table

[djonser]

First of all, I'm greatly looking forward to this. I've myself experimenting with Aspire and AWS CDK.

Separate Project

One thing that comes to mind after your most recent commit. Would it not be better to have the CDK parts remains as a separate project that references Aspire.Hosting.AWS? My thinking is that other first-party and third-party hosting packages can build upon Aspire.Hosting.AWS without having a dependency on CDK. While AWS CDK is my own preference there are other abstractions for CloudFormation.

Provisioning

I'm agree with @ReubenBond in that having the AppHost project both be a CDK Application and AppHost is the ideal user experience, at least for those of us who are already comfortable using the CLI. I currently don't see any issues with these two approaches living side-by-side.

My own implementation (fork of your fork) can be summarized as the following: I create a cdk.json pointing to AppHost.csproj and run cdk [command] like I would in any cdk project. When I debug, I check if cdk cli is running inside builder.AddStack(stack) and if the cli is not running I utilize this.AddAWSCloudFormationStack(stack.StackName). This loads all the stack outputs.

var builder = DistributedApplication.CreateBuilder(args)
    // AwsConfig is used to load outputs from the created stacks
    .WithAWSCDKCLI(awsConfig => awsConfig.WithProfile("default").WithRegion(RegionEndpoint.EUWest1));

var stack = new Stack(builder.App, "MyStack");
builder.AddStack(stack).SetDefaultStack(stack);

var bucket = new Bucket(stack, "Bucket");
var topic = new Topic(stack, "Topic");

builder.AddProject<Projects.MyApp>("app")
    .WithReference(builder.AddAWSSDKConfig().WithProfile("default").WithRegion(RegionEndpoint.EUWest1))
    // All ${Token[TOKEN.*]} are converted to a CfnOutput and uses the "default stack" as scope
    .WithEnvironment("BUCKET_NAME", bucket.BucketName)
    // Also possible to explicitly provide a scope for the CfnOutput
    .WithEnvironment(topic, "TOPIC_ARN", topic.TopicArn)
    // References previously created CfnOutput based on same TOKEN to prevent duplicate outputs
    .WithEnvironment(bucket, "BUCKET_NAME_AGAIN", bucket.BucketName);

// synthesis occurs when the CLI is executing and then returns before builder.Build() runs
builder.SynthOrBuildAndRun();

[vlesierse]

@djonser thank you for your feedback and mentioning some important points.

One thing that comes to mind after your most recent commit. Would it not be better to have the CDK parts remains as a separate project that references Aspire.Hosting.AWS? My thinking is that other first-party and third-party hosting packages can build upon Aspire.Hosting.AWS without having a dependency on CDK. While AWS CDK is my own preference there are other abstractions for CloudFormation.

This is for me still up for debate. At on one hand we would like to have an easy/out-of-the-box experience when you want to use AWS with .NET Aspire which make it easy to discover the available functionalities and API's. The CloudFormation Stack support is foundational for CDK and distributing them over multiple packages might make it harder to pick what is needed for your application. Especially if we would like to introduce convenient resource extensions like builder.AddBucket. On the other hand, CDK comes with different references (JSII, Constructs, AWS.CDK.Lib) which you don't need if you only want to deploy or reference a CloudFormation stack.

I'm agree with @ReubenBond in that having the AppHost project both be a CDK Application and AppHost is the ideal user experience, at least for those of us who are already comfortable using the CLI. I currently don't see any issues with these two approaches living side-by-side.

The current implementation support both the Aspire local development provisioning as provisioning the stack using cdk deploy with the cdk.json as you decribe. In publish mode it will always Synthesize the stack and write the reference to the assets in the manifest, but it does not provision.
cdk.json

{
  "app": "dotnet run -- --publisher manifest --output-path ./aspire-manifest.json",
  ...
}

Unfortunately with this deployment option you only deploy a part of your stack. The Project resources aren't taken into account yet and needs to be deployed differently when you would to publish your project. We are considering to leverage the AWS .NET Deployment Tools for this as Azure uses azd and for Kubernetes aspirate

I love the WithEnvironment syntax as additional option to WithReference for mapping construct information explicitly. 👍

builder.AddProject<Projects.MyApp>("app")
    // All ${Token[TOKEN.*]} are converted to a CfnOutput and uses the "default stack" as scope
    .WithEnvironment(bucket, "BUCKET_NAME", bucket.BucketName)

[djonser]

The CloudFormation Stack support is foundational for CDK and distributing them over multiple packages might make it harder to pick what is needed for your application.

My thinking was if other CloudFormation abstractions that is not CDK-based wanted to provide an Aspire package they could use the Aspire.Hosting.AWS package as a base. Also, Aspire.Hosting.AWS contains IAWSSDKConfig which establishes a way to handle this type of configuration in Aspire. If another package that does not directly deal with provisioning wants to use IAWSSDKConfig that package now also has a reference to CDK.Lib, JSII and Constructs.

Maybe I'm in the minority here having created a bunch of extensions and auto-completion seemingly always wants to pick the wrong IResource (hello API Gateway 😃 ).

I love the WithEnvironment syntax as additional option to WithReference for mapping construct information explicitly.

It really is the convenience extensions and inference that can be created when AWS CDK and Aspire is used together that is exciting. So much that could be derived from the shared context and reduce the amount of explicitly required configuration.

I've been experimenting with a little bit of everything and most recently extensions for Cognito. One thing I'm starting to believe is that if I'm to create constructs via inference (and re-using Aspire resource name as either a construct name or a construct name-prefix) is that it would be good to have a way to manage/track constructs in a way that does not require creation of Aspire Resources.

Regardless of Aspire parent resource the resource name must be unique. CloudFormation have less constraints for the resource name than Aspire have for resource names and having to specify both an Aspire resource name and Construct Id is inconvenient.

var userPoolId = stack.Node.TryGetContext("@acme/userPoolId");
var userPool = builder.ImportUserPoolById(stack, "UserPool", userPoolId, new UserPoolDefaults(/**/));

var api = builder.AddProject<Projects.Api>("api")
    .AsResourceServer("https://localhost", ["read", "write"])
    // Or with explicit scope and prefix for created constructs
    .AsResourceServer(scope, "idPrefix", "https://localhost", ["read", "write"], name: "api");

builder.AddProject<Projects.Worker>("worker")
    .AuthorizeAccess(api, ["read"]);

// Component wires up the Api Project.
builder.AddCognitoUserPoolAuthentication();
// ServiceDefaults used by Worker implements HttpClient parts.
builder.AddServiceDefaults();

[vlesierse]

@normj
Ok, the base of this PR is ready and I'll focus on the (code) documentation, unit tests and adding additional extensions for common AWS resources. (S3, SQS, SNS, Kinesis, etc). Some others to consider are OpenSearch, Elasticache, etc.
I think we also need to have SSM Parameters, Secrets Manager and AppConfig to support the application configuration patterns.

[RussKie]

❗ We can't pull packages from nuget.org.
This will have to be removed before this PR can get merged.

[vlesierse]

@normj, @davidfowl I've added some tests and refactored the provisioning to make is similar to the Azure resource provisioning. This way it will be extensible for future resources and separate clearly the different provisioning methods while facilitating code reuse. Please take another look and if OK, I can remove the draft status and mark this PR ready for review.

[normj]

I made it just through the new provisioning refactoring and added some comments. I'll get to more of later.

Also I see a lot of async methods not having Async suffixes. We should be consistent and have them.

[davidfowl]

@normj @vlesierse Please let us know when the PR is ready and what if any additional review you'd like. There's LOTS of new API here but we're deferring to you a bit for what the shape looks like. I'd love to make sure it looks like the rest of aspire and if there a new idioms that we can surface them for more discussion to see if they are AWS specific or if they should apply more generally.

That's not a reason to block on this PR but just an FYI.

[normj]

@davidfowl The thing I keep wondering about whether it fits with other Aspire hosting components is the extension method that adds the required CDK DI services. Users use the returned cdk type to build their CDK based resources. It is kind of a builder within a builder user experience for combining Aspire and CDK. It does give CDK a clear separation so we don't overload the Aspire builder and cause possible name collision confusion but I'm not sure if this pattern is used in other hosting components.

var cdk = builder.AddAWSCDK("cdk", "AspireStack").WithReference(awsConfig);

// Adds a custom stack and reference constructs as output
var stack = cdk.AddStack("stack", scope => new CustomStack(scope, "AspireStack-stack"));
stack.AddOutput("BucketName", s => s.Bucket.BucketName);

var topic = cdk.AddSNSTopic("topic");
var queue = cdk.AddSQSQueue("queue");

[vlesierse]

@davidfowl @normj The API pattern chosen here is more closely how you build AWS CDK applications, where you add the constructs to a stack. In previous iterations I abstracted away the App and Stack, but it requires some changes to add additional information to for example the DistributedApplicationExecutionContext as I always need the pass the scope when I create the CDK Construct. I'm Ok either way, but I was thinking to get some feedback first.

[normj]

The code needs a lot more comments for maintainers to understand what is all going on. Every pass I do takes a couple hours for me to reverse engineer the patterns that I think are cemented in your head as the implementer.

Also can we get the PR description updated with the current design. You can also add explanations to any design decisions being made. For example the resource parentage code that I see throughout is hard for me to follow without reading an explanation of the big picture what is going on.

I want to reduce the time it takes me to do a pass so I can review the PR more frequent and gets this merged. Now takes a couple hours and it is hard for me to block that time out.

[vlesierse]

@normj This new refactoring simplifies provisioning and reusing the CloudFormation code a lot. There is no need for a separate ICDKResource anymore and you can just simply add stacks. Also StackResource is now inheriting from CloudFormationTemplateResource which results in 99% code reuse and compatibility. I have included documentation in the code as much as possible.

var builder = DistributedApplication.CreateBuilder(args);

// Setup a configuration for the AWS .NET SDK.
var awsConfig = builder.AddAWSSDKConfig()
    .WithProfile("default")
    .WithRegion(RegionEndpoint.EUWest1);

var stack = builder.AddAWSCDKStack("stack").WithReference(awsConfig);

var topic = stack.AddSNSTopic("topic");
var queue = stack.AddSQSQueue("queue");
topic.AddSubscription(queue);

builder.AddProject<Projects.Frontend>("frontend")
    .WithEnvironment("AWS__Resources__ChatTopicArn", topic, t => t.TopicArn);
builder.AddProject<Projects.Processor>("processor")
    .WithEnvironment("AWS__Resources__ChatQueueUrl", queue, q => q.QueueUrl);

builder.Build().Run();

One thing this changes for CloudFormation templates is that the StackName is explicit property now. If it is not provided it default to the resource name with 'Aspire-' as prefix. If we doesn't want to break existing stack I can revert this.