Switch to using awsinfra for several examples. (pulumi#206)

This better shows how to create an ecs Service that is fully under the control of the user.
tracy-cho · Jan 9, 2019 · e3dd441 · e3dd441
1 parent 71225e5
commit e3dd441
Show file tree

Hide file tree

Showing 20 changed files with 606 additions and 55 deletions.
diff --git a/aws-js-containers/Pulumi.yaml b/aws-js-containers/Pulumi.yaml
@@ -0,0 +1,8 @@
+name: container-quickstart
+description: NGINX container example
+runtime: nodejs
+template:
+ config:
+ aws:region:
+ description: The AWS region to deploy into
+ default: us-west-2
diff --git a/aws-js-containers/README.md b/aws-js-containers/README.md
@@ -0,0 +1,65 @@
+[![Deploy](https://get.pulumi.com/new/button.svg)](https://app.pulumi.com/new)
+
+# Easy container example
+
+Companion to the tutorial [Provision containers on AWS](https://pulumi.io/quickstart/aws-containers.html).
+
+## Prerequisites
+
+To run this example, make sure [Docker](https://docs.docker.com/engine/installation/) is installed and running.
+
+## Running the App
+
+Note: some values in this example will be different from run to run. These values are indicated
+with `***`.
+
+1. Create a new stack:
+
+ ```
+ $ pulumi stack init containers-dev
+ ```
+
+1. Configure Pulumi to use an AWS region that supports Fargate. This is currently only available in `us-east-1`, `us-east-2`, `us-west-2`, and `eu-west-1`:
+
+ ```
+ $ pulumi config set aws:region us-west-2
+ ```
+
+1. Restore NPM modules via `npm install` or `yarn install`.
+
+1. Preview and deploy the app via `pulumi up`. The preview will take a few minutes, as it builds a Docker container. A total of 19 resources are created.
+
+ ```
+ $ pulumi up
+ ```
+
+1. View the endpoint URL, and run curl:
+
+ ```bash
+ $ pulumi stack output
+ Current stack outputs (1)
+ OUTPUT VALUE
+ hostname http:https://***.elb.us-west-2.amazonaws.com
+
+ $ curl $(pulumi stack output hostname)
+ <html>
+ <head><meta charset="UTF-8">
+ <title>Hello, Pulumi!</title></head>
+ <body>
+ <p>Hello, S3!</p>
+ <p>Made with ❤️ with <a href="https://pulumi.com">Pulumi</a></p>
+ </body></html>
+ ```
+
+1. To view the runtime logs from the container, use the `pulumi logs` command. To get a log stream, use `pulumi logs --follow`.
+
+ ```
+ $ pulumi logs --follow
+ Collecting logs for stack container-quickstart-dev since 2018-05-22T14:25:46.000-07:00.
+ 2018-05-22T15:33:22.057-07:00[ pulumi-nginx] 172.31.13.248 - - [22/May/2018:22:33:22 +0000] "GET / HTTP/1.1" 200 189 "-" "curl/7.54.0" "-"
+ ```
+
+## Clean up
+
+To clean up resources, run `pulumi destroy` and answer the confirmation question at the prompt.
+
diff --git a/aws-js-containers/app/Dockerfile b/aws-js-containers/app/Dockerfile
@@ -0,0 +1,2 @@
+FROM nginx
+COPY content /usr/share/nginx/html
diff --git a/aws-js-containers/app/content/favicon.png b/aws-js-containers/app/content/favicon.png
diff --git a/aws-js-containers/app/content/index.html b/aws-js-containers/app/content/index.html
@@ -0,0 +1,7 @@
+<html>
+ <head><meta charset="UTF-8">
+ <title>Hello, Pulumi!</title></head>
+<body>
+ <p>Hello, containers!</p>
+ <p>Made with ❤️ with <a href="https://pulumi.com">Pulumi</a></p>
+</body></html>
diff --git a/aws-js-containers/index.js b/aws-js-containers/index.js
@@ -0,0 +1,20 @@
+const awsx = require("@pulumi/aws-infra");
+
+let cluster = new awsx.ecs.Cluster("example", { });
+let listener= new awsx.elasticloadbalancingv2.NetworkListener("nginx", { port: 80 });
+let service = new awsx.ecs.FargateService("nginx", {
+ cluster,
+ desiredCount: 2,
+ taskDefinitionArgs: {
+ containers: {
+ nginx: {
+ image: awsx.ecs.Image.fromPath("./app"),
+ memory: 512,
+ portMappings: [listener],
+ },
+ },
+ },
+});
+
+// export just the hostname property of the container frontend
+exports.hostname = listener.endpoint().apply(e => `http:https://${e.hostname}`);
diff --git a/aws-js-containers/package.json b/aws-js-containers/package.json
@@ -0,0 +1,9 @@
+{
+ "name": "container-quickstart",
+ "main": "index.js",
+ "dependencies": {
+ "@pulumi/pulumi": "dev",
+ "@pulumi/aws": "dev",
+ "@pulumi/aws-infra": "dev"
+ }
+}
diff --git a/aws-ts-airflow/README.md b/aws-ts-airflow/README.md
@@ -20,12 +20,6 @@ For more information on how to run this example, see: https://pulumi.io/referenc
  $ pulumi config set aws:region us-east-1
  ```
 
-1. Enable ECS auto clustering:
-
- ```
- $ pulumi config set cloud-aws:ecsAutoCluster true
- ```
-
 1. Set the desired RDS password with:
 
  ```

diff --git a/aws-ts-airflow/index.ts b/aws-ts-airflow/index.ts
@@ -1,14 +1,27 @@
 import * as pulumi from "@pulumi/pulumi";
 import * as aws from "@pulumi/aws";
-import * as awscloud from "@pulumi/cloud-aws";
+import * as awsx from "@pulumi/aws-infra";
 
 let config = new pulumi.Config("airflow");
 const dbPassword = config.require("dbPassword");
 
-let securityGroupIds = [ awscloud.getCluster()!.securityGroupId! ];
+let vpc = awsx.ec2.Vpc.getDefault();
+
+// Create a basic cluster and autoscaling group
+let cluster = new awsx.ecs.Cluster("airflow", { vpc });
+let autoScalingGroup = cluster.createAutoScalingGroup("airflow", {
+ templateParameters: {
+ minSize: 20,
+ },
+ launchConfigurationArgs: {
+ instanceType: "t2.xlarge",
+ },
+});
+
+let securityGroupIds = cluster.securityGroups.map(g => g.id);
 
 let dbSubnets = new aws.rds.SubnetGroup("dbsubnets", {
- subnetIds: awscloud.getNetwork().subnetIds,
+ subnetIds: vpc.publicSubnetIds,
 });
 
 let db = new aws.rds.Instance("postgresdb", {
@@ -28,7 +41,7 @@ let db = new aws.rds.Instance("postgresdb", {
 });
 
 let cacheSubnets = new aws.elasticache.SubnetGroup("cachesubnets", {
- subnetIds: awscloud.getNetwork().subnetIds,
+ subnetIds: vpc.publicSubnetIds,
 });
 
 let cacheCluster = new aws.elasticache.Cluster("cachecluster", {
@@ -42,57 +55,80 @@ let cacheCluster = new aws.elasticache.Cluster("cachecluster", {
  securityGroupIds: securityGroupIds,
 });
 
-let environment = {
- "POSTGRES_HOST": db.endpoint.apply(e => e.split(":")[0]),
- "POSTGRES_PASSWORD": dbPassword,
-
- "REDIS_HOST": cacheCluster.cacheNodes.apply(n => n[0].address),
-
- "EXECUTOR": "Celery",
-};
-
-let airflowController = new awscloud.Service("airflowcontroller", {
- containers: {
- "webserver": {
- build: "./airflow-container",
- ports: [{ port: 8080, external: true, protocol: "http" }],
- environment: environment,
- command: [ "webserver" ],
- },
+let hosts = pulumi.all([db.endpoint.apply(e => e.split(":")[0]), cacheCluster.cacheNodes.apply(n => n[0].address)]);
+let environment = hosts.apply(([postgresHost, redisHost]) => [
+ { name: "POSTGRES_HOST", value: postgresHost },
+ { name: "POSTGRES_PASSWORD", value: dbPassword },
+ { name: "REDIS_HOST", value: redisHost },
+ { name: "EXECUTOR", value: "Celery" }
+]);
+
+let airflowControllerListener = new awsx.elasticloadbalancingv2.ApplicationListener("airflowcontroller", {
+ external: true,
+ port: 8080,
+ protocol: "HTTP",
+});
 
- "scheduler": {
- build: "./airflow-container",
- environment: environment,
- command: [ "scheduler" ],
+let airflowController = new awsx.ecs.EC2Service("airflowcontroller", {
+ cluster,
+ desiredCount: 1,
+ taskDefinitionArgs: {
+ containers: {
+ "webserver": {
+ image: awsx.ecs.Image.fromPath("./airflow-container"),
+ portMappings: [airflowControllerListener],
+ environment: environment,
+ command: [ "webserver" ],
+ memory: 128,
+ },
+
+ "scheduler": {
+ image: awsx.ecs.Image.fromPath("./airflow-container"),
+ environment: environment,
+ command: [ "scheduler" ],
+ memory: 128,
+ },
  },
  },
- replicas: 1,
 });
 
-let airflower = new awscloud.Service("airflower", {
- containers: {
- // If the container is named "flower", we create environment variables that start
- // with `FLOWER_` and Flower tries and fails to parse them as configuration.
- "notflower": {
- build: "./airflow-container",
- ports: [{ port: 5555, external: true, protocol: "http" }],
- environment: environment,
- command: [ "flower" ],
+let airflowerListener = new awsx.elasticloadbalancingv2.ApplicationListener("airflower", {
+ port: 5555,
+ external: true,
+ protocol: "HTTP"
+});
+
+let airflower = new awsx.ecs.EC2Service("airflower", {
+ cluster,
+ taskDefinitionArgs: {
+ containers: {
+ // If the container is named "flower", we create environment variables that start
+ // with `FLOWER_` and Flower tries and fails to parse them as configuration.
+ "notflower": {
+ image: awsx.ecs.Image.fromPath("./airflow-container"),
+ portMappings: [airflowerListener],
+ environment: environment,
+ command: [ "flower" ],
+ memory: 128,
+ },
  },
  },
 });
 
-let airflowWorkers = new awscloud.Service("airflowworkers", {
- containers: {
- "worker": {
- build: "./airflow-container",
- environment: environment,
- command: [ "worker" ],
- memory: 1024,
+let airflowWorkers = new awsx.ecs.EC2Service("airflowworkers", {
+ cluster,
+ desiredCount: 3,
+ taskDefinitionArgs: {
+ containers: {
+ "worker": {
+ image: awsx.ecs.Image.fromPath("./airflow-container"),
+ environment: environment,
+ command: [ "worker" ],
+ memory: 1024,
+ },
  },
  },
- replicas: 3,
 });
 
-export let airflowEndpoint = airflowController.defaultEndpoint.apply(e => e.hostname);
-export let flowerEndpoint = airflower.defaultEndpoint.apply(e => e.hostname);
+export let airflowEndpoint = airflowControllerListener.endpoint().apply(e => e.hostname);
+export let flowerEndpoint = airflowerListener.endpoint().apply(e => e.hostname);
diff --git a/aws-ts-airflow/package.json b/aws-ts-airflow/package.json
@@ -4,9 +4,8 @@
  "@types/node": "^10.1.2"
  },
  "dependencies": {
- "@pulumi/aws": "^0.16.4",
- "@pulumi/cloud": "^0.16.2",
- "@pulumi/cloud-aws": "^0.16.2",
- "@pulumi/pulumi": "^0.16.8"
+ "@pulumi/pulumi": "^0.16.9",
+ "@pulumi/aws": "^0.16.5",
+ "@pulumi/aws-infra": "dev"
  }
 }
diff --git a/aws-ts-url-shortener-cache-http/Pulumi.yaml b/aws-ts-url-shortener-cache-http/Pulumi.yaml
@@ -0,0 +1,20 @@
+name: url-shortener-cache-http
+runtime: nodejs
+description: URL shortener with cache and HttpServer
+template:
+ config:
+ aws:region:
+ description: The AWS region to deploy into
+ default: us-west-2
+ cloud:provider:
+ description: The cloud provider
+ default: aws
+ cloud-aws:useFargate:
+ description: Whether to use Fargate-based container compute
+ default: true
+ cloud-aws:functionIncludePaths:
+ description: Directory to include with the uploaded function code
+ default: www
+ redisPassword:
+ description: The Redis password
+ secret: true
diff --git a/aws-ts-url-shortener-cache-http/README.md b/aws-ts-url-shortener-cache-http/README.md
@@ -0,0 +1,72 @@
+[![Deploy](https://get.pulumi.com/new/button.svg)](https://app.pulumi.com/new)
+
+# Serverless URL Shortener with Redis Cache and HttpServer
+
+A sample URL shortener SPA that uses the high-level components. The example shows to combine serverless functions along with containers. This shows that you can create your own higher level
+abstractions for your own use, your team's, or to share with the community using your language's package manager.
+
+## Deploying and running the program
+
+Note: some values in this example will be different from run to run. These values are indicated
+with `***`.
+
+1. Create a new stack:
+
+ ```
+ $ pulumi stack init url-cache-testing
+ ```
+
+1. Configure Pulumi to use an AWS region that supports Fargate, which is currently only available in `us-east-1`, `us-east-2`, `us-west-2`, and `eu-west-1`:
+
+ ```
+ $ pulumi config set aws:region us-west-2
+ ```
+
+1. Set a value for the Redis password. The value can be an encrypted secret, specified with the `--secret` flag. If this flag is not provided, the value will be saved as plaintext in `Pulumi.url-cache-testing.yaml` (since `url-cache-testing` is the current stack name).
+
+ ```
+ $ pulumi config set --secret redisPassword S3cr37Password
+ ```
+
+1. Add the 'www' directory to the uploaded function code so it can be served from the http server:
+
+ ```
+ $ pulumi config set cloud-aws:functionIncludePaths www
+ ```
+
+1. Restore NPM modules via `npm install` or `yarn install`.
+
+1. Compile the program via `tsc` or `npm run build` or `yarn run build`.
+
+1. Preview and run the deployment via `pulumi update`. The operation will take about 5 minutes to complete.
+
+ ```
+ $ pulumi update
+ Previewing stack 'url-cache-testing'
+ ...
+
+ Updating stack 'url-cache-testing'
+ Performing changes:
+
+ #: Resource Type Name
+ 1: pulumi:pulumi:Stack url-shortener-cache-url-
+ ...
+ 49: aws:apigateway:Stage urlshortener
+
+ info: 49 changes performed:
+ + 49 resources created
+ Update duration: ***
+ ```
+
+1. To view the API endpoint, use the `stack output` command:
+
+ ```
+ $ pulumi stack output endpointUrl
+ https://***.us-east-1.amazonaws.com/stage/
+ ```
+
+1. Open this page in a browser and you'll see a single page app for creating and viewing short URLs.
+
+## Clean up
+
+To clean up resources, run `pulumi destroy` and answer the confirmation question at the prompt.