Merge pull request pulumi#778 from pulumi/zephyrz73/aws-ts-netlify-cm…

…s-and-oauth

Convert Netlify CMS project to an example

aws-ts-netlify-cms-and-oauth/README.md

@@ -0,0 +1,14 @@
+# About the CMS and OAuth
+Netlify CMS web apps and all the templates they have given on [Netlify CMS website](https://www.netlifycms.org/docs/start-with-a-template/) deployed on Netlify and lies inside the target repositories user would like to make change. However, in some case, we do not want the implementation detail of the CMS to locate in the target repositories and we want to deploy it on AWS instead of Netlify. This example shows how to do this.
+
+Both folder has README.md inside them here are some general thoughts:
+
+## ./cms
+- It contains implementation that made the CMS app a stand-alone React App that is not located inside the target repositories. Now it is able to make edits to another target repository that is under the same account. Moreover, the infrastructure deployes the cms app as a static website onto the AWS S3 and use AWS CloudFront to connect to the CDN and Certificate Manger to provide certificate.
+
+## ./cms-oauth
+Because we are deploying the CMS onto the AWS rather than Netlify, we could not use Netlify's Identity Service to retrieve Github tokens to access. Therefore we have build the [External OAuth Client](https://www.netlifycms.org/docs/external-oauth-clients/#header). We made some changes to the existing Golang OAuth Client example to make it work. Also, we deployed it on AWS by specify a Fargate Service and generated its domain and certificate as well.
+
+## How two part fit together
+Both cms and cms-oauth are deployed onto the AWS and have their own domains. In cms configuration yaml file cms/public/config.yml, we specify their domain in the site_domain (cms domain) and base_url (cms-oauth domain) for Neltify CMS to reference.
+See "Development Details" section of cms/README.md

aws-ts-netlify-cms-and-oauth/cms-oauth/.github/workflows/build-and-deploy.yml

@@ -0,0 +1,47 @@
+name: Deploy OAuth Provider
+on:
+ push:
+ branches:
+ - master
+ pull_request:
+ branches:
+ - master
+jobs:
+ deployOAuthProvider:
+ env: 
+ GOPATH: ${{ github.workspace }}
+ name: Install deps and update infrastructure
+ runs-on: ubuntu-latest
+ steps: 
+ - uses: actions/checkout@v2
+
+ - name: Install Pulumi CLI
+ uses: pulumi/action-install-pulumi-cli@releases/v1
+
+ - name: Install pulumi deps
+ run: yarn install --cwd infrastructure
+
+ - name: Preview infrastructure
+ if: ${{ github.event_name == 'pull_request' }}
+ env:
+ # Setting AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN is optional
+ # in case your organization's AWS setup requires it
+ AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+ AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+ AWS_SESSION_TOKEN: ${{ secrets.AWS_SESSION_TOKEN }}
+ PULUMI_STACK: ${{ secrets.PULUMI_STACK}}
+ PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
+ run:
+ pulumi preview --cwd infrastructure -s "${{ secrets.PULUMI_STACK }}"
+
+ - name: Deploy infrastructure
+ if: ${{ github.event_name == 'push' }}
+ env:
+ # Setting AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN is optional
+ # in case your organization's AWS setup requires it
+ AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+ AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+ AWS_SESSION_TOKEN: ${{ secrets.AWS_SESSION_TOKEN }}
+ PULUMI_STACK: ${{ secrets.PULUMI_STACK}}
+ PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
+ run: pulumi up --yes --cwd infrastructure -s "${{ secrets.PULUMI_STACK }}"

aws-ts-netlify-cms-and-oauth/cms-oauth/.gitignore

@@ -0,0 +1,9 @@
+.DS_Store
+Thumbs.db
+
+/.idea
+/.env*
+
+/bin
+/node_modules
+/tmp

aws-ts-netlify-cms-and-oauth/cms-oauth/Dockerfile

@@ -0,0 +1,18 @@
+
+FROM golang:1.13-alpine3.12 as buildenv
+RUN apk add --no-cache build-base git
+
+WORKDIR /go/src/github.com/pulumi/aws-ts-netlify-cms-and-oauth/cms-oauth
+COPY ./ ./
+RUN go mod download
+
+RUN go install ./
+RUN ls /go/bin
+
+FROM alpine:3.12
+
+RUN apk --no-cache add ca-certificates
+
+COPY --from=buildenv /go/bin/aws-ts-netlify-cms-and-oauth /go/bin/aws-ts-netlify-cms-and-oauth
+
+CMD ["/go/bin/netlify-cms-oauth-provider-go", "--logtostderr"]

aws-ts-netlify-cms-and-oauth/cms-oauth/Makefile

@@ -0,0 +1,20 @@
+TARGET_BIN = netlify-cms-oauth-provider
+TARGET_ARCH = amd64
+SOURCE_MAIN = main.go
+LDFLAGS = -s -w
+
+all: build
+
+build: build-darwin build-linux build-windows
+
+build-darwin:
+ CGO_ENABLED=0 GOOS=darwin GOARCH=$(TARGET_ARCH) go build -ldflags "$(LDFLAGS)" -o bin/$(TARGET_BIN)_darwin-amd64 $(SOURCE_MAIN)
+
+build-linux:
+ CGO_ENABLED=0 GOOS=linux GOARCH=$(TARGET_ARCH) go build -ldflags "$(LDFLAGS)" -o bin/$(TARGET_BIN)_linux-amd64 $(SOURCE_MAIN)
+
+build-windows:
+ CGO_ENABLED=0 GOOS=windows GOARCH=$(TARGET_ARCH) go build -ldflags "$(LDFLAGS)" -o bin/$(TARGET_BIN)_windows-amd64.exe $(SOURCE_MAIN)
+
+start:
+ go run $(SOURCE_MAIN)

aws-ts-netlify-cms-and-oauth/cms-oauth/README.md

@@ -0,0 +1,101 @@
+# About the Project
+This OAuth Server Project is connected with CMS project which deploy on AWS S3 rather than on Netlify. In this way, it requires us to create a OAuth Client Server for Netlify CMS. Netlify use the Netlify Identity Service which provides OAuth provider server. Based on [Netlify's instruction](https://www.netlifycms.org/docs/external-oauth-clients/) of customize this step we need to provide our own OAuth client.
+
+In this example, we are using [Netlify CMS's Github backends](https://www.netlifycms.org/docs/github-backend/) for CMS, but the OAuth Provider code enabled more types of backends Bitbucket and Gitlab. If you are using these [backends](https://www.netlifycms.org/docs/backends-overview/), simply update the callback url you are register Github OAuth Applicationc (See step 1 in the Getting Started section) to be https://{{the domain of your OAuth App}}/bitbucket/callback or https://{{the domain of your OAuth App}}/gitlab/callback
+
+## References
+The provider's content code is referencing to the [External OAuth Client example from Netlify CMS](https://www.netlifycms.org/docs/external-oauth-clients/). 
+Here are some reference:
+- @igk1972 [OAuth provider](https://github.com/igk1972/netlify-cms-oauth-provider-go) Thanks to Igor Kuznetsov for writing go code for OAuth Provider and it's frontend in file main.go. We updated the code in these ways:
+ - Now we have set the [Github scope](https://developer.github.com/apps/building-oauth-apps/understanding-scopes-for-oauth-apps/) (which variable specify what kind of access we want) to be public_repo (only permits read and write on the public repo). See line 132 of main.go for setting different scope.
+ - We also changed line 34 which fail to do JSON.stringify() the provided result
+ - We deleted "https://" when it is trying to concate with "/auth" because the environment variable HOST already contain "https://"
+ - We change line 158 the port that OAuth Server is listen to. Now the http is listen and serve for the port of the target group we set as Pulumi stack configuration and environment variable. The default value for the target group is 80 which is port for the local development. You could change the default port by specify the optional Pulumi config targetGroup Port.
+- pulumi's [hello fargate example](https://github.com/pulumi/examples/tree/master/aws-ts-hello-fargate) for connecting to AWS Fargate to adopt Docker setting in cloud
+- pulumi's [static website example](https://github.com/pulumi/examples/tree/master/aws-ts-static-website) for configuring certificate and obtain a subdomain for the provider server
+
+## File Structure
+- ./infrastructure
+ - Pulumi code with setting up AWS Fargate and the configuring certificate and domain
+- ./main.go the code for the provider itself and it's front end
+ - It is fetching the access token sent from Github API using Github's goth library. 
+- .github/workflow contain code for the workflow
+
+## Infrastructure
+The OAuth Client Server was deployed on AWS using Pulumi. The Pulumi code use AWS Certificate Manager to create certificate and validate it. It is using AWS ECS Fargate to read docker image and establish a Fargate Service. Then it is also creating Alias Record on Route53 for the OAuth Server.
+
+### Assume Role (Optional)
+It is recommended that you use an IAM role with more permissions in the _target_ AWS using a token for an IAM user in the _source_ account. To do this, you could refer to the [aws-ts-assume-role example](https://github.com/pulumi/examples/tree/master/aws-ts-assume-role) for more information. The example is available in multiple languages in our [examples repostiory](https://github.com/pulumi/examples).
+
+# Getting Start (Replace content in {{}} with correct informations)
+These steps are now automated using the Github Workflow. If you push to the master or merge a pull request, the OAuth Client Server would be automatically deployed. Open a new branch and push to the branch would only do a pulumi preview where the logs could be check on Github Actions. 
+
+### Step 1. Register OAuth Application in Github and Obtain Key and Secret 
+- Now it is using the OAuth Application in Pulumi's Github organization account
+- Steps are provided using this link https://docs.netlify.com/visitor-access/oauth-provider-tokens/#setup-and-settings
+- For the Home Page Url should be link to cms's website
+- For the Authorization callback URL enter https://{{the domain of your OAuth App}}/github/callback
+
+### Step 2. Fill in the pulumi configuration
+1. Make sure you are on the root directory of this repo.
+
+2. Get into the infrastructure folder and initialize a new stack
+```bash
+$ cd infrastructure
+$ pulumi stack init {{oauth-provider}} # any name you want for your pulumi stack
+```
+
+3. Set AWS Region
+```bash
+$ pulumi config set aws:region us-east-1
+```
+- It has to be set as us-east-1 because ACM certificate must be in the us-east-1 region.
+
+4. Set Target Domain of OAuth Provider
+```bash
+$ pulumi config set netlify-cms-oauth-provider-infrastructure:targetDomain {{"domain name of your oauth provider"}}
+```
+
+5. Set the Github Key and Secret (only do this if you want a personal test, the Github Key and Secret is now provided by the OAuth application in pulumi Github account)
+- change the {YOUR_GITHUB_KEY} and {YOUR_GITHUB_SECRET} with the key and secret obtain from Step 1.
+```bash
+$ pulumi config set netlify-cms-oauth-provider-infrastructure:githubKey {{YOUR_GITHUB_KEY}}
+$ pulumi config set --secret netlify-cms-oauth-provider-infrastructure:githubSecret
+$ {{YOUR_GITHUB_SECRET}}
+```
+- `--secret` tag is used to hash the secret so on the stack configuration yaml file it won't be shown
+- Don't directly append the secret to the command like this ` $ pulumi config set --secret netlify-cms-oauth-provider-infrastructure:githubKey {{YOUR_GITHUB_SECRET}} `
+because it might cause the secret be stored inside the command memory
+ - Only specify the key without name and hit ENTER key, then you are able to type secret on next line(won't actually show the value)
+- To make sure if key and secret is right do
+```bash
+$ pulumi config get netlify-cms-oauth-provider-infrastructure:githubKey
+$ pulumi config get netlify-cms-oauth-provider-infrastructure:githubSecret
+```
+
+### Step 3. Running Infrastructure
+```bash
+$ pulumi up
+```
+
+### Step 4. Config CMS
+You also need to add `base_url` to the backend section of your netlify-cms's config file. 
+
+Go to the cms repo which stores resource for CMS and on file public/config.yml add the base_url line with the oauth provider url
+
+```
+backend:
+ name: github
+ repo: user/repo # Path to your Github repository
+ branch: master # Branch to update
+ base_url: https://xxx # Path to ext auth provider
+```
+
+Then build use 
+```bash
+$ yarn build
+```
+and go to the infrastructure folder and do pulumi up to update changes
+
+
+

aws-ts-netlify-cms-and-oauth/cms-oauth/go.mod

@@ -0,0 +1,12 @@
+module github.com/zephyrz73/netlify-cms-oauth-provider-go
+
+go 1.14
+
+require (
+ github.com/davecgh/go-spew v1.1.1 // indirect
+ github.com/gorilla/pat v1.0.1
+ github.com/markbates/goth v1.64.2
+ github.com/pmezard/go-difflib v1.0.0 // indirect
+ golang.org/x/net v0.0.0-20200707034311-ab3426394381 // indirect
+ google.golang.org/appengine v1.6.6 // indirect
+)

aws-ts-netlify-cms-and-oauth/cms-oauth/go.sum

@@ -0,0 +1,42 @@
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
+github.com/golang/protobuf v1.3.1 h1:YF8+flBXS5eO826T4nzqPrxfhQThhXl0YzfuUPu4SBg=
+github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/gorilla/context v1.1.1 h1:AWwleXJkX/nhcU9bZSnZoi3h/qGYqQAGhq6zZe/aQW8=
+github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg=
+github.com/gorilla/mux v1.6.2 h1:Pgr17XVTNXAk3q/r4CpKzC5xBM/qW1uVLV+IhRZpIIk=
+github.com/gorilla/mux v1.6.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
+github.com/gorilla/pat v0.0.0-20180118222023-199c85a7f6d1/go.mod h1:YeAe0gNeiNT5hoiZRI4yiOky6jVdNvfO2N6Kav/HmxY=
+github.com/gorilla/pat v1.0.1 h1:OeSoj6sffw4/majibAY2BAUsXjNP7fEE+w30KickaL4=
+github.com/gorilla/pat v1.0.1/go.mod h1:YeAe0gNeiNT5hoiZRI4yiOky6jVdNvfO2N6Kav/HmxY=
+github.com/gorilla/securecookie v1.1.1 h1:miw7JPhV+b/lAHSXz4qd/nN9jRiAFV5FwjeKyCS8BvQ=
+github.com/gorilla/securecookie v1.1.1/go.mod h1:ra0sb63/xPlUeL+yeDciTfxMRAA+MP+HVt/4epWDjd4=
+github.com/gorilla/sessions v1.1.1 h1:YMDmfaK68mUixINzY/XjscuJ47uXFWSSHzFbBQM0PrE=
+github.com/gorilla/sessions v1.1.1/go.mod h1:8KCfur6+4Mqcc6S0FEfKuN15Vl5MgXW92AE8ovaJD0w=
+github.com/jarcoal/httpmock v0.0.0-20180424175123-9c70cfe4a1da/go.mod h1:ks+b9deReOc7jgqp+e7LuFiCBH6Rm5hL32cLcEAArb4=
+github.com/lestrrat-go/jwx v0.9.0/go.mod h1:iEoxlYfZjvoGpuWwxUz+eR5e6KTJGsaRcy/YNA/UnBk=
+github.com/markbates/going v1.0.0/go.mod h1:I6mnB4BPnEeqo85ynXIx1ZFLLbtiLHNXVgWeFO9OGOA=
+github.com/markbates/goth v1.64.2 h1:HDFwyuB6/ATU1USTvd/Rb3C9XE0VAxeuciSz+aUZHHA=
+github.com/markbates/goth v1.64.2/go.mod h1:hSFJFfH56BfFCX4+hBIxyd3o5VzuH5rNwKVRsFr/JPk=
+github.com/mrjones/oauth v0.0.0-20180629183705-f4e24b6d100c/go.mod h1:skjdDftzkFALcuGzYSklqYd8gvat6F1gZJ4YPVbkZpM=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
+github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
+golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
+golang.org/x/net v0.0.0-20200707034311-ab3426394381 h1:VXak5I6aEWmAXeQjA+QSZzlgNrpq9mjcfDemuexIKsU=
+golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
+golang.org/x/oauth2 v0.0.0-20180620175406-ef147856a6dd h1:QQhib242ErYDSMitlBm8V7wYCm/1a25hV8qMadIKLPA=
+golang.org/x/oauth2 v0.0.0-20180620175406-ef147856a6dd/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+google.golang.org/appengine v1.6.6 h1:lMO5rYAqUxkmaj76jAkRUvt5JZgFymx/+Q5Mzfivuhc=
+google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=

aws-ts-netlify-cms-and-oauth/cms-oauth/infrastructure/.gitignore

@@ -0,0 +1,2 @@
+/bin/
+/node_modules/

aws-ts-netlify-cms-and-oauth/cms-oauth/infrastructure/Pulumi.yaml

@@ -0,0 +1,3 @@
+name: netlify-cms-oauth-provider-infrastructure
+runtime: nodejs
+description: The infrastructure for the netlify-cms-oauth app