Skip to content

Commit

Permalink
netlify-cms pulumi deploy example
Browse files Browse the repository at this point in the history
  • Loading branch information
zephyrz73 committed Aug 21, 2020
1 parent 4a51963 commit 0b9c511
Show file tree
Hide file tree
Showing 45 changed files with 1,728 additions and 0 deletions.
14 changes: 14 additions & 0 deletions aws-ts-netlify-cms-and-oauth/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# About the CMS and OAuth
Netlify CMS web apps and all the templates they have given on [Netlify CMS website](https://www.netlifycms.org/docs/start-with-a-template/) deployed on Netlify and lies inside the target repositories user would like to make change. However, in some case, we do not want the implementation detail of the CMS to locate in the target repositories and we want to deploy it on AWS instead of Netlify. This example shows how to do this.

Both folder has README.md inside them here are some general thoughts:

## ./cms
- It contains implementation that made the CMS app a stand-alone React App that is not located inside the target repositories. Now it is able to make edits to another target repository that is under the same account. Moreover, the infrastructure deployes the cms app as a static website onto the AWS S3 and use AWS CloudFront to connect to the CDN and Certificate Manger to provide certificate.

## ./cms-oauth
Because we are deploying the CMS onto the AWS rather than Netlify, we could not use Netlify's Identity Service to retrieve Github tokens to access. Therefore we have build the [External OAuth Client](https://www.netlifycms.org/docs/external-oauth-clients/#header). We made some changes to the existing Golang OAuth Client example to make it work. Also, we deployed it on AWS by specify a Fargate Service and generated its domain and certificate as well.

## How two part fit together
Both cms and cms-oauth are deployed onto the AWS and have their own domains. In cms configuration yaml file cms/public/config.yml, we specify their domain in the site_domain (cms domain) and base_url (cms-oauth domain) for Neltify CMS to reference.
See "Development Details" section of cms/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: Deploy OAuth Provider
on:
push:
branches:
- master
pull_request:
branches:
- master
jobs:
deployOAuthProvider:
env:
GOPATH: ${{ github.workspace }}
name: Install deps and update infrastructure
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2

- name: Install Pulumi CLI
uses: pulumi/action-install-pulumi-cli@releases/v1

- name: Install pulumi deps
run: yarn install --cwd infrastructure

- name: Assume Role
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
IAM_ROLE_ARN: ${{ secrets.IAM_ROLE_ARN }}
EXTERNAL_ID: ${{ secrets.EXTERNAL_ID }}
run: |
unset AWS_SESSION_TOKEN
source ./scripts/assume-role.sh; assume_iam_role "${{secrets.IAM_ROLE_ARN}}" "cmsOAuthSession" "${{secrets.EXTERNAL_ID}}"
echo '::set-env name=AWS_ACCESS_KEY_ID::'$AWS_ACCESS_KEY_ID
echo '::set-env name=AWS_SECRET_ACCESS_KEY::'$AWS_SECRET_ACCESS_KEY
echo '::set-env name=AWS_SESSION_TOKEN::'$AWS_SESSION_TOKEN
- name: Preview infrastructure
if: ${{ github.event_name == 'pull_request' }}
env:
PULUMI_STACK: ${{ secrets.PULUMI_STACK}}
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
run:
pulumi preview --cwd infrastructure -s "${{ secrets.PULUMI_STACK }}"

- name: Deploy infrastructure
if: ${{ github.event_name == 'push' }}
env:
PULUMI_STACK: ${{ secrets.PULUMI_STACK}}
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
run: pulumi up --yes --cwd infrastructure -s "${{ secrets.PULUMI_STACK }}"
9 changes: 9 additions & 0 deletions aws-ts-netlify-cms-and-oauth/cms-oauth/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
.DS_Store
Thumbs.db

/.idea
/.env*

/bin
/node_modules
/tmp
18 changes: 18 additions & 0 deletions aws-ts-netlify-cms-and-oauth/cms-oauth/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@

FROM golang:1.13-alpine3.12 as buildenv
RUN apk add --no-cache build-base git

WORKDIR /go/src/github.com/zephyrz73/netlify-cms-oauth-provider-go
COPY ./ ./
RUN go mod download

RUN go install ./
RUN ls /go/bin

FROM alpine:3.12

RUN apk --no-cache add ca-certificates

COPY --from=buildenv /go/bin/netlify-cms-oauth-provider-go /go/bin/netlify-cms-oauth-provider-go

CMD ["/go/bin/netlify-cms-oauth-provider-go", "--logtostderr"]
20 changes: 20 additions & 0 deletions aws-ts-netlify-cms-and-oauth/cms-oauth/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
TARGET_BIN = netlify-cms-oauth-provider
TARGET_ARCH = amd64
SOURCE_MAIN = main.go
LDFLAGS = -s -w

all: build

build: build-darwin build-linux build-windows

build-darwin:
CGO_ENABLED=0 GOOS=darwin GOARCH=$(TARGET_ARCH) go build -ldflags "$(LDFLAGS)" -o bin/$(TARGET_BIN)_darwin-amd64 $(SOURCE_MAIN)

build-linux:
CGO_ENABLED=0 GOOS=linux GOARCH=$(TARGET_ARCH) go build -ldflags "$(LDFLAGS)" -o bin/$(TARGET_BIN)_linux-amd64 $(SOURCE_MAIN)

build-windows:
CGO_ENABLED=0 GOOS=windows GOARCH=$(TARGET_ARCH) go build -ldflags "$(LDFLAGS)" -o bin/$(TARGET_BIN)_windows-amd64.exe $(SOURCE_MAIN)

start:
go run $(SOURCE_MAIN)
107 changes: 107 additions & 0 deletions aws-ts-netlify-cms-and-oauth/cms-oauth/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# About the Project
This OAuth Server Project is connected with CMS project which deploy on AWS S3 rather than on Netlify. In this way, it requires us to create a OAuth Client Server for Netlify CMS. Netlify use the Netlify Identity Service which provides OAuth provider server. Based on [Netlify's instruction](https://www.netlifycms.org/docs/external-oauth-clients/) of customize this step we need to provide our own OAuth client.

## References
The provider's content code is referencing to the [External OAuth Client example from Netlify CMS](https://www.netlifycms.org/docs/external-oauth-clients/).
Here are some reference:
- @igk1972 [OAuth provider](https://github.com/igk1972/netlify-cms-oauth-provider-go) Thanks to Igor Kuznetsov for writing go code for OAuth Provider and it's frontend in file main.go. We updated the code in these ways:
- Now we have set the [Github scope](https://developer.github.com/apps/building-oauth-apps/understanding-scopes-for-oauth-apps/) (which variable specify what kind of access we want) to be public_repo (only permits read and write on the public repo). See line 132 of main.go for setting different scope.
- We also changed line 34 which fail to do JSON.stringify() the provided result
- We deleted "https://" when it is trying to concate with "/auth" because the environment variable HOST already contain "https://"
- We change line 158 the port that OAuth Server is listen to. Now the http is listen and serve for the port of the target group we set as Pulumi stack configuration and environment variable. The default value for the target group is 80 which is port for the local development. You could change the default port by specify the optional Pulumi config targetGroup Port.
- pulumi's [hello fargate example](https://github.com/pulumi/examples/tree/master/aws-ts-hello-fargate) for connecting to AWS Fargate to adopt Docker setting in cloud
- pulumi's [static website example](https://github.com/pulumi/examples/tree/master/aws-ts-static-website) for configuring certificate and obtain a subdomain for the provider server

## File Structure
- ./infrastructure
- Pulumi code with setting up AWS Fargate and the configuring certificate and domain
- ./main.go the code for the provider itself and it's front end
- It is fetching the access token sent from Github API using Github's goth library.
- .github/workflow contain code for the workflow
- scripts/assume-role.sh the script for assume role into the AWS admin account with IAM user role

## Deployment to AWS
The OAuth Client Server was deployed on AWS using Pulumi. The Pulumi code use AWS Certificate Manager to create certificate and validate it. It is using AWS ECS Fargate to read docker image and establish a Fargate Service. Then it is also creating Alias Record on Route53 for the OAuth Server.

# Getting Start (Replace content in {{}} with correct informations)
These steps are now automated using the Github Workflow. If you push to the master or merge a pull request, the OAuth Client Server would be automatically deployed. Open a new branch and push to the branch would only do a pulumi preview where the logs could be check on Github Actions.

### Step 1. Register OAuth Application in Github and Obtain Key and Secret
- Now it is using the OAuth Application in Pulumi's Github organization account
- Steps are provided using this link https://docs.netlify.com/visitor-access/oauth-provider-tokens/#setup-and-settings
- For the Home Page Url should be link to cms's website
- For the Authorization callback URL enter https://{{the domain of your OAuth App}}/github/callback

### Step 2. Fill in the pulumi configuration
1. Make sure you are on the root directory of this repo.

2. Get into the infrastructure folder and initialize a new stack
```bash
$ cd infrastructure
$ pulumi stack init {{oauth-provider}} # any name you want for your pulumi stack
```

3. Set AWS Region
```bash
$ pulumi config set aws:region us-east-1
```
- It has to be set as us-east-1 because ACM certificate must be in the us-east-1 region.

4. Set Target Domain of OAuth Provider
```bash
$ pulumi config set netlify-cms-oauth-provider-infrastructure:targetDomain {{"domain name of your oauth provider"}}
```

5. Set the Github Key and Secret (only do this if you want a personal test, the Github Key and Secret is now provided by the OAuth application in pulumi Github account)
- change the {YOUR_GITHUB_KEY} and {YOUR_GITHUB_SECRET} with the key and secret obtain from Step 1.
```bash
$ pulumi config set netlify-cms-oauth-provider-infrastructure:githubKey {{YOUR_GITHUB_KEY}}
$ pulumi config set --secret netlify-cms-oauth-provider-infrastructure:githubSecret
$ {{YOUR_GITHUB_SECRET}}
```
- `--secret` tag is used to hash the secret so on the stack configuration yaml file it won't be shown
- Don't directly append the secret to the command like this ` $ pulumi config set --secret netlify-cms-oauth-provider-infrastructure:githubKey {{YOUR_GITHUB_SECRET}} `
because it might cause the secret be stored inside the command memory
- Only specify the key without name and hit ENTER key, then you are able to type secret on next line(won't actually show the value)
- To make sure if key and secret is right do
```bash
$ pulumi config get netlify-cms-oauth-provider-infrastructure:githubKey
$ pulumi config get netlify-cms-oauth-provider-infrastructure:githubSecret
```


6. Don't forget to update AWS token before next step!!!!
If you are working with an organization and assume role is needed:
- you should ask your organization for creating new IAM user and providing credential for the user. Then after set credential as environment variable, assume the role of admin to your companies' production account use this:

```bash
$ source ./scripts/assume-role.sh; assume_iam_role "{{IAM_ROLE_ARN}}" "cmsSession" "{{EXTERNAL_ID}}"
```
IAM_ROLE_ARN and EXTERNAL_ID is the information provided by iam role we should assume. The script file in ./scripts/assume-role.sh contain assume_iam_role function

### Step 3. Running Infrastructure
```bash
$ pulumi up
```

### Step 4. Config CMS
You also need to add `base_url` to the backend section of your netlify-cms's config file.

Go to the cms repo which stores resource for CMS and on file public/config.yml add the base_url line with the oauth provider url

```
backend:
name: github
repo: user/repo # Path to your Github repository
branch: master # Branch to update
base_url: https://xxx # Path to ext auth provider
```

Then build use
```bash
$ yarn build
```
and go to the infrastructure folder and do pulumi up to update changes



12 changes: 12 additions & 0 deletions aws-ts-netlify-cms-and-oauth/cms-oauth/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
module github.com/zephyrz73/netlify-cms-oauth-provider-go

go 1.14

require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/gorilla/pat v1.0.1
github.com/markbates/goth v1.64.2
github.com/pmezard/go-difflib v1.0.0 // indirect
golang.org/x/net v0.0.0-20200707034311-ab3426394381 // indirect
google.golang.org/appengine v1.6.6 // indirect
)
42 changes: 42 additions & 0 deletions aws-ts-netlify-cms-and-oauth/cms-oauth/go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/golang/protobuf v1.3.1 h1:YF8+flBXS5eO826T4nzqPrxfhQThhXl0YzfuUPu4SBg=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/gorilla/context v1.1.1 h1:AWwleXJkX/nhcU9bZSnZoi3h/qGYqQAGhq6zZe/aQW8=
github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg=
github.com/gorilla/mux v1.6.2 h1:Pgr17XVTNXAk3q/r4CpKzC5xBM/qW1uVLV+IhRZpIIk=
github.com/gorilla/mux v1.6.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/pat v0.0.0-20180118222023-199c85a7f6d1/go.mod h1:YeAe0gNeiNT5hoiZRI4yiOky6jVdNvfO2N6Kav/HmxY=
github.com/gorilla/pat v1.0.1 h1:OeSoj6sffw4/majibAY2BAUsXjNP7fEE+w30KickaL4=
github.com/gorilla/pat v1.0.1/go.mod h1:YeAe0gNeiNT5hoiZRI4yiOky6jVdNvfO2N6Kav/HmxY=
github.com/gorilla/securecookie v1.1.1 h1:miw7JPhV+b/lAHSXz4qd/nN9jRiAFV5FwjeKyCS8BvQ=
github.com/gorilla/securecookie v1.1.1/go.mod h1:ra0sb63/xPlUeL+yeDciTfxMRAA+MP+HVt/4epWDjd4=
github.com/gorilla/sessions v1.1.1 h1:YMDmfaK68mUixINzY/XjscuJ47uXFWSSHzFbBQM0PrE=
github.com/gorilla/sessions v1.1.1/go.mod h1:8KCfur6+4Mqcc6S0FEfKuN15Vl5MgXW92AE8ovaJD0w=
github.com/jarcoal/httpmock v0.0.0-20180424175123-9c70cfe4a1da/go.mod h1:ks+b9deReOc7jgqp+e7LuFiCBH6Rm5hL32cLcEAArb4=
github.com/lestrrat-go/jwx v0.9.0/go.mod h1:iEoxlYfZjvoGpuWwxUz+eR5e6KTJGsaRcy/YNA/UnBk=
github.com/markbates/going v1.0.0/go.mod h1:I6mnB4BPnEeqo85ynXIx1ZFLLbtiLHNXVgWeFO9OGOA=
github.com/markbates/goth v1.64.2 h1:HDFwyuB6/ATU1USTvd/Rb3C9XE0VAxeuciSz+aUZHHA=
github.com/markbates/goth v1.64.2/go.mod h1:hSFJFfH56BfFCX4+hBIxyd3o5VzuH5rNwKVRsFr/JPk=
github.com/mrjones/oauth v0.0.0-20180629183705-f4e24b6d100c/go.mod h1:skjdDftzkFALcuGzYSklqYd8gvat6F1gZJ4YPVbkZpM=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20200707034311-ab3426394381 h1:VXak5I6aEWmAXeQjA+QSZzlgNrpq9mjcfDemuexIKsU=
golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/oauth2 v0.0.0-20180620175406-ef147856a6dd h1:QQhib242ErYDSMitlBm8V7wYCm/1a25hV8qMadIKLPA=
golang.org/x/oauth2 v0.0.0-20180620175406-ef147856a6dd/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
google.golang.org/appengine v1.6.6 h1:lMO5rYAqUxkmaj76jAkRUvt5JZgFymx/+Q5Mzfivuhc=
google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/bin/
/node_modules/
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
name: netlify-cms-oauth-provider-infrastructure
runtime: nodejs
description: The infrastructure for the netlify-cms-oauth app
Loading

0 comments on commit 0b9c511

Please sign in to comment.