Skip to content

Commit

Permalink
Examples of testing with Pulumi (pulumi#608)
Browse files Browse the repository at this point in the history
* .NET unit test
* Add an example of PaC-based testing
* Add a TypeScript unit test
* Add a python test
* Add an integration test
  • Loading branch information
mikhailshilkov committed Mar 24, 2020
1 parent d617a14 commit b95e810
Show file tree
Hide file tree
Showing 57 changed files with 2,112 additions and 19 deletions.
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ See the [Pulumi documentation](https://www.pulumi.com/docs/) for more details on
- [Twilio](#twilio)
- [Linode](#linode)
- [Packet](#packet)
- [Testing](#testing)

## AWS

Expand Down Expand Up @@ -358,3 +359,13 @@ Example | Description |
----- | --------- |
[Web Server](packet-py-webserver) | Build a web server on Packet.net.

## Testing

Example | Description |
----- | --------- |
[Unit Tests in TypeScript](testing-unit-ts) | Mock-based unit tests in TypeScript.
[Unit Tests in Python](testing-unit-python) | Mock-based unit tests in Python.
[Unit Tests in Go](testing-unit-go) | Mock-based unit tests in Go.
[Unit Tests in C#](testing-unit-cs) | Mock-based unit tests in C#.
[Testing with Policies](testing-pac-ts) | Tests based on Policy-as-Code in TypeScript.
[Integration Testing in Go](testing-integration) | Deploy-check-destroy tests in Go.
3 changes: 1 addition & 2 deletions aws-cs-fargate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ infrastructure. [`./Infra/Program.cs`](./Infra/Program.cs) defines the project's
## Prerequisites

* [Install Pulumi](https://www.pulumi.com/docs/get-started/install/)
* [Configure Pulumi to Use AWS](https://www.pulumi.com/docs/intro/cloud-providers/aws/setup/) (if your AWS CLI
is configured, this will just work)
* [Configure Pulumi to Use AWS](https://www.pulumi.com/docs/intro/cloud-providers/aws/setup/) (if your AWS CLI is configured, no further changes are required)
* [Install .NET Core 3](https://dotnet.microsoft.com/download)
* [Install Docker](https://docs.docker.com/install/)

Expand Down
3 changes: 1 addition & 2 deletions aws-go-fargate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,7 @@ This example is inspired by [Docker's Getting Started Tutorial](https://docs.doc
## Prerequisites

* [Install Pulumi](https://www.pulumi.com/docs/get-started/install/)
* [Configure Pulumi to Use AWS](https://www.pulumi.com/docs/intro/cloud-providers/aws/setup/) (if your AWS CLI
is configured, this will just work)
* [Configure Pulumi to Use AWS](https://www.pulumi.com/docs/intro/cloud-providers/aws/setup/) (if your AWS CLI is configured, no further changes are required)
* [Install Go](https://golang.org/doc/install)

## Running the Example
Expand Down
3 changes: 1 addition & 2 deletions aws-py-fargate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,7 @@ This example is inspired by [Docker's Getting Started Tutorial](https://docs.doc
## Prerequisites

* [Install Pulumi](https://www.pulumi.com/docs/get-started/install/)
* [Configure Pulumi to Use AWS](https://www.pulumi.com/docs/intro/cloud-providers/aws/setup/) (if your AWS CLI
is configured, this will just work)
* [Configure Pulumi to Use AWS](https://www.pulumi.com/docs/intro/cloud-providers/aws/setup/) (if your AWS CLI is configured, no further changes are required)

## Running the Example

Expand Down
3 changes: 1 addition & 2 deletions aws-ts-hello-fargate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@ databases, and so on.

- [Node.js](https://nodejs.org/en/download/)
- [Download and install the Pulumi CLI](https://www.pulumi.com/docs/get-started/install/)
- [Connect Pulumi with your AWS account](https://www.pulumi.com/docs/intro/cloud-providers/aws/setup/) (if your AWS CLI is
configured, this will just work)
- [Connect Pulumi with your AWS account](https://www.pulumi.com/docs/intro/cloud-providers/aws/setup/) (if your AWS CLI is configured, no further changes are required)

## Running the Example

Expand Down
3 changes: 1 addition & 2 deletions azure-go-webserver-component/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@ defining a `WebServer` class, we can hide many details (see [here](./webserver.t

- [Node.js](https://nodejs.org/en/download/)
- [Download and install the Pulumi CLI](https://www.pulumi.com/docs/get-started/install/)
- [Connect Pulumi with your Azure account](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/) (if your `az` CLI is
configured, this will just work)
- [Connect Pulumi with your Azure account](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/) (if your `az` CLI is configured, no further changes are required)

## Running the App

Expand Down
3 changes: 1 addition & 2 deletions azure-py-vm-scaleset/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ This example provisions a Scale Set of Linux web servers with nginx deployed, co

- [Node.js](https://nodejs.org/en/download/)
- [Download and install the Pulumi CLI](https://www.pulumi.com/docs/get-started/install/)
- [Connect Pulumi with your Azure account](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/) (if your `az` CLI is
configured, this will just work)
- [Connect Pulumi with your Azure account](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/) (if your `az` CLI is configured, no further changes are required)

## Running the App

Expand Down
2 changes: 1 addition & 1 deletion azure-ts-cosmosdb-logicapp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ At the time of writing, there is no native Pulumi resource for defining an API C

- [Node.js](https://nodejs.org/en/download/)
- [Download and install the Pulumi CLI](https://www.pulumi.com/docs/get-started/install/)
- [Connect Pulumi with your Azure account](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/) (if your `az` CLI is configured, this will just work)
- [Connect Pulumi with your Azure account](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/) (if your `az` CLI is configured, no further changes are required)

## Running the App

Expand Down
3 changes: 1 addition & 2 deletions azure-ts-vm-scaleset/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ This example provisions a Scale Set of Linux web servers with nginx deployed, co

- [Node.js](https://nodejs.org/en/download/)
- [Download and install the Pulumi CLI](https://www.pulumi.com/docs/get-started/install/)
- [Connect Pulumi with your Azure account](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/) (if your `az` CLI is
configured, this will just work)
- [Connect Pulumi with your Azure account](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/) (if your `az` CLI is configured, no further changes are required)

## Running the App

Expand Down
3 changes: 1 addition & 2 deletions azure-ts-webserver-component/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@ defining a `WebServer` class, we can hide many details (see [here](./webserver.t

- [Node.js](https://nodejs.org/en/download/)
- [Download and install the Pulumi CLI](https://www.pulumi.com/docs/get-started/install/)
- [Connect Pulumi with your Azure account](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/) (if your `az` CLI is
configured, this will just work)
- [Connect Pulumi with your Azure account](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/) (if your `az` CLI is configured, no further changes are required)

## Running the App

Expand Down
3 changes: 1 addition & 2 deletions azure-ts-webserver/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ This example provisions a Linux web server in an Azure Virtual Machine and gives

- [Node.js](https://nodejs.org/en/download/)
- [Download and install the Pulumi CLI](https://www.pulumi.com/docs/get-started/install/)
- [Connect Pulumi with your Azure account](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/) (if your `az` CLI is
configured, this will just work)
- [Connect Pulumi with your Azure account](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/) (if your `az` CLI is configured, no further changes are required)

## Running the App

Expand Down
4 changes: 4 additions & 0 deletions testing-integration/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.pulumi/
**/bin/
node_modules/

24 changes: 24 additions & 0 deletions testing-integration/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Integration Testing of Pulumi programs in Go

This integration test treats infrastucture deployed by a Pulumi program as a "black box". If deploys the infrastructure, retrieves an endpoint from stack outputs, sends an HTTP request to the endpoint, validates the response, and tears down the infrastructure again.

This test deploys a static website as an AWS S3 bucket and checks that the site is reachable.

## Prerequisites

1. [Install Go](https://golang.org/doc/install).
2. [Install Pulumi](https://www.pulumi.com/docs/get-started/install/).
3. [Configure Pulumi to Use AWS](https://www.pulumi.com/docs/intro/cloud-providers/aws/setup/) (if your AWS CLI is configured, no further changes are required).

## Running the tests

Run the tests:

```
$ go test
...
PASS
ok github.com/pulumi/examples/testing-integration 65.749s
```
118 changes: 118 additions & 0 deletions testing-integration/examples_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
// Copyright 2016-2020, Pulumi Corporation. All rights reserved.

package examples

import (
"fmt"
"io/ioutil"
"net/http"
"os"
"path"
"strings"
"testing"
"time"

"github.com/pulumi/pulumi/pkg/testing/integration"
"github.com/stretchr/testify/assert"
)

func TestS3Website(t *testing.T) {
cwd, err := os.Getwd()
if err != nil {
t.FailNow()
}

test := integration.ProgramTestOptions{
Dir: path.Join(cwd, "program"),
Quick: true,
SkipRefresh: true,
Config: map[string]string{
"aws:region": "us-west-1",
},
ExtraRuntimeValidation: func(t *testing.T, stack integration.RuntimeValidationStackInfo) {
assertHTTPResult(t, "http:https://"+stack.Outputs["websiteUrl"].(string), nil, func(body string) bool {
return assert.Contains(t, body, "Hello, Pulumi!")
})
},
}
integration.ProgramTest(t, &test)
}

func assertHTTPResult(t *testing.T, output interface{}, headers map[string]string, check func(string) bool) bool {
return assertHTTPResultWithRetry(t, output, headers, 5*time.Minute, check)
}

func assertHTTPResultWithRetry(t *testing.T, output interface{}, headers map[string]string, maxWait time.Duration, check func(string) bool) bool {
return assertHTTPResultShapeWithRetry(t, output, headers, maxWait, func(string) bool { return true }, check)
}

func assertHTTPResultShapeWithRetry(t *testing.T, output interface{}, headers map[string]string, maxWait time.Duration,
ready func(string) bool, check func(string) bool) bool {
hostname, ok := output.(string)
if !assert.True(t, ok, fmt.Sprintf("expected `%s` output", output)) {
return false
}

if !(strings.HasPrefix(hostname, "http:https://") || strings.HasPrefix(hostname, "https://")) {
hostname = fmt.Sprintf("http:https://%s", hostname)
}

startTime := time.Now()
count, sleep := 0, 0
for {
now := time.Now()
req, err := http.NewRequest("GET", hostname, nil)
if !assert.NoError(t, err) {
return false
}

for k, v := range headers {
// Host header cannot be set via req.Header.Set(), and must be set
// directly.
if strings.ToLower(k) == "host" {
req.Host = v
continue
}
req.Header.Set(k, v)
}

client := &http.Client{Timeout: time.Second * 10}
resp, err := client.Do(req)
if err == nil && resp.StatusCode == 200 {
if !assert.NotNil(t, resp.Body, "resp.body was nil") {
return false
}

// Read the body
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if !assert.NoError(t, err) {
return false
}

bodyText := string(body)

// Even if we got 200 and a response, it may not be ready for assertion yet - that's specific per test.
if ready(bodyText) {
// Verify it matches expectations
return check(bodyText)
}
}
if now.Sub(startTime) >= maxWait {
fmt.Printf("Timeout after %v. Unable to http.get %v successfully.", maxWait, hostname)
return false
}
count++
// delay 10s, 20s, then 30s and stay at 30s
if sleep > 30 {
sleep = 30
} else {
sleep += 10
}
time.Sleep(time.Duration(sleep) * time.Second)
fmt.Printf("Http Error: %v\n", err)
fmt.Printf(" Retry: %v, elapsed wait: %v, max wait %v\n", count, now.Sub(startTime), maxWait)
}

return false
}
11 changes: 11 additions & 0 deletions testing-integration/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
module github.com/pulumi/examples/testing-integration

go 1.13

require (
github.com/pkg/errors v0.8.1
github.com/pulumi/pulumi v1.13.0
github.com/stretchr/testify v1.4.1-0.20191106224347-f1bd0923b832
)

replace github.com/Azure/go-autorest => github.com/Azure/go-autorest v12.4.3+incompatible
Loading

0 comments on commit b95e810

Please sign in to comment.