A scenario-based API testing tool for HTTP/gRPC server.
Scenarigo is a scenario-based API testing tool for HTTP/gRPC server. It is written in Go and provides a plugin feature that enables you to extend by writing Go code. You can write test scenarios as YAML files and executes them.
title: get scenarigo repository
steps:
- title: GET https://api.github.com/repos/zoncoen/scenarigo
vars:
user: zoncoen
repo: scenarigo
protocol: http
request:
method: GET
url: "https://api.github.com/repos/{{vars.user}}/{{vars.repo}}"
expect:
code: OK
body:
name: "{{vars.repo}}"$ go install github.com/zoncoen/scenarigo/cmd/[email protected]Go to the releases page and download the zip file. Unpack the zip file, and put the binary to a directory in your $PATH.
You can download the latest command into the ./scenarigo directory with the following one-liner code. Place the binary ./scenarigo/scenarigo into your $PATH.
$ version=$(curl -s https://api.github.com/repos/zoncoen/scenarigo/releases/latest | jq -r '.tag_name') && \
go_version='go1.19.1' && \
curl -sLJ https://github.com/zoncoen/scenarigo/releases/download/${version}/scenarigo_${version}_${go_version}_$(uname)_$(uname -m).tar.gz -o scenarigo.tar.gz && \
mkdir ./scenarigo && tar -zxvf ./scenarigo.tar.gz -C ./scenarigo && rm scenarigo.tar.gzNotes: If you use the plugin mechanism, the scenarigo command and plugins must be built using the same version of Go.
You can generate a configuration file scenarigo.yaml via the following command.
$ scenarigo config initschemaVersion: config/v1
scenarios: [] # Specify test scenario files and directories.
pluginDirectory: ./gen # Specify the root directory of plugins.
plugins: # Specify configurations to build plugins.
plugin.so: # Map keys specify plugin output file path from the root directory of plugins.
src: ./path/to/plugin # Specify the source file, directory, or "go gettable" module path of the plugin.
output:
verbose: false # Enable verbose output.
colored: false # Enable colored output with ANSI color escape codes. It is enabled by default but disabled when a NO_COLOR environment variable is set (regardless of its value).
report:
json:
filename: ./report.json # Specify a filename for test report output in JSON.
junit:
filename: ./junit.xml # Specify a filename for test report output in JUnit XML format.scenarigo run executes test scenarios based on the configuration file.
schemaVersion: config/v1
scenarios:
- github.yamltitle: get scenarigo repository
steps:
- title: GET https://api.github.com/repos/zoncoen/scenarigo
vars:
user: zoncoen
repo: scenarigo
protocol: http
request:
method: GET
url: "https://api.github.com/repos/{{vars.user}}/{{vars.repo}}"
expect:
code: OK
body:
name: "{{vars.repo}}"$ scenarigo run
ok github.yaml 0.068sYou can see all commands and options by scenarigo help.
scenarigo is a scenario-based API testing tool.
Usage:
scenarigo [command]
Available Commands:
completion Generate the autocompletion script for the specified shell
config manage the scenarigo configuration file
help Help about any command
list list the test scenario files
plugin provide operations for plugins
run run test scenarios
version print scenarigo version
Flags:
-c, --config string specify configuration file path (read configuration from stdin if specified "-")
-h, --help help for scenarigo
--root string specify root directory (default value is the directory of configuration file)
Use "scenarigo [command] --help" for more information about a command.
You can write test scenarios easily in YAML.
A test scenario consists of some steps. A step represents an API request. The scenario steps will be run from top to bottom sequentially.
This simple example has a step that sends a GET request to https://example.com/message.
title: check /message
steps:
- title: GET /message
protocol: http
request:
method: GET
url: https://example.com/messageTo send a query parameter, add it directly to the URL or use the query field.
title: check /message
steps:
- title: GET /message
protocol: http
request:
method: GET
url: https://example.com/message
query:
id: 1You can use other methods to send data to your APIs.
title: check /message
steps:
- title: POST /message
protocol: http
request:
method: POST
url: https://example.com/message
body:
message: helloBy default, Scenarigo will send body data as JSON. If you want to use other formats, set the Content-Type header.
title: check /message
steps:
- title: POST /message
protocol: http
request:
method: POST
url: https://example.com/message
header:
Content-Type: application/x-www-form-urlencoded
body:
message: helloAvailable Content-Type header to encode request body is the following.
application/json(default)text/plainapplication/x-www-form-urlencoded
You can test your APIs by checking responses. If the result differs expected values, Scenarigo aborts the execution of the test scenario and notify the error.
title: check /message
steps:
- title: GET /message
protocol: http
request:
method: GET
url: https://example.com/message
query:
id: 1
expect:
code: OK
header:
Content-Type: application/json; charset=utf-8
body:
id: 1
message: helloScenarigo provides the original template string feature. It enables to store and reuse values in test scenarios.
The vars field defines variables that can be referred by template string like '{{vars.id}}'.
title: check /message
vars:
id: 1
steps:
- title: GET /message
protocol: http
request:
method: GET
url: https://example.com/message
query:
id: '{{vars.id}}'You can define "step" scope variables that can't be accessed from other steps.
title: check /message
steps:
- title: GET /message
vars:
- 1
protocol: http
request:
method: GET
url: https://example.com/message
query:
id: '{{vars[0]}}'Scenarigo has a plugin mechanism that enables you to add new functionalities you need by writing Go code.
This feature is based on Go's standard library plugin, which has the following limitations.
- Supported on Linux, FreeBSD, and macOS only.
- All plugins (and installed
scenarigocommand) must be built with the same version of the Go compiler and dependent packages.
Scenarigo loads built plugins at runtime and accesses any exported variable or function via template string.
See the official document for details of the plugin package.
A Go plugin is a main package with exported variables and functions.
package main
import "time"
var Layout = "2006-01-02"
func Today() string {
return time.Now().Format(Layout)
}You can use the variables and functions via template strings like below in your test scenarios.
{{plugins.date.Layout}}=>"2006-01-02"{{plugins.date.Today()}}=>"2022-02-22"
Scenarigo allows functions to return a value or a value and an error. The template string execution will fail if the function returns a non-nil error.
package main
import "time"
var Layout = "2006-01-02"
func TodayIn(s string) (string, error) {
loc, err := time.LoadLocation(s)
if err != nil {
return "", err
}
return time.Now().In(loc).Format(Layout), nil
}{{plugins.date.TodayIn("UTC")}}=>"2022-02-22"{{plugins.date.TodayIn("INVALID")}}=>failed to execute: {{plugins.date.TodayIn("INVALID")}}: unknown time zone INVALID
Go plugin can be built with go build -buildmode=plugin, but we recommend you use scenarigo plugin build instead. The wrapper command requires go command installed in your machine.
Scenarigo builds plugins according to the configuration.
schemaVersion: config/v1
scenarios:
- scenarios
pluginDirectory: ./gen # Specify the root directory of plugins.
plugins: # Specify configurations to build plugins.
date.so: # Map keys specify plugin output file path from the root directory of plugins.
src: ./plugins/date # Specify the source file, directory, or "go gettable" module path of the plugin..
├── plugins
│ └── date
│ └── main.go
├── scenarigo.yaml
└── scenarios
└── echo.yamlIn this case, the plugin will be built and written to date.so.
$ scenarigo plugin build.
├── gen
│ └── date.so # built plugin
├── plugins
│ └── date
│ ├── go.mod # generated automatically if not exists
│ └── main.go
├── scenarigo.yaml
└── scenarios
└── echo.yamlScenarigo checks the dependent packages of each plugin before building. If the plugins depend on a different version of the same package, Scenarigo overrides go.mod files by the maximum version to avoid the build error.
Now you can use the plugin in test scenarios.
title: echo
plugins:
date: date.so # relative path from "pluginDirectory"
steps:
- title: POST /echo
protocol: http
request:
method: POST
url: 'https://{{env.ECHO_ADDR}}/echo'
body:
message: '{{plugins.date.Today()}}'
expect:
code: 200Scenarigo can download source codes from remote repositories and build it with go get-able module query.
plugins:
uuid.so:
src: github.com/zoncoen-sample/scenarigo-plugins/uuid@latestplugin.RegisterSetup registers a setup function that will be called before running scenario tests once only. If the registered function returns a non-nil function as a second returned value, it will be executed after finished all tests.
package main
import (
"context"
"fmt"
"time"
"github.com/zoncoen/scenarigo/plugin"
secretmanager "cloud.google.com/go/secretmanager/apiv1"
secretmanagerpb "google.golang.org/genproto/googleapis/cloud/secretmanager/v1"
)
const (
projectName = "foo"
)
func init() {
plugin.RegisterSetup(setupClient)
}
var client *secretmanager.Client
func setupClient(ctx *plugin.Context) (*plugin.Context, func(*plugin.Context)) {
var err error
client, err = secretmanager.NewClient(context.Background())
if err != nil {
ctx.Reporter().Fatalf("failed to create secretmanager client: %v", err)
}
return ctx, func(ctx *plugin.Context) {
client.Close()
}
}
func GetSecretString(name string) (string, error) {
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
resp, err := client.AccessSecretVersion(ctx, &secretmanagerpb.AccessSecretVersionRequest{
Name: fmt.Sprintf("projects/%s/secrets/%s/versions/latest", projectName, name),
})
if err != nil {
return "", fmt.Errorf("failed to get secret: %v", err)
}
return string(resp.Payload.Data), nil
}plugins:
setup.so:
src: ./plugins/date # call "setupClient" before running test scenariosSimilarly, plugin.RegisterSetupEachScenario can register a setup function. The registered function will be called before each test scenario that uses the plugin.
package main
import (
"github.com/zoncoen/scenarigo/plugin"
"github.com/google/uuid"
)
func init() {
plugin.RegisterSetupEachScenario(setRunID)
}
func setRunID(ctx *plugin.Context) (*plugin.Context, func(*plugin.Context)) {
return ctx.WithVars(map[string]string{
"runId": uuid.NewString(),
}), nil
}title: echo
plugins:
setup: setup.so # call "setRunID" before running this test scenario
steps:
- title: POST /echo
protocol: http
request:
method: POST
url: 'https://{{env.ECHO_ADDR}}/echo'
header:
Run-Id: '{{vars.runId}}'
body:
message: hello
expect:
code: 200Generally, a step represents sending a request in Scenarigo. However, you can use a Go's function as a step with the plugin.
package main
import (
"github.com/zoncoen/scenarigo/plugin"
"github.com/zoncoen/scenarigo/schema"
)
var Nop = plugin.StepFunc(func(ctx *plugin.Context, step *schema.Step) *plugin.Context {
ctx.Reporter().Log("nop step")
return ctx
})title: nop
plugins:
step: step.so
steps:
- title: nop step
ref: '{{plugins.step.Nop}}'Scenarigo enables you to define a function that takes arguments in YAML for readability. It is called the "Left Arrow Function" since its syntax {{funcName <-}}.
package main
import (
"errors"
"fmt"
"github.com/zoncoen/scenarigo/plugin"
)
var CoolFunc plugin.LeftArrowFunc = &fn{}
type fn struct{}
type arg struct {
Foo string `yaml:"foo"`
Bar string `yaml:"bar"`
Baz string `yaml:"baz"`
}
func (_ *fn) UnmarshalArg(unmarshal func(interface{}) error) (interface{}, error) {
var a arg
if err := unmarshal(&a); err != nil {
return nil, err
}
return &a, nil
}
func (_ *fn) Exec(in interface{}) (interface{}, error) {
a, ok := in.(*arg)
if !ok {
return nil, errors.New("arg must be a arg")
}
return fmt.Sprintf("foo: %s, bar: %s, baz: %s", a.Foo, a.Bar, a.Baz), nil
}title: echo
plugins:
cool: cool.so
steps:
- title: POST /echo
protocol: http
request:
method: POST
url: 'https://{{env.ECHO_ADDR}}/echo'
body:
message:
'{{plugins.cool.CoolFunc <-}}':
foo: 1
bar: 2
baz: 3
expect:
code: 200