Skip to content

Commit

Permalink
set RequireService to true for benchmarks (pulumi#1598)
Browse files Browse the repository at this point in the history
RequireService used to be the default for ProgramTest, in older
pulumi/pkg versions. However when pkg got updated in
pulumi@3d889a9#diff-8af729bc718ea271aecd1ceaae6d16e2119d7dfbe6a21b6e6f793b3b92fc65d1R10,
running against the service started to be hidden behind a flag for
better test isolation.

We do however want to run benchmarks against the service, so set
RequireService to true here to get that behaviour back.

/cc @t0yv0 @justinvp
  • Loading branch information
tgummerer committed Feb 21, 2024
1 parent 7d6a4a6 commit 8fbc061
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions misc/test/performance_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ func TestAccAwsGoS3Folder(t *testing.T) {
return assert.Contains(t, body, "Hello, world!")
})
},

}
test := getAWSBase(t).With(opts)
programTestAsBenchmark(t, benchmark, test)
Expand Down Expand Up @@ -224,6 +223,7 @@ func programTestAsBenchmark(
// measurements.
t.Run("prewarm", func(t *testing.T) {
prewarmOptions := test.With(integration.ProgramTestOptions{
RequireService: true,
SkipRefresh: true,
SkipEmptyPreviewUpdate: true,
SkipExportImport: true,
Expand All @@ -239,7 +239,8 @@ func programTestAsBenchmark(
// Run with --tracing to record measured data.
t.Run("benchmark", func(t *testing.T) {
finalOptions := test.With(bench.ProgramTestOptions()).With(integration.ProgramTestOptions{
NoParallel: true,
RequireService: true,
NoParallel: true,
})
integration.ProgramTest(t, &finalOptions)
})
Expand Down

0 comments on commit 8fbc061

Please sign in to comment.