Panic when attempting to mock SelectObjectContentEventStream for unit testing

[thesilentg]

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• I've gone though Developer Guide and API reference
• I've checked AWS Forums and StackOverflow for answers
• I've searched for previous similar issues and didn't find any solution

Describe the bug
I'm using SelectObjectContent in my code. Example below:

	queryResp, err := s.s3.SelectObjectContentWithContext(ctx, queryRequest)
	if err != nil {
		return errors.Wrap(err, "failed to call select object")
	}
	defer func() {
		 _ = queryResp.EventStream.Close()
	}()

As part of unit testing, I would like mock out the S3 client. I've created my own fakeS3Client which conforms to the s3iface.S3API interface. The SelectObjectContentWithContext method returns a fake stream reader which satisfies SelectObjectContentEventStreamReader and a fake iocloser which satisfies io.Closer.

func (f *fakeS3Client) SelectObjectContentWithContext(ctx aws.Context, input *s3.SelectObjectContentInput, opts ...request.Option) (*s3.SelectObjectContentOutput, error) {
        // Details omitted for brevity

	return &s3.SelectObjectContentOutput{
		EventStream: &s3.SelectObjectContentEventStream{
			Reader:       fakeStreamReader{event: &s3.RecordsEvent{Payload: payload}},
			StreamCloser: fakeIOCloser{},
		},
	}, nil
}

However, when trying to run the testing code, there is a panic inside of the deferred queryResp.EventStream.Close(). The cleaned stack trace looks like:

github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi.(*OnceError).Err()
	github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi/error.go:50 
github.com/aws/aws-sdk-go/service/s3.(*SelectObjectContentEventStream).Err()
	github.com/aws/aws-sdk-go/service/s3/api.go:9733 
github.com/aws/aws-sdk-go/service/s3.(*SelectObjectContentEventStream).Close()
	github.com/aws/aws-sdk-go/service/s3/api.go:9714 

Diving into this code more, it appears that SelectObjectContentEventStream has a non-exported field err which is a pointer to eventstreamapi.OnceError. When the deferred queryResp.EventStream.Close() is called, it calls es.err.Err(), which panics for a nil es.err. However, there is no way to set the value of this non-exported field, or call newSelectObjectContentEventStream, as this is also non-exported. What is the expectation here? I see three paths forward:

1. Modify my application code to remove the call to queryResp.EventStream.Close(). Given the documentation for this method, this seems dangerous.
2. Modify my application code to include a deferred recover to catch this during unit testing. Also not ideal, as this is only necessitated because of the internal implementation details of the SelectObjectContentEventStream, which could change at any time
3. (Desired) Update the SelectObjectContentEventStream to validate that ecs.err is non-nil before calling Err()

Version of AWS SDK for Go?
1.33.1

Version of Go (go version)?
go version go1.14.3 darwin/amd64

[diehlaws]

Hi @thesilentg, thanks for bringing this to our attention and I do apologize for the delay in response on our end. We're investigating this behavior, and will update the issue once we have additional information on the matter.

[jasdel]

Thanks for reporting this issue @thesilentg, and providing a detailed debug. I merged in change #3473 which exports the NewSelectObjectContentEventStream constructor function for SelectObjectContentEventStream. This will ensure that any internal state behavior that is needed by the SelectObjectContentEventStream will be initialized.

The constructor also takes functional arguments to configure the Reader and stream closer at the same time.

return &s3.SelectObjectContentOutput{
	EventStream: s3.NewSelectObjectContentEventStream(func(o *s3.SelectObjectContentEventStream) {
		o.Reader = myStreamReader
		o.StreamCloser = myStreamCloser
	}),
}