Testing: "samples/" folder in swagger-codegen-generators

[jmini]

For the moment the only integration tests we have are located in the swagger-codegen repo. There are a lot of *.sh and *.bat scripts under bin/ that calls the cli tool and that generate code in samples/. Then each generated project is tested. This is great and this effort should continue.

---

At swagger-codegen-generators level we need something else between unit tests and the mentioned integration tests. We need to see the impact of a change in the Codegen (and in the abstract Codegen like DefaultCodegenConfig) in the generated code (for each supported framework).

I propose following approach:

We create a Unit Test class (say GenerateSampleTest) containing one test case (say generateXxxxTest()) for each kind of generator we want to support (by genarator I mean Codegen class & library combination. For example for JavaClientCodegen we need multiple test cases to support the different libraries). This calls the generator programatically (no dependency on cli tool) and generate code in a subfolder of the samples/ folder at the root of the swagger-codegen-generators repository (similar to the integration test suite in the swagger-codegen repo).

The idea is that after a change, the GenerateSampleTest Test will be called (I guess everybody runs a mvn verify locally). The idea is to control the changes in the samples/ folder and to commit them inside the same pull request.

The spec used as input should be in the openapi: 3.0.1 format. It should also be extended when new buggy pattern are detected to be as complete as possible. Over the last weeks, following issues were fixed for special cases:

• [JAVA] Array of Strings as requestBody of a POST request in OAS3 swagger-codegen#7592
• [JAXRS] Format not respected for primitive datatype Definition models swagger-codegen#7754
• Nested array data type missing import and fails in build swagger-codegen#7812
• jaxrs-cxf-client - Generate error in @ApiResponse when the response class is not specified swagger-codegen#7853
• [Java] Missing import of class OrderEnum when using nested Array in spec.  swagger-codegen#7918
• [JAVA] duplicates fields in the generated code #9

The input spec used by the GenerateSampleTest should contains one example of each cases described in theses bug reports. Yes it will be a monster spec, but this is the point. For more realistic example, we have the integration test suite in swagger-codegen.

At CI (Travis) level, the idea is to break the build if after the execution of the Maven Build (that will run the GenerateSampleTest) the samples/ folder contains changes that are not committed. This way we ensure that the folder is updated with each Pull Request.

---

Before I start to implement it, I would like to know what you think.

cc: @HugoMario, @webron, @wing328