Incompatibility between JSON Schema validation & OpenAPI specification

[JoeAlamo]

Hey there,

I have been testing this package and I think I have came across an issue which I believe is the result of an incompatibility between the JSON Schema validation being performed, and the OpenAPI specification.

If I have an OpenAPI response with some nullable attributes, I might have that defined like:

...
schema:
  properties:
    first_name:
      type: string
      example: 'Joe'
    last_name:
      type: string
      example: 'Bloggs'
    favourite_colour:
      type: string
      example: 'Red'
      nullable: true

using nullable: true, in accordance with OpenAPI 3.0 spec.

If in my tests, I have a response for that endpoint like the following:

{
  "first_name": "Jane",
  "last_name": "Doe",
  "favourite_colour": null
}

I will receive a Opis\JsonSchema\ValidationError with keyword = "favourite_colour" and keywordArgs = ["expected" => "string", "used" => "null"].

This error is a valid error, from a JsonSchema point of view. To mark something as nullable in JsonSchema, you provide type as an array, such as type: [string, null].

However, whilst that would satisfy JsonSchema, that is invalid according to OpenAPI 3.0 specification.

There may be more cases like that, but that is the only one I have came across.

This feels like quite a fundamental problem. The only solutions I can think of are:

• convert the provided OpenAPI spec to a JSON schema on the fly, and continue using Opis\JsonSchema validation tool
• use an OpenAPI message validator like https://github.com/thephpleague/openapi-psr7-validator instead of Opis\JsonSchema validator

I do hope there is something I've been doing wrong to get these errors but there more I've looked at it, the Json Schema / OpenAPI incompatibility has seemed like the cause!

[hotmeteor]

Ya, very interesting discovery. That's a bit of a inconsistency for sure.

Let me dig into this for a bit and see what I come up with. Thanks.

[hotmeteor]

@JoeAlamo So, there's been extensive discussion on this topic by OpenAPI spec contributors. There's even a proposal to clarify the usage of nullable: https://github.com/OAI/OpenAPI-Specification/blob/master/proposals/003_Clarify-Nullable.md

Feel free to read it all, but the part that pertains to this issue is:

What is the correct translation of a nullable schema from OpenAPI into an equivalent JSON Schema?

A nullable type should translate into a type array with two string elements: the name of the type specified in the Schema Object, and "null".

More simply, using nullable is not really the right way to go, and you should be doing something like this instead:

"type" : ["string", "null"]

[JoeAlamo]

Yep, bit of an ongoing saga!

But therein lines the conundrum - if you modify your OpenAPI 3.0 spec file to have "type": ["string", "null"] then according to OpenAPI Spec 3.0.3 it is invalid - https://swagger.io/specification/#data-types

And any other tooling you are using for OpenAPI will break (SwaggerUI, linters, other doc generation tools etc.). Stuck between a rock and a hard place :/

This is being addressed in OpenAPI 3.1, but that will take some time to be finalised and tooling vendors to offer support (some more info in https://apisyouwonthate.com/blog/openapi-v31-and-json-schema-2019-09 and OAI/OpenAPI-Specification#1977).

The only solutions I can think of to work with OpenAPI 3.0 as is are the ones I listed in the OP

[hotmeteor]

I suppose I could detect if you're using < OAS 3.1 and treat it specially. Would that work?

[JoeAlamo]

I think that could work (perhaps targeting 3.0 specifically? I am not familiar with differences between OpenAPI 3.0 and 2.0, and any JSON Schema things related to that, but assume there may be some). I just don't know how big of a change that would be.

https://github.com/thephpleague/openapi-psr7-validator looks promising, but it seems Laravel and PSR-7 aren't the best of friends - and the types of error messages could be very different to what you get from Opis\JsonSchema

[hotmeteor]

Ya I think I'll stick with Opis\JsonSchema for now and see if I can get this to work in a not too hacky way.

[hotmeteor]

@JoeAlamo

Here's a PR for adding support for nullable for 3.0.x specs. Basically, it checks if that property is set in that version of spec doc and if so it changes the type to be ['string', 'null'] (or whatever the original value was).

It doesn't apply to 3.1.x specs, since nullable is deprecated and you should be sending an array in the first place.

Code of interest: https://github.com/hotmeteor/spectator/pull/20/files#diff-a98ad63f96e6bc68609a93c68c484bd021aee76560a9c5fc3a0f59c5b75359a0R163

[hotmeteor]

@JoeAlamo Do you want to take that PR for a spin and see if it does what you expect?