More and complete examples for serialization styles

[slinkydeveloper]

OAS 3 include more complex serialization styles than Swagger 2.0 . Can you include in the spec more complete examples for every in/serialization style/explode/type combination on how a request is supposed to be done? Also where parameters need to be encoded or not. For example:

Path parameters

style	exploded	example path	example uri request	example array uri request	example object uri request
matrix	true	/{color}	/;color=blue	/;color=blue,black,brown	/;color=R,100,G,200,B,150

Cookie parameters

style	exploded	example cookie header	example array cookie header	example object cookie header
form	true	Cookie: color=blue	Cookie: color=blue; color=red; color=green	Cookie: R=100; G=100; B=200

Query parameters

style	exploded	example uri	example array uri	example object uri
form	false	/path?color=blue	/path?color=blue%2Cblack%2Cbrown	/path?color=R%2C100%2CG%2C200%2CB%2C150

Of course this examples can be included outside the spec markdown, but they can be really useful for tool implementers. Thank you

[MikeRalphson]

A good case for a revised Wiki (see OAI/OpenAPI-Specification#1249)

[darrelmiller]

@slinkydeveloper I attempted to make the serialization rules a subset of RFC 6570's rules. https://tools.ietf.org/html/rfc6570#section-3.2.7 The idea was that you should could use an existing URI Template library to do the serialization for you. Just construct a template based on the operation definitions and "ta-da" no work to do. Let me know if there are places I failed to do that. No-one wants to have to re-write the code to deal with the crazy URL serialization rules if they don't have to.

So, if you want to know how a parameter should be serialized, map the styles to prefixes, the explode to the * suffix and the allowReserved to the + prefix and try out the template in your favourite URI Template library.

[slinkydeveloper]

Okay i'll give a try thank you so much!

[hkosova]

I've published serialization examples at https://swagger.io/docs/specification/serialization/. Hope it helps!

[slinkydeveloper]

That's awesome thank you so much!

In the project I'm working on, I've created a "unit test generator" (specific for my project) based on an oas 3 spec. This generator actually takes all operations declared in an oas 3 spec and generates a specific test to validate the correct parsing of parameters on server side. With some small changes this can be a complete server libraries/frameworks compatibility test tool.
For example this is the input of the generator and this is the output
In my case, this tool helped me a lot to check compatibility of my work with oas 3

You are interested to a tool like this?

[MikeRalphson]

So, if you want to know how a parameter should be serialized, map the styles to prefixes, the explode to the * suffix and the allowReserved to the + prefix and try out the template in your favourite URI Template library.

@darrelmiller does RFC6570 specify 'prefixes' (specifically, multiple ones) or a single 'operator'?

If an in:query parameter must use a ? or & operator, how is this squared with allowReserved only applying to in:query parameters where /foo/{+bar} appears to be a valid RFC6570 template, but /foo{?+bar} does not?

[darrelmiller]

@MikeRalphson You can only have one prefix and it causes a bunch of issues. And you hit on one of them there. The only workaround is to make bar a required query parameter.
/foo?bar={+bar}

This means OAS has more flexibility because we can make a query parameter not required and have it allowReserved. To be honest, I couldn't decide if this was a good or a bad thing.

The allowReserved was disallowed from path parameters specifically to prevent the / from introducing new segments, therefore making path matching non-deterministic. Same reason that we don't allow {/foo}

Until we fix the path matching problem we are going to have a mismatch between us and 6570. And we also need to decide if we are going to limit ourselves by what 6570 can't do.

[MikeRalphson]

@darrelmiller thanks, that's helpful.

[jwalton]

@darrelmiller So, if I have a parameter that's {name: 'param', in: 'query', style: 'form', allowReserved: false}, this is essentially the URI Template {?param}, and it matches "param=a%2Cb,c" as ["a,b", "c"].

If I take this same example and change allowReserved: true, then it's essentially {+param}, so "param=a%2Cb,c" now decodes as "a,b,c" (because the "," doesn't turn this into a list anymore). Correct?

If that is correct, it would be nice it this was spelled out a little more explicitly in the spec - the first time I read through, I read the description of allowReserved and through "I'm on the server side - I don't need to worry about this flag; if there's reserved characters in there, they'll already be there by the time it gets to me." :P I didn't realize this actually changed the way the parameter was encoded.

[jwalton]

Oh, wait, you can still have lists with RFC 6570 reserved expansions? How can I parse a comma delimited list when it's allowed to have commas in the values? 😢

[darrelmiller]

@jwalton Sadly you are in fairly uncharted territory here. The allowReserved tells what to do when serializing the parameter values. i.e. don't percent-encode them. I'm not sure what it can tell us about parsing parameter values from a URL.

I have done some work in my UriTemplate libary for parsing comma delimited lists and I just extracted the comma list as single value. See here

I think it is impossible to tell from a template and the final URL whether the input was "1,2,3" or [1,2,3] so we may be forced to extra all parameter values as simple text and then use schema information to translate them into the appropriate types.

BTW, it is awesome that you are digging deep into this stuff. Sorry, it is so painful, but if we can find a clear set of guidance for extracting parameters from URIs based on uri templates that would be great.

[jwalton]

I'm working for a node.js startup in Ottawa. I started looking at OpenAPI 3, then realized there's no node.js support for it and most of the node.js swagger/v2 packages seem to not be actively maintained, so I started looking at 2, then I thought "3 is way better... How hard can it be to write a router and a parameter parser?"

And now I'm finding out. :)

I was debating making allowReserved basically just bypass all the URL template stuff and pass the string value straight through. For a lot of values, you can skip the overhead; like, GET /pet/{petId}, you're never going to have more than one petId in there. If there's a "," in petId you'll probably catch it later when you try to find the pet and it doesn't exist, so you don't really need the overhead of pct-encoding any ","s on the client, then scanning through the petId for ","s on the server to make sure it isn't a list. Kind of makes me wish there was a 'string' style that just passed the value through unmolested. :P

Of course then you couldn't rely on a nice URI Template library to do all the heavy lifting for you... But then the only URI template library I can find for node.js that has support for extracting values hasn't been updated in two years either. -_-

[handrews]

But then the only URI template library I can find for node.js that has support for extracting values hasn't been updated in two years either.

It's a six-year-old standard and it's not that complex, so how many updates would you really expect?

[jwalton]

Well, I wouldn't at all mind if this was merged, or this because they both sound kind of important. :) I suppose I could fork it...

[handrews]

We could ask Geraint if he's going to keep maintaining it or would like to hand it off to someone (meaning you, not me, although I'd help out in a pinch). I'm not sure how much he pays attention to GitHub notifications but he responds to email (at the address on the JSON Schema and Relative JSON Pointer specs)

[handrews]

Since the main question here is more examples, I'm going to un-assign it as I will not have time to add extensive examples in 3.0.4. Although I will leave it in the milestone for now in case someone else feels inspired. This may be better migrated to the learn.openapis.org repo, though.

As for the other questions that came up, they are being tracked in other issues or addressed by a variety of PRs:

• Appendix on RFC6570-derived behavior + allowReserved (3.0.4) OpenAPI-Specification#3818
• Appendix on converting data types to strings (3.0.4) OpenAPI-Specification#3840
• Warnings for header and cookie serialization (3.0.4) OpenAPI-Specification#3841
• Appendix for percent-encoding concerns (3.0.4) OpenAPI-Specification#3859
• How are conflicts between allowReserved and RFC 3986 handled? OpenAPI-Specification#3759
• Should style: form examples include ? prefix? If so, what about spaceDelimited, pipeDelimited, and deepObject? OpenAPI-Specification#3849

[handrews]

We are moving supplementary examples to the learn.openapis.org site, so I am going to transfer this issue so it's easier to see what issues are for the separate examples like these and what are for the examples embedded in the spec. See this issue for the decision:

• Move examples to learn.openapis.org or spec.openapis.org OpenAPI-Specification#3854

Obviously the specification PRs and issues listed in the previous comment will stay open here, and will actually cover a lot of what was asked (most notably a lot of detail about RFC6570 equivalence in OAI/OpenAPI-Specification#3818 and percent-encoding in OAI/OpenAPI-Specification#3859). This is just being moved to the Learn site to inform any further documentation that we can add without the difficulties of fitting it into the spec.

[rafalkrupinski]

In 3.0 do path parameters serialized with matrix or label style need to be mentioned in the path (as ;{param}, but not as {;param}, since that's the RFC notation)?

Not obvious, since they don't look like they're building the path (not separated with /)

[handrews]

@rafalkrupinski OAS path templating just uses names which match as-is to the Parameter Object with the exact name and in: path. Since we reference RFC6570 for style: matrix, the expansion will produce a ; at the beginning of the expanded value. You do not need to put it either inside or outside of the {}.

The only place where the RFC6570 expansion needs modifying is when you use style: form for a form-urlencoded body, where you need to strip off the leading ? (this is now stated directly in the Encoding Object text in 3.0.4 and 3.1.1).

I have not see any questions about this after going through all of the issues from the last several years in detail, so I think it's probably OK. We do not, at any point, talk about RFC6570 for the Path Templating notation, which is described in its own section near the top of the spec.