Reference local files, by schema/property id

[brettstack]

I'm currently using the ajv lib and some custom logic to dereference my schemas, and then I found this lib which will probably do a better job. Is there a way to add schemas so this lib is aware of them? My JSON schemas reference by id, so for example in ajv I'm loading all my schemas schemas.forEach(schema => ajv.addSchema(schema)) and then trying to parse the $refs. Is there something similar I can do for this lib? Or is the example in the docs the only way "$ref": "schemas/people/Bruce-Wayne.json"?

Thanks

[JamesMessinger]

I'm not 100% sure what you mean. Can you provide an example?

There's no need to pre-load the schemas, because JSON Schema $Ref Parser automatically finds the $referenced schemas and loads them itself. Does that answer your question?

[brettstack]

I use ids for $ref instead of schemas/common.json#/definitions/email, so my schema needs to know about all the other schemas (I need to load them manually) before parsing since it doesn't know of their location to load them in.

//common.json
{
  "$schema": "http:https://json-schema.org/draft-04/schema#",
  "id": "common",
  "definitions": {
    "email": {
      "id": "email",
      "type": "string",
      "format": "email"
    }
}

//user.json
{
  "$schema": "http:https://json-schema.org/draft-04/schema#",
  "id": "user",
  "type": "object",
  "properties": {
    "email": {
      "$ref": "email"
    }
  }
}

[JamesMessinger]

Ah, I see what you mean. Couple of problems though:

1. I'm not sure that you're using id and $ref correctly here. The JSON Schema specification allows the id keyword to alter the resolution scope of JSON References ($refs). But the resolution scope must still be a valid URI, not just a keyword, like "email".
2. JSON Schema $Ref Parser is an implementation of the JSON Reference specification and the JSON Pointer specification, not the JSON Schema specification. Thus, the id keyword isn't supported, because that's not part of JSON Reference or JSON Pointer.

All that said... I might be willing to add support for the id keyword at some point in the future, if there's enough demand for it. But I want to hold-off on that for now, because it could be very confusing to start supporting part of the JSON Schema specification but not all of it

[gnimmelf]

Sorry for co-opting, but if I have understood correctly;

It would be very nice though, to use the id to dynamically alter the referencing scope of relative $refs prior to parsing.

My scenario is that that the schemas come out of a db mirroring the fs, but the $refs are relative to the fs schema root, not taking into account the diff from the cwd. It is of course easily amended by calculating the full path to the schema to dereference, but it feels awkward to re-read it using file io.

Perhaps to make it optional and off by default, as an extension or an alternative parser/resolver would be viable, and not confuse too much on part of the specifications?

So this is my +1 (for someone) to make it possible.

Thanks for an awesome module.

[Relequestual]

ajv has this sort of functionality.

You can add any number of schemas to the instance of the validator after specifying the subject schema, and then references will resolve based on the $id of other schemas that have been added, before making any HTTP calls.

The use of $id and $ref in the above schema is not exactly how it's supposed to work. The $id must be a URI, and those $ids are not.

URI resolution is HARD to understand! I've tried. It's easier to think about in terms of a link on a web page. If the URL of the page you are on X, and a link on that page is Y, how does that apply? It's the same with $ref resolution in a schema file.

What's needed to resolve this issue?

• There needs to be an interface that allows the user to add JSON Schema files to an index to be used later.
  Preferably, "load in all schemas in this directory and descendant directories"; I have 4 levels of schemas in a spec I've created.

• Any combining or bundling needs to be aware of $id resetting the base URI for that schema.
  I don't know how this is currenly handled, but you are already able to make HTTP requests to resolve schemas, and this could be many layers deep. I'm assuming this already works correctly. I've not checked the tests (I should do that).

• When attempting to transclude a referenced schema, look up the URL to see if it's already included in the index, and if so, use that JSON Schema in stead of making an HTTP request.

Based on the above and some initial digging, the code for loading and storing an index of JSON Schema files could PROBABLY be lifted from ajv, and resolution COULD be achieved by creating a new resolver which is always run first if additional files have been loaded this way.

This also applies schemas which are defined with URNs. Uncommon, but in use.

If you want a sample set of schemas to work with, feel free to use these.

---

A note on termonology:
Strictly speaking, dereferencing is the way in which you determin what a reference is refurring to, because the process involves URI resolution.

Including one schema in another schema is called "transclusion".

Helpful distinction when you're trying to talk about two similar but subtily different events.

[5amfung]

Like @brettstack, I have a set of schema files that are not accessible through http but only accessible from a file directory. These schema are unfortunately referenced by their $id. I know $id is not supported as explained above. But is there a way for me to provide the schema instead of the parser to go downloading it through http? Perhaps a custom parser?

@James-Messinger Any pointer would be very much appreciated.

[Relequestual]

@5amfung It looks like custom resolvers are supported: https://github.com/James-Messinger/json-schema-ref-parser/blob/master/docs/plugins/resolvers.md

If you're willing to write a resolver plugin which replaces the HTTP resolver, please do, and make a PR for it.

An issue with that solution is that schemas which use a mixture of addressable and non addressable URLs would run into problem. Maybe you could add provision in your resolver to only resolve specific domains, otherwise pass through the URL to the standard resolver.

[Relequestual]

If you have time to spend on this, then I'm happy to test and confirm my end, as I have a need, but currently no resource to apply to the issue.

[jayarjo]

@5amfung have you resolved your problem in some way?

Having schemas stored locally is such a natural usage scenario, that I wonder how it hasn't been solved one way or another until now. I was trying to use them in NodeJS app, so simply planned to load them from local dir and validate some data structures.

[philsturgeon]

I think this is part of the greater $id rewrites needed to get them up to date with the latest changes in 2019-09: #145

[philsturgeon]

Let's combine efforts on $id resolution on #145 to avoid duplicating any conversations. I'll take some of Ben's comments over.

[Relequestual]

Let's combine efforts on $id resolution on #145 to avoid duplicating any conversations. I'll take some of Ben's comments over.

The problem for older drafts is different to 2019-09 and above. I'm in the process of writing a blog post about this.