References to references resolve relative to self, not baseUrl

[evergreen-lee-campbell]

Given the schema, mySchema.json:

{ "properties": { "thing": { "$ref": "./schemas/thing.json" } } }

Where thing.json is:

{ "properties": { "inner_thing": { "$ref": "./schemas/inner_thing.json" } } }

Attempting to deference mySchema.json at baseUrl json/, results in the dereference function observing the baseUrl for mySchema.json, but not for thing.json, leading to trying to read the file inner_thing.json at: json/schemas/json/schemas/inner_thing.json.

As such, there is no way to resolve schema structure whereby there are schemas that reference either of thing.json or inner_thing.json. Could a resolver option be included that says "always resolve relative to the supplied cwd"?

[manuscriptmastr]

Just noticed this as well:

Folder structure

• {rootDir}
  • node_modules
  • package.json
  • schemas
    • base.json
    • child.json
    • grandchild.json

Schemas

// base.json
{
  "type": "object",
  "properties": {
    "child": {
      "allOf": [{ "$ref": "schemas/child.json" }],
      // ...
    }
  }
}

// child.json
{
  "type": "object",
  "properties": {
    "grandChild": {
      "allOf": [{ "$ref": "schemas/grandchild.json" }],
      // ...
    }
  }
}

// grandchild.json
{
  "type": "object",
  "properties": {
    "firstName": {
      "type": string
    }
  }
}

Expected behavior is that calling $RefParser.dereference() with base.json resolves both $refs to filepaths {rootDir}/schemas/{child|grandchild}.json and returns:

{
  "type": "object",
  "properties": {
    "child": {
      "type": "object",
      "properties": {
        "grandChild": {
          "type": "object",
          "properties": {
            "firstName": {
              "type": "string"
          }
        }
      }
    }
  }
}

Actual behavior: calling $RefParser.dereference correctly resolves {rootDir}/schemas/child.json, but then attempts to resolve filepath {rootDir}/schemas/schemas/grandchild.json, which throws an error:

{
  stack: 'ResolverError: Error opening file "{rootDir}/schemas/schemas/grandchild.json" \n' +
    "ENOENT: no such file or directory, open '{rootDir}/schemas/schemas/grandchild.json'\n" +
    '    at ReadFileContext.callback ({rootDir}/node_modules/@apidevtools/json-schema-ref-parser/lib/resolvers/file.js:52:20)\n' +
    '    at FSReqCallback.readFileAfterOpen [as oncomplete] (fs.js:265:13)',
  code: 'ERESOLVER',
  message: 'Error opening file "{rootDir}/schemas/schemas/grandchild.json" \n' +
    "ENOENT: no such file or directory, open '{rootDir}/schemas/schemas/grandchild.json'",
  source: '{rootDir}/schemas/schemas/grandchild.json',
  path: null,
  toJSON: [Function: toJSON],
  ioErrorCode: 'ENOENT',
  name: 'ResolverError',
  toString: [Function: toString]
}

[MoJo2600]

I can confirm this. Here is a minimal example which shows this behavior.

[JudeMurphy]

Was about to post a long comment about running into this as well, but @manuscriptmastr, seems to have covered it perfectly.

[evergreen-lee-campbell]

FWIW, I ended up with the following workaround, defining a resolver which is tried in the event that the file parser fails:

const globalParserOptions: Options = {
    continueOnError: true,
    dereference: {
        circular: true
    },
    resolve: {
        file: fixedRootDirectoryResolver
    }
};

and:

const fixedRootDirectoryResolver: ResolverOptions = {
    order: 101,
    canRead: true,
    async read(file: FileInfo) {
        let schema = await dereference(
            globalBaseUrl,
            JSON.parse(fs.readFileSync(file.url).toString('utf8')),
            globalParserOptions
        );

        return JSON.stringify(schema, null, 4);
    }
};

Such that it just calls "dereference" again, with the existing options, thus resolving from the original root directory.

[MoJo2600]

I will definitely try this as a workaround @evergreen-lee-campbell. It would solve a lot of issues for me.

[shennan]

Will there be a fix to this or is this considered expected behaviour?

[jmm]

@evergreen-lee-campbell when you say...

at baseUrl json/

...do you mean you're calling it like dereference(baseUrl, schema, options) (which unfortunately doesn't seem to be documented)? And if so, what are you passing as baseUrl?

In any case, my expectation would be for that to be used as the base URI for the top-level schema document and then for relative refs from that schema to result in different base URIs for the referenced schemas.

Using @manuscriptmastr's example:

{rootDir}/
├── node_modules
├── package.json
└── schemas/
    ├── base.json
    ├── child.json
    └── grandchild.json

(Diagram courtesy of https://github.com/nfriend/tree-online)

For purposes of simplification let's say the base is specified as an actual URI file:https:///rootDir. This is what I would expect:

File: schemas/base.json
Base URI: file:https:///rootDir (because it was explicitly specified)
Refs:

• Input:schemas/child.json
  Resolved: file:https:///rootDir/schemas/child.json

File: schemas/child.json
Base URI: file:https:///rootDir/schemas/child.json
Refs:

• Input: schemas/grandchild.json
• Resolved: file:https:///rootDir/schemas/schemas/grandchild.json

My expectation would not be for the initial base URI to be used to resolve all refs throughout the tree by default, though having an option to make it work like that would be fine. So resolution of that kind of relative path makes sense to me.

What I would like though (and maybe a custom resolver is the solution -- haven't looked at that yet) is to be able to have root relative resolution such that refs like /something.json would be resolved relative to the initial filesystem path.

[darcyrush]

I can confirm this happens to grandchildren and older referenced files where the baseUrl is different to the currently referenced file;

const rawSchema: JSONSchema | undefined = await readSchema(schemaWithRefPointingToAnotherSchemaWithRef);
const baseUrl: string = '/full/path/to/cwd/including/trailing/slash/';
const parsedSchema: JSONSchema = await $RefParser.dereference(baseUrl, rawSchema, {});

Luckily, I was actually looking for this functionality, since I am creating a library where the schema folder will not be inside the cwd, so thank you for pointing me in the right direction @jmm. In my case the bug replicates the behaviour I need.

const rawSchema: JSONSchema | undefined = await readSchema(currentSchema);
const parsedSchema: JSONSchema = await $RefParser.dereference(currentSchema, rawSchema, {});

[darcyrush]

Also, it would be great if the eventual fix would add a parameter to the resolve options, as this library is used in a lot of other JSON Schema and OpenAPI libraries and they usually expose the options object of this library.

Currently I have to create a complete dereferenced JSONSchema using the above logic and use that schema file for the rest of my API generation and validation toolchain as there is no way to declare what kind of $ref logic is desired via the options object.

[getlarge]

I also encountered the same issue when bundling some schemas located in the same folder.
Just to add my 2 cents on top of @evergreen-lee-campbell proposed workaround, instead of dereferencing all schema references, which has the drawback of generating lots of duplicate when bundling multiple schemas, I simply removed the duplicates segments in the resolved path. Which gives something like :

const fixedRootDirectoryResolver: ResolverOptions = {
  order: 101,
  canRead: true,
  read(file) {
    const fileUrl = [...new Set(file.url.split('/'))].join('/');
    return readFileSync(fileUrl, 'utf-8');
  },
};

const globalParserOptions: ParserOptions = {
  continueOnError: true,
  dereference: {
    circular: true,
  },
  resolve: {
    file: fixedRootDirectoryResolver,
  },
};

That's bricolage, but it solved my problem!

[jonluca]

https://github.com/APIDevTools/json-schema-ref-parser/pull/305/files