Future-proofing rulesets

[P0lip]

The current design of rulesets is slightly troublesome.
The list of user-facing problems includes, but not limits itself to:

• no feature parity between CLI <-> browser usage of Spectral
• if custom functions are in use, the distribution becomes very tricky, as there's no way to generate a single file Spectral could import. This will likely become important for shared style guides
• versioning is somewhat troublesome
• they're vulnerable to breaking changes because there's no way they could lock a version of Spectral util, etc.
• difficult/impossible to use external libraries in a browser env
• not being able to specify your own format (if you're a CLI user)

The other side is us, engineers, that have quite a bit of burden to deal with:

• separate flows for CLI / browsers
• shenanigans needed to load a custom function

Moreover, current rulesets have a set of features that are very uncommonly used, but do add a notable amount of complexity:

• $refs (we do use them internally, but I doubt others do)
• functionOptions can be validated using a JSON schema

I could probably find more issues and flaws with the current state of things, but these are all that come to my mind at this very moment.

I'd like to propose a revision of the current design, that would address most of the issues described above, and at the same time would reduce the burden we have to deal with.

{
  "rules": {
    "foo": {
      "then": {
        "function": "truthy"
      }
    }
  }
}

{
  "extends": ["https://stoplight.com/spectral-rulesets/oas2.json"]
}

import { isObject } from "https://cdn.jsdelivr.net/npm/lodash-es/+esm";
// obviously we could serve these from our own server / tenant server (on-prem installations)
import { truthy, schema } from "https://cdn.jsdelivr.net/npm/@stoplight/spectral-functions/+esm";
import { alphabetical } from "https://cdn.jsdelivr.net/npm/@stoplight/[email protected]/+esm"; // you can stick to an older version if you want to for some reason. That's fine
import { oasRuleset } from "https://cdn.jsdelivr.net/npm/@stoplight/spectral-rulesets/+esm";
import { oas2 } from "https://cdn.jsdelivr.net/npm/@stoplight/spectral-formats/+esm";

import { verifyType } from './verifyType.mjs';

const $SCHEMA_DRAFT_2020_XX_REGEX = /^https?:\/\/json-schema.org\/draft\/2020-\d\d\/(?:hyper-)?schema#?$/;

const JSONSchemaDraft2020_XX = document => isObject(document) && "$schema" in document && $SCHEMA_DRAFT_2020_XX_REGEX.test(document.$schema);

export default {
  formats: [oas2, oas3],
  // this would no longer accept functions & functionOptions, but the rest would be almost the same (with the exception it does not support $refs, and functions have to be provided)
  extends: [oasRuleset],
  rules: {
    "valid-rule": {
      given: "$.info",
      then: {
        function: truthy, // instead of 'truthy'
      },
    },
    "only-new-json-schema": {
      formats: [JSONSchemaDraft2020_XX],
      given: "$..type",
      then: {
        function: verifyType,
      },
    }
  },
};

As you can see, the syntax is pretty much the same.
What's different is that you need to explicitly load formats or rulesets, but thanks to that you get control over versioning.
All the inheritance rules would be kept untouched, so all/off/recommended, etc. would all stay the same

Proposed file extension: /^\.?spectral\.mjs$/. mjs cause of Node.js.

Breaking Changes

• no custom functions in JSON/YAML rulesets
• custom functions need to be valid ESM

Benefits

With such an approach, any time we make a change to a ruleset or/and a function, we can quite easily roll it out to everyone.
If someone does want to stick to a particular version, it's as easy as specifying a version in the import.
Furthermore, most of the issues I listed above are solved.

Potential Challenges

Adjusting Studio Spectral ruleset form - I'll be able to do without any issues by working on ESTree rather than trying to dispatch JSON patches.
So basically, the flow would be:

• parse JS file with acorn
• work on the tree
• use astring and write contents to disk

Some migration for existing users.

Links

#845
#1329
#1561

[tillig]

This looks awesome, and definitely addresses #1329, so thanks!

Something to consider, maybe not as minimum-viable-product, but in the design: I gather the import X from "url" isn't part of Node so much as the module loader you choose. (I could be very wrong about that, my information is about five minutes of Google there.) I work primarily in a financial services environment, so there's literally nothing we have that doesn't require authentication of some sort. We also have proxies and security things to get around. It would be nice if the loader could somehow be configured to support authentication and proxy handling.

But, again, not super important on day one since we can just embed all the code right in the file, totally self-contained, which is sweet and would solve so much.

[philsturgeon]

This seems like it'll make life a lot less confusing for people working on custom functions and custom rulesets in general. Can we provide any tooling around this, like eslint or some other code checking tools to assist them with the process? As followup work ofc.

It also seems like it opens the door for spectral to get auto-fix logic, as rules could declare fixer functions too like eslint too. cc @mnaumanali94

[P0lip]

We also have proxies and security things to get around. It would be nice if the loader could somehow be configured to support authentication and proxy handling.

We have the same set of issues at Stoplight here, so no worries about this one. We'll have to sort it out.

This seems like it'll make life a lot less confusing for people working on custom functions and custom rulesets in general.

That's the primary hope. Instead of using a somewhat proprietary syntax, we'll try to use a more standard approach more people should be familiar with and can also find plenty of resources about if in doubt.
At the same time, however, we are really careful with making significant changes. IMHO most of the rulesets shouldn't require any migration, and those that do should be straightforward to migrate using some automated script we provide.

Can we provide any tooling around this, like eslint or some other code checking tools to assist them with the process? As followup work ofc.

We'll certainly try to provide some tooling around this, because we'll need it for our own rulesets, thus hopefully we can share some of the concepts we used, to ensure that a) the majority of the community rulesets are more or less consistent b) rulesets are easier to author & test c) writing powerful rules requires less effort.

[mnaumanali94]

It also seems like it opens the door for spectral to get auto-fix logic, as rules could declare fixer functions too like eslint too.

Yes! I'm looking forward to us building that out once we have this out of the door.