Ability to embed custom functions in ruleset YAML/JSON

[tillig]

User story.
As a Spectral custom ruleset developer, I can put my custom rule functions in the same file as the ruleset YAML/JSON, so that I can more easily distribute my custom rules.

Is your feature request related to a problem?
I'm trying to write a rule that mandates all schema objects have examples defined, but the complexity of where you can define various examples (at the property level, at the object level, etc.) makes it nigh impossible to do without a custom function. However, while we have a .spectral.yaml in our various projects, there is some organizational reluctance to start distributing the custom ruleset as two or more files - one with the rules, one with the function(s). Keeping them both in sync is seen as a challenge, and adding more dot-files to repos is something we're pretty careful about. It's getting crowded in there.

Describe the solution you'd like
A way to embed the custom functions in the ruleset YAML/JSON file, similar to how you can embed inline script in an azure-pipelines.yml.

Using the example in the docs I imagine it might look like this:

functions:
- equals
  - properties:
      value:
        type: string
        description: Value to check equality for
  - code: |
      module.exports = (targetVal, opts) => {
        const { value } = opts;

        if (targetVal !== value) {
          return [
            {
              message: `Value must equal {value}.`,
            },
          ];
        }
      };
rules:
  my-rule:
    message: "{{error}}"
    given: "$.info"
    then:
      function: "equals"
      functionOptions:
        value: "abc"

Granted, this could make for some larger YAML files, but most folks aren't poking around in there anyway.

From a dev experience standpoint it'd be easy enough to put together some sort of build chain that "assembles" the inline code from separate JS to make testing or whatever easier, but the portability would be there after build, sort of like a "webpack for Spectral rulesets."

[philsturgeon]

Hello again @tillig!

This is a really interesting idea. For now we've simply accepted that if you wanna get "Advanced Mode" you can use NPM, but I have always loved the simplicity of sharing a ruleset as a URL...

@P0lip what do you think about this as a feature, and what do you think about this from a security perspective?
Man in the Middle attacks are not a bigger concern. Could be we only allow custom functions if it happened to be loaded over HTTPS or something... which still isn't bulletproof but does cut out some vectors for nonsense.

[tillig]

Interesting, I didn't think about sharing it as a URL; I was sort of stuck on "embed the functions in the YAML." URL might be an interesting way to go. May still be worth having a fallback for embedding in the YAML.

The default mechanism for rulesets appears to already fall back to unpkg so maybe share via URL isn't as hard to implement?

[philsturgeon]

Oh well you talked about portability, I suppose I'm not sure of your use case.

Currently YAML is handy for easily making local rulesets, and it's easy for distributing via a URL:

extends:
- https://example.com/company-ruleset.yaml

It's one of a few ways to share rulesets and its certainly the easiest, but has no support for custom functions. For that we recommend NPM modules.

If you aren't trying to distribute, then you've just gotta make a custom function file.

# functions/abc.js

module.exports = (targetVal) => {
  if (targetVal !== 'hello') {
    return [
      {
        message: 'Value must equal "hello".',
      },
    ];
  }
};

Is there any reason this needs to move to the YAML if its not for making "rulesets on URLs" work?

[tillig]

Consider an environment where:

• You are not allowed to publish custom rules as a public npm package.
• There is not an internal npm repository to which you may publish rules.
• All internally developed source code access requires authentication, such that simply putting a URL in to extend a base ruleset or reference some JS source isn't possible without the tool also supporting authentication.

Think government, banking, that sort of environment.

In this sort of an environment, if you have a custom rule that requires a function because it can't be expressed entirely in JSONPath, you need to create a custom function. It's awesome that Spectral allows this, and it gets me most of the way to where I need to be.

However, once you introduce a custom function, you need to keep that .spectral.yaml and the set of custom functions in line. Which "version" of the ruleset are you using? Did you get the one with the updated function? Hard to say. Here - replace your .spectral.yaml file with this one, clear out your .spectral folder, and then drop these .js files in there.

That's kind of painful. It'd be nice if the .spectral.yaml had more of the portability that azure-pipelines.yaml has. If you have a powershell task, like the one in your build, instead of requiring the script to always be in some external file, it allows you to specify the script inline. The upgrade story is a lot easier here. Need the latest? Drop this file in. Done. No extra steps. What changed recently? Diff one file. Done. No extra steps.

That's the level of portability I was hoping for here.

You could take this a step further, too. Like, you can configure Spectral with YAML or JSON, but what if it was like ESLint and allowed configuration via .js? .spectral.js could evaluate to export config and custom functions, and the whole thing could be one self contained thing. (I recognize that ESLint doesn't let you define custom rules in its JS config format, at least that I'm aware of, but the idea that you could use JS as configuration rather than simply just static YAML or JSON means... you could do that.)

Does that help clarify what I'm thinking about?

[philsturgeon]

I gotcha, you want to be able to share a single file, without worrying about copying and pasting multiple files, maybe not over a URL but easily copy/paste/tweakable, where no NPM package would work (public or private), and git submodules are a pain in the backside, and who has time for rsync or whatever.

I can see this being a use case in locked down environments like those you mentioned: banking, gov, etc.

If we enable it, we have to think about the security aspects of it happening over a URL like I mentioned, and consider what limitations we put in place (HTTPS only for example).

We also have to think about how/if we can enable this throughout the rest of Stoplight Platform (like Studio Web) and what security implications that might have.

Let me chat it over with @P0lip (who happens to be sat next to me! 🇵🇱 🙌).

Also @tillig if we were to proceed, is this something you might have time to work on, if you need it for work?

[tillig]

I probably wouldn't be able to get to it in the immediate future, but if time does permit I could give it a shot. Current work project is a 20 person project with two people on it and I'm in the middle of the Autofac v6 rollout, too, so I'll need to finish that up first.

Might be good to talk over designs so there's less exploratory work when it gets addressed. It could also allow this to be broken down into smaller chunks that could be addressed individually.

For example, here's one way single-file config could be done:

• Enable .spectral.js configuration. Similar to how ESLint allows you to have a module that exports the configuration. Maybe .js configuration is the only format that supports custom functions - that way you don't have to deal with eval on inline code.
• Allow the function in a custom rule to be the actual function, not just a string. If it's a string, do the current work; if it's a function, just use it as-is.

...I think that's it? Maybe I'm missing some details - "I don't know what I don't know" sorts of things. Your subject matter expertise could help a lot in that respect. "Don't forget to _____ in the _____ function because it'll break things if you forget!"

Additional items like the URL-based include could also, then, be separate. But having a sort of plan broken down like that would help make things more "bite-sized" so incremental progress can be made. I can almost guarantee I won't have time to help if it all has to be done in One Big Pull Request.

[P0lip]

@tillig
Putting the security considerations aside, I think this idea sounds pretty interesting and it's worth considering.
Some time ago, I was thinking about providing a way to "bundle" your ruleset, which is somewhat similar to what you describe.
We kind of support it internally (see generate-assets task in package.json and the output it produces), but it's implemented in a different way than you mentioned. We use these assets in Studio, but they are loaded fully offline via require call in plain JS code, so we don't need to worry about anything, as we fully trust the code.
Either way, it wouldn't be terribly hard to expose that publicly and have a new CLI method like bundle as well as some JS API method.
Speaking of potential security implications - I have no suggestions yet, but we can take that into account at some point.

[P0lip]

Hey @tillig!
I compiled a short design doc of the v6 rulesets in #1615.
That new approach should let you accomplish what you describe over here, as it'll be possible to pass a JS file as a ruleset, thus you can use any bundler, such as Rollup.js or similar.
I'll close this one in the meantime - I linked this issue there, to ensure the new ruleset design achieves the requirements brought up here. Please track the progress in #1615.
Thanks for the suggestion!