Relative paths in [nitpick.styles].includes remote style should be resolved against the remote URL

[mjpieters]

Problem

Defining a preset that reuses other rules is arduous because relative paths are not supported. E.g. hosting a series of nitpick rules at https://coding-standards.example.com/python/* and a preset that includes the rules at https://coding-standards.example.com/presets/python.toml requires that we repeat the base URL in every single entry:

[nitpick.styles]
include = [
  "https://coding-standards.example.com/python/black.toml",
  "https://coding-standards.example.com/python/absent",
  "https://coding-standards.example.com/python/bandit",
  "https://coding-standards.example.com/python/black",
  "https://coding-standards.example.com/python/flake8",
  "https://coding-standards.example.com/python/hooks",
  "https://coding-standards.example.com/python/isort",
  "https://coding-standards.example.com/python/mypy",
  "https://coding-standards.example.com/python/poetry",
]

This makes it hard to re-home the files somewhere else and error prone. The problem is compounded when you use private github project URLs with a PAT and a tag reference; every link has to have the token reference and branch named:

[nitpick.styles]
include = [
  "gh:https://$GITHUB_TOKEN@org/coding-standards@v1/python/black.toml",
  "gh:https://$GITHUB_TOKEN@org/coding-standards@v1/python/absent",
  "gh:https://$GITHUB_TOKEN@org/coding-standards@v1/python/bandit",
  "gh:https://$GITHUB_TOKEN@org/coding-standards@v1/python/black",
  "gh:https://$GITHUB_TOKEN@org/coding-standards@v1/python/flake8",
  "gh:https://$GITHUB_TOKEN@org/coding-standards@v1/python/hooks",
  "gh:https://$GITHUB_TOKEN@org/coding-standards@v1/python/isort",
  "gh:https://$GITHUB_TOKEN@org/coding-standards@v1/python/mypy",
  "gh:https://$GITHUB_TOKEN@org/coding-standards@v1/python/poetry",
]

If I want to test changes to the styles before publishing I can't, not without first having to rewrite every include to point to the staging version or local filesystem.

Possible solution

Include references should be resolved relative to a base URL for the style file. So a file located at https://coding-standards.example.com/presets/python.toml, with the contents:

[nitpick.styles]
include = [
  "../python/black.toml",
  "../python/absent",
  "../python/bandit",
  "../python/black",
  "../python/flake8",
  "../python/hooks",
  "../python/isort",
  "../python/mypy",
  "../python/poetry",
]

results in the paths being resolved as relative against the https://coding-standards.example.com/presets/python.toml base URL for that file. It would work exactly the same for a Github URL; the base would be gh:https://$GITHUB_TOKEN@org/coding-standards@v1/presets/python.toml in that case.

This would also allow me to point nitpick to a local filesystem copy of my styles-under-development or hosted with a simple python -m http.server web server, for example, and all the includes would automatically be taken from the same location.

[andreoliwa]

Hello again.

Yes, these remote/local/merged configs have been a problem...

I think this can be done, I don't know how easy it is.
I used full paths because I had issues trying to keep state while merging style files.

And this part of the Nitpick code is not actually pretty (it's super ugly in fact).
I'll take a look and see what can be done.

[mjpieters]

I have an implementation for this now, but I'll need to clean it up a little, add a few more tests and update the documentation. I'm aiming for that to be done tomorrow.

It does rip out the current "subsequent style entries are relative to the first URI" functionality I found in the code, because it conflicts with the design I set out in this FR and I found that behaviour to be confusing and counter to how URI resolving works in other contexts such as HTML.

The implementation is otherwise relatively straightforward; after loading a style file the raw include values are normalised based against the current documentation URI.

[mjpieters]

One thing to decide on: how to handle symlinks.

The existing tests and implementation appear to assume that the real, absolute path on disk (all symlinks resolved) is the base for relative paths. A case could be made for the original path the be used however, as that can give you more flexibility with altering what is included based on where you put a symlink to a style file with includes.

I don't have a good use case ready for that scenario, however. It is also something that could be made configurable on a per file basis if the need arises in future.

[andreoliwa]

It does rip out the current "subsequent style entries are relative to the first URI" functionality I found in the code, because it conflicts with the design I set out in this FR and I found that behaviour to be confusing and counter to how URI resolving works in other contexts such as HTML.

Yes, this was a bad design while I was trying to shorten the declaration of styles.
Your solution is better.

The implementation is otherwise relatively straightforward; after loading a style file the raw include values are normalised based against the current documentation URI.

Sounds good. Just like HTML (as you mentioned above): a page with relative images inside it.

I don't have a good use case ready for that scenario, however. It is also something that could be made configurable on a per file basis if the need arises in future.

I didn't think of symlinks before because I didn't (don't) have any scenario either.
Just choose what makes more sense now. 👍🏻
If this becomes a problem, someone will create an issue.

I don't even consider it a breaking change because I don't think anyone is relying on symlinks.
I changed behaviours before (e.g. #421) and no one complained.
So I assume no one was affected, or they adjusted their workflow when they read the change log.

On a side note, I wish I could have some kind of telemetry to know which features are being used or not... 🤷🏻‍♂️

[mjpieters]

Just choose what makes more sense now. 👍🏻

That’s tempting. The simpler implementation is to ignore symlinks :-) That said: I’m very happy with the current delegation pattern where the file fetcher is using the normalisation step to expand ~ in a path and so that’s a natural location to also call os.realpath() anyway so it isn’t arduous.

[andreoliwa]

The simpler implementation is to ignore symlinks :-)

Works for me. 🙂

Let's postpone the decision for when someone has a use case for symlinks. 👍🏻

[github-actions]

🎉 This issue has been resolved in version 0.32.0 🎉

The release is available on:

• v0.32.0
• GitHub release

Your semantic-release bot 📦🚀