Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse configuration file II: using serde, toml, merge & envy instead of config #111

Closed
wants to merge 7 commits into from

Conversation

robinkrahl
Copy link
Collaborator

In pull request #101, I used the config crate to parse and merge the configuration from files and from the environment. config is apparently no longer maintained and uses an old version of the nom crate. This PR replaces config with serde and toml for the configuration file parsing, merge for the merging of different configuration sources, and envy for reading the configuration from the environment. Thus we no longer need the nom dependency and have more control over the dependency versions.

Note that we still have a separate Config struct that is manually merged with the Args struct. We could also use the Args struct directly, but the question is how to handle the subcommand enums. We could just skip them, or we could read the configuration for all subcommands into a map and then select the variant based on the CLI arguments.

In the first version of the patch, I used the app_dirs crate to locate the config directories, but it was unmaintained and had some annoying bugs and limitations (andybarron/app-dirs-rs#34, andybarron/app-dirs-rs#12). We switched to the directories crate, but this crate does not support system configuration files. Now there is a maintained fork of app_dirs, app_dirs2, and I prepared patches for the two app_dirs issues (app-dirs-rs/app_dirs2#8, app-dirs-rs/app_dirs2#12). Once they are merged and released, I would move back to app_dirs2.

@robinkrahl robinkrahl requested a review from d-e-s-o August 9, 2020 13:52
@d-e-s-o
Copy link
Owner

d-e-s-o commented Aug 9, 2020

Thanks for the pull request Robin! I'll take a look this week.

Copy link
Owner

@d-e-s-o d-e-s-o left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for the pull request. I took a first look. I have a few comments, but overall looks very good to me! I'll sleep over it and give it another look later this week.

CHANGELOG.md Outdated Show resolved Hide resolved
Cargo.toml Outdated Show resolved Hide resolved
src/args.rs Show resolved Hide resolved
src/config.rs Outdated Show resolved Hide resolved
src/config.rs Outdated Show resolved Hide resolved
src/config.rs Outdated Show resolved Hide resolved
src/config.rs Show resolved Hide resolved
src/error.rs Outdated Show resolved Hide resolved
src/config.rs Outdated Show resolved Hide resolved
doc/nitrocli.1 Show resolved Hide resolved
@robinkrahl
Copy link
Collaborator Author

Thanks for the review! Maybe I should have made it clearer that this is just a draft with many rough edges. But I first wanted to reach consensus on the general approach before I polish the code.

@d-e-s-o
Copy link
Owner

d-e-s-o commented Aug 12, 2020

Thanks for the review! Maybe I should have made it clearer that this is just a draft with many rough edges. But I first wanted to reach consensus on the general approach before I polish the code.

Oh, well, looked good to me (although I recognized it's a draft). The only other thing I can think of are are tests. E.g., to check the priorities of the different ways of setting configuration options.

@robinkrahl
Copy link
Collaborator Author

If we want to restrict the configuration to the top-level options, we could refactor these options into a GlobalArgs struct. Then Args could use this struct with #[structopt(flatten)], the configuration could be deserialized into this struct too and we could automatically merge the configuration into the CLI arguments using the derived Merge trait.

This would allow us to remove the manual merge code. The drawback is that we would have to make bigger changes if we want to add configuration for subcommand options in the future. What do you think?

@robinkrahl
Copy link
Collaborator Author

Fixed some of the issues, rebased on devel.

@d-e-s-o
Copy link
Owner

d-e-s-o commented Aug 14, 2020

Thanks. Will take a look on the weekend or so.

src/args.rs Outdated Show resolved Hide resolved
@d-e-s-o
Copy link
Owner

d-e-s-o commented Aug 15, 2020

I think it looks great!

@d-e-s-o
Copy link
Owner

d-e-s-o commented Aug 15, 2020

Make it ready for review when you feel like it.

@d-e-s-o
Copy link
Owner

d-e-s-o commented Aug 15, 2020

Btw., feel free to merge commits adding a new crate as a dependency into the first one using it. I had separate commits for that in the past because of the subrepo magic we had (that checked in the respective source code). But without that I'd say it just adds noise and obfuscates why a dependency is introduced. But if you see value then keep it.

@robinkrahl
Copy link
Collaborator Author

  • Changed str::parse to DeviceModel::from_str.
  • Squashed dependency commits.

@robinkrahl
Copy link
Collaborator Author

Also, I have been thinking about dropping the directories/app-dirs2 dependency and using xdg directly:

  • On Linux systems, we want to use the XDG Base Directory Specification anyway.
  • There is a debate whether command-line applications should use the library paths or the XDG Base Directory Specification on macOS, see e. g. this discussion on stackoverflow.com.
  • As far as I know, there are no common conventions for command-line applications on Windows.

Users who prefer the macOS library or Windows AppData paths could still set the XDG environment variables. What do you think?

@d-e-s-o
Copy link
Owner

d-e-s-o commented Aug 26, 2020

Also, I have been thinking about dropping the directories/app-dirs2 dependency and using xdg directly:
[...]
Users who prefer the macOS library or Windows AppData paths could still set the XDG environment variables. What do you think?

I'd be fine with that, as long as it still works on those operating systems (which I can't test). So if we were to use xdg on, say, Windows (and assuming no configuration on the user's side), the configuration would be expected at %USERPROFILE%/.config/nitrocli, is that correct?

@robinkrahl
Copy link
Collaborator Author

robinkrahl commented Aug 26, 2020 via email

@robinkrahl
Copy link
Collaborator Author

Rebased onto topic/anyhow.

This patch implements basic configuration handling that reads a
configuration file and stores the parsed data in the ExecCtx and RunCtx
structs.  It supports three configuration items:
- model (previously only --model)
- no_cache (previously only NITROCLI_NO_CACHE)
- verbosity (previously only --verbose)
This patch uses the envy crate to parse the environment.  A variable
NITROCLI_KEY can be used to overwrite the configuration for *key*.  This
has the side effect that the NITROCLI_NO_CACHE variable is evaluated as
a boolean variable (instead of only checking whether it is set).  We
also accept two new variables, NITROCLI_MODEL and NITROCLI_VERBOSITY.
This patch uses the directories crate to query the appropriate path for
the configuration files.  For Linux, paths according to the XDG Base
Directory Specification are used.

Note that directories does not yet support the XDG_CONFIG_DIRS variable
for system-wide configuration files.  Therefore we only use a user
configuration file.
This patch adds a simple configuration file that demonstrates the syntax
and contains some documentation.  I suggest to ship this file together
with nitrocli and to install it e. g. in the  /usr/share/doc/nitrocli
directory.  This patch also adds a simple test case that makes sure that
the example file is parsed correctly.
This patch adds a new --no-cache option that corresponds to the
NITROCLI_NO_CACHE environment variable and the no_cache configuration.
This makes the user interface more consistent as all configuration items
are now backed by both an environment variable and a command-line
option.
This patch updates the man page for the last changes:
- new option --no-cache
- changes to the environment variables
- configuration files
@robinkrahl
Copy link
Collaborator Author

  • Rebased onto devel.
  • Use merge 0.1.0 instead of Git version.

I think I addressed all comments, so this should be ready for review and merge.

@robinkrahl robinkrahl marked this pull request as ready for review September 1, 2020 14:56
@d-e-s-o
Copy link
Owner

d-e-s-o commented Sep 1, 2020

Awesome! Thanks Robin. I'll have a final look later today and then we should be good to merge.

@d-e-s-o d-e-s-o self-assigned this Sep 2, 2020
@d-e-s-o
Copy link
Owner

d-e-s-o commented Sep 2, 2020

All merged now with a few small changes. Thanks again!

@d-e-s-o d-e-s-o closed this Sep 2, 2020
@robinkrahl robinkrahl mentioned this pull request Sep 3, 2020
@robinkrahl robinkrahl deleted the config-2 branch September 10, 2020 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants