The python argparse module is powerful but eventually apps can
easily accumulate a huge number of arguments that are a pain to
specify every time. Thus, the simple answer to that is to add a
configuration file. There are other python modules that try to merge
argparse with configuration files, but none of them supports
hierarchies well -- and if you're going to accept a configuration
file, it should (IMHO) be well structured where modules can contain
their own sections, etc. The ArgparseWithConfig class provides this
interface.
It is intended as a drop-in replacement (a super class) of argparse,
but I have no doubt there are missing features. Basic command options
work, as do argument_groups. More is likely needed beyond that.
Arguments are processed in the following order, regardless of ordering on the command line:
- defaults for the application
- configuration file (
--config) loaded overriding defaults (`--set-default``) - command line options are parsed, overriding both 1 and 2
pip install argparse-with-config
Just like standard argparse, but now there are some extra options:
from argparse_with_config import ArgumentParserWithConfig
parser = ArgumentParserWithConfig()
parser.add_argument(
"-d", "--dog", default="spike", help="bogus"
)
parser.add_argument(
"-c", "--cat", default="mittens", help="cat name", config_path="kitty"
)
args = parser.parse_args(["-d", "spot"])
print(args)
# Namespace(config=None, set=None, dog='spot', cat='mittens')
print(parser.config)
# {'dog': 'spot', 'kitty': 'mittens'}Note how the config tokens are mapped from the additional config_path flag.
What's better is that you can have (endless) sub-dicts with a better structure to isolate needed components together:
from argparse_with_config import ArgumentParserWithConfig
parser = ArgumentParserWithConfig()
parser.add_argument(
"-d", "--dog", default="spike", help="bogus", config_path="animals.dog"
)
parser.add_argument(
"-c", "--cat", default="mittens", help="cat name", config_path="animals.kitty"
)
args = parser.parse_args(["-d", "spot"])
print(args)
# Namespace(config=None, set=None, dog='spot', cat='mittens')
print(parser.config)
# {'animals': {'dog': 'spot', 'kitty': 'mittens'}}Note that the base Namespace is still the same, but the config now has a lot more structure to it.
The above is basically also equivalent to:
from argparse_with_config import ArgumentParserWithConfig parser = ArgumentParserWithConfig()
group = parser.add_argument_group("animals", config_path="animals")
group.add_argument(
"-d", "--dog", default="spike", help="bogus", config_path="dog"
)
group.add_argument(
"-c", "--cat", default="mittens", help="cat name", config_path="kitty"
)
args = parser.parse_args(["-d", "spot"])
print(args)
# Namespace(config=None, set=None, dog='spot', cat='mittens')
print(parser.config)
# {'animals': {'dog': 'spot', 'kitty': 'mittens'}}By default, two new arguments will be added to the command line:
- --config FILE...: loads configuration from a YAML configuration file
- --set name=val: Evaluates each expression for a left/right pair
Consider this yaml file:
---
bogus: 5000
animals:
zebra: Marty
silent:
ninja: deadly
something:
wicked:
thisway: comesAnd this code base:
from argparse_with_config import ArgumentParserWithConfig
parser = ArgumentParserWithConfig()
group = parser.add_argument_group("animals", config_path="animals")
group.add_argument(
"-d", "--dog", default="spike", help="bogus", config_path="dog"
)
group.add_argument(
"-c", "--cat", default="mittens", help="cat name", config_path="kitty"
)
args = parser.parse_args(["-d", "goodboy", "--config", "test.yml"])
print(parser.config)
# {'animals': {'dog': 'goodboy', 'kitty': 'mittens'}}Note that command line options always override configuration files, which are expected to be general defaults. Ordering does not matter. Thus, even though the --dog flag occurs before the --config flag, the --dog flag is given preference.
from argparse_with_config import ArgumentParserWithConfig
parser = ArgumentParserWithConfig()
group = parser.add_argument_group("animals", config_path="animals")
group.add_argument(
"-d", "--dog", default="spike", help="bogus", config_path="dog"
)
group.add_argument(
"-c", "--cat", default="mittens", help="cat name", config_path="kitty"
)
args = parser.parse_args(["--dog", "goodboy", "--config", "test.yml"])
print(parser.config)
# {'animals': {'dog': 'goodboy', 'kitty': 'mittens'}}This also works with the --set-default flag:
from argparse_with_config import ArgumentParserWithConfig
parser = ArgumentParserWithConfig()
group = parser.add_argument_group("animals", config_path="animals")
group.add_argument(
"-d", "--dog", default="spike", help="bogus", config_path="dog"
)
group.add_argument(
"-c", "--cat", default="mittens", help="cat name", config_path="kitty"
)
args = parser.parse_args(["--set-default", "animals.cat=paws"])
print(parser.config)
# {'animals': {'kitty': 'paws', 'dog': 'spike'}}There is a huge amount lef to do, but it is in a basic usable state today.
Left:
- more testing with other
argparsefeatures - support many more
argparsesub-classes where needed - support reading multiple config types, including
TOMLand maybejson - make
parse_argsreturn a super class ofNamespacewith a.configattribute? - remove/fix some of the grosser hacks -- much is clean, but there are a couple of nasty hacks.
- argparse_config
- Uses a generic config structure -- I wanted something much more complex at times.
- A useful stack overflow and partial argument parsing
- (there was at least one more that I've lost track of)