pypi ConfigArgParse 1.10.0

53 minutes ago

Behavior changes

Config file and env var lists no longer swallow subcommands

ConfigArgParse hands config file and env var values to argparse by adding them to the command line. For an option with nargs='+' or nargs='*', files = [a, b] becomes --files a b, and before 1.10.0 it was added just before the subcommand. So prog run was parsed as prog --files a b run: argparse read run as a third element in the list, and the subcommand silently never ran (files=['a', 'b', 'run'], command=None).

Now:

  • If an option comes before the subcommand, the list is added before that option, which ends the list: prog -v run gives files=['a', 'b'] and treats run as a subcommand.
  • Otherwise, if no other top-level options are specified, parsing stops with an error saying to set the list on the command line instead, or to specify another option before the subcommand.

List elements that argparse would read as an option or an args file are now an error

For an option with nargs (eg. nargs='+'), a config file or env var list is added to the command line as separate args after the option, so files = [a, --admin] becomes --files a --admin. Argparse then reads --admin as an option rather than a file name. In 1.9.0 that silently set admin=True, even from an env var when --admin has no env var of its own. In the same way, with fromfile_prefix_chars='@', files = [a, @extra_args.txt] read more args from extra_args.txt.

Now a list element that starts with - (or another of the parser's prefix_chars) or with one of its fromfile_prefix_chars stops parsing with an error that names the key. A lone - and negative numbers such as -2 still work.

Command-line values now win over config file and env var values in more cases

A config file or env var value is meant to be skipped when the same option is on the command line. 1.9.0 only spotted the option when it was written out in full, so it missed other forms argparse accepts and applied both values:

  • An abbreviated name: prog --ta e with tag = c in a config file gave tag=['c', 'e'] for an append option. Now tag=['e'].
  • Bundled short flags: prog -vv with verbose = 2 gave verbose=4 for a count option. Now verbose=2.
  • A value attached to a short option: prog -Ifoo with include = bar gave include=['bar', 'foo'] for an append option. Now include=['foo'].
  • An abbreviated name in a mutually exclusive group: with --level and --quiet in the group, prog --lev 3 with quiet = true failed with "argument --level: not allowed with argument --quiet". Now the config file's quiet is skipped, as it already was for prog --level 3.

Options that keep a single value weren't affected by these, since the command-line value came last and won anyway.

Options inside an args file now win too. With fromfile_prefix_chars='@', prog @args.txt, where args.txt holds --name y, gave the config file's name='x'; now it gives name='y'. A config file path given in an args file (-c my.conf) is now opened; before, it was ignored.

Options that belong to a subcommand or to REMAINDER args no longer block config file values

1.9.0 also counted options that argparse doesn't read as this parser's own. With name = x in a config file, 1.9.0 dropped it (name=None) in each of these cases, and 1.10.0 gives name='x':

  • prog run --name y, where the subcommand run has its own --name.
  • prog cmd --name y, where everything from cmd on goes to a nargs=REMAINDER positional.
  • prog --exec cmd --name y, where --exec takes nargs=REMAINDER.

Likewise, a -h after -- or inside REMAINDER args (prog -- -h, prog cmd -h) made 1.9.0 skip config files entirely, as if help had been asked for. Config files are now read.

BooleanOptionalAction values from config files and env vars

For parser.add_argument('--feature', action=argparse.BooleanOptionalAction), a config file key can be written in its no- form, but 1.9.0 did the opposite of what it says: no-feature = true gave feature=True, and no-feature = false gave feature=False. Now no-feature = true gives feature=False, like --no-feature on the command line, and no-feature = false gives feature=True.

An option with only short names, such as -f and -F, has no no- form, so nothing on the command line can set it to false. When an env var asked for false anyway, 1.9.0 set it to true, or crashed with IndexError if the option had a single name. It now stops with an error such as "FEATURE can't set feature to False: none of -f/-F does".

Other changes that can turn a previously accepted input into an error

  • Writing out a config file (with write_config_file() or an arg added with is_write_out_config_file_arg=True) now raises ValueError when the file would read back differently. In 1.9.0:
    • In the default format, the string [1, 2] was written as name = [1, 2], which reads back as a list.
    • With TomlConfigParser, a string containing a backslash, such as a\x41, was written without escaping it, so the file couldn't be read back at all.
    • TomlConfigParser([]) and IniConfigParser([], ...), which have no section to read from, wrote files that read back as nothing.
  • verbose = -1 for a count option used to be ignored and is now an error.
  • With exit_on_error=False, errors from config files and env vars raise argparse.ArgumentError instead of exiting.
  • Calling YAMLConfigFileParser().serialize() directly with an object such as a Path raises an error, instead of writing a !!python/object tag its own loader can't read.

Other reading changes

  • Default config format: name #value (no =) now sets the flag name and treats the rest as a comment. name = #value is unchanged.
  • Wildcards in default_config_files are read in alphabetical order, so which file wins no longer depends on the file system.
  • IniConfigParser(['my_tool', 'DEFAULT']): the named section now wins over [DEFAULT].
  • Written list values are JSON (#366). Current versions read them unchanged, but ConfigArgParse older than 1.3 keeps the quote marks.

New

  • Python 3.15 is now tested and listed as supported. On 3.15, parser.print_help(file) no longer writes terminal color codes into file when stdout is a terminal but file is not (eg. a log file).
  • parse_intermixed_args() and parse_known_intermixed_args() now read config files and env vars like parse_args() does. Before, Python 3.12 and later ignored them, and 3.8 to 3.11 added an env var value even when the command line set that option.
  • nargs=1 options accept a list from config files and env vars (size = [5]).
  • configargparse.already_on_command_line(), the helper that checks whether an option is already on a command line, takes an optional parser= argument. With it, the helper follows the rules above: it recognizes abbreviations, bundled short flags and attached values, and ignores args that go to a subcommand or to REMAINDER args. Calls without it work as before.

Fixes

Writing config files (each checked by writing a file and reading it back)

  • List values read back unchanged (#366): ['a,b', 'c'] used to read back as three items. Non-ASCII text is written as is, and list elements may contain newlines, # or ;.
  • Options given in abbreviated or bundled form (--gen hg38, -vv) are written; they used to be left out.
  • Values a type= function converted are written as the original string, so type=json.loads and FileType values read back.
  • Enum members, tuple defaults of multi-value options, nargs=1 values, append_const, and append/count options with defaults all read back as they were written.
  • Defaults that can't be read back (eg. default=sys.stdout, empty lists), defaults of mutually exclusive options, and the config file path option are left out of written files.
  • YAML writes Path and IntEnum values as plain values. The default format and IniConfigParser quote values that would otherwise read back differently (tests from #368), and IniConfigParser writes an empty list as [].
  • An arg that writes the config file with type=pathlib.Path no longer crashes after writing.

Reading config files and env vars

  • A false value for a BooleanOptionalAction with a short alias, such as ("-f", "--feature"), set it to true (#367).
  • An option value that equals a subcommand name (--target build build), prog run -- a, and a REMAINDER positional holding an option now get the config and env var args in the right place.
  • The config file path is found in a bundle of short flags (-vc conf), no longer confused with an option whose name starts the same (--set next to --settings), and found with allow_abbrev=False respected.
  • IniConfigParser strips a value on the line after key =, and line numbers in "Unexpected line N" errors start at 1.
  • A YAML key read as a bool or number (yes: 1), a stray % in an IniConfigParser value, flag = [a, b] for a store_true option, and verbose = lots for a count option now give a clear error instead of a traceback.
  • A TOML file whose section path runs through a plain value (tool = "x"), an empty or comment-only YAML file, and a bytes path in default_config_files no longer crash.

Help and print_values()

  • Calling format_help() more than once gives the same text; later calls used to drop the config file paragraph.
  • Env var names from auto_env_var_prefix show in the help before anything is parsed.
  • CompositeConfigParser's help labels ConfigparserConfigFileParser instead of showing ??, and IniConfigParser's help says keys are lowercased.
  • print_values() writes to the current sys.stdout, shows @file args expanded, and no longer lists a positional's default when it was given.

Full changelog: v1.9.0...v1.10.0

Don't miss a new ConfigArgParse release

NewReleases is sending notifications on new releases.