Behavior changes
Config file and env var lists no longer swallow subcommands
ConfigArgParse hands config file and env var values to argparse by adding them to the command line. For an option with nargs='+' or nargs='*', files = [a, b] becomes --files a b, and before 1.10.0 it was added just before the subcommand. So prog run was parsed as prog --files a b run: argparse read run as a third element in the list, and the subcommand silently never ran (files=['a', 'b', 'run'], command=None).
Now:
- If an option comes before the subcommand, the list is added before that option, which ends the list:
prog -v rungivesfiles=['a', 'b']and treatsrunas a subcommand. - Otherwise, if no other top-level options are specified, parsing stops with an error saying to set the list on the command line instead, or to specify another option before the subcommand.
List elements that argparse would read as an option or an args file are now an error
For an option with nargs (eg. nargs='+'), a config file or env var list is added to the command line as separate args after the option, so files = [a, --admin] becomes --files a --admin. Argparse then reads --admin as an option rather than a file name. In 1.9.0 that silently set admin=True, even from an env var when --admin has no env var of its own. In the same way, with fromfile_prefix_chars='@', files = [a, @extra_args.txt] read more args from extra_args.txt.
Now a list element that starts with - (or another of the parser's prefix_chars) or with one of its fromfile_prefix_chars stops parsing with an error that names the key. A lone - and negative numbers such as -2 still work.
Command-line values now win over config file and env var values in more cases
A config file or env var value is meant to be skipped when the same option is on the command line. 1.9.0 only spotted the option when it was written out in full, so it missed other forms argparse accepts and applied both values:
- An abbreviated name:
prog --ta ewithtag = cin a config file gavetag=['c', 'e']for anappendoption. Nowtag=['e']. - Bundled short flags:
prog -vvwithverbose = 2gaveverbose=4for acountoption. Nowverbose=2. - A value attached to a short option:
prog -Ifoowithinclude = bargaveinclude=['bar', 'foo']for anappendoption. Nowinclude=['foo']. - An abbreviated name in a mutually exclusive group: with
--leveland--quietin the group,prog --lev 3withquiet = truefailed with "argument --level: not allowed with argument --quiet". Now the config file'squietis skipped, as it already was forprog --level 3.
Options that keep a single value weren't affected by these, since the command-line value came last and won anyway.
Options inside an args file now win too. With fromfile_prefix_chars='@', prog @args.txt, where args.txt holds --name y, gave the config file's name='x'; now it gives name='y'. A config file path given in an args file (-c my.conf) is now opened; before, it was ignored.
Options that belong to a subcommand or to REMAINDER args no longer block config file values
1.9.0 also counted options that argparse doesn't read as this parser's own. With name = x in a config file, 1.9.0 dropped it (name=None) in each of these cases, and 1.10.0 gives name='x':
prog run --name y, where the subcommandrunhas its own--name.prog cmd --name y, where everything fromcmdon goes to anargs=REMAINDERpositional.prog --exec cmd --name y, where--exectakesnargs=REMAINDER.
Likewise, a -h after -- or inside REMAINDER args (prog -- -h, prog cmd -h) made 1.9.0 skip config files entirely, as if help had been asked for. Config files are now read.
BooleanOptionalAction values from config files and env vars
For parser.add_argument('--feature', action=argparse.BooleanOptionalAction), a config file key can be written in its no- form, but 1.9.0 did the opposite of what it says: no-feature = true gave feature=True, and no-feature = false gave feature=False. Now no-feature = true gives feature=False, like --no-feature on the command line, and no-feature = false gives feature=True.
An option with only short names, such as -f and -F, has no no- form, so nothing on the command line can set it to false. When an env var asked for false anyway, 1.9.0 set it to true, or crashed with IndexError if the option had a single name. It now stops with an error such as "FEATURE can't set feature to False: none of -f/-F does".
Other changes that can turn a previously accepted input into an error
- Writing out a config file (with
write_config_file()or an arg added withis_write_out_config_file_arg=True) now raisesValueErrorwhen the file would read back differently. In 1.9.0:- In the default format, the string
[1, 2]was written asname = [1, 2], which reads back as a list. - With
TomlConfigParser, a string containing a backslash, such asa\x41, was written without escaping it, so the file couldn't be read back at all. TomlConfigParser([])andIniConfigParser([], ...), which have no section to read from, wrote files that read back as nothing.
- In the default format, the string
verbose = -1for acountoption used to be ignored and is now an error.- With
exit_on_error=False, errors from config files and env vars raiseargparse.ArgumentErrorinstead of exiting. - Calling
YAMLConfigFileParser().serialize()directly with an object such as aPathraises an error, instead of writing a!!python/objecttag its own loader can't read.
Other reading changes
- Default config format:
name #value(no=) now sets the flagnameand treats the rest as a comment.name = #valueis unchanged. - Wildcards in
default_config_filesare read in alphabetical order, so which file wins no longer depends on the file system. IniConfigParser(['my_tool', 'DEFAULT']): the named section now wins over[DEFAULT].- Written list values are JSON (#366). Current versions read them unchanged, but ConfigArgParse older than 1.3 keeps the quote marks.
New
- Python 3.15 is now tested and listed as supported. On 3.15,
parser.print_help(file)no longer writes terminal color codes intofilewhen stdout is a terminal butfileis not (eg. a log file). parse_intermixed_args()andparse_known_intermixed_args()now read config files and env vars likeparse_args()does. Before, Python 3.12 and later ignored them, and 3.8 to 3.11 added an env var value even when the command line set that option.nargs=1options accept a list from config files and env vars (size = [5]).configargparse.already_on_command_line(), the helper that checks whether an option is already on a command line, takes an optionalparser=argument. With it, the helper follows the rules above: it recognizes abbreviations, bundled short flags and attached values, and ignores args that go to a subcommand or to REMAINDER args. Calls without it work as before.
Fixes
Writing config files (each checked by writing a file and reading it back)
- List values read back unchanged (#366):
['a,b', 'c']used to read back as three items. Non-ASCII text is written as is, and list elements may contain newlines,#or;. - Options given in abbreviated or bundled form (
--gen hg38,-vv) are written; they used to be left out. - Values a
type=function converted are written as the original string, sotype=json.loadsandFileTypevalues read back. - Enum members, tuple defaults of multi-value options,
nargs=1values,append_const, andappend/countoptions with defaults all read back as they were written. - Defaults that can't be read back (eg.
default=sys.stdout, empty lists), defaults of mutually exclusive options, and the config file path option are left out of written files. - YAML writes
PathandIntEnumvalues as plain values. The default format andIniConfigParserquote values that would otherwise read back differently (tests from #368), andIniConfigParserwrites an empty list as[]. - An arg that writes the config file with
type=pathlib.Pathno longer crashes after writing.
Reading config files and env vars
- A false value for a
BooleanOptionalActionwith a short alias, such as("-f", "--feature"), set it to true (#367). - An option value that equals a subcommand name (
--target build build),prog run -- a, and a REMAINDER positional holding an option now get the config and env var args in the right place. - The config file path is found in a bundle of short flags (
-vc conf), no longer confused with an option whose name starts the same (--setnext to--settings), and found withallow_abbrev=Falserespected. IniConfigParserstrips a value on the line afterkey =, and line numbers in "Unexpected line N" errors start at 1.- A YAML key read as a bool or number (
yes: 1), a stray%in anIniConfigParservalue,flag = [a, b]for astore_trueoption, andverbose = lotsfor acountoption now give a clear error instead of a traceback. - A TOML file whose section path runs through a plain value (
tool = "x"), an empty or comment-only YAML file, and abytespath indefault_config_filesno longer crash.
Help and print_values()
- Calling
format_help()more than once gives the same text; later calls used to drop the config file paragraph. - Env var names from
auto_env_var_prefixshow in the help before anything is parsed. CompositeConfigParser's help labelsConfigparserConfigFileParserinstead of showing??, andIniConfigParser's help says keys are lowercased.print_values()writes to the currentsys.stdout, shows@fileargs expanded, and no longer lists a positional's default when it was given.
Full changelog: v1.9.0...v1.10.0