Create variable key/value pairs with argparse (python)

I’m using argparse module to set my command line options. I’m also using a dict as a config in my application. Simple key/value store.

What I’m looking for is a possibility to override JSON options using command line arguments, without defining all possible arguments in advance. Something like --conf-key-1 value1 --conf-key-2 value2, which would create a dict {'key_1': 'value1','key_2': 'value2'} (‘-‘ in the argument is replaced by ‘_’ in the dict). Then I can combine this dict with my JSON config (dict).

So basically I would like to define --conf-* as an argument, where * can be any key and what comes after is the value.

I did find configargparse module, but as far as I can see I start with a dict I already use.

Any ideas how I could approach this?

Answer

The first thing I’d try is use parse_known_args to handle other arguments, and handle the list of extras with my on routine. Adding the ‘–conf-‘ handling to argparse would be more work.

argv = '--conf-key-1 value1 --conf-key-2 value2'.split()
p = argparse.ArgumentParser()
args, extras = p.parse_known_args(argv)

def foo(astr):
    if astr.startswith('--conf-'):
        astr = astr[7:]
    astr = astr.replace('-','_')
    return astr

d = {foo(k):v for k,v in zip(extras[::2],extras[1::2])}
# {'key_1': 'value1', 'key_2': 'value2'}

The extras parsing could be more robust – making sure that there are proper pairs, rejecting badly formed keys, handling =.

Another approach would be to scan sys.argv for --conf- strings, and use those to construct add_argument statements.

keys = [k for k in argv if k.startswith('--conf-')]
p = argparse.ArgumentParser()
for k in keys:
    p.add_argument(k, dest=foo(k))
print vars(p.parse_args(argv))

---

If you would accept ‘–conf key1 value1 –conf key2 value2 …’ as the input, you could define

parser.add_argument('--conf', nargs=2, action='append')

which would produce:

namespace('conf': [['key1','value1'],['key2','value2']])

which could easily be turned into a dictionary. Or a custom Action could use setattr(namespace, values[0], values[1]) to enter the key/value pairs directly into the namespace.

I believe there have been SO question(s) about accepting ‘”key1:value” “key2:value2″‘ inputs.