GNU Parallel and Python atexit

I was trying to run a python script with GNU parallel. Everything seems to work, except for the atexit routine used inside the python script. It seems, after ctrl+c, parallel is killing the python process without giving python a chance to call the registered atexit routine. How to make parallel a bit nicer towards the child processes?

Here is an example to show the behaviour.

test_signal.py:

#!/usr/bin/env python3

import time
import sys
import atexit

def cleanup():
    print('cleanup called', flush=True)

atexit.register(cleanup)

time.sleep(60)
print('completed process', sys.argv[1])

Tested with the command:

chmod +x test_signal.py
./test_signal.py 1 # this works as expected when using ^C
parallel -j 4 ./test_signal.py {} ::: $(seq 1 12) # this one does not

Answer

GNU Parallel kills jobs as specified in --termseq. It defaults to:

TERM,200,TERM,100,TERM,50,KILL,25

So SIGTERM, wait 200 ms, SIGTERM, wait 100 ms, SIGTERM, wait 50 ms, SIGKILL, wait 25 ms. If the process dies while waiting, GNU Parallel ignores the rest of the sequence – no need to kill a dead horse.

Change --termseq to the signals you want sent and the time between them.

CTRL-C sends SIGINT, so this should work:

# Give 1 second to clean up, then kill
parallel --termseq INT,1000,KILL,25 ...

Just be aware that you will not see the output unless you use --ungroup.