asyncio add_signal_handler not catching sigint or sigterm

I’m having some trouble debugging an issue, I have an asyncio project and I would like it to shutdown gracefully.

import asyncio
import signal

async def clean_loop(signal, loop):
        print("something")
        tasks = [t for t in asyncio.all_tasks() if t is not
                asyncio.current_task()]
        [task.cancel() for task in tasks]
        
        await asyncio.gather(*tasks, return_exceptions=True)
        loop.stop()

def main():
    loop = asyncio.get_event_loop()
    signals = (signal.SIGTERM, signal.SIGINT)
    for s in signals:
        loop.add_signal_handler(s, lambda s=s: asyncio.create_task(clean_loop(s, loop)))

    task = loop.create_task(run(some_code))
    loop.call_later(60, task.cancel)
    try:
        loop.run_until_complete(task)
    except asyncio.CancelledError:
        pass
    finally:
        loop.close()

if __name__ == "__main__":
    main()

When I run my code and send a KeyboardInterrupt or TERM signal from a different screen, nothing seems to happen and it doesn’t look like clean_loop is being called.

EDIT: I think I was able to isolate the problem a bit more, the code that I’m running within some_code which has an infinte loop and contains another asyncio.gather(*tasks) within it, when I comment it out I am able to catch the signal and clean_loop runs. Could anyone explain why this conflict is happening?

Answer

If you’re on a Unix-like systems,

“””Add a handler for a signal. UNIX only.

it should work fine. The only thing is clean_loop itself is a task that getting destroyed with loop.stop() and because of that you see:

Task was destroyed but it is pending!
task: <Task pending name='Task-2' coro=<clean_loop() running at ...> wait_for=<_GatheringFuture finished result=[CancelledError('')]>>

It shouldn’t be important though because it’s just there to cancel tasks.

I’ve made some slight changes that have nothing to do with the actual question:

import asyncio
import signal


async def run():
    for i in range(4):
        print(i)
        await asyncio.sleep(2)


async def clean_loop(signal, loop):
    print("-----------------------handling signal")

    tasks = asyncio.all_tasks() - {asyncio.current_task()}
    for task in tasks:
        task.cancel()
    print("--------------------------reached here")
    await asyncio.gather(*tasks, return_exceptions=True)
    loop.stop()


def main():
    loop = asyncio.new_event_loop()
    signals = (signal.SIGTERM, signal.SIGINT)
    for s in signals:
        loop.add_signal_handler(s, lambda s=s: asyncio.create_task(clean_loop(s, loop)))

    task = loop.create_task(run())
    loop.call_later(60, task.cancel)
    try:
        loop.run_until_complete(task)
    except asyncio.CancelledError:
        pass
    finally:
        loop.close()


if __name__ == "__main__":
    main()

output:

0
1
2
^C-----------------------handling signal
--------------------------reached here
Task was destroyed but it is pending!
task: <Task pending name='Task-2' coro=<clean_loop() running at ...> wait_for=<_GatheringFuture finished result=[CancelledError('')]>>

According to your edit:

Documentation of add_signal_handler says:

The callback will be invoked by loop, along with other queued callbacks and runnable coroutines of that event loop.

Just like other coroutines it should be invoked by the loop in a cooperative way. In other words, a coroutine should give the control back to the event loop so that event loop can run another coroutine.

In your case, your coroutine which has infinite loop prevents event loop from running another coroutines especially the the callback which registered through add_signal_handler. It doesn’t cooperate! That’s why you though it didn’t work. It sat idle in queue waiting for run.