How to access captured arguments / closure of python instance method objects?

Tags: , ,



The idea is to implement the Observer pattern in a non leaking / autocleanup fashion. Therefor the instance method objects should be removed when the the associated object is cleaned up by the gc.
My original idea was to only store weak references to the instance method objects with a finalizer to call a cleanup routine.

class Observeable:
    def __init__(self):
        self._callbacks: list = []
        self._dirty = False

    def add_callback(self, callback):
        finalize(callback, self._set_dirty)
        self._callbacks.append(ref(callback))

    def trigger_callbacks(self, *args, **kwargs):
        if self._dirty:
            self._cleanup_callbacks()
        for callback in self._callbacks:
            callback()(*args, **kwargs)

    def _set_dirty(self):
        self._dirty = True

    def _cleanup_callbacks(self):
        for callback in self._callbacks:
            if not callback():
                self._callbacks.remove(callback)
        self._dirty = False

However as it turns out the approach is conceptually flawed as the lifetime of the instance method object is not bound to the associated object.

This leads me to the idea to extract the self parameter from the closure of the instance method objects and bind to its lifetime. This can of course be done by passing a second argument to add_callback, however it would be cleaner to extract it from the closure.

As I wasn’t able to find any usefull information of how the closure is stored in the function object, I have a few questions for you guys.

  1. Is my thinking even right? Storing a instance method objects will prevent the behind object from being cleaned up automatically, right?
  2. Is it possible to extract the self parameter / general closure from function objects?
  3. Is there a more “official” name for function objects? The python docu also just called it like that, however there seems to be no usefull information about its more low level implementation available.

I hope my problem / questions are clear, thanks for the help in advance!

Answer

@fountainhead mostly answered it with his comments so I will summarize it shortly.

A function object from a method is officially called “instance method object” (https://docs.python.org/3/reference/datamodel.html, search for method object)

Therefor 1. it will prevent the associated object from being cleaned up by the gc. 2. It can be accessed by __self__ and 3. is already answered above.

To make the Observable Class working like I intended it, I basically save the function name and a weakref to __self__ and then call it with getattr(). Works like a charm!

from weakref import finalize, ref
from functools import partial


class Observeable:
    def __init__(self):
        self._methods: list = []
        self._functions: list = []
        self._dirty = False

    def add_callback_methode(self, callback):
        """
        Adds a callback from a method.
        The callback will automatically be cleanup,
         when its associated objects gets out of scope,
          therefor also not preventing the gc from cleaning it up.
        """
        finalize(callback.__self__, self._set_dirty)
        self._methods.append((callback.__name__, ref(callback.__self__)))

    def trigger_callbacks(self, *args, **kwargs):
        """Forwards the given args and kwargs to all callbacks"""
        if self._dirty:
            self._cleanup_callbacks()
        for callback in self._methods:
            getattr(callback[1](), callback[0])(*args, **kwargs)

    def _set_dirty(self):
        self._dirty = True

    def _cleanup_callbacks(self):
        for callback in self._methods:
            if not callback[1]():
                self._methods.remove(callback)
        self._dirty = False


Source: stackoverflow