How to troubleshoot `super()` calls finding incorrect type and obj?

I have a decorator in my library which takes a user’s class and creates a new version of it, with a new metaclass, it is supposed to completely replace the original class. Everything works; except for super() calls:

class NewMeta(type):
 pass

def deco(cls):
 cls_dict = dict(cls.__dict__)
 if "__dict__" in cls_dict:
   del cls_dict["__dict__"]
 if "__weakref__" in cls_dict:
   del cls_dict["__weakref__"]
 return NewMeta(cls.__name__, cls.__bases__, cls_dict)

@deco
class B:
 def x(self):
  print("Hi there")

@deco
class A(B):
 def x(self):
  super().x()

Using this code like so, yields an error:

>>> a = A()
>>> a.x()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in x
TypeError: super(type, obj): obj must be an instance or subtype of type

Some terminology:

• The source code class A as produced by class A(B):.
• The produced class A*, as produced by NewMeta(cls.__name__, cls.__bases__, cls_dict).

A is established by Python to be the type when using super inside of the methods of A*. How can I correct this?

There’s some suboptimal solutions like calling super(type(self), self).x, or passing cls.__mro__ instead of cls.__bases__ into the NewMeta call (so that obj=self always inherits from the incorrect type=A). The first is unsustainable for end users, the 2nd pollutes the inheritance chains and is confusing as the class seems to inherit from itself.

Python seems to introspect the source code, or maybe stores some information to automatically establish the type, and in this case, I’d say it is failing to do so;

How could I make sure that inside of the methods of A A* is established as the type argument of argumentless super calls?

Answer

The argument-free super uses the __class__ cell, which is a regular function closure.

Data Model: Creating the class object

__class__ is an implicit closure reference created by the compiler if any methods in a class body refer to either __class__ or super.

>>> class E:
...    def x(self):
...        return __class__  # return the __class__ cell
...
>>> E().x()
__main__.E
>>> # The cell is stored as a __closure__
>>> E.x.__closure__[0].cell_contents is E().x() is E
True

Like any other closure, this is a lexical relation: it refers to class scope in which the method was literally defined. Replacing the class with a decorator still has the methods refer to the original class.

---

The simplest fix is to explicitly refer to the name of the class, which gets rebound to the newly created class by the decorator.

@deco
class A(B):
    def x(self):
        super(A, self).x()

Alternatively, one can change the content of the __class__ cell to point to the new class:

def deco(cls):
    cls_dict = dict(cls.__dict__)
    cls_dict.pop("__dict__", None)
    cls_dict.pop("__weakref__", None)
    new_cls = NewMeta(cls.__name__, cls.__bases__, cls_dict)
    for method in new_cls.__dict__.values():
        if getattr(method, "__closure__", None) and method.__closure__[0].cell_contents is cls:
            method.__closure__[0].cell_contents = new_cls
    return new_cls