(JIT) Compilation of Python code with FFI library calls

I’m using Python with a library that uses cppyy to get access to a ton of C++ functions natively in Python. However, the calls to cppyy methods take a lot of time, and looping in Python with a library call means that overhead becomes a serious issue. Here’s an example of what I mean:

import ROOT  # Library based on cppyy

def some_function():
    for i in range(10_000_000):
        ROOT.some_method(i)  # A little bit of overhead * 10_000_000 is a lot of overhead

This code would be really fast if I was using the ROOT library directly in C++, but Python is really slow.

Is there a way to JIT compile this to run quickly, where the compiled version doesn’t use the Python runtime (like numba nopython=True)?

Answer

There will have to be very little work done in some_method for the cppyy overhead to matter … Note that there is also the overhead of the method lookup (i.e. the ROOT. bit) and the iterator protocol in Python itself isn’t negligible either if cppyy overhead isn’t (timing an empty loop, I find it’s 20% of the total overhead right there).

To avoid all that if this is a tight inner loop, just invert it and run it in C++. E.g. compare these:

import cppyy
import time

N = 10000000

cppyy.cppdef("""
void cppmethod(int i) {
   /* empty */
}""")

f = cppyy.gbl.cppmethod

ts = time.perf_counter()
for i in range(N):
    f(i)
te = time.perf_counter()-ts
print("python loop:", te)

cppyy.cppdef("""
template<typename T>
void call_cppmethod(T f, int N) {
    for (int i = 0; i < N; ++i) f(i);
}""")

ts = time.perf_counter()
cppyy.gbl.call_cppmethod(f, N)
te = time.perf_counter()-ts
print("callback loop:", te)

On my laptop the callback is 100x faster than the Python loop. However, if I do something in cppmethod, eg. calling a trigonometric function, the difference already drops down to 10x (with, again, a good chunk being just the python loop).

Just to be sure, the above use of a template for the callback works in cppyy proper. ROOT, however, contains a fork of cppyy that is quite out of date. There you can’t use the template and will have to be explicit about the function type:

cppyy.cppdef("""
void call_cppmethod(void (*f)(int), int N) {
    for (int i = 0; i < N; ++i) f(i);
}""")

Likely that problem will go away at some point in the future if/when ROOT updates their fork, but for now it is what it is.