Skip to content
Advertisement

How to use memory_profiler (python module) with class methods?

I want to profile time and memory usage of class method. I didn’t find an out of box solution for this (are there such modules?), and I decided to use timeit for time profiling and memory_usage from memory_profiler module.

I faced a problem of profiling methods with memory_profiler. I’ve tried different variants, and none of them worked.

When I try to use partial from functools, I get this error:

JavaScript

By the way, exactly the same approach works fine with timeit function.

When I try to use lambda as was I got this error:

JavaScript

How can I handle class methods with memory_profiler?

PS: I have memory-profiler (0.26) (installed with pip).

UPD: It’s actually bug. You can check status here: https://github.com/pythonprofilers/memory_profiler/issues/47

Advertisement

Answer

If you want to see the change in memory allocated to the Python VM, you can use psutil. Here is a simple decorator using psuil that will print the change in memory:

JavaScript

In all likelihood, the output you will see will say no change in memory. This is because for small allocations, the Python VM may not need to request more memory from the OS. If you allocate a large array, you will see something different:

JavaScript

Here you should see some change in memory.

If you want to see how much memory is used by each allocated object, the only tool I know of is heapy

JavaScript

I have not used it in a long time, so I recommend experimenting and reading the documentation. Note that for an application using a large amount of memory, it can be extremely slow to calculate this information.

User contributions licensed under: CC BY-SA
1 People found this is helpful
Advertisement