Capturing print output from shared library called from python with ctypes module

I am working with a shared library that is being called through the ctypes module. I would like to redirect the stdout associated with this module to a variable or a file that I can access in my program. However ctypes uses a separate stdout from sys.stdout.

I’ll demonstrate the problem I am having with libc. If anyone is copying and pasting the code they might have to change the filename on line 2.

import ctypes
libc = ctypes.CDLL('libc.so.6')

from cStringIO import StringIO
import sys
oldStdOut = sys.stdout
sys.stdout = myStdOut = StringIO()

print 'This text gets captured by myStdOut'
libc.printf('This text fails to be captured by myStdOutn')

sys.stdout = oldStdOut
myStdOut.getvalue()

Is there any way I can capture the stdout that is associated with the ctypes loaded shared library?

Answer

We can use os.dup2() and os.pipe() to replace the entire stdout file descriptor (fd 1) with a pipe we can read from ourselves. You can do the same thing with stderr (fd 2).

This example uses select.select() to see if the pipe (our fake stdout) has data waiting to be written, so we can print it safely without blocking execution of our script.

As we are completely replacing the stdout file descriptor for this process and any subprocesses, this example can even capture output from child processes.

import os, sys, select

# the pipe would fail for some reason if I didn't write to stdout at some point
# so I write a space, then backspace (will show as empty in a normal terminal)
sys.stdout.write(' b')
pipe_out, pipe_in = os.pipe()
# save a copy of stdout
stdout = os.dup(1)
# replace stdout with our write pipe
os.dup2(pipe_in, 1)

# check if we have more to read from the pipe
def more_data():
        r, _, _ = select.select([pipe_out], [], [], 0)
        return bool(r)

# read the whole pipe
def read_pipe():
        out = ''
        while more_data():
                out += os.read(pipe_out, 1024)

        return out

# testing print methods
import ctypes
libc = ctypes.CDLL('libc.so.6')

print 'This text gets captured by myStdOut'
libc.printf('This text fails to be captured by myStdOutn')

# put stdout back in place 
os.dup2(stdout, 1)
print 'Contents of our stdout pipe:'
print read_pipe()