Calling C++ function which accepts and returns std::string from Python

I’m trying to call C++ function using ctypes. The function declaration looks like

std::string some_function( const std::string &param )

As Python can not interact with C++ directly, I have created a C-wrapper for this function as follows

const char *my_function( const char *param )
{
  std::string result = some_function( std::string( param ) );
  char *ret = new char[result.length() + 1];
  strcpy(ret, result.c_str());
  return ret;
}

And Python wrapper

def my_function(param):
    func = lib.my_function
    func.argtypes = [ctypes.c_char_p]
    func.restype = ctypes.c_char_p
    result = func(ctypes.c_char_p(param))
    return result

But when I try to call Python wrapper with some bytes passed as param, for example

my_function(b'GPx00x01xe6x10x00x00x01x01x00x00x00x1exxcbxa16lxf1xbfpxe6xaaxc89x81xdd?')

It fails with the following error

terminate called after throwing an instance of 'std::length_error'
  what():  basic_string::_M_create

I’m not very good in C and C++. Can you help me to figure out how to create correct wrapper? Thanks.

Answer

Since your data contains embedded nulls, c_char_p won’t work as it assumes the returned char* is null-terminated and converts the data up to the first null found to a bytes object. std::string as used also makes that assumption when pass a char* only, so it needs the data length as well.

To manage a data buffer with null content, the size of the input data must be passed in and the size of the output data must be returned. You’ll also have to manage freeing the data allocated in the C++ code.

The below code demonstrates everything:

test.cpp – compiled with cl /EHsc /W4 /LD test.cpp with Microsoft Visual Studio.

#include <string>
#include <string.h>

// Windows DLLs need functions to be explicitly exported
#ifdef _WIN32
#   define API __declspec(dllexport)
#else
#   define API
#endif

std::string some_function(const std::string& param) {
    return param + std::string("somex00thing", 10);
}

// pLength is used as an input/output parameter.
// Pass in input length and get output length back.
extern "C" API
const char* my_function(const char* param, size_t* pLength) {
    auto result = some_function(std::string(param, *pLength));
    auto reslen = result.length();
    auto res = new char[reslen];
    // Use memcpy to handle embedded nulls
    memcpy_s(res, reslen, result.data(), reslen);
    *pLength = reslen;
    return res;
}

extern "C" API
void my_free(const char* param) {
    delete [] param;
}

test.py

import ctypes as ct

dll = ct.CDLL('./test')

# Note: A restype of ct.c_char_p is converted to a bytes object
#       and access to the returned C pointer is lost.  Use
#       POINTER(c_char) instead to retain access and allow it
#       to be freed later.
dll.my_function.argtypes = ct.c_char_p, ct.POINTER(ct.c_size_t)
dll.my_function.restype = ct.POINTER(ct.c_char)
dll.my_free.argtypes = ct.c_char_p,
dll.my_free.restype = None

def my_function(param):
    # wrap size in a ctypes object so it can be passed by reference
    size = ct.c_size_t(len(param))
    # save the returned pointer
    p = dll.my_function(param, ct.byref(size))
    # slice pointer to convert to an explicitly-sized bytes object
    result = p[:size.value]
    # now the pointer can be freed...
    dll.my_free(p)
    # and the bytes object returned
    return result

result = my_function(b'datax00more')
print(result)

Output:

b'datax00moresomex00thing'