AutoWIG: Wrapping very large C++ libraries in Python automatically

Most of Python and R scientific packages incorporate compiled scientific libraries to speed up the execution of the code needed for high-performance computing and to reuse legacy libraries.

Several semi-automatic solutions exist to wrap these compiled libraries: SWIG, Cython, Boost.Python. However, the process of wrapping a large C++ library is cumbersome and time consuming, mainly due some high-level constructs that have no equivalent in Python (template, complex iterators, ...).

In this talk, we introduce AutoWIG, a Python package that enables full C++ introspection using LLVM/Clang technologies. Default strategies have been designed to transform any C++ construct into Python, using Boost.Python for instance. Based on the introspection, a set of classes, methods, namespaces are retrieve and Boost.Python code is generated using the Mako template engine.

Our approach is automatic, extensible, and applies to complex C++ libraries, composed of thousands of classes or incorporating modern meta-programming constructs. For instance, we wrap with AutoWIG the full Clang library to obtain full introspection on C++ code such as template classes. Jupyter notebooks and Docker images render AutoWIG easy to pick up and play with. The source code is hosted on GitHub and binaries can be installed with Conda. Designed as a library, this package can easily be integrated with compiler toolchains, such as SCons, to ease the development process of teams. A more detailed description of this package can be read on this preprint.

During the talk, we will describe AutoWIG main features and cover how you can generate Python bindings for your C++ libraries illustrating by various Jupyter notebooks. In particular, we will explain how we have bootstrapped the wrapping of LLVM/Clang in Python, using libclang, to provide unavailable features such as introspection on C++ templates from Python. An example of the wrapping of C++ linear algebra and statistical libraries to benefit of state-of-the-art graphical model learning algorithms will be discussed in depth.