In our last thrilling adventure, I mentioned a few ways I didn’t put Python into my app. Having settled on Boost.Python, the documentation naturally contained a few simple examples to get me started, and some very obscure reference material, but relatively little of the intermediate stuff. You know, like basically all documentation for anything that has some nonobvious gotchas, it never makes it easy to figure out YOUR problem when you don’t know enough about it to understand what’s going on. This post covers a few specific warts that I lost time on. They won’t necessarily be what you lost time on, but that are the examples that would have shaved a week off of my project if I had found them all on one place at the right moment, so maybe you will find them useful. Here’s the simple example from the docs:
#include <boost/python.hpp> BOOST_PYTHON_MODULE(hello_ext) { using namespace boost::python; def("greet", greet); }
So, let’s start with BOOST_PYTHON_MODULE(foo). According to module.hpp, that resolves to BOOST_PYTHON_MODULE_INIT, which can have a slightly different definition depending on platform. And that definition contains some other preprocessor macros, which have some other macros. It’s quite a few layers to pick apart what’s actually going on, even if you are willing to sit down and try to read Boost code, which is something you should generally try to avoid doing. So, what does it do that you actually care about? It creates a function called void initfoo() that runs executes the stuff in the module block at run time. Initially, after a brief glance over the docs I had assumed that the “boost::python::def()” was doing some magic to generate classes at compile time, and I didn’t realise that it was just a function that executed at runtime. So, you can stick any C++ in there that you like, such as ‘std::cout<< “When does this happen? << std::endl;’ when you are trying to figure out what is going on, or do something “interesting.” And, if you are embedding Python in your app, rather than building a standalone module, you will need to deal with that function yourself. Which isn’t so bad, except that in order to mesh with what the Python libs expect since Python is written in C, you need to be aware that the function initfoo() is declared as ‘extern “C”.’ At least if the module declaration and your actual embedding setup are happening in different places. Which is never the case in teh simple examples in the docs, but in teh context of a larger application that will definitely be an issue.
Okay, so moving on to dealing with classes. The next slightly fancier mini example shows exposing a simple class to Python.
#include <boost/python.hpp> using namespace boost::python; BOOST_PYTHON_MODULE(hello) { class_<World>("World") .def("greet", &World::greet) .def("set", &World::set); }
But if you have, for example, a Node class that can reference a Graph, which the Graph class can also reference a Node, you might try this:
bpy::class_<Node>("node").def("getGraph", &playback::node::getGraph);
bpy::class_<Graph>("nodalGraph").def("getNode", &playback::nodalGraph::getNode);
And then you’ll spend some time angry about a rather vague compiler error, and lose half a day after you through things were going swimmingly based on how easy the trivial example built. You may even start looking at all the other options I looked at in the first blog post and see if maybe they will work better than Boost. But fear not. The problem is that we can’t create a method of Node that refers to a Graph until we first create the Python binding for Graph. Of course, by that logic we also can’t do the binding for Graph first since it has a method that deals with Node which would need to be bound first. Continue to fear not! It’s a little nonobvious in the trivial example, but binding a class and binding the methods of the class don’t need to happen at the same time. boost::python::class_<foo>(“bar”) constructs an instance of the template class named “class_” which is templated with type “foo” and exposed to Python with the name “bar”. (In practice you’ll usually want foo and bar to be the same name.) That instance is something that we can hang on to. Which means we can make a bunch of them for all the various classes that we need to wrap.
auto nodeBinding = bpy::class_<NodeWrap, boost::noncopyable>("node", bpy::no_init);
auto graphBinding = bpy::class_<GraphWrap, boost::noncopyable>("nodalGraph", bpy::no_init);
nodeBinding.def("getGraph", &playback::node::getGraph, bpy::return_internal_reference<>());
graphBinding.def("getNode", &playback::nodalGraph::getNode, bpy::return_internal_reference<>());
See, that’s not so bad, right? I am using noncopyable and no_init because I want to create these things only on the well manicured C++ side. A Node should ALWAYS be made by the Graph, and the Graph should always live in a project, etc. I just wanted Python to have access to existing objects, so that was the easiest way to do it given things like private constructors, and no copy constructor. I’m only about 10% sure that return_internal_reference is actually correct for my use of returning bare pointers. I’m still slogging my way through some understanding there.
So, what about when the methods you are trying to bind are ambiguous? Or you need to return arrays of stuff? Well, you need member function pointers when you have methods with the same name. Which is something that I use rarely enough I think they look really weird. This is another thing that took me a while to figure out because I got confused by the docs, and I couldn’t find a useful example. BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS sure sounds like it would have something to make the pain go away, but that didn’t turn out to be what I needed. So, we’ll start with the declaration of a generic parameter class in the header:
class Parameter : public SomeBase {
public:
explicit Parameter(OutputType T, std::string parameterName, QObject *parent);
explicit Parameter(OutputType T, std::string parameterName);
virtual void setValue(int newValue);
virtual void setValue(double newValue);
virtual void setValue(Color newValue);
virtual void setValue(std::string newValue);
std::vector<std::string> getTags();
private:
// Some implementation nonsense to arcane to consider exposing to mere Python scripters.
}
Then this is the Python binding for that class. Note that one of those returns a vector of strings. You’ll need to bind every template variation of vector that you use, even for standard stuff like strings, so I include my binding of a vector of strings using the vector indexing suite for the sake of a slightly more complete example.
auto parameterBinding = bpy::class_<core::Parameter, boost::noncopyable>("Parameter", bpy::init<core::OutputType, std::string>());bpy::class_<StringList>("StringList") .def(bpy::vector_indexing_suite<StringList>() );parameterBinding.def("setValue", static_cast<void (core::Parameter::*)(int)>(&core::Parameter::setValue) ) .def("setValue", static_cast<void (core::Parameter::*)(double)>(&core::Parameter::setValue) ) .def("setValue", static_cast<void (core::Parameter::*)(core::Color)>(&core::Parameter::setValue) ) .def("setValue", static_cast<void (core::Parameter::*)(std::string)>(&core::Parameter::setValue) ) .def("getTags", &core::Parameter::getTags);
Note that the original C++ class has two constructors, but I only expose one to Python. That’s fine. So far, I haven’t really needed to expose every last constructor that seems useful in C++ to the scripting engine. YMMV, just pick the constructor you like based on the types it uses as above. Then wait to see if anybody files any support tickets about actually needing another constructor. Then I have a bunch of different versions of setValue() which take different types and do different things and have the same name. It’s admittedly not something to hold up as a great example of perfect design, but you will see similar things in real C++ projects, so now you have an example. Doesn’t that pointer to member function syntax seem gross? I think “static_cast<void (core::Parameter::*)(double)>(&core::Parameter::setValue)” belongs in a god damned zoo. But it does work.
And since we have figured out how to do bindings for a bunch of cool stuff now, that brings us to actually doing the Python embedding. All of the trivial examples are based on building a module to run from standalone Python. But if you want to script an existing app, it’s not always practical to build your app as a giant python module and have the python side do all the driving. In my case, I had an existing native C++ app that I wanted to add scripting features to. So, the next post will be about some of that magic. In the context of an app with multiple modules to be exposed, and how to layer things so that the UI can trigger Python, but Python can also expose the UI in a cross platform way. You knew it was going to be spread across three posts when you saw the first one was titled “An unexpected Journey!”
Will Rosecrans Blog 