CPython is a C-based implementation of Python. It even contains an API. This allows CPython to be used as an external library and linked with a separate application written in C. Functions defined by an “embedding” program may be called via Python script, running on the embedded interpreter. As such, Blender itself embeds a Python interpreter, providing access to some of its own functions. Python-callable functions must be registered using the Python API. This is done by Blender’s python module.
The Blender python Module
Blender makes use of Python in multiple ways. One focus in this chapter is the python module and how its code calls the interpreter. We also look at function registration for the mathutils Python API module.
Because Python is object-oriented, Blender creates “built-in” Python modules and classes, some of whose methods map to Blender’s internal C structs (as in the case of struct bContext) and internal module APIs, for example, the bmesh module. mathutils’s mathematical objects, such as vectors and matrices, are assigned a Python class. Math utility functions from blenlib become methods for these same Python classes.
Source Files and Directories
![../images/495914_1_En_5_Chapter/495914_1_En_5_Fig1_HTML.jpg](../images/495914_1_En_5_Chapter/495914_1_En_5_Fig1_HTML.jpg)
Top-level directory in the python module. rna_dump.py and simple_enum_gen.py generate C source code. In simple_enum_gen.py, you can see calls that generate PyModule_AddObject(), a Python API function. BPY_extern.h supplies prototypes for the python module API. BPY_extern_clog.h is used for logging
![../images/495914_1_En_5_Chapter/495914_1_En_5_Fig2_HTML.jpg](../images/495914_1_En_5_Chapter/495914_1_En_5_Fig2_HTML.jpg)
Partial dependency graph for the subdirectories of the python module. Arrows point in the direction of the dependency
The python module provides a Python API for parts of the codebase. There are internal dependencies between subdirectories as well. One salient example is source/blender/python/intern’s dependency on source/blender/python/generic/.
The python module API
The python module API prototypes are provided by source/blender/python/BPY_extern.h. As with other modules, the API is called either inside or outside of the module. A sample of the BPY_* functions is shown in Listing 5-1.
Snippet from BPY_extern.h, representing part of the module’s API. Function names are in boldface
Extending and Embedding Python
Before we go into the mechanics of the python module, we need to understand how to run the Python interpreter using the CPython API. Python’s primary documentation related to this is “Embedding Python in Another Application.”1 Most of this information can be distilled to just a number of steps.
Note that CPython API prototypes are in Python.h. A program using the CPython API must contain the preprocessor directive #include <Python.h>, assuming Python is in the system path.
When building Blender, there is a CMake variable that can be set to use the Python installation or to download binaries to be placed in the build directory under [build directory]/bin/share/blender/[Blender version]/python/.
Adding Python Extensions
PyModule_New()
PyModule_Create()
PyModule_AddObject()
PyImport_ExtendInittab()
PyModule_New() and PyModule_Create() both are used to add an extension module. For instance, Blender’s Python mathutils module is added by PyModule_Create(), while its bpy module is via PyModule_New(). PyModule_New() was introduced in version 3.3 of the CPython, after PyModule_Create(). You will see both functions used for creating new modules in Blender.
PyModule_AddObject() adds a class to a module. For example, the mathutils.Vector is added as a built-in Python class using PyModule_AddObject(). PyImport_ExtendInittab() registers modules to be included in Python’s initialization table (i.e., Inittab from the function name). This table contains extension modules to be registered when Python is initialized.
PyObject
PyTypeObject
PyMethodDef
PyModuleDef
PyCFunction
All Python objects are “derived” from PyObject. PyObject structs are “extended,” by adding PyObject_HEAD to a user-defined struct.
In C, there is no explicit language mechanism for inheritance. By adding common fields to the beginning of a user-defined struct, we can treat these structs as having inherited those fields. In CPython, user-defined structs starting with PyObject_HEAD are treated as objects with the base type PyObject. We saw an example of this in Chapter 2, for “DNA” types. There, the ID struct is placed at the beginning of each “DNA” type. This method of “inheritance” is used extensively in Blender’s codebase.
PyObject_HEAD is expanded by the preprocessor into two fields. One field points to an object of PyTypeObject, the other a count of the object’s references.2
PyTypeObject contains, in addition to function pointers for an object’s type, a PyVarObject_HEAD_INIT macro similar to PyObject_HEAD. PyTypeObject also has a field for a PyObject’s size.
Some Python methods, for example, __init__(), are added as fields to a PyTypeObject, the one using it as its type definition. Other types of methods must be added to a PyMethodDef object. The PyTypeObject.tb_methods field points to a methods table (an object of type PyMethodDef). Note that there is a separate PyGetSetDef methods table, used extensively in the implementation of the mathutils.Vector class.
The PyMethodDef and PyModuleDef objects of the mathutils extension module (source/blender/python/mathutils/mathutils.c). mathutils does not contain functions defined in the mathutils namespace alone. All functionality is attached to its various classes or functions in its submodules (geometry, bvhtree, etc.)
Py_Initialize()
Py_Finalize()
PyEval_EvalCode()
Abbreviated initialization table in source/blender/python/intern/bpy_interface.c. This table is used to add extensions to the list of “built-in” modules, before a call to Py_Initialize(). Each entry in bpy_internal_modules is structs, with an identifier for the module and a function pointer to the initialization function PyMODINIT_FUNC. Each of these modules is then initialized with member classes, functions, etc
BPY_execute_filepath() and BPY_execute_text() both call PyEval_EvalCode(), which in turn calls python_script_exec(). python_script_exec() is defined in source/blender/python/intern/bpy_interface.c. If the script has not already been compiled, then Py_CompileStringObject() is first executed. The resulting Python bytecode is placed in Text’s compiled field. source/blender/python/bpy.c and source/blender/python/bpy_interface.c (among others) make extensive use of the CPython API.
The mathutils Extension Module
![../images/495914_1_En_5_Chapter/495914_1_En_5_Fig3_HTML.jpg](../images/495914_1_En_5_Chapter/495914_1_En_5_Fig3_HTML.jpg)
Dependencies of source/blender/python/mathutils/. There are separate bmesh nodes in the graph. The red bmesh node represents the external dependency source/blender/bmesh/. The black bmesh node represents source/blender/python/bmesh/, a dependency within the python module
![../images/495914_1_En_5_Chapter/495914_1_En_5_Fig4_HTML.jpg](../images/495914_1_En_5_Chapter/495914_1_En_5_Fig4_HTML.jpg)
The mathutils directory from the python module. The directory contains the implementation of the mathutils Python module. mathutils features math utilities representing objects such as vectors, matrices, and quaternions
Implementation of the mathutils.Vector Class
In this section, we go over the implementation of the mathutils.Vector Python class. It is a good example to illustrate how Blender implements an extension module’s class.
VectorObject
The BASE_MATH_MEMBERS macro (defined in source/blender/python/mathutils/mathutils.h). PyObject_VAR_HEAD is included in BASE_MATH_MEMBERS, and shown in boldface
The VectorObject struct. This is the “derived” PyObject for mathutils.Vector. BASE_MATH_MEMBERS is shown in boldface
Adding Methods to vector_Type
The vector_Type definition for mathutils.Vector. tp_alloc is NULL. This is significant because Vector_new() must allocate a VectorObject. Since tp_alloc is NULL, a call will be made to _PyObject_GC_New()—a CPython API function—using the BASE_MATH_NEW macro
The C functions that implement the getter and setter are Vector_length_get and Vector_length_set, respectively. The struct also contains the Python docstring, displayed when the Python’s built-in help() is called in a script. The last field of the PyGetSetDef struct is left NULL, as it is the optional closure slot.
Vector_new()’s Implementation
Abbreviated Vector_new() from python/mathutils/mathutils_Vector.c. Vector_new() creates VectorObjects. Notice that it calls blenlib’s math utility copy_vn_fl() , shown in boldface
Once called, Vector_new() assigns its local variable vec, to a dynamically allocated array of floats via PyMem_Malloc(). The allocation uses the CPython API, not Blender’s intern/guardedalloc library (see Chapter 1).
copy_vn_fl() from source/blender/blenlib/intern/math_vector.c. The function starts at the last component of the array and decrements the tar pointer down to the first entry of vec, assigning each entry with the parameter val
source/blender/python/mathutils/mathutils_Vector.h’s prototypes for instantiating VectorObjects . These are used for a copy constructor, swizzling operators, etc. source/blender/python/mathutils/mathutils_Vector.c contains the function definitions
Vector_CreatePyObject_alloc() and an abbreviated Vector_CreatePyObject_wrap(). Subsequent calls for allocation are in boldface
The BASE_MATH_NEW macro defined in source/blender/python/mathutils/mathutils.h. BASE_MATH_NEW issues a call to either the tp_alloc method or _PyObject_GC_New(), a CPython function
Exploration of other extension classes and __new__() method implementations reveal differences with Vector_new(). mathutils.Vector is not tied to a Blender “DNA” type. Extension classes that have corresponding “DNA” are likely to use Blender “RNA.”
Vector_normalize()’s Implementation
The PyMethodDef for Vector.normalize()
Vector_normalize(), which defines the Python extension method Vector.normalize(). The blendlib function normalize_vn() is in boldface
Other mathutils.Vector Methods
Abbreviated Vector_methods[] from mathutils_Vector.c. The sentinel and the “Fill” method registration are both shown in boldface
Submodules in mathutils
Example of the mathutils.noise submodule initialization and function registration (from source/blender/python/mathutils/mathutils_noise.c). The submodule initialization function is PyInit_mathutils_noise(), shown in boldface. M_Noise_methods[] is abbreviated for brevity
Abbreviated bpy_internal_modules[] and BPY_python_start() from source/blender/python/intern/bpy_interface.c
Summary
In this chapter, we discussed many of the fundamentals of Python extension, as handled in the python module. However, we did not discuss issues related to multithreading, while running the Python interpreter. A mutual exclusion lock (mutex) is necessary, so that the CPython library obtains the “global interpreter lock” (GIL), when working with shared PyObjects. This ensures that PyObjects are not updated by more than one running interpreter. The GIL is managed by various functions of the CPython API.
The python module also uses the CPython API to provide extensions to the Python interpreter. These extensions can be called from a Python script executed by Blender. We saw key parts of mathutils.Vector’s implementation in Blender’s codebase and how C functions are called when a script invokes mathutils.Vector’s methods. This process is similar for all Blender Python extension classes.