SWAMPy

✎ CARVE FLESH

EDITING: cporting.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
 <meta http-equiv="X-UA-Compatible" content="IE=Edge" />
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <title>Porting Extension Modules to Python 3 &#8212; Python 3.6.7 documentation</title>
 <link rel="stylesheet" href="../_static/pydoctheme.css" type="text/css" />
 <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
 <script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
 <script type="text/javascript" src="../_static/jquery.js"></script>
 <script type="text/javascript" src="../_static/underscore.js"></script>
 <script type="text/javascript" src="../_static/doctools.js"></script>
 <script type="text/javascript" src="../_static/sidebar.js"></script>
 <link rel="search" type="application/opensearchdescription+xml"
 title="Search within Python 3.6.7 documentation"
 href="../_static/opensearch.xml"/>
 <link rel="author" title="About these documents" href="../about.html" />
 <link rel="index" title="Index" href="../genindex.html" />
 <link rel="search" title="Search" href="../search.html" />
 <link rel="copyright" title="Copyright" href="../copyright.html" />
 <link rel="next" title="Curses Programming with Python" href="curses.html" />
 <link rel="prev" title="Porting Python 2 Code to Python 3" href="pyporting.html" />
 <link rel="shortcut icon" type="image/png" href="../_static/py.png" />
 <link rel="canonical" href="https://docs.python.org/3/howto/cporting.html" />
 
 <script type="text/javascript" src="../_static/copybutton.js"></script>

</head><body> 
 <div class="related" role="navigation" aria-label="related navigation">
 <h3>Navigation</h3>
 <ul>
 <li class="right" style="margin-right: 10px">
 <a href="../genindex.html" title="General Index"
 accesskey="I">index</a></li>
 <li class="right" >
 <a href="../py-modindex.html" title="Python Module Index"
 >modules</a> |</li>
 <li class="right" >
 <a href="curses.html" title="Curses Programming with Python"
 accesskey="N">next</a> |</li>
 <li class="right" >
 <a href="pyporting.html" title="Porting Python 2 Code to Python 3"
 accesskey="P">previous</a> |</li>
 <li><img src="../_static/py.png" alt=""
 style="vertical-align: middle; margin-top: -1px"/></li>
 <li><a href="https://www.python.org/">Python</a> &#187;</li>
 <li>
 <a href="../index.html">3.6.7 Documentation</a> &#187;
 </li>

<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Python HOWTOs</a> &#187;</li>
 <li class="right">

<div class="inline-search" style="display: none" role="search">
 <form class="inline-search" action="../search.html" method="get">
 <input placeholder="Quick search" type="text" name="q" />
 <input type="submit" value="Go" />
 <input type="hidden" name="check_keywords" value="yes" />
 <input type="hidden" name="area" value="default" />
 </form>
 </div>
 <script type="text/javascript">$('.inline-search').show(0);</script>
 |
 </li>

</ul>
 </div>

<div class="document">
 <div class="documentwrapper">
 <div class="bodywrapper">
 <div class="body" role="main">
 
 <div class="section" id="porting-extension-modules-to-python-3">
<h1>Porting Extension Modules to Python 3<a class="headerlink" href="#porting-extension-modules-to-python-3" title="Permalink to this headline">¶</a></h1>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">author:</th><td class="field-body">Benjamin Peterson</td>
</tr>
</tbody>
</table>
<div class="topic">
Abstract
Although changing the C-API was not one of Python 3’s objectives,
the many Python-level changes made leaving Python 2’s API intact
impossible. In fact, some changes such as <a class="reference internal" href="../library/functions.html#int" title="int"><code class="xref py py-func docutils literal notranslate">int()</code></a> and
<code class="xref py py-func docutils literal notranslate">long()</code> unification are more obvious on the C level. This
document endeavors to document incompatibilities and how they can
be worked around.
</div>
<div class="section" id="conditional-compilation">
<h2>Conditional compilation<a class="headerlink" href="#conditional-compilation" title="Permalink to this headline">¶</a></h2>
The easiest way to compile only some code for Python 3 is to check
if <code class="xref c c-macro docutils literal notranslate">PY_MAJOR_VERSION</code> is greater than or equal to 3.
<div class="highlight-c notranslate"><div class="highlight"><pre>#if PY_MAJOR_VERSION &gt;= 3
#define IS_PY3K
#endif
</pre></div>
</div>
API functions that are not present can be aliased to their equivalents within
conditional blocks.
</div>
<div class="section" id="changes-to-object-apis">
<h2>Changes to Object APIs<a class="headerlink" href="#changes-to-object-apis" title="Permalink to this headline">¶</a></h2>
Python 3 merged together some types with similar functions while cleanly
separating others.
<div class="section" id="str-unicode-unification">
<h3>str/unicode Unification<a class="headerlink" href="#str-unicode-unification" title="Permalink to this headline">¶</a></h3>
Python 3’s <a class="reference internal" href="../library/stdtypes.html#str" title="str"><code class="xref py py-func docutils literal notranslate">str()</code></a> type is equivalent to Python 2’s <code class="xref py py-func docutils literal notranslate">unicode()</code>; the C
functions are called <code class="docutils literal notranslate">PyUnicode_*</code> for both. The old 8-bit string type has become
<a class="reference internal" href="../library/stdtypes.html#bytes" title="bytes"><code class="xref py py-func docutils literal notranslate">bytes()</code></a>, with C functions called <code class="docutils literal notranslate">PyBytes_*</code>. Python 2.6 and later provide a compatibility header,
<code class="file docutils literal notranslate">bytesobject.h</code>, mapping <code class="docutils literal notranslate">PyBytes</code> names to <code class="docutils literal notranslate">PyString</code> ones. For best
compatibility with Python 3, <code class="xref c c-type docutils literal notranslate">PyUnicode</code> should be used for textual data and
<code class="xref c c-type docutils literal notranslate">PyBytes</code> for binary data. It’s also important to remember that
<code class="xref c c-type docutils literal notranslate">PyBytes</code> and <code class="xref c c-type docutils literal notranslate">PyUnicode</code> in Python 3 are not interchangeable like
<code class="xref c c-type docutils literal notranslate">PyString</code> and <code class="xref c c-type docutils literal notranslate">PyUnicode</code> are in Python 2. The following example
shows best practices with regards to <code class="xref c c-type docutils literal notranslate">PyUnicode</code>, <code class="xref c c-type docutils literal notranslate">PyString</code>,
and <code class="xref c c-type docutils literal notranslate">PyBytes</code>.
<div class="highlight-c notranslate"><div class="highlight"><pre>#include &quot;stdlib.h&quot;
#include &quot;Python.h&quot;
#include &quot;bytesobject.h&quot;

/* text example */
static PyObject *
say_hello(PyObject *self, PyObject *args) {
 PyObject *name, *result;

if (!PyArg_ParseTuple(args, &quot;U:say_hello&quot;, &amp;name))
 return NULL;

result = PyUnicode_FromFormat(&quot;Hello, %S!&quot;, name);
 return result;
}

/* just a forward */
static char * do_encode(PyObject *);

/* bytes example */
static PyObject *
encode_object(PyObject *self, PyObject *args) {
 char *encoded;
 PyObject *result, *myobj;

if (!PyArg_ParseTuple(args, &quot;O:encode_object&quot;, &amp;myobj))
 return NULL;

encoded = do_encode(myobj);
 if (encoded == NULL)
 return NULL;
 result = PyBytes_FromString(encoded);
 free(encoded);
 return result;
}
</pre></div>
</div>
</div>
<div class="section" id="long-int-unification">
<h3>long/int Unification<a class="headerlink" href="#long-int-unification" title="Permalink to this headline">¶</a></h3>
Python 3 has only one integer type, <a class="reference internal" href="../library/functions.html#int" title="int"><code class="xref py py-func docutils literal notranslate">int()</code></a>. But it actually
corresponds to Python 2’s <code class="xref py py-func docutils literal notranslate">long()</code> type—the <a class="reference internal" href="../library/functions.html#int" title="int"><code class="xref py py-func docutils literal notranslate">int()</code></a> type
used in Python 2 was removed. In the C-API, <code class="docutils literal notranslate">PyInt_*</code> functions
are replaced by their <code class="docutils literal notranslate">PyLong_*</code> equivalents.
</div>
</div>
<div class="section" id="module-initialization-and-state">
<h2>Module initialization and state<a class="headerlink" href="#module-initialization-and-state" title="Permalink to this headline">¶</a></h2>
Python 3 has a revamped extension module initialization system. (See
<a class="pep reference external" href="https://www.python.org/dev/peps/pep-3121">PEP 3121</a>.) Instead of storing module state in globals, they should
be stored in an interpreter specific structure. Creating modules that
act correctly in both Python 2 and Python 3 is tricky. The following
simple example demonstrates how.
<div class="highlight-c notranslate"><div class="highlight"><pre>#include &quot;Python.h&quot;

struct module_state {
 PyObject *error;
};

#if PY_MAJOR_VERSION &gt;= 3
#define GETSTATE(m) ((struct module_state*)PyModule_GetState(m))
#else
#define GETSTATE(m) (&amp;_state)
static struct module_state _state;
#endif

static PyObject *
error_out(PyObject *m) {
 struct module_state *st = GETSTATE(m);
 PyErr_SetString(st-&gt;error, &quot;something bad happened&quot;);
 return NULL;
}

static PyMethodDef myextension_methods[] = {
 {&quot;error_out&quot;, (PyCFunction)error_out, METH_NOARGS, NULL},
 {NULL, NULL}
};

#if PY_MAJOR_VERSION &gt;= 3

static int myextension_traverse(PyObject *m, visitproc visit, void *arg) {
 Py_VISIT(GETSTATE(m)-&gt;error);
 return 0;
}

static int myextension_clear(PyObject *m) {
 Py_CLEAR(GETSTATE(m)-&gt;error);
 return 0;
}

static struct PyModuleDef moduledef = {
 PyModuleDef_HEAD_INIT,
 &quot;myextension&quot;,
 NULL,
 sizeof(struct module_state),
 myextension_methods,
 NULL,
 myextension_traverse,
 myextension_clear,
 NULL
};

#define INITERROR return NULL

PyMODINIT_FUNC
PyInit_myextension(void)

#else
#define INITERROR return

void
initmyextension(void)
#endif
{
#if PY_MAJOR_VERSION &gt;= 3
 PyObject *module = PyModule_Create(&amp;moduledef);
#else
 PyObject *module = Py_InitModule(&quot;myextension&quot;, myextension_methods);
#endif

if (module == NULL)
 INITERROR;
 struct module_state *st = GETSTATE(module);

st-&gt;error = PyErr_NewException(&quot;myextension.Error&quot;, NULL, NULL);
 if (st-&gt;error == NULL) {
 Py_DECREF(module);
 INITERROR;
 }

#if PY_MAJOR_VERSION &gt;= 3
 return module;
#endif
}
</pre></div>
</div>
</div>
<div class="section" id="cobject-replaced-with-capsule">
<h2>CObject replaced with Capsule<a class="headerlink" href="#cobject-replaced-with-capsule" title="Permalink to this headline">¶</a></h2>
The <code class="xref c c-type docutils literal notranslate">Capsule</code> object was introduced in Python 3.1 and 2.7 to replace
<code class="xref c c-type docutils literal notranslate">CObject</code>. CObjects were useful,
but the <code class="xref c c-type docutils literal notranslate">CObject</code> API was problematic: it didn’t permit distinguishing
between valid CObjects, which allowed mismatched CObjects to crash the
interpreter, and some of its APIs relied on undefined behavior in C.
(For further reading on the rationale behind Capsules, please see <a class="reference external" href="https://bugs.python.org/issue5630">bpo-5630</a>.)
If you’re currently using CObjects, and you want to migrate to 3.1 or newer,
you’ll need to switch to Capsules.
<code class="xref c c-type docutils literal notranslate">CObject</code> was deprecated in 3.1 and 2.7 and completely removed in
Python 3.2. If you only support 2.7, or 3.1 and above, you
can simply switch to <code class="xref c c-type docutils literal notranslate">Capsule</code>. If you need to support Python 3.0,
or versions of Python earlier than 2.7,
you’ll have to support both CObjects and Capsules.
(Note that Python 3.0 is no longer supported, and it is not recommended
for production use.)
The following example header file <code class="file docutils literal notranslate">capsulethunk.h</code> may
solve the problem for you. Simply write your code against the
<code class="xref c c-type docutils literal notranslate">Capsule</code> API and include this header file after
<code class="file docutils literal notranslate">Python.h</code>. Your code will automatically use Capsules
in versions of Python with Capsules, and switch to CObjects
when Capsules are unavailable.
<code class="file docutils literal notranslate">capsulethunk.h</code> simulates Capsules using CObjects. However,
<code class="xref c c-type docutils literal notranslate">CObject</code> provides no place to store the capsule’s “name”. As a
result the simulated <code class="xref c c-type docutils literal notranslate">Capsule</code> objects created by <code class="file docutils literal notranslate">capsulethunk.h</code>
behave slightly differently from real Capsules. Specifically:
<blockquote>
<div><ul class="simple">
<li>The name parameter passed in to <a class="reference internal" href="../c-api/capsule.html#c.PyCapsule_New" title="PyCapsule_New"><code class="xref c c-func docutils literal notranslate">PyCapsule_New()</code></a> is ignored.</li>
<li>The name parameter passed in to <a class="reference internal" href="../c-api/capsule.html#c.PyCapsule_IsValid" title="PyCapsule_IsValid"><code class="xref c c-func docutils literal notranslate">PyCapsule_IsValid()</code></a> and
<a class="reference internal" href="../c-api/capsule.html#c.PyCapsule_GetPointer" title="PyCapsule_GetPointer"><code class="xref c c-func docutils literal notranslate">PyCapsule_GetPointer()</code></a> is ignored, and no error checking
of the name is performed.</li>
<li><a class="reference internal" href="../c-api/capsule.html#c.PyCapsule_GetName" title="PyCapsule_GetName"><code class="xref c c-func docutils literal notranslate">PyCapsule_GetName()</code></a> always returns NULL.</li>
<li><a class="reference internal" href="../c-api/capsule.html#c.PyCapsule_SetName" title="PyCapsule_SetName"><code class="xref c c-func docutils literal notranslate">PyCapsule_SetName()</code></a> always raises an exception and
returns failure. (Since there’s no way to store a name
in a CObject, noisy failure of <a class="reference internal" href="../c-api/capsule.html#c.PyCapsule_SetName" title="PyCapsule_SetName"><code class="xref c c-func docutils literal notranslate">PyCapsule_SetName()</code></a>
was deemed preferable to silent failure here. If this is
inconvenient, feel free to modify your local
copy as you see fit.)</li>
</ul>
</div></blockquote>
You can find <code class="file docutils literal notranslate">capsulethunk.h</code> in the Python source distribution
as <a class="reference external" href="https://github.com/python/cpython/tree/3.6/Doc/includes/capsulethunk.h">Doc/includes/capsulethunk.h</a>. We also include it here for
your convenience:
<div class="highlight-c notranslate"><div class="highlight"><pre>#ifndef __CAPSULETHUNK_H
#define __CAPSULETHUNK_H

#if ( (PY_VERSION_HEX &lt; 0x02070000) \
 || ((PY_VERSION_HEX &gt;= 0x03000000) \
 &amp;&amp; (PY_VERSION_HEX &lt; 0x03010000)) )

#define __PyCapsule_GetField(capsule, field, default_value) \
 ( PyCapsule_CheckExact(capsule) \
 ? (((PyCObject *)capsule)-&gt;field) \
 : (default_value) \
 ) \

#define __PyCapsule_SetField(capsule, field, value) \
 ( PyCapsule_CheckExact(capsule) \
 ? (((PyCObject *)capsule)-&gt;field = value), 1 \
 : 0 \
 ) \

#define PyCapsule_Type PyCObject_Type

#define PyCapsule_CheckExact(capsule) (PyCObject_Check(capsule))
#define PyCapsule_IsValid(capsule, name) (PyCObject_Check(capsule))

#define PyCapsule_New(pointer, name, destructor) \
 (PyCObject_FromVoidPtr(pointer, destructor))

#define PyCapsule_GetPointer(capsule, name) \
 (PyCObject_AsVoidPtr(capsule))

/* Don&#39;t call PyCObject_SetPointer here, it fails if there&#39;s a destructor */
#define PyCapsule_SetPointer(capsule, pointer) \
 __PyCapsule_SetField(capsule, cobject, pointer)

#define PyCapsule_GetDestructor(capsule) \
 __PyCapsule_GetField(capsule, destructor)

#define PyCapsule_SetDestructor(capsule, dtor) \
 __PyCapsule_SetField(capsule, destructor, dtor)

/*
 * Sorry, there&#39;s simply no place
 * to store a Capsule &quot;name&quot; in a CObject.
 */
#define PyCapsule_GetName(capsule) NULL

static int
PyCapsule_SetName(PyObject *capsule, const char *unused)
{
 unused = unused;
 PyErr_SetString(PyExc_NotImplementedError,
 &quot;can&#39;t use PyCapsule_SetName with CObjects&quot;);
 return 1;
}

#define PyCapsule_GetContext(capsule) \
 __PyCapsule_GetField(capsule, descr)

#define PyCapsule_SetContext(capsule, context) \
 __PyCapsule_SetField(capsule, descr, context)

static void *
PyCapsule_Import(const char *name, int no_block)
{
 PyObject *object = NULL;
 void *return_value = NULL;
 char *trace;
 size_t name_length = (strlen(name) + 1) * sizeof(char);
 char *name_dup = (char *)PyMem_MALLOC(name_length);

if (!name_dup) {
 return NULL;
 }

memcpy(name_dup, name, name_length);

trace = name_dup;
 while (trace) {
 char *dot = strchr(trace, &#39;.&#39;);
 if (dot) {
 *dot++ = &#39;\0&#39;;
 }

if (object == NULL) {
 if (no_block) {
 object = PyImport_ImportModuleNoBlock(trace);
 } else {
 object = PyImport_ImportModule(trace);
 if (!object) {
 PyErr_Format(PyExc_ImportError,
 &quot;PyCapsule_Import could not &quot;
 &quot;import module \&quot;%s\&quot;&quot;, trace);
 }
 }
 } else {
 PyObject *object2 = PyObject_GetAttrString(object, trace);
 Py_DECREF(object);
 object = object2;
 }
 if (!object) {
 goto EXIT;
 }

trace = dot;
 }

if (PyCObject_Check(object)) {
 PyCObject *cobject = (PyCObject *)object;
 return_value = cobject-&gt;cobject;
 } else {
 PyErr_Format(PyExc_AttributeError,
 &quot;PyCapsule_Import \&quot;%s\&quot; is not valid&quot;,
 name);
 }

EXIT:
 Py_XDECREF(object);
 if (name_dup) {
 PyMem_FREE(name_dup);
 }
 return return_value;
}

#endif /* #if PY_VERSION_HEX &lt; 0x02070000 */

#endif /* __CAPSULETHUNK_H */
</pre></div>
</div>
</div>
<div class="section" id="other-options">
<h2>Other options<a class="headerlink" href="#other-options" title="Permalink to this headline">¶</a></h2>
If you are writing a new extension module, you might consider <a class="reference external" href="http://cython.org/">Cython</a>. It translates a Python-like language to C. The
extension modules it creates are compatible with Python 3 and Python 2.
</div>
</div>

</div>
 </div>
 </div>
 <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
 <div class="sphinxsidebarwrapper">
 <h3><a href="../contents.html">Table Of Contents</a></h3>
 <ul>
<li><a class="reference internal" href="#">Porting Extension Modules to Python 3</a><ul>
<li><a class="reference internal" href="#conditional-compilation">Conditional compilation</a></li>
<li><a class="reference internal" href="#changes-to-object-apis">Changes to Object APIs</a><ul>
<li><a class="reference internal" href="#str-unicode-unification">str/unicode Unification</a></li>
<li><a class="reference internal" href="#long-int-unification">long/int Unification</a></li>
</ul>
</li>
<li><a class="reference internal" href="#module-initialization-and-state">Module initialization and state</a></li>
<li><a class="reference internal" href="#cobject-replaced-with-capsule">CObject replaced with Capsule</a></li>
<li><a class="reference internal" href="#other-options">Other options</a></li>
</ul>
</li>
</ul>

<h4>Previous topic</h4>
 <a href="pyporting.html"
 title="previous chapter">Porting Python 2 Code to Python 3</a>
 <h4>Next topic</h4>
 <a href="curses.html"
 title="next chapter">Curses Programming with Python</a>
 <div role="note" aria-label="source link">
 <h3>This Page</h3>
 <ul class="this-page-menu">
 <li><a href="../bugs.html">Report a Bug</a></li>
 <li>
 <a href="https://github.com/python/cpython/blob/3.6/Doc/howto/cporting.rst"
 rel="nofollow">Show Source
 </a>
 </li>
 </ul>
 </div>
 </div>
 </div>
 <div class="clearer"></div>
 </div> 
 <div class="related" role="navigation" aria-label="related navigation">
 <h3>Navigation</h3>
 <ul>
 <li class="right" style="margin-right: 10px">
 <a href="../genindex.html" title="General Index"
 >index</a></li>
 <li class="right" >
 <a href="../py-modindex.html" title="Python Module Index"
 >modules</a> |</li>
 <li class="right" >
 <a href="curses.html" title="Curses Programming with Python"
 >next</a> |</li>
 <li class="right" >
 <a href="pyporting.html" title="Porting Python 2 Code to Python 3"
 >previous</a> |</li>
 <li><img src="../_static/py.png" alt=""
 style="vertical-align: middle; margin-top: -1px"/></li>
 <li><a href="https://www.python.org/">Python</a> &#187;</li>
 <li>
 <a href="../index.html">3.6.7 Documentation</a> &#187;
 </li>

<li class="nav-item nav-item-1"><a href="index.html" >Python HOWTOs</a> &#187;</li>
 <li class="right">

</ul>
 </div> 
 <div class="footer">
 &copy; <a href="../copyright.html">Copyright</a> 2001-2023, Python Software Foundation.
 
 The Python Software Foundation is a non-profit corporation.
 <a href="https://www.python.org/psf/donations/">Please donate.</a>
 
 Last updated on Dec 18, 2023.
 <a href="../bugs.html">Found a bug</a>?
 
 Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.7.6.
 </div>

</body>
</html>

CANCEL

Name	Type	Size	Modified	Actions
↩ ..	DIR	—	—
📄 argparse.html	HTML	79.5 KB	2023-12-18 11:36	EDIT
📄 clinic.html	HTML	128.3 KB	2023-12-18 11:36	EDIT
📄 cporting.html	HTML	40.9 KB	2023-12-18 11:36	EDIT
📄 curses.html	HTML	57.5 KB	2023-12-18 11:36	EDIT
📄 descriptor.html	HTML	58.4 KB	2023-12-18 11:36	EDIT
📄 functional.html	HTML	154.1 KB	2023-12-18 11:36	EDIT
📄 index.html	HTML	8.6 KB	2023-12-18 11:36	EDIT
📄 instrumentation.html	HTML	29.5 KB	2023-12-18 11:36	EDIT
📄 ipaddress.html	HTML	37.9 KB	2023-12-18 11:36	EDIT
📄 logging-cookbook.html	HTML	293.9 KB	2023-12-18 11:36	EDIT
📄 logging.html	HTML	111.2 KB	2023-12-18 11:36	EDIT
📄 pyporting.html	HTML	45.7 KB	2023-12-18 11:36	EDIT
📄 regex.html	HTML	147.9 KB	2023-12-18 11:36	EDIT
📄 sockets.html	HTML	39.7 KB	2023-12-18 11:36	EDIT
📄 sorting.html	HTML	46.1 KB	2023-12-18 11:36	EDIT
📄 unicode.html	HTML	70.5 KB	2023-12-18 11:36	EDIT
📄 urllib2.html	HTML	70.3 KB	2023-12-18 11:36	EDIT