Skip to content

GH-134160: Prefer multi-phase initialisation in docs #134764

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 60 additions & 45 deletions Doc/c-api/module.rst
Original file line number Diff line number Diff line change
Expand Up @@ -137,9 +137,15 @@
(where the initialization function is added using :c:func:`PyImport_AppendInittab`).
See :ref:`building` or :ref:`extending-with-embedding` for details.

The initialization function can either pass a module definition instance
to :c:func:`PyModule_Create`, and return the resulting module object,
or request "multi-phase initialization" by returning the definition struct itself.
To perform ':ref:`multi-phase initialization <multi-phase-initialization>`'
(:pep:`489`), the initialization function must return a pointer to the
:c:type:`module definition struct <PyModuleDef>`.
This allows Python to determine which capabilities the module supports *before*
it is executed, as creation and initialization are split,
similarly to the :py:meth:`!__new__` and :py:meth:`!__init__` on classes.

The legacy method (prior to Python 3.5) to specify an extension module is
':ref:`single-phase initialization <single-phase-initialization>`'.

.. c:type:: PyModuleDef

Expand Down Expand Up @@ -189,7 +195,7 @@

An array of slot definitions for multi-phase initialization, terminated by
a ``{0, NULL}`` entry.
When using single-phase initialization, *m_slots* must be ``NULL``.
When using legacy single-phase initialization, *m_slots* must be ``NULL``.

.. versionchanged:: 3.5

Expand Down Expand Up @@ -249,52 +255,22 @@
.. versionchanged:: 3.9
No longer called before the module state is allocated.

Single-phase initialization
...........................

The module initialization function may create and return the module object
directly. This is referred to as "single-phase initialization", and uses one
of the following two module creation functions:

.. c:function:: PyObject* PyModule_Create(PyModuleDef *def)

Create a new module object, given the definition in *def*. This behaves
like :c:func:`PyModule_Create2` with *module_api_version* set to
:c:macro:`PYTHON_API_VERSION`.


.. c:function:: PyObject* PyModule_Create2(PyModuleDef *def, int module_api_version)

Create a new module object, given the definition in *def*, assuming the
API version *module_api_version*. If that version does not match the version
of the running interpreter, a :exc:`RuntimeWarning` is emitted.

Return ``NULL`` with an exception set on error.

.. note::

Most uses of this function should be using :c:func:`PyModule_Create`
instead; only use this if you are sure you need it.

Before it is returned from in the initialization function, the resulting module
object is typically populated using functions like :c:func:`PyModule_AddObjectRef`.

.. _multi-phase-initialization:

Multi-phase initialization
..........................

An alternate way to specify extensions is to request "multi-phase initialization".
The preferred method to specify extensions is to request "multi-phase initialization".
Extension modules created this way behave more like Python modules: the
initialization is split between the *creation phase*, when the module object
is created, and the *execution phase*, when it is populated.
The distinction is similar to the :py:meth:`!__new__` and :py:meth:`!__init__` methods
of classes.

Unlike modules created using single-phase initialization, these modules are not
singletons: if the *sys.modules* entry is removed and the module is re-imported,
a new module object is created, and the old module is subject to normal garbage
collection -- as with Python modules.
Unlike modules created using the legacy single-phase initialization mechanism,
these modules are not singletons: if the *sys.modules* entry is removed and
the module is re-imported, a new module object is created, and the old module
is subject to normal garbage collection -- as with Python modules.
Comment on lines -287 to +273
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this shouldn't be a comparison. If we're calling single-phase “legacy” (which finally seems to be the cunsensus), then this section should describe what modules are, and the single-phase section should describe the eldritch weirdness of that method.
Should I try to reword it that way? (I'd like to avoid merge conflicts.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is existing mention of single-phase as 'legacy', though admittedly in a niche location: https://docs.python.org/3/c-api/init.html#c.PyInterpreterConfig.check_multi_interp_extensions

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I try to reword it that way? (I'd like to avoid merge conflicts.)

@encukou feel free to push to this branch.

By default, multiple modules created from the same definition should be
independent: changes to one should not affect the others.
This means that all state should be specific to the module object (using e.g.
Expand All @@ -320,7 +296,12 @@
.. versionadded:: 3.5

The *m_slots* member of the module definition must point to an array of
``PyModuleDef_Slot`` structures:
:c:type:`PyModuleDef_Slot` structures, terminated by a slot with id 0.

See :PEP:`489` for more details on multi-phase initialization.

Module slots
............

.. c:type:: PyModuleDef_Slot

Expand All @@ -334,8 +315,6 @@

.. versionadded:: 3.5

The *m_slots* array must be terminated by a slot with id 0.

The available slot types are:

.. c:macro:: Py_mod_create
Expand Down Expand Up @@ -446,7 +425,43 @@

.. versionadded:: 3.13

See :PEP:`489` for more details on multi-phase initialization.
.. _single-phase-initialization:

Single-phase initialization
...........................

.. attention::
Single-phase initialization is a legacy mechanism to initialize extension
modules, with known drawbacks and design flaws. Extension module authors
are encouraged to use multi-phase initialization instead.

The module initialization function may create and return the module object
directly. This is referred to as "single-phase initialization", and uses one
of the following two module creation functions, returning the resulting
module object:

.. c:function:: PyObject* PyModule_Create(PyModuleDef *def)

Create a new module object, given the definition in *def*. This behaves

Check warning on line 445 in Doc/c-api/module.rst

View workflow job for this annotation

GitHub Actions / Docs / Docs

c:macro reference target not found: PYTHON_API_VERSION [ref.macro]
like :c:func:`PyModule_Create2` with *module_api_version* set to
:c:macro:`PYTHON_API_VERSION`.


.. c:function:: PyObject* PyModule_Create2(PyModuleDef *def, int module_api_version)

Create a new module object, given the definition in *def*, assuming the
API version *module_api_version*. If that version does not match the version
of the running interpreter, a :exc:`RuntimeWarning` is emitted.

Return ``NULL`` with an exception set on error.

.. note::

Most uses of this function should be using :c:func:`PyModule_Create`
instead; only use this if you are sure you need it.

Before it is returned from in the initialization function, the resulting module
object is typically populated using functions like :c:func:`PyModule_AddObjectRef`.

Low-level module creation functions
...................................
Expand Down Expand Up @@ -677,8 +692,8 @@
.. versionadded:: 3.13


Module lookup
^^^^^^^^^^^^^
Module lookup (single-phase initialization)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Single-phase initialization creates singleton modules that can be looked up
in the context of the current interpreter. This allows the module object to be
Expand Down
10 changes: 7 additions & 3 deletions Doc/howto/free-threading-extensions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Extensions that use multi-phase initialization (i.e.,
module definition. If your extension supports older versions of CPython,
you should guard the slot with a :c:data:`PY_VERSION_HEX` check.

::
.. code-block:: c

static struct PyModuleDef_Slot module_slots[] = {
...
Expand All @@ -70,13 +70,17 @@ you should guard the slot with a :c:data:`PY_VERSION_HEX` check.
Single-Phase Initialization
...........................

Extensions that use single-phase initialization (i.e.,
Extensions that use legacy single-phase initialization (i.e.,
:c:func:`PyModule_Create`) should call :c:func:`PyUnstable_Module_SetGIL` to
indicate that they support running with the GIL disabled. The function is
only defined in the free-threaded build, so you should guard the call with
``#ifdef Py_GIL_DISABLED`` to avoid compilation errors in the regular build.

::
Note that this function is part of the :pep:`unstable C API <689>`, meaning
that it may change without warning between minor releases. Where possible,
prefer using multi-phase initialization with the ``Py_mod_gil`` slot.

.. code-block:: c

static struct PyModuleDef moduledef = {
PyModuleDef_HEAD_INIT,
Expand Down
Loading