Skip to content

Make multiprocessing import local to support pyodide #7519

Open
@twiecki

Description

@twiecki

Description

We almost have PyMC working natively under pyodide. But upon import we get an error importing multiprocessing which isn't included in pyodide:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[5], line 1
----> 1 import pymc as pm
      2 pm.__version__

File /lib/python3.12/site-packages/pymc/__init__.py:72
     70 from pymc.pytensorf import *
     71 from pymc.sampling import *
---> 72 from pymc.smc import *
     73 from pymc.stats import *
     74 from pymc.step_methods import *

File /lib/python3.12/site-packages/pymc/smc/__init__.py:16
      1 #   Copyright 2024 The PyMC Developers
      2 #
      3 #   Licensed under the Apache License, Version 2.0 (the "License");
   (...)
     12 #   See the License for the specific language governing permissions and
     13 #   limitations under the License.
     15 from pymc.smc.kernels import IMH, MH
---> 16 from pymc.smc.sampling import sample_smc
     18 __all__ = ("sample_smc",)

File /lib/python3.12/site-packages/pymc/smc/sampling.py:21
     18 import warnings
     20 from collections import defaultdict
---> 21 from concurrent.futures import ProcessPoolExecutor, wait
     22 from typing import Any
     24 import cloudpickle

File /lib/python312.zip/concurrent/futures/__init__.py:44, in __getattr__(name)
     41 global ProcessPoolExecutor, ThreadPoolExecutor
     43 if name == 'ProcessPoolExecutor':
---> 44     from .process import ProcessPoolExecutor as pe
     45     ProcessPoolExecutor = pe
     46     return pe

File /lib/python312.zip/concurrent/futures/process.py:55
     52 # This import is required to load the multiprocessing.connection submodule
     53 # so that it can be accessed later as `mp.connection`
     54 import multiprocessing.connection
---> 55 from multiprocessing.queues import Queue
     56 import threading
     57 import weakref

File /lib/python312.zip/multiprocessing/queues.py:23
     19 import errno
     21 from queue import Empty, Full
---> 23 import _multiprocessing
     25 from . import connection
     26 from . import context

ModuleNotFoundError: No module named '_multiprocessing'

If we made that optional, it will work out of the box.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions