tripwire backfires in debian stable environment: test_basic fails

[yarikoptic]

building backport for debian stable and this only test fails on me:

=================================== FAILURES ===================================
__________________________________ test_basic __________________________________

pkg_name = 'nottriedbefore', min_version = None

    def assert_bad(pkg_name, min_version=None):
        pkg, have_pkg, setup = optional_package(pkg_name, min_version=min_version)
        assert not have_pkg
        assert isinstance(pkg, TripWire)
        with pytest.raises(TripWireError):
>           pkg.a_method

nibabel/tests/test_optpkg.py:29: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <nibabel.tripwire.TripWire object at 0x7f2041b628d0>
attr_name = 'a_method'

    def __getattr__(self, attr_name):
        """ Raise informative error accessing attributes """
>       raise TripWireError(self._msg)
E       nibabel.tripwire.TripWireError: We need package nottriedbefore for these functions, but ``import nottriedbefore`` raised non ImportError could be thrown by some malfunctioning module upon import, and optional_package should catch it too

nibabel/tripwire.py:51: TripWireError

During handling of the above exception, another exception occurred:

    def test_basic():
        # We always have os
        assert_good('os')
        # Subpackage
        assert_good('os.path')
        # We never have package _not_a_package
        assert_bad('_not_a_package')
    
        # setup_module imports unittest, so make sure we don't disrupt that
        orig_import = builtins.__import__
        def raise_Exception(*args, **kwargs):
            if args[0] == 'unittest':
                return orig_import(*args, **kwargs)
            raise Exception(
                "non ImportError could be thrown by some malfunctioning module "
                "upon import, and optional_package should catch it too")
        with mock.patch.object(builtins, '__import__', side_effect=raise_Exception):
>           assert_bad('nottriedbefore')

nibabel/tests/test_optpkg.py:51: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
nibabel/tests/test_optpkg.py:29: in assert_bad
    pkg.a_method
/usr/lib/python3/dist-packages/_pytest/_code/code.py:405: in __init__
    import _pytest._code
/usr/lib/python3.7/unittest/mock.py:960: in __call__
    return _mock_self._mock_call(*args, **kwargs)
/usr/lib/python3.7/unittest/mock.py:1026: in _mock_call
    result = effect(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ('_pytest._code', {'CO_VARARGS': 4, 'CO_VARKEYWORDS': 8, 'Code': <class '_pytest._code.code.Code'>, 'ExceptionChainRepr': <class '_pytest._code.code.ExceptionChainRepr'>, ...}, None, None, 0)
kwargs = {}

    def raise_Exception(*args, **kwargs):
        if args[0] == 'unittest':
            return orig_import(*args, **kwargs)
        raise Exception(
>           "non ImportError could be thrown by some malfunctioning module "
            "upon import, and optional_package should catch it too")
E       Exception: non ImportError could be thrown by some malfunctioning module upon import, and optional_package should catch it too

nibabel/tests/test_optpkg.py:48: Exception

not familiar with that tripwire and how Exception should have become a TripWire error. Test passes fine on more recent systems. Here we have pytest 3.10.1 and python 3.7.3-2+deb10u2 .

any clues on how to make test more resilient or is there a real issue revealed here?

[effigies]

Can you describe how to produce an environment that reproduces this?

[yarikoptic]

very original one would be the one following elderly http://neuro.debian.net/blog/2012/2012-04-14_ndtools.html

I've reproduced in docker so smth like docker run -it --rm neurodebian:nd100 in which you do

    sed -i -e 's,#deb-src,deb-src,g' /etc/apt/sources.list.d/neurodebian.sources.list
    cd /tmp
   sed -ie 's,^deb \(.*\),deb \1\ndeb-src \1,g' /etc/apt/sources.list
   apt-get update -qqq
   apt-get install -y git python3-pytest python3-packaging
   apt-get build-dep nibabel
   git clone git://github.com/nipy/nibabel
   cd nibabel/
   python3 -m pytest nibabel/tests/test_optpkg.py

[effigies]

Okay. It looks like pytest is monkeypatching some deep Python internals, and there are function-local imports inside their code that are triggering this mocked __import__.

The simple solution would be to allow any _pytest import to pass through, just like unittest, but that also feels a bit fragile, since they could add other imports later. Maybe we can make the mock more targeted.

[effigies]

This patch should work:

diff --git a/nibabel/tests/test_optpkg.py b/nibabel/tests/test_optpkg.py
index 925180ce..2bc70221 100644
--- a/nibabel/tests/test_optpkg.py
+++ b/nibabel/tests/test_optpkg.py
@@ -39,14 +39,14 @@ def test_basic():
     # We never have package _not_a_package
     assert_bad('_not_a_package')
 
-    # setup_module imports unittest, so make sure we don't disrupt that
+    # Only disrupt imports for "nottriedbefore" package
     orig_import = builtins.__import__
     def raise_Exception(*args, **kwargs):
-        if args[0] == 'unittest':
-            return orig_import(*args, **kwargs)
-        raise Exception(
-            "non ImportError could be thrown by some malfunctioning module "
-            "upon import, and optional_package should catch it too")
+        if args[0] == 'nottriedbefore':
+            raise Exception(
+                "non ImportError could be thrown by some malfunctioning module "
+                "upon import, and optional_package should catch it too")
+        return orig_import(*args, **kwargs)
     with mock.patch.object(builtins, '__import__', side_effect=raise_Exception):
         assert_bad('nottriedbefore')

How far back should we patch?

[effigies]

This was originally written for nose, not unittest/pytest, so I don't think we need to backport all the way to 2.5.x, so just 3.2.x.

[yarikoptic]

while in that environment, although more trivial - transferred another issue from nipype: https://github.com/nipy/nibabel/issues/984