Lock when importing nipype.interfaces.spm in version 0.10.0

[m-guggenmos]

I just encountered a deadlock (or alike) after updating to 0.10.0. This lock (by which I simply mean that the code doesn't proceed) happens already at the import of nipype.interfaces.spm when running the following code:

from nipype.interfaces.matlab import MatlabCommand
MatlabCommand.set_default_paths("/opt/matlab/64bit/R2015a/toolbox/spm12")
MatlabCommand.set_default_matlab_cmd("/opt/matlab/64bit/R2015a/bin/matlab -nodesktop -nosplash")
import nipype.interfaces.spm as spm

This issue does not occur for me on 0.9.2 (here the import of nipype.interfaces.spm is executed in less than a second). I have switched back and forth a couple of times now between 0.10.0 and 0.9.2, and the pattern is consistent. Upon pressing Ctrl+C, I get the following output in 0.10.0: http://pastebin.com/9S1BJjsw

Of note, a similar lock was previously reported for 0.9.1 in these two links:

http://stackoverflow.com/questions/24544726/subprocess-get-stucks-at-poller-poll
https://groups.google.com/forum/#!topic/nipy-user/ICJzyP6sLek

Versions: Ubuntu 15.04 / Python 2.7.8 / Matlab R2015a / SPM12

[satra]

@monade - could you please update to the latest master and try?

in the past nipype used to create a lock file in the home directory. could also try deleting the lock file if there is one? (~/.nipype/nipype.json)

[m-guggenmos]

@satra:

Calling pip install https://github.com/nipy/nipype/zipball/master (correct?) installed 0.11.0.dev0. The problem still persists: http://pastebin.com/1QM0JDEs

(nb: there was no such file in the .nipype directory)

[satra]

@monade - i see that you are using matlab. are you on a debian/ubuntu system?

if so could you try setting the matlab command to this (replacing the variables):

$pathtomatlabdir/bin/$platform/MATLAB

[m-guggenmos]

@satra - thanks for your suggestions. I tried it and it did not change the outcome.

[m-guggenmos]

Now it gets a bit more complicated. I can reproduce the above behavior on two Ubuntu machines running Matlab R2015a and R2015b. However, on the computer with Matlab R2015b, which has a pretty identical setup otherwise, I now end up in a similar lock when executing the following minimal working example (using 0.9.2):

from nipype.interfaces.matlab import MatlabCommand
MatlabCommand.set_default_paths('/opt/matlab/R2015b/toolbox/spm12')
MatlabCommand.set_default_matlab_cmd("/opt/matlab/R2015b/bin/glnxa64/MATLAB -nodesktop -nosplash")
import nipype.interfaces.spm as spm # <-- here it hangs with 0.10.0

smooth = spm.Smooth()
smooth.inputs.in_files = '/data/mpr/1010/mpr/s1010-mpr.nii'
smooth.inputs.fwhm = 6
smooth.run() # <-- here it hangs with 0.9.2 with Matlab R2015b

When pressing Ctrl+C I get: http://pastebin.com/MvhrU3vh

The code runs through on the computer with Matlab R2015a.

Could this be an issue with very recent Matlab versions?

[satra]

that's a possibility - i'll test it later tonight on our cluster. i have mostly been using 2013, but will check with the latest versions.

[m-guggenmos]

I think i have an idea about the failure to import SPM-related stuff from nipy. Although I am setting the matlab command with MatlabCommand.set_default_matlab_cmd(), this does not set os.environ['MATLABCMD']. However, the import code tries to infer the location of the matlab binary from this environment variable. Setting the environment variable manually seems to help with this first issue at first glance.

[m-guggenmos]

In fact, it seems that setting the environment variable solves the other problem as well. Will report back after running a couple of tests tomorrow.

[m-guggenmos]

All issues were indeed resolved by setting the environment variable.

[satra]

i'm on a centos server, and the following sequence works fine and no MATLABCMD variable is set:

from nipype.interfaces.matlab import MatlabCommand
MatlabCommand.set_default_paths('/cm/shared/openmind/spm/spm12/spm12_r6225/')
MatlabCommand.set_default_matlab_cmd("/cm/shared/mit/matlab/matlab-2015a/bin/matlab")
import nipype.interfaces.spm as spm

(i don't have 2015b to test)

In [10]: spm.Info.version()
Out[10]: 
{'name': 'SPM12',
 'path': '/cm/shared/openmind/spm/spm12/spm12_r6225',
 'release': '6225'}

In [11]: os.environ['MATLABCMD']
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-11-35bc0f58a02b> in <module>()
----> 1 os.environ['MATLABCMD']

/om/user/satra/envs/testenv/lib/python2.7/UserDict.pyc in __getitem__(self, key)
     21         if hasattr(self.__class__, "__missing__"):
     22             return self.__class__.__missing__(self, key)
---> 23         raise KeyError(key)
     24     def __setitem__(self, key, item): self.data[key] = item
     25     def __delitem__(self, key): del self.data[key]

KeyError: 'MATLABCMD'

[m-guggenmos]

Strange, did you add /cm/shared/mit/matlab/matlab-2015a/bin/ to the $PATH environment variable or did you symlink to /usr/bin or alike? The point being that in the critical section of the code (interfaces.spm.base, function version(), line 158ff in my case), first the environment variable is checked and second "matlab" is called without path info (which afaik then would have to be in $PATH, which it is not in my case):

try:
    matlab_cmd = os.environ['MATLABCMD']
except KeyError:
    matlab_cmd = 'matlab -nodesktop -nosplash'

I also noted that the variable matlab_cmd could in fact be passed to version(), but i think it is not passed during the initial import of spm.

[satra]

try this. make sure you:

1. don't have an environment variable called MATLABCMD
2. don't import MatlabCommand

then do just the following (note the matlab command doesn't have any args).

import nipype.interfaces.spm as spm
spm.SPMCommand.set_mlab_paths(matlab_cmd='path/to/bin/matlab',
                                                            paths='/path/to/spm12')d
SPMCommand().version()

the SPMCommand setting ensures that any spm node imported after this call will be run with those variables. if this works you should be all set.

[satra]

actually spm.Info.version() won't work, so i updated it.

[m-guggenmos]

this doesn't work, because the import of spm already calls the version() function

edit: saw your second comment just now, will try with the new master