terminal_output

[djarecka]

Summary

A question on neurostar motivated me to check terminal_output and I'm not sure if I understand how this should work. For example, I set terminal_output="file" for node with BET and I can see that the file output.nipype is created, but I it's removed before the node ends. In addition, I think the file is empty before removing anyway and the output is sent to my terminal.

Can someone please explain me what is the expected behavior.

[satra]

the file is removed because it's not needed downstream from a node. if you run it as an interface the file will stay. BET i don't think produces any stdout

the only files kept by a node are listed here:
https://github.com/nipy/nipype/blob/master/nipype/pipeline/engine/utils.py#L1411

any stdout/error is available via the runtime field of the result from an execution of an interface/node

[djarecka]

So this is not a bug and terminal_output is not meant to save the output/error for later references?

So this will not help with this qst?

[satra]

@djarecka - that is currently true. however there are some interfaces like fsl.ImageStats, where we interpret the terminal output into an output key.

we could consider whether runtime information more generally can be used as an input to something else, but that is something we currently don't support.

however i think the information from a node can be suppressed (which i think is the question) in different ways:

1. changing logging levels (this will suppress everything)
2. for spm, unfortunately this comes from the matlab interface: https://github.com/nipy/nipype/blob/master/nipype/interfaces/matlab.py#L161

which defaults to allatonce. this would need to be configurable to use file instead.

[spanta28]

Hi Satra, thanks for the reply. So I am using spm12 standalone and matlab mcr.
I changed my /opt/miniconda/envs/default/lib/python3.5/site-packages/nipype/interfaces/matlab.py
def _run_interface(self, runtime):
self.terminal_output = 'file'

Then it gives me the foll. error:
Python 3.5.1 | packaged by conda-forge | (default, Jun 29 2016, 12:18:14)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.

from nipype.interfaces import spm
spm.SPMCommand.set_mlab_paths(matlab_cmd='/opt/spm12/run_spm12.sh /opt/mcr/v92 script', use_mcr=True)
Traceback (most recent call last):
File "", line 1, in
File "/opt/miniconda/envs/default/lib/python3.5/site-packages/nipype/interfaces/spm/base.py", line 316, in set_mlab_paths
use_mcr=use_mcr)
File "/opt/miniconda/envs/default/lib/python3.5/site-packages/nipype/interfaces/spm/base.py", line 231, in getinfo
out = sd._strip_header(out.runtime.stdout)
File "/opt/miniconda/envs/default/lib/python3.5/site-packages/nipype/utils/spm_docs.py", line 53, in _strip_header
index = doc.index(hdr)
ValueError: substring not found

My spm12 standalone and mcr is running fine with the 'allatonce' option, but prints things to stdout.
Let me know what I can do here. Also for 1) let me know how I can change the logging levels, that would be helpful.

Thanks

[satra]

@spanta28 - that may be a bug - @djarecka - could you take a look at this within the nipype tutorial container?

regarding logging: http://nipype.readthedocs.io/en/latest/users/config_file.html#enabling-logging-to-file

also look at the debug section. if you set logging to CRITICAL only errors will be posted.

[djarecka]

@satra - just confirming that I can see a similar error after changing self.terminal_output.

[spanta28]

@satra I tried the logging acc. to
http://nipype.readthedocs.io/en/latest/users/config_file.html#enabling-logging-to-file
Still its printing out nipype outputs as before to stdout

[spanta28]

I will try the debug CRITICAL thing now

[spanta28]

This worked for me in suppressing nipype.workflow to stdout
from nipype import logging
logging.getLogger('nipype.workflow').setLevel('CRITICAL')

Thanks @satra

[djarecka]

I've been debugging the ValueError, it looks like that results["stdout"] reads stream only for allatonce, otherwise it's empty.
At the end strip_header is called with an empty string and this gives the error.

@satra - any suggestions what should be changed? Is it ok that results["stdout"] is empty and I only should change strip_header, so it works for an empty string?

[satra]

@djarecka - perhaps write a few tests with a shell script that write out to both stdout and stderr and see that all the terminal_output options return the appropriate values.