SelectFiles doesn't accept lists as input

[Shotgunosine]

Summary

The documentation for SelectFiles indicates that you should be able to pass lists as inputs for the fields defined the template string. When I try that, I get the error: TypeError: unsupported format string passed to list.__format__

Actual behavior

Crashes with the error: TypeError: unsupported format string passed to list.__format__

Expected behavior

SelectFiles accepts an subject_id as a list and pull the anatomicals from the test session for both subject 1 and subject 2. As shown in the documentation.

How to replicate the behavior

Run the following snippet in the Nipype Tutorial container: miykael/nipype_tutorial:latest

Script/Workflow details

Running in the Nipype Tutorial container: miykael/nipype_tutorial:latest

from nipype import Node, SelectFiles
template = {'anat': 'sub-{subject_id:02d}/ses-{ses_name}/anat/*_T1w.nii.gz'}
sf = Node(SelectFiles(template, 
                      force_lists=True,
                      base_directory='/data/ds000114/',
                      sort_filelist=True),
          name='selectfile')
sf.inputs.subject_id = [1,2]
sf.inputs.ses_name = 'test'
sf.run()

produces:

181004-18:58:46,481 nipype.workflow INFO:
	 [Node] Setting-up "selectfile" in "/tmp/tmpu4a95r0i/selectfile".
181004-18:58:46,496 nipype.workflow INFO:
	 [Node] Running "selectfile" ("nipype.interfaces.io.SelectFiles")
181004-18:58:46,513 nipype.workflow WARNING:
	 [Node] Error on "selectfile" (/tmp/tmpu4a95r0i/selectfile)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-1-7559abb2aeb6> in <module>()
      8 sf.inputs.subject_id = [1,2]
      9 sf.inputs.ses_name = 'test'
---> 10 sf.run()

/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py in run(self, updatehash)
    469 
    470         try:
--> 471             result = self._run_interface(execute=True)
    472         except Exception:
    473             logger.warning('[Node] Error on "%s" (%s)', self.fullname, outdir)

/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py in _run_interface(self, execute, updatehash)
    553             self._update_hash()
    554             return self._load_results()
--> 555         return self._run_command(execute)
    556 
    557     def _load_results(self):

/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py in _run_command(self, execute, copyfiles)
    633         logger.info(message)
    634         try:
--> 635             result = self._interface.run(cwd=outdir)
    636         except Exception as msg:
    637             result.runtime.stderr = '%s\n\n%s'.format(

/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/interfaces/base/core.py in run(self, cwd, ignore_exception, **inputs)
    521             runtime = self._run_interface(runtime)
    522             runtime = self._post_run_hook(runtime)
--> 523             outputs = self.aggregate_outputs(runtime)
    524         except Exception as e:
    525             import traceback

/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/interfaces/base/core.py in aggregate_outputs(self, runtime, needed_outputs)
    595         """
    596 
--> 597         predicted_outputs = self._list_outputs()
    598         outputs = self._outputs()
    599         if predicted_outputs:

/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/interfaces/io.py in _list_outputs(self)
   1414 
   1415             # Fill in the template and glob for files
-> 1416             filled_template = template.format(**info)
   1417             filelist = glob.glob(filled_template)
   1418 

TypeError: unsupported format string passed to list.__format__

Please put URL to code or code here (if not too long).

Platform details:

{'commit_hash': '%h',
 'commit_source': 'archive substitution',
 'networkx_version': '2.1',
 'nibabel_version': '2.3.0',
 'nipype_version': '1.1.3-dev',
 'numpy_version': '1.15.1',
 'pkg_path': '/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype',
 'scipy_version': '1.1.0',
 'sys_executable': '/opt/miniconda-latest/envs/neuro/bin/python',
 'sys_platform': 'linux',
 'sys_version': '3.6.6 | packaged by conda-forge | (default, Jul 26 2018, '
                '09:53:17) \n'
                '[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]',
 'traits_version': '4.6.0'}

Execution environment

Choose one

• Container miykael/nipype_tutorial:latest, environment [conda env:neuro]

[mgxd]

@Shotgunosine lists should still be allowed, the problem is that you are trying to format a list with :02d. If you change your template to

template = {'anat': 'sub-{subject_id}/ses-{ses_name}/anat/*_T1w.nii.gz'}

it should remove the error

[kimsin98]

This is more of an enhancement suggestion, but from what I can observe, SelectFiles doesn't actually support list inputs like DataGrabber.

Given a list as argument, DataGrabber accesses each element in the list, puts it into the template, then matches files. Any lists of strings/numbers behaves as expected.

However, SelectFiles just puts the whole list's string representation into the template. Then, glob module's [ ] handling takes over.

The problem is that, for glob, [ ] defines a character set a la regex. Except in cases with single digits like [0,1,2], character sets behave very differently from lists. For example, sub-[10,11,12]* actually matches sub-1*, sub-0*, sub-2* NOT sub-10*, sub-11*, sub-12* as most users might expect.

This could easily lead to unexpected errors and confusion. I suggest either

1. implement proper list support like DataGrabber or
2. change SelectFiles documentation to clarify that it does NOT support list input and maybe direct users to Python's glob pattern expansion documentation.

[effigies]

I think the latter would be reasonable. SelectFiles was written specifically to be a simpler alternative to DataGrabber, and changing its semantics would break things for a lot of users.