Description
from rick herrick @ XNAT:
I was thinking more about nipype last night, especially invocation steps and what it would take to integrate nipype as a parallel to or replacement for our current pipeline architecture. One of the biggest issues that we run into with our current pipeline architecture is the “fire and forget” nature of the pipeline processes. Simply put, when it’s time to launch a processing pipeline, we basically create a database entry, launch the process, and hope that it finishes. The discussion list is rife with fixes for this issue, e.g. how to clear database entries, relaunch the process, check process status (you can’t J), etc.
Providing this sort of spawn and management capability would be a huge advantage in both easing the adoption of a processing technology and providing a toolset for monitoring, profiling, and debugging processing jobs once they’ve been created. What I’m thinking would be fantastic would be a Django-based front-end for nipype (just because of the ease of integration from Django to python modules installed on the system; Jython or something else might work as a bridge from Tomcat) that exposes all of the configuration and process management interfaces as (at least initially) REST calls. A typical transaction would be something like:
· /nipype/process/{tool}/launch/{params}
The request body could contain JSON or XML to specify parameters for the process launch, including resources to be processed, etc.
The response entity would contain process ID to allow for status monitoring.
· /nipype/process/{id}/status/
The response entity could contain current status (in-process, completed, error, etc.), aggregate statistics, resource paths for completed jobs, etc.
And if the REST API can be sufficiently standardized and abstracted, that would allow for later implementation of tools in other environments if, for whatever reason, someone wanted to have Freesurfer and BRAINS launched from, e.g., Ruby or Java. The only thing that would need to change for the client would be the server address. Thus an API defined for launching Freesurfer would specify a tool ID and XML, JSON, and hashmap (for simple HTML forms support) interfaces for tool parameters.