Closed
Description
Hi everyone,
I'm trying to run the examples and having some issues. It seems like torchscript fails to load the model that comes with the AI Spleen Seg Data example. I wonder if it's because I don't have an NVIDIA GPU (I'm trying this on a Macbook), and maybe the model was trained using CUDA.
I get this error:
DEBUG:__main__.AISpleenSegApp:Begin compose
DEBUG:__main__.AISpleenSegApp:End compose
DEBUG:__main__.AISpleenSegApp:Begin run
Going to initiate execution of operator DICOMDataLoaderOperator
Executing operator DICOMDataLoaderOperator (Process ID: 51436, Operator ID: 18939fcb-44ac-4b32-9329-1c5f83d246f6)
Done performing execution of operator DICOMDataLoaderOperator
Going to initiate execution of operator DICOMSeriesSelectorOperator
Executing operator DICOMSeriesSelectorOperator (Process ID: 51436, Operator ID: 7ea4fcfb-399a-46f1-98e4-73736df15273)
Done performing execution of operator DICOMSeriesSelectorOperator
Going to initiate execution of operator DICOMSeriesToVolumeOperator
Executing operator DICOMSeriesToVolumeOperator (Process ID: 51436, Operator ID: e5275132-befe-4ada-9b8b-31cd415ad5c0)
Done performing execution of operator DICOMSeriesToVolumeOperator
Going to initiate execution of operator SpleenSegOperator
Executing operator SpleenSegOperator (Process ID: 51436, Operator ID: 1272eced-c34e-4012-a1c8-87fb662e35f3)
False
cpu
Traceback (most recent call last):
File "app.py", line 75, in <module>
app_instance.run()
File "app.py", line 37, in run
super().run(*args, **kwargs)
File "/Users/Erik/opt/anaconda3/lib/python3.8/site-packages/monai/deploy/core/application.py", line 429, in run
executor_obj.run()
File "/Users/Erik/opt/anaconda3/lib/python3.8/site-packages/monai/deploy/core/executors/single_process_executor.py", line 125, in run
op.compute(op_exec_context.input_context, op_exec_context.output_context, op_exec_context)
File "/Users/Erik/Projects/LNQ/monai-deploy-app-sdk/examples/apps/ai_spleen_seg_app/spleen_seg_operator.py", line 87, in compute
infer_operator.compute(op_input, op_output, context)
File "/Users/Erik/opt/anaconda3/lib/python3.8/site-packages/monai/deploy/operators/monai_seg_inference_operator.py", line 180, in compute
d[self._pred_dataset_key] = sliding_window_inference(
File "/Users/Erik/opt/anaconda3/lib/python3.8/site-packages/monai/inferers/utils.py", line 130, in sliding_window_inference
seg_prob = predictor(window_data, *args, **kwargs).to(device) # batched patch segmentation
File "/Users/Erik/opt/anaconda3/lib/python3.8/site-packages/monai/deploy/core/models/model.py", line 220, in __call__
if self.predictor:
File "/Users/Erik/opt/anaconda3/lib/python3.8/site-packages/monai/deploy/core/models/torch_model.py", line 50, in predictor
self._predictor = torch.jit.load(self.path).eval()
File "/Users/Erik/opt/anaconda3/lib/python3.8/site-packages/torch/jit/_serialization.py", line 161, in load
cpp_module = torch._C.import_ir_module(cu, str(f), map_location, _extra_files)
NotImplementedError: Could not run 'aten::empty_strided' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::empty_strided' is only available for these backends: [CPU, Meta, BackendSelect, Named, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, UNKNOWN_TENSOR_TYPE_ID, AutogradMLC, AutogradHPU, AutogradNestedTensor, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, Tracer, Autocast, Batched, VmapMode].
CPU: registered at aten/src/ATen/RegisterCPU.cpp:16286 [kernel]
Meta: registered at aten/src/ATen/RegisterMeta.cpp:9460 [kernel]
BackendSelect: registered at aten/src/ATen/RegisterBackendSelect.cpp:609 [kernel]
Named: registered at ../aten/src/ATen/core/NamedRegistrations.cpp:7 [backend fallback]
ADInplaceOrView: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:60 [backend fallback]
AutogradOther: registered at ../torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradCPU: registered at ../torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradCUDA: registered at ../torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradXLA: registered at ../torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at ../torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradMLC: registered at ../torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradHPU: registered at ../torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradNestedTensor: registered at ../torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradPrivateUse1: registered at ../torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradPrivateUse2: registered at ../torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradPrivateUse3: registered at ../torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
Tracer: registered at ../torch/csrc/autograd/generated/TraceType_0.cpp:9750 [kernel]
Autocast: fallthrough registered at ../aten/src/ATen/autocast_mode.cpp:255 [backend fallback]
Batched: registered at ../aten/src/ATen/BatchingRegistrations.cpp:1019 [backend fallback]
VmapMode: fallthrough registered at ../aten/src/ATen/VmapModeRegistrations.cpp:33 [backend fallback]```
It looks like in the Torchscript FAQ, they address the fact that the model may need to be saved twice, once for GPU inference, and one for CPU inference: https://pytorch.org/docs/stable/jit.html#frequently-asked-questions
So I was wondering if that Google Drive link for the Spleen Seg Data needs to be updated to include a CPU version of the model.
Thanks,
Erik
_Originally posted by @swederik in https://github.com/Project-MONAI/monai-deploy-app-sdk/discussions/156_