RuntimeError: Number of dpcpp devices should be greater than zero!

[axel588]

Hello,
I used the gpu configuration
oneAPI is installed correctly
I am in a python virtual environment ai_tr
I have this issue with Pytorch, the two import are on the top of the file :
import torch
import intel_extension_for_pytorch as ipex

I runned: source ${ONEAPI_ROOT}/setvars.s
with output :

(ai_tr) axel@Artishima:~/ai_tr/cod$ source ${ONEAPI_ROOT}/setvars.sh

:: WARNING: setvars.sh has already been run. Skipping re-execution.
   To force a re-execution of setvars.sh, use the '--force' option.
   Using '--force' can result in excessive use of your environment variables.

usage: source setvars.sh [--force] [--config=file] [--help] [...]
  --force        Force setvars.sh to re-run, doing so may overload environment.
  --config=file  Customize env vars using a setvars.sh configuration file.
  --help         Display this help message and exit.
  ...            Additional args are passed to individual env/vars.sh scripts
                 and should follow this script's arguments.

  Some POSIX shells do not accept command-line options. In that case, you can pass
  command-line options via the SETVARS_ARGS environment variable. For example:

  $ SETVARS_ARGS="ia32 --config=config.txt" ; export SETVARS_ARGS
  $ . path/to/setvars.sh

  The SETVARS_ARGS environment variable is cleared on exiting setvars.sh.

With --force :

(ai_tr) axel@Artishima:~/ai_tr/cod$ source ${ONEAPI_ROOT}/setvars.sh --force

:: initializing oneAPI environment ...
   -bash: BASH_VERSION = 5.1.16(1)-release
   args: Using "$@" for setvars.sh arguments: --force
:: advisor -- latest
:: ccl -- latest
:: compiler -- latest
:: dal -- latest
:: debugger -- latest
:: dev-utilities -- latest
:: dnnl -- latest
:: dpcpp-ct -- latest
:: dpl -- latest
:: ipp -- latest
:: ippcp -- latest
:: mkl -- latest
:: mpi -- latest
:: tbb -- latest
:: vpl -- latest
:: vtune -- latest
:: oneAPI environment initialized ::

But this error keep appearing whenether I try to run my training python file:
xpu /home/axel/ai_tr/lib/python3.10/site-packages/intel_extension_for_pytorch/xpu/lazy_init.py:73: UserWarning: DPCPP Device count is zero! (Triggered internally at /build/intel-pytorch-extension/csrc/gpu/runtime/Device.cpp:120.) _C._initExtension() /home/axel/ai_tr/lib/python3.10/site-packages/torch/nn/modules/module.py:985: UserWarning: dpcppSetDevice: device_id is out of range (Triggered internally at /build/intel-pytorch-extension/csrc/gpu/runtime/Device.cpp:159.) return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) Traceback (most recent call last): File "/home/axel/ai_tr/cod/train.py", line 190, in <module> m = model.to(device) File "/home/axel/ai_tr/lib/python3.10/site-packages/torch/nn/modules/module.py", line 987, in to return self._apply(convert) File "/home/axel/ai_tr/lib/python3.10/site-packages/torch/nn/modules/module.py", line 639, in _apply module._apply(fn) File "/home/axel/ai_tr/lib/python3.10/site-packages/torch/nn/modules/module.py", line 662, in _apply param_applied = fn(param) File "/home/axel/ai_tr/lib/python3.10/site-packages/torch/nn/modules/module.py", line 985, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) RuntimeError: Number of dpcpp devices should be greater than zero!

Everything related to mkl is installed correctl and path are set correctly and working, I am on ubuntu 22.04 using torch 13.1 on WSL 2 on windows 11 with intel drivers installed on windows 11 on Arc 770 with i9 13900K.

The error is trigerred here :

#the line below is triggering the error
m = model.to(device)
m = ipex.optimize(m)
# print the number of parameters in the model
print(sum(p.numel() for p in m.parameters())/1e6, 'M parameters')

# create a PyTorch optimizer
optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)```

Also the name optimize is a weird naming.

It does seems to be an out of range issue, I have no idea how to solve this issue.

[axel588]

I updated to pytorch 1.13.1 instead of 1.13.a0 ... should update documentation,

I have this error :

Traceback (most recent call last):
  File "/home/axel/ai_tr/cod/train.py", line 2, in <module>
    import intel_extension_for_pytorch as ipex
  File "/home/axel/ai_tr/lib/python3.10/site-packages/intel_extension_for_pytorch/__init__.py", line 2, in <module>
    from . import cpu
  File "/home/axel/ai_tr/lib/python3.10/site-packages/intel_extension_for_pytorch/cpu/__init__.py", line 2, in <module>
    from . import runtime
  File "/home/axel/ai_tr/lib/python3.10/site-packages/intel_extension_for_pytorch/cpu/runtime/__init__.py", line 3, in <module>
    from .multi_stream import MultiStreamModule, get_default_num_streams, \
  File "/home/axel/ai_tr/lib/python3.10/site-packages/intel_extension_for_pytorch/cpu/runtime/multi_stream.py", line 4, in <module>
    import intel_extension_for_pytorch._C as core
ImportError: /home/axel/ai_tr/lib/python3.10/site-packages/intel_extension_for_pytorch/lib/libintel-ext-pt-gpu.so: undefined symbol: _ZNK5torch8autograd4Node4nameB5cxx11Ev

Much weirder is the package it asked to install :

(ai_tr) axel@Artishima:~/ai_tr/cod$ python -m pip install torch==1.13.1 -f https://developer.intel.com/ipex-whl-stable-xpu
Looking in links: https://developer.intel.com/ipex-whl-stable-xpu
DEPRECATION: The HTML index page being used (https://www.intel.com/content/dam/develop/external/us/en/documents/ipex/whl-stable-xpu.html) is not a proper HTML 5 document. This is in violation of PEP 503 which requires these pages to be well-formed HTML 5 documents. Please reach out to the owners of this index page, and ask them to update this index page to a valid HTML 5 document. pip 22.2 will enforce this behaviour change. Discussion can be found at https://github.com/pypa/pip/issues/10825
Collecting torch==1.13.1
  Downloading torch-1.13.1-cp310-cp310-manylinux1_x86_64.whl (887.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 887.5/887.5 MB 2.0 MB/s eta 0:00:00
Collecting nvidia-cuda-nvrtc-cu11==11.7.99
  Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.0/21.0 MB 65.7 MB/s eta 0:00:00
Collecting nvidia-cudnn-cu11==8.5.0.96
  Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 557.1/557.1 MB 6.5 MB/s eta 0:00:00
Collecting nvidia-cuda-runtime-cu11==11.7.99
  Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 849.3/849.3 KB 46.9 MB/s eta 0:00:00
Collecting nvidia-cublas-cu11==11.10.3.66
  Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.1/317.1 MB 18.9 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions in /home/axel/ai_tr/lib/python3.10/site-packages (from torch==1.13.1) (4.4.0)
Collecting wheel
  Downloading wheel-0.38.4-py3-none-any.whl (36 kB)
Requirement already satisfied: setuptools in /home/axel/ai_tr/lib/python3.10/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch==1.13.1) (59.6.0)
Installing collected packages: wheel, nvidia-cuda-nvrtc-cu11, nvidia-cuda-runtime-cu11, nvidia-cublas-cu11, nvidia-cudnn-cu11, torch
  Attempting uninstall: torch
    Found existing installation: torch 1.13.0a0+gitb1dde16
    Uninstalling torch-1.13.0a0+gitb1dde16:
      Successfully uninstalled torch-1.13.0a0+gitb1dde16
Successfully installed nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 torch-1.13.1 wheel-0.38.4

I reinstalled from wheel pytorch :

pip install torch-1.13.0a0+gitb1dde16-cp310-cp310-linux_x86_64.whl

still the same error : number of dpcpp devices ..

[jingxu10]

@sanchitintel Would you help to check this issue?

[gujinghui]

Please install the torch 1.13.0 provided in below page.
https://github.com/intel/intel-extension-for-pytorch/releases/tag/v1.13.10%2Bxpu

[asirvaiy]

@axel588 could you tell, what else are you importing before line m =model.to(device)?
If you are importing matplotlib.pyplot before it, try removing it.

[sanchitintel]

Hi @Alex588, can you please share your full script?
Please ensure that the lines import torch and import intel_extension_for_pytorch are present before any other module's imports in your python script. We had encountered a similar issue & have been using such a workaround.

[kns1966]

I've seen the same error under WSL2 + Ubuntu 22.04 (Arc a770) with the GPU installation. Most recently, I tried the official releases, listed above, for python 3.9.x and 3.10.x.

In addition, running on the CPU will return a warning about the number of dpcpp devices. It also just crashed my machine. Perhaps available memory was exhausted.

[nathanodle]

Check that you have resizable BAR enabled in the system BIOS

[charitarthchugh]

I am facing the same issue. I have checked if ReBAR is enabled using lspci -v.

The code that I am using is the starter code in the readme for the GPU. I am also using the recommended packages as well from the README.
System Details:
Fedora 38
i7-12700H
ArcA370M
Appropriate Torchvision package (0.14.0) was installed with --no-check-deps

[jingxu10]

Could you run https://github.com/intel/intel-extension-for-pytorch/raw/master/scripts/collect_env.py and share the outputs?

[charitarthchugh]

Sure! here is the output: https://pastebin.com/ZFcLesPB

[jingxu10]

please use oneapi basekit 2023.0 with 1.13.10. Also, it seems like you don't have level-0 installed. Please install it as well. Driver version better to be 540, as shown in the installation guide.

DPCPP runtime version: 2023.1.0 <====================
MKL version: 2023.1.0 <===========================
GPU models and configuration: 
 
Intel OpenCL ICD version: 23.05.25593.18-1.fc38
Level Zero version: N/A <==========================

dnf/yum install -y intel-opencl level-zero intel-level-zero-gpu

By the way, we will have a new release soon. Probably you can try the new version directly soon.

[charitarthchugh]

I am trying to install level-zero. There seems to be no repos for fedora, so would I be able to install the RHEL one?

[jingxu10]

Since Fedora is not in the list of our verified OSs, we cannot provide support unfortunately.
But personally I would recommend you to have a try with the RHEL one.