unable to pip install jiter==0.5.0

[laserjim]

I'm trying to install jiter==0.5.0 using the JVM graalpy EA (24.1.0-ea.23) am running into the following error:

  × Building wheel for jiter (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [79 lines of output]
      Running `maturin pep517 build-wheel -i /root/.venv/bin/graalpy --compatibility off`
      🔗 Found pyo3 bindings
      🐍 Found GraalPy 3.11 at /root/.venv/bin/graalpy
      📡
         Compiling wyz v0.5.1
         Compiling lexical-parse-integer v0.8.6
         Compiling num-bigint v0.4.5
         Compiling getrandom v0.2.15
         Compiling zerocopy v0.7.34
         Compiling indoc v2.0.5
         Compiling unindent v0.2.3
         Compiling funty v2.0.0
         Compiling bitvec v1.0.1
         Compiling lexical-parse-float v0.8.5
         Compiling smallvec v1.13.2
         Compiling pyo3-macros-backend v0.22.0
         Compiling pyo3-ffi v0.22.0
         Compiling pyo3 v0.22.0
         Compiling jiter v0.5.0 (/tmp/pip-install-d9upxpk2/jiter_1f9ad73d0d774e0886b2183f3fbba9a7/crates/jiter)
         Compiling pyo3-macros v0.22.0
      error[E0425]: cannot find function, tuple struct or tuple variant `PyUnicode_KIND` in module `pyo3::ffi`
         --> crates/jiter/src/py_string_cache.rs:207:33
          |
      207 |       debug_assert_eq!(pyo3::ffi::PyUnicode_KIND(ptr), pyo3::ffi::PyUnicode_1BYTE_KIND);
          |                                   ^^^^^^^^^^^^^^ help: a function with a similar name exists: `PyUnicode_Fill`
          |
         ::: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-ffi-0.22.0/src/cpython/unicodeobject.rs:579:5
          |
      579 | /     pub fn PyUnicode_Fill(
      580 | |         unicode: *mut PyObject,
      581 | |         start: Py_ssize_t,
      582 | |         length: Py_ssize_t,
      583 | |         fill_char: Py_UCS4,
      584 | |     ) -> Py_ssize_t;
          | |___________________- similarly named function `PyUnicode_Fill` defined here
      
      error[E0425]: cannot find function, tuple struct or tuple variant `PyUnicode_DATA` in module `pyo3::ffi`
         --> crates/jiter/src/py_string_cache.rs:208:31
          |
      208 |       let data_ptr = pyo3::ffi::PyUnicode_DATA(ptr).cast();
          |                                 ^^^^^^^^^^^^^^ help: a function with a similar name exists: `PyUnicode_Fill`
          |
         ::: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-ffi-0.22.0/src/cpython/unicodeobject.rs:579:5
          |
      579 | /     pub fn PyUnicode_Fill(
      580 | |         unicode: *mut PyObject,
      581 | |         start: Py_ssize_t,
      582 | |         length: Py_ssize_t,
      583 | |         fill_char: Py_UCS4,
      584 | |     ) -> Py_ssize_t;
          | |___________________- similarly named function `PyUnicode_Fill` defined here
      
      For more information about this error, try `rustc --explain E0425`.
      error: could not compile `jiter` (lib) due to 2 previous errors
      warning: build failed, waiting for other jobs to finish...
      💥 maturin failed
        Caused by: Failed to build a native library through cargo
        Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_ENVIRONMENT_SIGNATURE="graalpy-3.11-64bit" PYO3_PYTHON="/root/.venv/bin/graalpy" PYTHON_SYS_EXECUTABLE="/root/.venv/bin/graalpy" "cargo" "rustc" "--features" "pyo3/extension-module" "--message-format" "json-render-diagnostics" "--manifest-path" "/tmp/pip-install-d9upxpk2/jiter_1f9ad73d0d774e0886b2183f3fbba9a7/crates/jiter-python/Cargo.toml" "--release" "--lib" "--crate-type" "cdylib"`
      Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/root/.venv/bin/graalpy', '--compatibility', 'off'] returned non-zero exit status 1
      [end of output]

Any ideas?

[laserjim]

Looks like pydantic-core==2.20.1 has exactly the same issue, so this is not jiter specific. Presumably anything that depends on pyo3-ffi will encounter this error?

[msimacek]

It is jiter specific after all, pydantic-core uses jiter (via a cargo dependency). I'm going to make a patch for jiter on our side and try to submit it upstream. For pydantic-core, changing the cargo dependency is quite tricky to do on our side, so I don't plan to do that at the moment unless you really that version. We have a patch for pydantic-core==2.10.1 and that one should work for now. pydantic-core has a very fast release cycle, so hopefully, my jiter patch will be merged and released within a couple of weeks.

[laserjim]

I have no reason to believe pydantic-core==2.10.1 wouldn't be sufficient, but 2.20.1 is pulled in as a transitive dependency of pydantic 2.8.2 (pydantic 2.8.2 depends on pydantic-core==2.20.1).

I tried setting the dependency in my requirements.txt, which gave me something like:

The conflict is caused by:
    The user requested pydantic-core==2.10.1
    pydantic 2.8.2 depends on pydantic-core==2.20.1

What would be the easiest path forward here? Should I create a patch file that tricks pydantic to use an older version of pydantic-core? Is there a documented/easy workflow for creating such patch files for graal? Or should I try downgrading pydantic also? Is there an easy way to see which version of pydantic matches pydantic-core==2.10.1? My concern is that if I downgrade pydantic, I'm going to run into the same problem recursively because I'm pretty sure the pydantic dependency comes in from the openai dependency and also the litellm dependency. It starts to become kindof a mess if I start downgrading a large number of dependencies. At some point, it seems likely one of these version dependency rewrites will actually create a real conflict, and the more kosher solution would be to properly patch pydantic-core==2.20.1? I honestly don't know, and would defer to your expertise here.

Is there an easy/documented workflow for creating python package patches for graalpy? If so, I'm happy to try my hand at it. But please bear with me, as my knowledge of the python ecosystem is quite limited.

[msimacek]

The problem with patching pydantic-core is that pydantic-core doesn't actually contain the code that we need to patch. That code is downloaded by cargo (the rust build system) during the build as a dependency and we have no good way to hook into that process because that happens after we patch, in an external tool that's not even in python.

I tried pip install openai litellm pydantic-core==2.10.1 and it installed fine (locally with my jiter patch), so please wait for the jiter patch to be merged.

[laserjim]

This worked great! Thank you @msimacek!