Pydantic warnings at every base64 embedding call for Async Azure

[ishaan-jaff]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

I see several warnings with this:

UserWarning: Pydantic serializer warnings:
  Expected `list[float]` but got `str` - serialized value may not be as expected
  return self.__pydantic_serializer__.to_json

This is impacting LiteLLM users too: BerriAI/litellm#1187

To Reproduce

import openai

    client = openai.AsyncAzureOpenAI(
        api_key=os.environ["AZURE_API_KEY"],
        azure_endpoint=os.environ["AZURE_API_BASE"]
    )

    async def _test():
        response = await client.embeddings.create(
                model="azure-embedding-model",
                input=["write a litellm poem"],
                encoding_format="base64"

        )

        print(response)

        response = response.model_dump_json()

        print(response)
        import json
        response = json.loads(response)
        print(response)
    import asyncio
    asyncio.run(_test())

Code snippets

No response

OS

macOS

Python version

3.10

Library version

1.6.1

[ishaan-jaff]

I suspect it's caused due to this:

openai-python/src/openai/types/embedding.py

Line 12 in f1c7d71

embedding: List[float]

[rattrayalex]

cc @kristapratico is azure returning nonconformant data here?

[kristapratico]

I'm seeing the warning repro for both OpenAI and Azure API. A string is returned for the embedding in the base64 case.

>>> import openai
>>> client = openai.OpenAI()
>>> 
>>> response = client.embeddings.create(model="text-embedding-ada-002", input="hello world", encoding_format="base64")
>>> r = response.model_dump_json()
/home/krpratic/.pyenv/versions/11/lib/python3.11/site-packages/pydantic/main.py:352: UserWarning: Pydantic serializer warnings:
  Expected `list[float]` but got `str` - serialized value may not be as expected
  return self.__pydantic_serializer__.to_json(

>>> import openai
>>> client = openai.AzureOpenAI()
>>> 
>>> response = client.embeddings.create(model="text-embedding-ada-002", input="hello world", encoding_format="base64")
>>> r = response.model_dump_json()
/home/krpratic/.pyenv/versions/11/lib/python3.11/site-packages/pydantic/main.py:352: UserWarning: Pydantic serializer warnings:
  Expected `list[float]` but got `str` - serialized value may not be as expected
  return self.__pydantic_serializer__.to_json(

Despite the warning, it does appear to serialize the model correctly.

[ishaan-jaff]

Despite the warning, it does appear to serialize the model correctly.

correct

For this issue, can we fix the warning message ? a lot of our users see this warning in their logs

[kristapratico]

@ishaan-jaff I'll let @rattrayalex answer, but just a suggestion - you can pass warnings=False into model_dump_json to silence the warning in logs.

>>> import openai
>>> client = openai.AzureOpenAI()
>>>
>>> response = client.embeddings.create(model="text-embedding-ada-002", input="hello world", encoding_format="base64")
>>> r = response.model_dump_json(warnings=False)

[rattrayalex]

Ah, got it – thanks @kristapratico!

[rattrayalex]

@ishaan-jaff for further context, is it your goal/intent to have the embeddings formatted as a list of floats, or as a string? If you don't pass that param (and have numpy installed), we'll request it as a string for efficiency, but then convert it back to a list of floats for you. This way, you won't get any type errors or warnings. Would that work well for you, or do you specifically want the base64 encoding in the resulting json?

[ishaan-jaff]

@rattrayalex I need the embeddings formatted as a string

Context: I'm building an OpenAI proxy server, the openai sdk requests base64 encoded embeddings from the proxy, the proxy should respond in embeddings formatted as a string. Goal of this issue is to get rid of the warning messages I'm seeing

[rattrayalex]

Got it, thanks @ishaan-jaff, that makes sense. As Krista shared, your best bet in the short term is response.model_dump_json(warnings=False) – does that work for you?

EDIT: out of curiosity, why do you want them as a string?

[ishaan-jaff]

does that work for you?

Not really, That suppresses the warning caused by model_dump_json but I still see the error raised here: https://github.com/BerriAI/litellm/blob/0864713b620d590295f5018da414860ab657a648/litellm/proxy/proxy_server.py#L1511. Does the OpenAI python SDK define some type of deserialization function that's getting used by FastAPI ?

EDIT: out of curiosity, why do you want them as a string?
(sorry if this is a complicated explanation) - if it helps I can share a diagram of what we're doing too

• LiteLLM provides a proxy call LLMs
• on the LiteLLM Proxy, we use the OpenAI python client to call Azure, OpenAI
• Users use the OpenAI python client to call the LiteLLM Proxy
• For embeddings, the OpenAI python client, requests embeddings as a string. It sends this request to the LiteLLM Proxy, which needs to respond back as a string

[rattrayalex]

Why does LiteLLM expect embeddings to be formatted as a string, do you know?