CRITICAL BUG: images.generate does not work on Azure OpenAI

[nicolaschaillan]

        openai_api_version = "2023-09-01-preview"
        client = AzureOpenAI(azure_endpoint = openai_api_base, api_key=openai_api_key, api_version=openai_api_version)

        response = client.images.generate(
            prompt=prompt,
            size=size,
            n=n
        )

Fails with:

{'error': {'code': '404', 'message': 'Resource not found'}}

[kristapratico]

Hi @nicolaschaillan, Azure OpenAI Dall-E is not supported in the v1 SDK yet, but support will be added soon.

https://learn.microsoft.com/en-us/azure/ai-services/openai/dall-e-quickstart?tabs=command-line&pivots=programming-language-python#install-the-python-sdk

[nicolaschaillan]

This should be documented. Once folks migrate it's impossible to go back with all those changes and now I'm left with no option for our DALL E deployment... Not good! Any ETA for the support?

We have hundreds of users on Azure OpenAI using it.

[kristapratico]

@nicolaschaillan I will share as soon as I have an ETA. As a stop-gap, it is possible to access Azure OpenAI DALL-E by passing in a httpx custom transport:

import time
import json
import httpx
import openai


class CustomHTTPTransport(httpx.HTTPTransport):
    def handle_request(
        self,
        request: httpx.Request,
    ) -> httpx.Response:
        if "images/generations" in request.url.path and request.url.params[
            "api-version"
        ] in [
            "2023-06-01-preview",
            "2023-07-01-preview",
            "2023-08-01-preview",
            "2023-09-01-preview",
            "2023-10-01-preview",
        ]:
            request.url = request.url.copy_with(path="/openai/images/generations:submit")
            response = super().handle_request(request)
            operation_location_url = response.headers["operation-location"]
            request.url = httpx.URL(operation_location_url)
            request.method = "GET"
            response = super().handle_request(request)
            response.read()

            timeout_secs: int = 120
            start_time = time.time()
            while response.json()["status"] not in ["succeeded", "failed"]:
                if time.time() - start_time > timeout_secs:
                    timeout = {"error": {"code": "Timeout", "message": "Operation polling timed out."}}
                    return httpx.Response(
                        status_code=400,
                        headers=response.headers,
                        content=json.dumps(timeout).encode("utf-8"),
                        request=request,
                    )

                time.sleep(int(response.headers.get("retry-after", 10)))
                response = super().handle_request(request)
                response.read()

            if response.json()["status"] == "failed":
                error_data = response.json()
                return httpx.Response(
                    status_code=400,
                    headers=response.headers,
                    content=json.dumps(error_data).encode("utf-8"),
                    request=request,
                )

            result = response.json()["result"]
            return httpx.Response(
                status_code=200,
                headers=response.headers,
                content=json.dumps(result).encode("utf-8"),
                request=request,
            )
        return super().handle_request(request)


client = openai.AzureOpenAI(
    azure_endpoint="<azure_endpoint>",
    api_key="<api_key>",
    api_version="<api_version>",
    http_client=httpx.Client(
        transport=CustomHTTPTransport(),
    ),
)
image = client.images.generate(prompt="a cute baby seal")

print(image.data[0].url)

Once support is officially added, you can remove the http_client keyword argument and the custom transport class from your code and all should work as expected. Let me know if that helps.

[nicolaschaillan]

Thanks! Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Krista Pratico ***@***.***> Sent: Monday, November 6, 2023 9:16:52 PM To: openai/openai-python ***@***.***> Cc: Nicolas Chaillan ***@***.***>; Mention ***@***.***> Subject: Re: [openai/openai-python] CRITICAL BUG: images.generate does not work on Azure OpenAI (Issue #692) @nicolaschaillan<https://github.com/nicolaschaillan> I will share as soon as I have an ETA. As a stop-gap, it is possible to access Azure OpenAI DALL-E by passing in a httpx custom transport: import time import json import httpx import openai class CustomHTTPTransport(httpx.HTTPTransport): def handle_request( self, request: httpx.Request, ) -> httpx.Response: if "images/generations" in request.url.path and request.url.params[ "api-version" ] in [ "2023-06-01-preview", "2023-07-01-preview", "2023-08-01-preview", "2023-09-01-preview", "2023-10-01-preview", ]: request.url = request.url.copy_with(path="/openai/images/generations:submit") response = super().handle_request(request) operation_location_url = response.headers["operation-location"] request.url = httpx.URL(operation_location_url) request.method = "GET" response = super().handle_request(request) response.read() timeout_secs: int = 120 start_time = time.time() while response.json()["status"] not in ["succeeded", "failed"]: if time.time() - start_time > timeout_secs: raise openai.OpenAIError("Operation polling timed out.") time.sleep(int(response.headers.get("retry-after")) or 10) response = super().handle_request(request) response.read() if response.json()["status"] == "failed": error_data = response.json()["error"] message = error_data.get("message", "Operation failed") code = error_data.get("code") raise openai.OpenAIError(message, code) result = response.json()["result"] return httpx.Response( status_code=200, headers=response.headers, content=json.dumps(result).encode("utf-8"), request=request, ) client = openai.AzureOpenAI( azure_endpoint="<azure_endpoint>", api_key="<api_key>", api_version="<api_version>", http_client=httpx.Client( transport=CustomHTTPTransport(), ), ) image = client.images.generate(prompt="a cute baby seal") print(image.data[0].url) Once support is officially added, you can remove the http_client keyword argument and the custom transport class from your code and all should work as expected. Let me know if that helps. — Reply to this email directly, view it on GitHub<#692 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABK2VBVJHUV4EEZ35TIUMQ3YDGKZJAVCNFSM6AAAAAA7AJI3A6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJXGE4DCNZUGM>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[ashjeanbird]

Hey @kristapratico, I made an update to the docs per @nicolaschaillan suggestion. Can you please take a look to see if it works and if there is anything else that I should add? Happy to add any additional details.

[StephenHodgson]

It is because msft adds an additional :submit part to the end of the uri they specify for generation.
You can easily append it to your endpoint request as needed.

I also added feedback to msft to remove this uri part:

https://feedback.azure.com/d365community/idea/f2204961-d97f-ee11-a81c-6045bdb7ea56

[kristapratico]

@ashjeanbird I'll take a look.

@StephenHodgson yes, the Azure Dall-E endpoint contains a :submit for the initial call. However, it's not enough to just append this to the endpoint. The initial call returns a status monitor which has to be checked until the final result is ready. This code shows how to workaround this for now: #692 (comment)

[StephenHodgson]

Yeah I don't understand why Microsoft went so far different with this endpoint implementation. Super frustrating.

[ashjeanbird]

@kristapratico thank you for checking out my PR #766. I have made a change to reflect the known issues verbiage link and the associated workaround.

[kristapratico]

@nicolaschaillan @StephenHodgson @ashjeanbird Azure OpenAI now supports Dall-E 3. See this Quickstart for details. It is supported on version 1.2.4 or greater of the openai library.

Please note that if you wish to continue to use Dall-E 2 you must either use the 0.28.1 version of the library or use 1.X version with the workaround.

[nicolaschaillan]

@nicolaschaillan @StephenHodgson @ashjeanbird Azure OpenAI now supports Dall-E 3. See this Quickstart for details. It is supported on version 1.2.4 or greater of the openai library.

Please note that if you wish to continue to use Dall-E 2 you must either use the 0.28.1 version of the library or use 1.X version with the workaround.

This doesn't work whatsoever on my East US API...

{'error': {'code': 'DeploymentNotFound', 'message': 'The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.'}}

Any clue why?

[kristapratico]

Are you using Dall-E 2 or 3?

Dall-E 3 is currently only available on SwedenCentral: https://learn.microsoft.com/en-us/azure/ai-services/openai/dall-e-quickstart?tabs=dalle3%2Ccommand-line&pivots=programming-language-python

You'll need to deploy the dall-e-3 model (you can do this from the Azure OpenAI Studio) and then target the deployment name in the method call:

result = client.images.generate(
    model="dalle3", # the name of your DALL-E 3 deployment
    prompt="a close-up of a bear walking through the forest",
)

[nicolaschaillan]

I tried both. Both don't work.

I use openai_api_version = "2023-12-01-preview"

model = "dalle2"

Fails.

We can't use Sweden for the U.S government work obviously, this should be better documented... list of supported models should be in a single location and easy to find. This is just a plain mess/nightmare.