Add support for token providers in the native OpenAI client

[samuelwcm]

Confirm this is a feature request for the Python library and not the underlying OpenAI API.

• This is a feature request for the Python library

Describe the feature or improvement you're requesting

Our project utilizes both OpenAI and Azure OpenAI APIs, which our organisation manages through an API gateway. The gateway provides its own short-lived JWTs and serves as a mediator between clients and these APIs, with some organisational logic over the top.

For Azure APIs, we use the AzureOpenAI.azure_ad_token_provider parameter to automatically refresh access tokens. Unfortunately, OpenAI's standard clients lack this feature.

As a workaround, we have written a custom class extending openai.AsyncOpenAI, with api_key defined as an @property getter to handle token refreshes. However, this workaround is not ideal due to the extra maintenance required. Ideally, OpenAI's clients would natively support token providers the same way as the Azure ones do.

Would there be interest in adding this feature? I'm willing to make this contribution myself!

Additional context

No response

[rattrayalex]

Thanks for sharing your use-case!

May I ask what's wrong with using the AzureOpenAI class?

[pamelafox]

FYI, I'm able to create something like this by using the api_key parameter/property, since that actually sets a Bearer header behind the scenes.

import logging
import os

from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv(override=True)
logging.basicConfig(level=logging.DEBUG)

credential = DefaultAzureCredential()
openai_token_provider = get_bearer_token_provider(credential, "https://cognitiveservices.azure.com/.default")

azure_openai_service = os.environ["AZURE_OPENAI_SERVICE"]
deployment_name = os.environ["AZURE_OPENAI_DEPLOYMENT"]

client = OpenAI(
    base_url=f"https://{azure_openai_service}.openai.azure.com/openai/deployments/{deployment_name}",
    api_key=openai_token_provider(),
    default_query={"api-version": os.environ["AZURE_OPENAI_API_VERSION"]},
)

# We must set the api_key each time we issue a new request, to make sure our token is refreshed as needed
client.api_key = openai_token_provider()
result = client.chat.completions.create(
    model=None, # this must be specified, but doesn't need a value
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant.",
        },
        {
            "role": "user",
            "content": "What is the capital of the United States?",
        },
    ])