Intermittent Connection Aborted Error on Moderation Endpoint

[jackadair]

Describe the bug

I'll leave this specific to my use case but as others have reported on the forums there is a intermittent error:

Error communicating with OpenAI: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

In our case we are encountering this on the Moderation endpoint. Only started happening within the last week or so.

To Reproduce

1. Make Request to Moderation endpoint using openai.Moderation.create()
2. It may work it may not, our users have found that this most often happens at the beginning of a conversation thread, the fact that it is so intermittent is what is making me believe this is a server error on OpenAI's part.

Code snippets

try:
        input = body["messages"][-1]["content"]
        moderation = openai.Moderation.create(input=input, model='text-moderation-latest')
        moderation_flagged = moderation["results"][0]["flagged"]
    except Exception as e:
        print(e) #error is received here
        return EventSourceResponse(error_generate(message="Error: Unable to moderate prompt", status_code=500))

OS

macOS/Windows

Python version

Python v3.9.16

Library version

0.27.2

[kaikaiyao]

Same here. Mostly only happens at the beginning of the session. If there's no solution to this yet is there any workaround for this?

[jackadair]

@kaikaiyao My only solution so far is to just retry the request if I get the 104 error from the moderation endpoint.

[thefedoration]

I get this too, but on the ChatCompletion endpoint. Seems to occasionally work fine and occasionally fail to connect.

[ra312]

Maybe just maintain a pool of connections and re-route between available connections for one user ?

[decoherencer]

Maybe just maintain a pool of connections and re-route between available connections for one user ?

how would one do that if I am currently doing new async session creation on every request?

[rattrayalex]

Closing as stale.