Is there a way to prevent the API from returning the same answers in different outputs of the same HTTP response?

[nbro10]

Describe the feature or improvement you're requesting

When I make a request to the OpenAI API (specifically, the chat endpoint) with this package, I expect that different outputs, in the same response, are different. There's no reason for expecting different outputs to be the same - you could just copy them. This problem happens so often and it's so annoying. Even if we increase the temperature, this issue still occurs, i.e. the model may always return the same answer in different outputs. We should never pay for getting the same output in the same HTTP response multiple times. It would be like asking someone (a human) to come up with 5 different descriptions of a product, but 3 of them are repetitions.

So, is there a way to prevent the API from returning the same answer in different outputs? I think the answer is no.

If not, could this be implemented in some way in the backend? It would also be great if this feature allowed us to make the comparison by ignoring cases (some standardized form) or maybe using some kind of similarity metric (but this is more advanced).

Clearly, we need this to avoid making unnecessary calls and spending unnecessary credits.

Prompt engineering doesn't really work in many cases. It's trial-and-error combined with headaches. What we need is the API or the backend to enforce the different outputs to contain different outputs/answers, otherwise, we'd better just remove the possibility to return multiple outputs because, at least, that's less misleading.

Additional context

No response

[logankilpatrick]

Hey! We don't support this today but you could try to build a work around using the presence penalty and passing one response to the next call: https://platform.openai.com/docs/api-reference/chat/create#presence_penalty

[nbro10]

@logankilpatrick I think you should absolutely support this feature, so you should reopen this issue. Why would you want duplicate texts in the same response? It makes no sense. We can just duplicate the texts. So, either remove the parameter n (in that case we are responsible for making multiple requests and try to get different texts) or implement this feature. Can you clarify why one would ever need to use n > 1, if the texts were/are always or often the same? I cannot see a use case other than this makes use spend more credits than what we should without being able to control it. However, if you could ensure that texts are different in the same answer we will minimize the number of requests we make and this will not only make us spend no more than what we should fairly spend but also your APIs will be less overloaded with requests. Requests may take longer, but this is expected, if you request multiple texts. This can be implemented at the library or API level. Probably, the models should even be fine-tuned to do this by default.