Skip to content

Commit 4e61673

Browse files
fpagnybene2k1RoRoJ
authored
feat(genapi): Update faq about maximum output tokens (#5071)
* feat(genapi): Update faq about maximum output tokens * Update pages/generative-apis/faq.mdx Co-authored-by: Rowena Jones <36301604+RoRoJ@users.noreply.github.com> --------- Co-authored-by: Benedikt Rollik <brollik@scaleway.com> Co-authored-by: Rowena Jones <36301604+RoRoJ@users.noreply.github.com>
1 parent 5c3cc91 commit 4e61673

File tree

1 file changed

+7
-0
lines changed

1 file changed

+7
-0
lines changed

pages/generative-apis/faq.mdx

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,13 @@ Yes, API rate limits define the maximum number of requests a user can make withi
120120

121121
Refer to our dedicated [documentation](/generative-apis/reference-content/rate-limits/) for more information on rate limits.
122122

123+
## Can I increase maximum output (completion) tokens for a model?
124+
No, you cannot increase maximum output tokens above [limits for each models](https://www.scaleway.com/en/docs/generative-apis/reference-content/supported-models/) in Generative APIs.
125+
These limits are in place to protect you against:
126+
- Long generation which may be ended by an HTTP timeout. Limits are designed to ensure a model will send its HTTP response in less than 5 minutes.
127+
- Uncontrolled billing, as several models are known to be able to enter infinite generation loops (specific prompts can make the model generate the same sentence over and over, without stopping at all).
128+
If you require higher maximum output tokens, you can use [Managed Inference](https://console.scaleway.com/inference/deployments) where these limts do not apply (as your bill will be limited by the size of your deployment).
129+
123130
## What is the model lifecycle for Generative APIs?
124131
Scaleway is dedicated to updating and offering the latest versions of generative AI models, while ensuring older models remain accessible for a significant time, and also ensuring the reliability of your production applications. Learn more in our [model lifecycle policy](/generative-apis/reference-content/model-lifecycle/).
125132

0 commit comments

Comments
 (0)