Closed
Description
Expected Behavior
The example server has a /tokenize
endpoint which converts strings to tokens. I am storing the response from this endpoint to calculate the token length of an input (for context window reasons) so I'd also like a /detokenize
to convert this back into a string for prompting reasons.
Current Behavior
No /detokenize
endpoint.
Metadata
Metadata
Assignees
Labels
No labels