-
Notifications
You must be signed in to change notification settings - Fork 101
Update inference specification for Hugging Face's rerank task #4417
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Update inference specification for Hugging Face's rerank task #4417
Conversation
Following you can find the validation results for the APIs you have changed.
You can validate these APIs yourself by using the |
…ace's-rerank-task # Conflicts: # output/schema/schema-serverless.json # output/schema/schema.json
Following you can find the validation results for the APIs you have changed.
You can validate these APIs yourself by using the |
…ace's-rerank-task
Following you can find the validation results for the APIs you have changed.
You can validate these APIs yourself by using the |
waiting for #4383 to be merged since it changes the same files |
…ace's-rerank-task
Following you can find the validation results for the APIs you have changed.
You can validate these APIs yourself by using the |
…ification-for-Hugging-Face's-rerank-task
7b97888
to
e73e977
Compare
Following you can find the validation results for the APIs you have changed.
You can validate these APIs yourself by using the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's add the task_settings
definition. Here's an example: https://github.com/elastic/elasticsearch-specification/blob/main/specification/inference/put_cohere/PutCohereRequest.ts#L76
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you, Jonathan. Done
Following you can find the validation results for the APIs you have changed.
You can validate these APIs yourself by using the |
Following you can find the validation results for the APIs you have changed.
You can validate these APIs yourself by using the |
…ace's-rerank-task
Following you can find the validation results for the APIs you have changed.
You can validate these APIs yourself by using the |
…ace's-rerank-task
Following you can find the validation results for the APIs you have changed.
You can validate these APIs yourself by using the |
@@ -40,7 +40,7 @@ import { TaskType } from '@inference/_types/TaskType' | |||
* * ELSER (`sparse_embedding`) | |||
* * Google AI Studio (`completion`, `text_embedding`) | |||
* * Google Vertex AI (`rerank`, `text_embedding`) | |||
* * Hugging Face (`text_embedding`) | |||
* * Hugging Face (`chat_completion`, `completion`, `rerank`, `sparse_embedding`, `text_embedding`) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jonathan-buttner could you please confirm that we want to add sparse_embedding task here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@davidkyle do we want to document hugging face support for sparse embedding aka elser? I don't think we ever released elser on hugging face for customers right?
completion | ||
completion, | ||
rerank, | ||
sparse_embedding, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jonathan-buttner Please confirm if we want to include sparse_embedding into a list of supported HF tasks. Because HF models don't provide sparse embeddings support out of the box.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It wasn't in scope of the related PR so I removed sparse_embedding in advance.
Initially I added it due to seeing it as a supported task type for HF in the codebase, namely, in HuggingFaceService
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AFAIC it is used for internal elasticsearch elser integration, so I wouldn't be adding it since it was proven by my investigation that it is impossible to use HF sparse embedding models out of the box.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, Jan. That's removed
export enum HuggingFaceTaskType { | ||
chat_completion, | ||
completion, | ||
rerank, | ||
sparse_embedding, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jonathan-buttner same thing here
…ace's-rerank-task
Following you can find the validation results for the APIs you have changed.
You can validate these APIs yourself by using the |
…ace's-rerank-task
Following you can find the validation results for the APIs you have changed.
You can validate these APIs yourself by using the |
Extended Task Support:
Added:
Add Hugging Face Rerank support PR