TermSuggest options should be an array

[LukasSeglias]

Java API client version

8.1.1

Java version

11

Elasticsearch Version

8.1.0

Problem description

I try to execute a search requesting a term-suggester:

{
  "query": {
    "bool": {
      "must": [
        {
          "multi_match": {
            "analyzer": "full_analyzer",
            "fields": [
              "searchText^1",
              "searchTextBoosted^4.0",
            ],
            "operator": "and",
            "query": "Stabi",
            "type": "cross_fields"
          }
        }
      ]
    }
  },
  "from": 0,
  "size": 20,
  "sort": [
    {
      "_score": {
        "order": "desc"
      }
    }
  ],
  "suggest": {
    "name.suggest": {
      "term": {
        "field": "searchTextBoosted",
        "suggest_mode": "missing"
      }
    },
    "text": "Stabi"
  }
}

The result looks good:

{
  "took": 10,
  "timed_out": false,
  "_shards": {
    "total": 3,
    "successful": 3,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "max_score": 3.4410129,
    "hits": [
      {
        "_index": "my-index",
        "_id": "BP-xtn8BLc_6fF5-zBDh",
        "_score": 3.4410129,
        "_source": {
          // stuff ...
        }
      }
    ]
  },
  "suggest": {
    "term#name.suggest": [
      {
        "text": "stabi",
        "offset": 0,
        "length": 5,
        "options": [
          {
            "text": "stab",
            "score": 0.75,
            "freq": 2
          },
          {
            "text": "stand",
            "score": 0.6,
            "freq": 2
          }
        ]
      },
      {
        "text": "stab",
        "offset": 0,
        "length": 5,
        "options": []
      }
    ]
  }
}

But the Java Client is not happy, because it thinks that the "options"-field in the term-suggester's result should be an object rather than an array. This causes the following error:
co.elastic.clients.json.UnexpectedJsonEventException: Unexpected JSON event 'START_ARRAY' instead of '[START_OBJECT, KEY_NAME]' at co.elastic.clients.json.JsonpUtils.ensureAccepts(JsonpUtils.java:103) ~[elasticsearch-java-8.1.1.jar:?]

This is probably due to the code-generation that this library uses. I think that the specification is incorrect.
Looking at the options of a completion-suggester (https://github.com/elastic/elasticsearch-specification/blob/7b8304b279b50f87239c0d83262a1f0fd2963a2a/specification/_global/search/_types/suggester.ts#L51):

export class CompletionSuggest<TDocument> extends SuggestBase {
  options: CompletionSuggestOption<TDocument>[]
}

And looking at the options of a term-suggester (https://github.com/elastic/elasticsearch-specification/blob/7b8304b279b50f87239c0d83262a1f0fd2963a2a/specification/_global/search/_types/suggester.ts#L65):

export class TermSuggest extends SuggestBase {
  options: TermSuggestOption
}

This looks wrong, the result that elasticsearch returns is clearly an Array of options.

PS: PhraseSuggest looks wrong too (https://github.com/elastic/elasticsearch-specification/blob/7b8304b279b50f87239c0d83262a1f0fd2963a2a/specification/_global/search/_types/suggester.ts#L58):

export class PhraseSuggest extends SuggestBase {
  options: PhraseSuggestOption
}

Cheers
Lukas

[jbaiera]

I've verified the same issue on my client, version 8.1.2 - Any requests using suggestion functionality need to be run through the old HLRC until this is fixed.

[mcremolini]

Hi,
I found the same issue in Java Client 8.1.3
This is my code excerpt

client.search(request -> request.
index(indexName).
suggest(suggest -> suggest.suggesters("suggest", value -> value.
term(v -> v.
field("ocr_text").maxTermFreq(Float.valueOf("10")).suggestMode(SuggestMode.Always))).text("test")), HashMap.class);

[fabriziofortino]

The same happens with 7.17.3

[jhnesk]

I have the same issue with 7.17.4
Is there any workaround?

[jbaiera]

@jhnesk I don't believe there are any workarounds in-client at the moment. If you are transitioning from the old High Level Rest Client to the new Java client, you may need to continue using HLRC for suggestion based workloads until this is fixed up.

@swallez Is there something that we should be looking at in the ES project regarding these response specifications to get this working?

[swallez]

@jbaiera the definition of TermSuggest in the API specification has to be fixed so that options is an array. We should also check if this applies to other suggesters. This will then be used to update the generated code in the Java API client.

[LukasSeglias]

I created an issue in the specification repository: elastic/elasticsearch-specification#1775

[sorhent]

Hello,
I faced this issue when migrating from the old HLRC client to this new Java API client and for the moment I am blocked in the migration process because of this issue.
The environment I'm using is Java 17 and ES Java API Client 8.2.3
I hope it will be fixed soon.
Regards