MultiSearch() throws UnexpectedTransportException: Unable to deserialize union.

[LionelVallet]

Elastic.Clients.Elasticsearch version: 8.1.1

Elasticsearch version: 8.8.0

.NET runtime version: 7.0.5

Operating system version: Debian 11

Description of the problem including expected versus actual behavior:
Multiget() with unknown id throws UnexpectedTransportException: Unable to deserialize union.

Steps to reproduce:

var multiSearchResponse = await _elasticsearchClient.MultiSearchAsync<Place>(r =>
{
    foreach (var fullTextQuery in fullTextQueries)
    {
        r.AddSearch(new SearchRequestItem(new MultisearchBody { Size = 1, Query = BuildQuery(fullTextQuery) }));
    }
});

Expected behavior
Get a valid Multisearch response

DebugInformation:

Elastic.Transport.UnexpectedTransportException: Unable to deserialize union.
 ---> System.Text.Json.JsonException: Unable to deserialize union.
   at Elastic.Clients.Elasticsearch.Serialization.UnionConverter.DerivedUnionConverterInner`3.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/Serialization/UnionConverter.cs:line 119
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ContinueDeserialize[TValue](ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.JsonSerializer.ReadFromStreamAsync[TValue](Stream utf8Json, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.SetBodyAsync[TResponse](ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.ToResponseAsync[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, Dictionary`2 headers, Stream responseStream, String mimeType, Int64 contentLength, IReadOnlyDictionary`2 threadPoolStats, IReadOnlyDictionary`2 tcpStats, CancellationToken cancellationToken)
   at Elastic.Transport.HttpTransportClient.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultRequestPipeline`1.CallProductEndpointAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultHttpTransport`1.RequestAsync[TResponse](HttpMethod method, String path, PostData data, RequestParameters requestParameters, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Elastic.Transport.DefaultHttpTransport`1.ThrowUnexpectedTransportException[TResponse](Exception killerException, List`1 seenExceptions, RequestData requestData, TResponse response, RequestPipeline pipeline)
   at Elastic.Transport.DefaultHttpTransport`1.RequestAsync[TResponse](HttpMethod method, String path, PostData data, RequestParameters requestParameters, CancellationToken cancellationToken)
   at Elastic.Clients.Elasticsearch.ElasticsearchClient.<>c__DisplayClass40_0`3.<<DoRequestAsync>g__SendRequest|0>d.MoveNext() in /_/src/Elastic.Clients.Elasticsearch/Client/ElasticsearchClient.cs:line 381

[LionelVallet]

It looks similar to #7169

[LionelVallet]

If I add a MultisearchHeader in the SearchRequestItems, the method works as expected.

var multiSearchResponse = await _elasticsearchClient.MultiSearchAsync<Place>(r =>
{
    foreach (var fullTextQuery in fullTextQueries)
    {
        r.AddSearch(new SearchRequestItem(
            new MultisearchHeader { Index = "my-index" },
            new MultisearchBody { Size = 1, Query = BuildQuery(fullTextQuery) }));
    }
});

I think there is still a response deserialization error that prevents retrieving the server exception message.

[flobernd]

Hi @LionelVallet, does that bug still exist in 8.13.x?

[LionelVallet]

Hi @flobernd, I don't have time to manage the breaking changes of version 8.13 this month, but I can answer you next month.

[LionelVallet]

Hi @flobernd, with the same code and the 8.13.11 package, it throws this exception:

var multiSearchResponse = await _elasticsearchClient.MultiSearchAsync<Place>(r =>
{
    foreach (var fullTextQuery in fullTextQueries)
    {
        r.AddSearches(new SearchRequestItem(
            new MultisearchHeader { Index = "my-index" },
            new MultisearchBody { Size = 1, Query = BuildQuery(fullTextQuery) }));
    }
});

Unsuccessful (400) low level call on POST: /places/_msearch?pretty=true&error_trace=true&typed_keys=true
 Exception: Request failed to execute. Call: Status code 400 from: POST /places/_msearch?pretty=true&error_trace=true&typed_keys=true. ServerError: Type: parse_exception Reason: "request body or source parameter is required"

# Audit trail of this API call:
 - [1] BadResponse: Node: https://es.cloud/ Took: 00:00:01.0842137
# OriginalException: Elastic.Transport.TransportException: Request failed to execute. Call: Status code 400 from: POST /places/_msearch?pretty=true&error_trace=true&typed_keys=true. ServerError: Type: parse_exception Reason: "request body or source parameter is required"
   at Elastic.Transport.DistributedTransport`1.HandleTransportException(RequestData data, Exception clientException, TransportResponse response)
   at Elastic.Transport.DistributedTransport`1.FinalizeResponse[TResponse](RequestData requestData, RequestPipeline pipeline, List`1 seenExceptions, TResponse response)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, HttpMethod method, String path, PostData data, RequestParameters requestParameters, OpenTelemetryData openTelemetryData, CancellationToken cancellationToken)
# Request:

# Response:
{
  "error" : {
    "root_cause" : [
      {
        "type" : "parse_exception",
        "reason" : "request body or source parameter is required",
        "stack_trace" : "org.elasticsearch.ElasticsearchParseException: request body or source parameter is required\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.RestRequest.contentOrSourceParam(RestRequest.java:557)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.action.search.RestMultiSearchAction.parseMultiLineRequest(RestMultiSearchAction.java:216)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.action.search.RestMultiSearchAction.parseRequest(RestMultiSearchAction.java:162)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.action.search.RestMultiSearchAction.parseRequest(RestMultiSearchAction.java:115)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.action.search.RestMultiSearchAction.prepareRequest(RestMultiSearchAction.java:88)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:82)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.RestController$1.onResponse(RestController.java:452)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.RestController$1.onResponse(RestController.java:446)\n\tat"
      }
    ],
    "type" : "parse_exception",
    "reason" : "request body or source parameter is required",
    "stack_trace" : "org.elasticsearch.ElasticsearchParseException: request body or source parameter is required\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.RestRequest.contentOrSourceParam(RestRequest.java:557)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.action.search.RestMultiSearchAction.parseMultiLineRequest(RestMultiSearchAction.java:216)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.action.search.RestMultiSearchAction.parseRequest(RestMultiSearchAction.java:162)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.action.search.RestMultiSearchAction.parseRequest(RestMultiSearchAction.java:115)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.action.search.RestMultiSearchAction.prepareRequest(RestMultiSearchAction.java:88)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:82)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.RestController$1.onResponse(RestController.java:452)\n\tat org.elasticsearch.server@8.13.2/org.elasticsearch.rest.RestController$1.onResponse(RestController.java:446)\n\tat"
  },
  "status" : 400
}

[flobernd]

Hi @LionelVallet, thanks for testing.

It seems like the new generator solved the initial issue, but caused other issues:

• The descriptor should implement IStreamSerializable (just like the plain request)
• The descriptor should not implement an inference constructor (because index names are on a per-item basis)

I'll try to provide a fix later this week!