Skip to content

IndexSettingsConverter tries to deserialize routing_path as collection of string even if there is a string in raw JSON response #8233

Closed
@matteocalisti

Description

@matteocalisti

Elastic.Clients.Elasticsearch version: 8.13.15, 8.14.0, 8.14.1

Elasticsearch version: 8.14.0

.NET runtime version: net6.0

Operating system version: Windows 10

Description of the problem including expected versus actual behavior:

What happens: when trying to load index informations via Elastic.Clients.Elasticsearch.IndexManagement.IndicesNamespacedClient.GetAsync(indexName), deserialization of IndexSettings fails because the property RoutingPath is expected to be an ICollection<string>, IndexSettingsConverter tries to deserialize the property as an ICollection<string>, but the raw response from Elasticsearch backend has a string in place of the ICollection. The exception is shown below.

Desired behaviour: Converter correctly deserializes the string to a list of strings.

Somewhat similar to #8190 , but is this case I cannot apply the attribute SingleOrManyCollectionConverterAttribute to the property or apply another converter to IndexSettings because it already has the attribute

[JsonConverter(typeof(IndexSettingsConverter))]
public sealed partial class IndexSettings

At the moment I'm trying to use the low level client to get the response and deserialize it with custom code.

Steps to reproduce:
1.
2.
3.

Expected behavior
A clear and concise description of what you expected to happen.

Provide ConnectionSettings (if relevant):

Provide DebugInformation (if relevant):

# FailureReason: Unrecoverable/Unexpected BadResponse while attempting GET on http://127.0.0.1:59317/.ds-data-stream-unittest_d113a864-9211-4099-8f9c-26762aca2b03_timeseries-2024.06.11-000001
 - [1] BadResponse: Node: http://127.0.0.1:59317/ Exception: JsonException Took: 00:00:00.6247743
# Audit exception in step 1 BadResponse:
System.Text.Json.JsonException: The JSON value could not be converted to System.Collections.Generic.ICollection`1[System.String]. Path: $ | LineNumber: 0 | BytePositionInLine: 6.
   at System.Text.Json.ThrowHelper.ThrowJsonException_DeserializeUnableToConvertValue(Type propertyType)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at Elastic.Clients.Elasticsearch.IndexManagement.IndexSettingsConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Types/IndexManagement/IndexSettings.g.cs:line 303
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at Elastic.Clients.Elasticsearch.IndexManagement.IndexSettingsConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Types/IndexManagement/IndexSettings.g.cs:line 129
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonDictionaryConverter`3.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TDictionary& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](Utf8JsonReader& reader, JsonSerializerOptions options)
   at Elastic.Clients.Elasticsearch.Serialization.DictionaryResponseConverterFactory.ResolvableDictionaryResponseConverterInner`3.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch.Shared/Serialization/DictionaryResponseConverter.cs:line 78
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize(ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.SetBodyCoreAsync[TResponse](Boolean isAsync, ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.ToResponseAsync[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, Dictionary`2 headers, Stream responseStream, String mimeType, Int64 contentLength, IReadOnlyDictionary`2 threadPoolStats, IReadOnlyDictionary`2 tcpStats, CancellationToken cancellationToken)
   at Elastic.Transport.HttpRequestInvoker.RequestCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultRequestPipeline`1.CallProductEndpointCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, HttpMethod method, String path, PostData data, RequestParameters requestParameters, OpenTelemetryData openTelemetryData, CancellationToken cancellationToken)
# Inner Exception: The JSON value could not be converted to System.Collections.Generic.ICollection`1[System.String]. Path: $ | LineNumber: 0 | BytePositionInLine: 6.
System.Text.Json.JsonException: The JSON value could not be converted to System.Collections.Generic.ICollection`1[System.String]. Path: $ | LineNumber: 0 | BytePositionInLine: 6.
   at System.Text.Json.ThrowHelper.ThrowJsonException_DeserializeUnableToConvertValue(Type propertyType)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at Elastic.Clients.Elasticsearch.IndexManagement.IndexSettingsConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Types/IndexManagement/IndexSettings.g.cs:line 303
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at Elastic.Clients.Elasticsearch.IndexManagement.IndexSettingsConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Types/IndexManagement/IndexSettings.g.cs:line 129
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonDictionaryConverter`3.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TDictionary& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](Utf8JsonReader& reader, JsonSerializerOptions options)
   at Elastic.Clients.Elasticsearch.Serialization.DictionaryResponseConverterFactory.ResolvableDictionaryResponseConverterInner`3.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch.Shared/Serialization/DictionaryResponseConverter.cs:line 78
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize(ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.SetBodyCoreAsync[TResponse](Boolean isAsync, ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.ToResponseAsync[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, Dictionary`2 headers, Stream responseStream, String mimeType, Int64 contentLength, IReadOnlyDictionary`2 threadPoolStats, IReadOnlyDictionary`2 tcpStats, CancellationToken cancellationToken)
   at Elastic.Transport.HttpRequestInvoker.RequestCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultRequestPipeline`1.CallProductEndpointCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, HttpMethod method, String path, PostData data, RequestParameters requestParameters, OpenTelemetryData openTelemetryData, CancellationToken cancellationToken)
# Exception:
Elastic.Transport.UnexpectedTransportException: The JSON value could not be converted to System.Collections.Generic.ICollection`1[System.String]. Path: $ | LineNumber: 0 | BytePositionInLine: 6.
 ---> System.Text.Json.JsonException: The JSON value could not be converted to System.Collections.Generic.ICollection`1[System.String]. Path: $ | LineNumber: 0 | BytePositionInLine: 6.
   at System.Text.Json.ThrowHelper.ThrowJsonException_DeserializeUnableToConvertValue(Type propertyType)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at Elastic.Clients.Elasticsearch.IndexManagement.IndexSettingsConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Types/IndexManagement/IndexSettings.g.cs:line 303
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at Elastic.Clients.Elasticsearch.IndexManagement.IndexSettingsConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Types/IndexManagement/IndexSettings.g.cs:line 129
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonDictionaryConverter`3.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TDictionary& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](Utf8JsonReader& reader, JsonSerializerOptions options)
   at Elastic.Clients.Elasticsearch.Serialization.DictionaryResponseConverterFactory.ResolvableDictionaryResponseConverterInner`3.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch.Shared/Serialization/DictionaryResponseConverter.cs:line 78
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize(ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.SetBodyCoreAsync[TResponse](Boolean isAsync, ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.ToResponseAsync[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, Dictionary`2 headers, Stream responseStream, String mimeType, Int64 contentLength, IReadOnlyDictionary`2 threadPoolStats, IReadOnlyDictionary`2 tcpStats, CancellationToken cancellationToken)
   at Elastic.Transport.HttpRequestInvoker.RequestCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultRequestPipeline`1.CallProductEndpointCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, HttpMethod method, String path, PostData data, RequestParameters requestParameters, OpenTelemetryData openTelemetryData, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Elastic.Transport.DistributedTransport`1.ThrowUnexpectedTransportException[TResponse](Exception killerException, List`1 seenExceptions, RequestData requestData, TResponse response, RequestPipeline pipeline)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, HttpMethod method, String path, PostData data, RequestParameters requestParameters, OpenTelemetryData openTelemetryData, CancellationToken cancellationToken)
   at [...]

Raw response (taken from Kibana)

{
	".ds-data-stream-unittest_b47fda94-ce27-4eef-ae25-4d54f296afe8_timeseries-2024.06.11-000001": {
		"aliases": {},
		"mappings": {
			"dynamic": "true",
			"_data_stream_timestamp": {
				"enabled": true
			},
			"_source": {
				"mode": "synthetic"
			},
			"properties": {
				"@timestamp": {
					"type": "date_nanos",
					"format": "strict_date_optional_time_nanos"
				},
				"code": {
					"type": "keyword",
					"time_series_dimension": true
				}
			}
		},
		"settings": {
			"index": {
				"mode": "time_series",
				"codec": "best_compression",
				"routing": {
					"allocation": {
						"include": {
							"_tier_preference": "data_hot"
						}
					}
				},
				"hidden": "true",
				"number_of_shards": "3",
				"time_series": {
					"end_time": "2024-06-11T15:08:40.000Z",
					"start_time": "2024-06-11T12:38:40.000Z"
				},
				"provided_name": ".ds-data-stream-unittest_b47fda94-ce27-4eef-ae25-4d54f296afe8_timeseries-2024.06.11-000001",
				"creation_date": "1718116720832",
				"number_of_replicas": "1",
				"routing_path": "code",
				"uuid": "L9GG3ZXNRC6WnIkZAdoHNA",
				"version": {
					"created": "8505000"
				}
			}
		},
		"data_stream": "data-stream-unittest_b47fda94-ce27-4eef-ae25-4d54f296afe8_timeseries"
	}
}

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions