JsonConverter error with AutoDateHistogram aggregation

[kabcampbell]

NEST/Elasticsearch.Net version:
NEST 6.8.0
NEST.JsonNetSerializer 6.8.0

Elasticsearch version:
Elastic Cloud - Elasticsearch version 6.8.0

Description of the problem including expected versus actual behavior:

When trying to use the AutoDateHistogram aggregation, the following exception is thrown:

Inner Exception Message:

Expression '1y' is invalid
Parameter name: timeUnit

Inner Exception Stack Trace:

   at Nest.Time.ParseExpression(String timeUnit)
   at Nest.Time..ctor(String timeUnit)
   at Nest.AggregateJsonConverter.GetMultiBucketAggregate(JsonReader reader, JsonSerializer serializer)
   at Nest.AggregateJsonConverter.ReadAggregate(JsonReader reader, JsonSerializer serializer)
   at Nest.AggregateDictionaryConverter.ParseAggregate(JsonReader reader, JsonSerializer serializer, String name, Dictionary`2 dictionary)
   at Nest.AggregateDictionaryConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Elastic.Internal.JsonNet.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Elastic.Internal.JsonNet.JsonSerializer.Deserialize[T](JsonReader reader)
   at Nest.InternalSerializer.<DeserializeAsync>d__23`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Elasticsearch.Net.ResponseBuilder.<SetBodyAsync>d__6`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Elasticsearch.Net.ResponseBuilder.<ToResponseAsync>d__3`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Elasticsearch.Net.HttpConnection.<RequestAsync>d__4`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Elasticsearch.Net.RequestPipeline.<CallElasticsearchAsync>d__57`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Elasticsearch.Net.Transport`1.<RequestAsync>d__15`1.MoveNext()

Expected behavior would be that the request succeeds. The equivalent cURL request succeeds:

{
    "aggs" : {
        "modDate" : {
            "auto_date_histogram" : {
                "field" : "modDate"
            }
        }
    }
}

This is the JSON response to my successful request:

{
  "took" : 162,
  "timed_out" : false,
  "_shards" : {
    "total" : 3,
    "successful" : 3,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 4089559,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "modDate" : {
      "buckets" : [
        {
          "key_as_string" : "2013-01-01T00:00:00.000Z",
          "key" : 1356998400000,
          "doc_count" : 32
        },
        {
          "key_as_string" : "2014-01-01T00:00:00.000Z",
          "key" : 1388534400000,
          "doc_count" : 617
        },
        {
          "key_as_string" : "2015-01-01T00:00:00.000Z",
          "key" : 1420070400000,
          "doc_count" : 183
        },
        {
          "key_as_string" : "2016-01-01T00:00:00.000Z",
          "key" : 1451606400000,
          "doc_count" : 3479
        },
        {
          "key_as_string" : "2017-01-01T00:00:00.000Z",
          "key" : 1483228800000,
          "doc_count" : 1948427
        },
        {
          "key_as_string" : "2018-01-01T00:00:00.000Z",
          "key" : 1514764800000,
          "doc_count" : 555748
        },
        {
          "key_as_string" : "2019-01-01T00:00:00.000Z",
          "key" : 1546300800000,
          "doc_count" : 1268034
        },
        {
          "key_as_string" : "2020-01-01T00:00:00.000Z",
          "key" : 1577836800000,
          "doc_count" : 313039
        }
      ],
      "interval" : "1y"
    }
  }
}

Steps to reproduce:
Here is my SearchDescriptor:

var searchDescriptor = new SearchDescriptor<dynamic>()
                                            .Index(_indexName)
                                            .AllTypes()
                                            .Routing(_routingID.ToUpper())
                                            .Size(0)
                                            .Aggregations(a => a.AutoDateHistogram("modDate", d => d.Field("modDate")));

Provide ConnectionSettings (if relevant):

var settings = new ConnectionSettings(pool, sourceSerializer: (builtin, s) =>
                                                  new JsonNetSerializer(builtin, s,
                                                     () => new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore, DefaultValueHandling = DefaultValueHandling.Include }));

Provide DebugInformation (if relevant):

# FailureReason: Unrecoverable/Unexpected BadResponse while attempting POST on https://6c1316d8b538420380cb471e0b1583cf.us-east-1.aws.found.io:9243/prjs_5_3_1923_0/_search?pretty=true&typed_keys=true&routing=KALABS5
 - [1] BadResponse: Node: https://6c1316d8b538420380cb471e0b1583cf.us-east-1.aws.found.io:9243/ Exception: ArgumentException Took: 00:00:00.4229550
# Audit exception in step 1 BadResponse:
System.ArgumentException: Expression '1y' is invalid
Parameter name: timeUnit
   at Nest.Time.ParseExpression(String timeUnit)
   at Nest.Time..ctor(String timeUnit)
   at Nest.AggregateJsonConverter.GetMultiBucketAggregate(JsonReader reader, JsonSerializer serializer)
   at Nest.AggregateJsonConverter.ReadAggregate(JsonReader reader, JsonSerializer serializer)
   at Nest.AggregateDictionaryConverter.ParseAggregate(JsonReader reader, JsonSerializer serializer, String name, Dictionary`2 dictionary)
   at Nest.AggregateDictionaryConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Elastic.Internal.JsonNet.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Elastic.Internal.JsonNet.JsonSerializer.Deserialize[T](JsonReader reader)
   at Nest.InternalSerializer.DeserializeAsync[T](Stream stream, CancellationToken cancellationToken)
   at Elasticsearch.Net.ResponseBuilder.SetBodyAsync[TResponse](ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elasticsearch.Net.ResponseBuilder.ToResponseAsync[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, IEnumerable`1 warnings, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elasticsearch.Net.HttpConnection.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   at Elasticsearch.Net.RequestPipeline.CallElasticsearchAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   at Elasticsearch.Net.Transport`1.RequestAsync[TResponse](HttpMethod method, String path, CancellationToken cancellationToken, PostData data, IRequestParameters requestParameters)
# Inner Exception: Expression '1y' is invalid
Parameter name: timeUnit
System.ArgumentException: Expression '1y' is invalid
Parameter name: timeUnit
   at Nest.Time.ParseExpression(String timeUnit)
   at Nest.Time..ctor(String timeUnit)
   at Nest.AggregateJsonConverter.GetMultiBucketAggregate(JsonReader reader, JsonSerializer serializer)
   at Nest.AggregateJsonConverter.ReadAggregate(JsonReader reader, JsonSerializer serializer)
   at Nest.AggregateDictionaryConverter.ParseAggregate(JsonReader reader, JsonSerializer serializer, String name, Dictionary`2 dictionary)
   at Nest.AggregateDictionaryConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Elastic.Internal.JsonNet.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Elastic.Internal.JsonNet.JsonSerializer.Deserialize[T](JsonReader reader)
   at Nest.InternalSerializer.DeserializeAsync[T](Stream stream, CancellationToken cancellationToken)
   at Elasticsearch.Net.ResponseBuilder.SetBodyAsync[TResponse](ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elasticsearch.Net.ResponseBuilder.ToResponseAsync[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, IEnumerable`1 warnings, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elasticsearch.Net.HttpConnection.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   at Elasticsearch.Net.RequestPipeline.CallElasticsearchAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   at Elasticsearch.Net.Transport`1.RequestAsync[TResponse](HttpMethod method, String path, CancellationToken cancellationToken, PostData data, IRequestParameters requestParameters)
# Exception:
Elasticsearch.Net.UnexpectedElasticsearchClientException: Expression '1y' is invalid
Parameter name: timeUnit ---> System.ArgumentException: Expression '1y' is invalid
Parameter name: timeUnit
   at Nest.Time.ParseExpression(String timeUnit)
   at Nest.Time..ctor(String timeUnit)
   at Nest.AggregateJsonConverter.GetMultiBucketAggregate(JsonReader reader, JsonSerializer serializer)
   at Nest.AggregateJsonConverter.ReadAggregate(JsonReader reader, JsonSerializer serializer)
   at Nest.AggregateDictionaryConverter.ParseAggregate(JsonReader reader, JsonSerializer serializer, String name, Dictionary`2 dictionary)
   at Nest.AggregateDictionaryConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Elastic.Internal.JsonNet.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Elastic.Internal.JsonNet.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Elastic.Internal.JsonNet.JsonSerializer.Deserialize[T](JsonReader reader)
   at Nest.InternalSerializer.DeserializeAsync[T](Stream stream, CancellationToken cancellationToken)
   at Elasticsearch.Net.ResponseBuilder.SetBodyAsync[TResponse](ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elasticsearch.Net.ResponseBuilder.ToResponseAsync[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, IEnumerable`1 warnings, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elasticsearch.Net.HttpConnection.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   at Elasticsearch.Net.RequestPipeline.CallElasticsearchAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   at Elasticsearch.Net.Transport`1.RequestAsync[TResponse](HttpMethod method, String path, CancellationToken cancellationToken, PostData data, IRequestParameters requestParameters)
   --- End of inner exception stack trace ---
   at Elasticsearch.Net.Transport`1.RequestAsync[TResponse](HttpMethod method, String path, CancellationToken cancellationToken, PostData data, IRequestParameters requestParameters)
   at Nest.ElasticClient.Nest.IHighLevelToLowLevelDispatcher.DispatchAsync[TRequest,TQueryString,TResponse,TResponseInterface](TRequest request, CancellationToken cancellationToken, Func`3 responseGenerator, Func`4 dispatch)
   at KnowledgeArchitecture.ElasticLib.Services.FilterValuesService.GetFilterFieldValuesFromElasticForField(IProfileEntityListField field, Boolean returnDebugInfo, IEnumerable`1 filtersToApply) in C:\VS Projects\Synthesis6\ElasticLibProfilesStandard\Services\FilterValuesService.cs:line 88

[russcam]

@kabcampbell can you capture the JSON response that raises this exception and add it here, please?

[kabcampbell]

@kabcampbell can you capture the JSON response that raises this exception and add it here, please?

Are you referring to the response that I receive from a successful request? If so, I added that in the issue above. For unsuccessful requests, it is throwing the error without giving me anything other than the debug information above.

[russcam]

The JSON response from the NEST or curl call is what I was after, thanks 👍

[MaJaHa95]

It looks like this is still happening in NEST 7.7.1.

Is the issue just that TimeUnit doesn't have a Year value? It looks like maybe months would be an issue as well, although I haven't tested that.

[russcam]

@MaJaHa95 The Interval property for AutoDateHistogramAggregate should be a DateMathTime rather than a Time, which supports all the interval units that auto date histogram supports