High allocation rate in JsonValue.to

[mgorpenko]

Java API client version

7.17.9

Java version

19

Elasticsearch Version

7.17.5

Problem description

In our service, allocation rate with Java api client is dramatically higher that it used to be with an old high-level rest client: 550 mb/s vs 330 mb/s (please, find the graph below). Now, it takes more than 44% of all service allocations.

Most of these allocations are done while response deserialization. And especially, while JsonValue.to call.

Our service sends a lot of search requests to Elasticsearch. We have Source disabled and use doc value fields to get data. Actually, each hit has 2 fields only (and we may have up to 200 hits per request). And the allocation rate looks a little bit frantic.

I'm not sure what is the reason exactly. But the issue is definitely in the JsonDataImpl.to method. If I instead of JsonData.to method use JsonData.toString and then parse this string in my code locally, it works fine. You can see in the second graph below that this workaround solves completely the allocation issue.

I'm also attaching an excerpt from the flame graph.

Could you please look at it? Do you have plans to optimize JsonDataImpl? I see the comment in the code of JsonDataImpl: "// FIXME: inefficient roundtrip through a string. Should be replaced by an Event buffer structure." May it be related?

Allocation rate when switching from new the client to the old client and back to the new one

Allocation rate when switching (using new client) from JsonData.toString -> JsonData.to -> JsonData.toString

Flame graph

[swallez]

Thanks for the detailed report @mgorpenko. This is most probably related to the "FIXME" you mention. We'll investigate and see if there's an easy workaround before going into a more permanent correction of the issue.

[swallez]

@mgorpenko can you provide a bit more context so that we can investigate in the right direction:

• what is the type used in the search call, i.e. the 2nd parameter in client.search(request, SomeType.class)?
• you say you use doc value fields to get the data. Does this mean you use hit.fields() and not hit.source()?
• can you provide a code snippet showing how you use the search results, and in particular your usage of JsonData.to?

Thanks.

[mgorpenko]

@swallez

• we use String.class there just to write something (not sure what should be there if no source)
• right, we use hit.fields() only. We don't save the source in Elastic
• a code snippet:

public class ElasticField<T> {
  
  private String fieldName;
  private Class<T> type;

  public ElasticField(String fieldName, Class<T> type) {
    this.fieldName = fieldName;
    this.type = type;
  }

  public T parseValue(Hit<?> esHit) {
    var esData = esHit.fields().get(fieldName);
    T[] values = (T[]) esData.to(type.arrayType());
    return values[0];
  }
}

public class Parser {  
  private static final ElasticField<Long> field = new ElasticField<>("field1", Long.class);
  
  public Long parse(Hit<?> esHit) {
    return field.parseValue(esHit);
  }
}

[swallez]

Thanks a lot for this information. There's no immediate workaround, but we will work on improving JsonData.to() (resolving the FIXME in JsonDataImpl.getParser()) for the next release.

One additional request: when you "parse this string in my code locally, it works fine". Can you also provide a code snippet for this, so that we can compare?

Also, "we use String.class there just to write something": you can use Void.class, which will just ignore whatever is returned in _source, if present.

[mgorpenko]

@swallez

Thank you for your reply.

Our local workaround was not as generic:

public class Parser {

  public Long parse(final Hit<?> esHit) {
    final var value = esHit.fields().get("field1").toString();
    return Long.parseLong(value.substring(1, value.length() - 1));
  }
}

Honestly, later, we got slightly better performance using toJson() method instead of toString:

return esHit.fields().get("field1").toJson().asJsonArray().getJsonNumber(0).longValue();

[swallez]

Thanks. The toJson().asJsonArray() approach is indeed the fastest workaround for now since JsonDataImpl stores a JsonValue internally.

However this will change with the optimization work, since JsonDataImpl will use a compact JSON event buffer that can then be deserialized efficiently to any type using the generic JsonData.to().