diff --git a/docs/_releases.asciidoc b/docs/_releases.asciidoc deleted file mode 100644 index f8980014d..000000000 --- a/docs/_releases.asciidoc +++ /dev/null @@ -1,11 +0,0 @@ -:is_main_branch: - -// Non man branches should include the following (8.1 exemplified below) -// // Versions for the 8.1.x releases -// :v8_1_0_released: -// :v8_1_1_released: - -// // Handle version bump in the docs build -// ifeval::["{version}" == "v8.1.2"] -// :v8_1_2_released: -// endif::[] diff --git a/docs/api-conventions/blocking-and-async.asciidoc b/docs/api-conventions/blocking-and-async.asciidoc deleted file mode 100644 index 41a880002..000000000 --- a/docs/api-conventions/blocking-and-async.asciidoc +++ /dev/null @@ -1,19 +0,0 @@ -[[blocking-and-async]] -=== Blocking and asynchronous clients - -API clients come in two flavors: blocking and asynchronous. All methods on -asynchronous clients return a standard `CompletableFuture`. - -Both flavors can be used at the same time depending on your needs, sharing the -same transport object: - -["source","java"] --------------------------------------------------- -ElasticsearchTransport transport = ... - -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[blocking-and-async] --------------------------------------------------- - -Although we won't go in deeper details on asynchronous programming in Java, remember to handle failures of asynchronous tasks. It's easy to overlook them and have errors go unnoticed. - -{doc-tests-blurb} diff --git a/docs/api-conventions/building-objects.asciidoc b/docs/api-conventions/building-objects.asciidoc deleted file mode 100644 index db3e3e493..000000000 --- a/docs/api-conventions/building-objects.asciidoc +++ /dev/null @@ -1,68 +0,0 @@ -[[building-objects]] -=== Building API objects - -[discrete] -==== Builder objects - -All data types in the {java-client} are immutable. Object creation uses the -https://www.informit.com/articles/article.aspx?p=1216151&seqNum=2[builder pattern] -that was popularized in *Effective Java* in 2008. - -["source","java"] --------------------------------------------------- -ElasticsearchClient client = ... -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[builders] --------------------------------------------------- - -Note that a builder should not be reused after its `build()` method has been -called. - -[discrete] -==== Builder lambda expressions - -Although this works nicely, having to instantiate builder classes and call the -`build()` method is a bit verbose. So every property setter in the {java-client} also -accepts a lambda expression that takes a newly created builder as a parameter -and returns a populated builder. The snippet above can also be written as: - -["source","java"] --------------------------------------------------- -ElasticsearchClient client = ... -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[builder-lambdas] --------------------------------------------------- - -This approach allows for much more concise code, and also avoids importing -classes (and even remembering their names) since types are inferred from the -method parameter signature. - -Note in the above example that builder variables are only used to start a chain -of property setters. The names of these variables are therefore unimportant and -can be shortened to improve readability: - -["source","java"] --------------------------------------------------- -ElasticsearchClient client = ... -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[builder-lambdas-short] --------------------------------------------------- - -Builder lambdas become particularly useful with complex nested queries like the -one below, taken from the -{ref}/query-dsl-intervals-query.html[intervals query API documentation]. - -This example also highlights a useful naming convention for builder parameters in -deeply nested structures. For lambda expressions with a single argument, Kotlin -provides the implicit `it` parameter and Scala allows use of `_`. This can be approximated -in Java by using an underscore or a single letter prefix followed by a number representing the depth -level (i.e. `_0`, `_1`, or `b0`, `b1` and so on). Not only does this remove the need to create -throw-away variable names, but it also improves code readability. Correct indentation -also allows the structure of the query to stand out. - -["source","java"] --------------------------------------------------- -ElasticsearchClient client = ... -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[builder-intervals] --------------------------------------------------- -<1> Search results will be mapped to `SomeApplicationData` instances to -be readily available to the application. - -{doc-tests-blurb} diff --git a/docs/api-conventions/exceptions.asciidoc b/docs/api-conventions/exceptions.asciidoc deleted file mode 100644 index ef1c7863c..000000000 --- a/docs/api-conventions/exceptions.asciidoc +++ /dev/null @@ -1,15 +0,0 @@ -[[exception-conventions]] -=== Exceptions - -Client methods can throw two kinds of exceptions: - -* Requests that were received by the {es} server but that were rejected -(validation error, server internal timeout exceeded, etc) will produce an -`ElasticsearchException`. This exception contains details about the error, -provided by {es}. - -* Requests that failed to reach the server (network error, server unavailable, -etc) will produce a `TransportException`. That exception's cause is the exception -thrown by the lower-level implementation. In the case of the `RestClientTransport` -it will be a `ResponseException` that contains the low level HTTP response. - diff --git a/docs/api-conventions/index.asciidoc b/docs/api-conventions/index.asciidoc deleted file mode 100644 index d30023609..000000000 --- a/docs/api-conventions/index.asciidoc +++ /dev/null @@ -1,32 +0,0 @@ -[[api-conventions]] -== API conventions - -The {java-client} uses a very consistent code structure, using modern code -patterns that make complex requests easier to write and complex responses easier -to process. The sections below explain these in details. - -* <> -* <> -* <> -* <> -* <> -* <> -* <> -ifdef::is_main_branch,v8_1_1_released,v7_17_2_released[] -* <> -endif::is_main_branch,v8_1_1_released,v7_17_2_released[] -* <> - -include::package-structure.asciidoc[] -include::method-naming.asciidoc[] -include::blocking-and-async.asciidoc[] -include::building-objects.asciidoc[] -include::lists-and-maps.asciidoc[] -include::variant-types.asciidoc[] -include::object-lifecycles.asciidoc[] - -ifdef::is_main_branch,v8_1_1_released,v7_17_2_released[] -include::loading-json.asciidoc[] -endif::is_main_branch,v8_1_1_released,v7_17_2_released[] - -include::exceptions.asciidoc[] diff --git a/docs/api-conventions/lists-and-maps.asciidoc b/docs/api-conventions/lists-and-maps.asciidoc deleted file mode 100644 index 117295b50..000000000 --- a/docs/api-conventions/lists-and-maps.asciidoc +++ /dev/null @@ -1,40 +0,0 @@ -[[lists-and-maps]] -=== Lists and maps - -[discrete] -==== Additive builder setters - -Properties of type `List` and `Map` are exposed by object builders as a set of overloaded -additive-only methods that _update_ the property value, by appending to lists and adding -new entries to maps (or replacing existing ones). - -Object builders create immutable objects, and this also applies to list and map properties -that are made immutable at object construction time. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[collections] --------------------------------------------------- - -[discrete] -==== List and map values are never `null` - -The {es} API has a lot of optional properties. For single-valued properties, the {java-client} -represents missing optional values as `null`. Applications therefore have to null-check -optional values before using them. - -For lists and maps however, applications often only care about whether they're empty or not, -or even just iterate on their content. Using `null` values is then cumbersome. To avoid this, -{java-client} collection properties are never `null`, and missing optional collections are -returned as an empty collection. - -If you ever need to distinguish between a missing (undefined) optional collection and an -effectively-empty collection returned by {es}, the `ApiTypeHelper` class provides a utility -method to distinguish them: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[optional-collections] --------------------------------------------------- - -{doc-tests-blurb} diff --git a/docs/api-conventions/loading-json.asciidoc b/docs/api-conventions/loading-json.asciidoc deleted file mode 100644 index 5c50f4a1a..000000000 --- a/docs/api-conventions/loading-json.asciidoc +++ /dev/null @@ -1,87 +0,0 @@ -[[loading-json]] -=== Creating API objects from JSON data - -ifeval::["{minor-version}" == "7.17"] -NOTE: This feature was added in version 7.17.2 -endif::[] - -ifeval::["{minor-version}" == "8.1"] -NOTE: This feature was added in version 8.1.1 -endif::[] - -A common workflow during application development with Elasticsearch is to use the Kibana Developer Console to interactively prepare and test queries, aggregations, index mappings and other complex API calls. This results in working JSON snippets that you may want to use in your application. - -As translating these JSON snippets to Java code can be time-consuming and error-prone, most of the data classes in the {java-client} can be loaded from JSON text: object builders have `withJson()` methods that populate the builder from raw JSON. This also allows you to combine dynamically loaded JSON with programmatic construction of objects. - -Under the hood, the `withJson()` methods call the object's deserializer. The JSON text's structure and value types therefore have to be correct for the target data structure. Using `withJson()` keeps the strong typing guarantees of the {java-client}. - -[discrete] -==== Examples - -[discrete] -===== Loading an index definition from a resource file - -Consider a resource file `some-index.json` containing an index definition: - -["source", "json"] --------------------------------------------------- -{ - "mappings": { - "properties": { - "field1": { "type": "text" } - } - } -} --------------------------------------------------- - -You can create an index from that definition as follows: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/api_conventions/LoadingJsonTest.java[load-index] --------------------------------------------------- -<1> open an input stream for the JSON resource file. -<2> populate the index creation request with the resource file contents. - -[discrete] -===== Ingesting documents from JSON files - -Similarly, you can read documents to be stored in {es} from data files: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/api_conventions/LoadingJsonTest.java[ingest-data] --------------------------------------------------- -<1> when calling `withJson()` on data structures that have generic type parameters, these generic types will be considered to be `JsonData`. - -[discrete] -===== Creating a search request combining JSON and programmatic construction - -You can combine `withJson()` with regular calls to setter methods. The example below loads the query part of a search request from a `String` and programmatically adds an aggregation. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/api_conventions/LoadingJsonTest.java[query] --------------------------------------------------- -<1> loads the query from the JSON string. -<2> adds the aggregation. -<3> since this is an aggregation we don't care about result documents and set their target class to `Void`, meaning they will just be ignored. Note that setting `size` to zero actually prevents any document from being returned. - -[discrete] -===== Creating a search request from multiple JSON snippets - -The `withJson()` methods are partial deserializers: the properties loaded from the JSON will set property values or replace the previous ones, but will not reset other properties not found in the JSON input. You can use this to combine multiple JSON snippets to build complex search requests. In the example below, we combine separate definitions of a query that selects some documents and an aggregation that is run on the results of this query. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/api_conventions/LoadingJsonTest.java[query-and-agg] --------------------------------------------------- -<1> set max number of returned document to 100 for queries. -<2> we do not want any matching document in aggregations. -<3> loads the query part of the request. -<4> loads the aggregation part of the request (overwrites `size` from the query). -<5> additional request properties set programmatically. - -Notice that order matters when the JSON snippets have some common properties: just as when setting property values programmatically, the last value that is set for a property overwrites the previous one. - -{doc-tests-blurb} diff --git a/docs/api-conventions/method-naming.asciidoc b/docs/api-conventions/method-naming.asciidoc deleted file mode 100644 index 704f8a9c3..000000000 --- a/docs/api-conventions/method-naming.asciidoc +++ /dev/null @@ -1,15 +0,0 @@ -[[method-naming]] -=== Method naming conventions - -Classes in the {java-client} contain two kinds of methods and properties: - -* Methods and properties that are part of the API, such as -`ElasticsearchClient.search()` or `SearchResponse.maxScore()`. They are derived -from their respective names in the {es} JSON API using the standard Java -`camelCaseNaming` convention. - -* Methods and properties that are part of the framework on which the Java API -Client is built, such as `Query._kind()`. These methods and properties are -prefixed with an underscore to both avoid any naming conflicts with API names, -and as an easy way to distinguish the API from the framework. - diff --git a/docs/api-conventions/object-lifecycles.asciidoc b/docs/api-conventions/object-lifecycles.asciidoc deleted file mode 100644 index a4500ce0b..000000000 --- a/docs/api-conventions/object-lifecycles.asciidoc +++ /dev/null @@ -1,31 +0,0 @@ -[[object-lifecycles]] -=== Object life cycles and thread safety - -There are five kinds of objects in the {java-client} with different life cycles: - - -**Object mapper**:: -Stateless and thread-safe, but can be costly to create. -It’s usually a singleton that is created at application startup and used to -create the transport. - -**Transport**:: -Thread-safe, holds network resources through the underlying HTTP client. A -transport object is associated with an {es} cluster and has to be explicitly -closed to release the underlying resources such as network connections. - -**Clients**:: -Immutable, stateless and thread-safe. -These are very lightweight objects that just wrap a transport and provide API -endpoints as methods. Closing a client closes the underlying transport. - -**Builders**:: -Mutable, non thread-safe. -Builders are transient objects that should not be reused after calling -`build()`. - -**Requests & other API objects**:: -Immutable, thread-safe. -If your application uses the same request or same parts of a request over and -over, these objects can be prepared in advance and reused across multiple calls -over multiple clients with different transports. diff --git a/docs/api-conventions/package-structure.asciidoc b/docs/api-conventions/package-structure.asciidoc deleted file mode 100644 index f51d0f476..000000000 --- a/docs/api-conventions/package-structure.asciidoc +++ /dev/null @@ -1,26 +0,0 @@ -[[package-structure]] -=== Package structure and namespace clients - -The {es} API is large and is organized into feature groups, as can be seen in -the {ref}/rest-apis.html[{es} API documentation]. - -The {java-client} follows this structure: feature groups are called “namespaces”, -and each namespace is located in a subpackage of -`co.elastic.clients.elasticsearch`. - -Each of the namespace clients can be accessed from the top level {es} client. The -only exceptions are the “search” and “document” APIs which are located in the `core` -subpackage and can be accessed on the main {es} client object. - -The snippet below shows how to use the indices namespace client to create an -index (the lambda syntax is explained in <>): - -["source","java"] --------------------------------------------------- -// Create the "products" index -ElasticsearchClient client = ... -client.indices().create(c -> c.index("products")); --------------------------------------------------- - -Namespace clients are very lightweight objects that can be created on the fly. - diff --git a/docs/api-conventions/variant-types.asciidoc b/docs/api-conventions/variant-types.asciidoc deleted file mode 100644 index 99f602e9d..000000000 --- a/docs/api-conventions/variant-types.asciidoc +++ /dev/null @@ -1,106 +0,0 @@ -[[variant-types]] -=== Variant types - -The {es} API has a lot of variant types: queries, aggregations, field mappings, -analyzers, and so on. Finding the correct class name in such large collections -can be challenging. - -The {java-client} builders make this easy: the builders for variant types, such as -`Query`, have methods for each of the available implementations. We’ve seen this -in action above with `intervals` (a kind of query) and `allOf`, `match` and -`anyOf` (various kinds of intervals). - -This is because variant objects in the {java-client} are implementations of a -“tagged union”: they contain the identifier (or tag) of the variant they hold -and the value for that variant. For example, a `Query` object can contain an -`IntervalsQuery` with tag `intervals`, a `TermQuery` with tag `term`, and so on. -This approach allows writing fluent code where you can let the IDE completion -features guide you to build and navigate complex nested structures: - -Variant builders have setter methods for every available implementation. They -use the same conventions as regular properties and accept both a builder lambda -expression and a ready-made object of the actual type of the variant. Here’s an -example to build a term query: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[variant-creation] --------------------------------------------------- -<1> Choose the `term` variant to build a term query. -<2> Build the terms query with a builder lambda expression. -<3> Build the `Query` that now holds a `TermQuery` object of kind `term`. - -Variant objects have getter methods for every available implementation. These -methods check that the object actually holds a variant of that kind and return -the value downcasted to the correct type. They throw an `IllegalStateException` -otherwise. This approach allows writing fluent code to traverse variants. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[variant-navigation] --------------------------------------------------- - -Variant objects also provide information on the variant kind they currently hold: - -* with `is` methods for each of the variant kinds: `isTerm()`, `isIntervals()`, `isFuzzy()`, etc. - -* with a nested `Kind` enumeration that defines all variant kinds. - -This information can then be used to navigate down into specific variants after checking -their actual kind: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[variant-kind] --------------------------------------------------- -<1> Test if the variant is of a specific kind. -<2> Test a larger set of variant kinds. -<3> Get the kind and value held by the variant object. - -[discrete] -[[variant-types-custom]] -==== Custom extensions provided by {es} plugins - -{es} accepts plugins that can extend the available variants for a number of types. This includes queries, aggregations, text analyzers and tokenizers, ingest processors, etc. - -The {java-client} classes for these types accept a `_custom` variant in addition to the builtin ones. This allows you to use these plugin-defined extensions by providing arbitrary JSON in requests, and also receive arbitrary JSON produced by the plugins in responses. - -In the examples below we use a hypothetical plugin that adds a `sphere-distance` aggregation that groups documents containing 3D coordinates according to their distance to a reference location. - -To create a custom aggregation, use the `_custom()` aggregation type and provide its identifier, defined by the plugin, and parameters. The parameters can be any object or value that can be serialized to JSON. In the example below we use a simple map: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[custom-variant-creation] --------------------------------------------------- -<1> Parameters for the custom aggregation. -<2> Create a custom aggregation named `neighbors` of kind `sphere-distance` with its parameters. - -The results of custom variants are returned as raw JSON represented by a `JsonData` object. You can then traverse the JSON tree to get the data. Since this is not always convenient, you can also define classes that represent that JSON data and deserialize them from the raw JSON. - -Traversing the JSON tree: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[custom-variant-navigation-json] --------------------------------------------------- -<1> Use `Void` if you're only interested in aggregation results, not search hits (see also <>). -<2> Get the `neighbors` aggregation result as custom JSON result. -<3> Traverse the JSON tree to extract the result data. - -Using a class that represents the custom aggregation results: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[custom-variant-navigation-typed] --------------------------------------------------- -<1> Deserialize the custom JSON to a dedicated `SphereDistanceAggregate` class. - -Where `SphereDistanceAggregate` can be defined as follows: -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/api_conventions/ApiConventionsTest.java[custom-variant-types] --------------------------------------------------- - - -{doc-tests-blurb} diff --git a/docs/docset.yml b/docs/docset.yml new file mode 100644 index 000000000..0c29a1da5 --- /dev/null +++ b/docs/docset.yml @@ -0,0 +1,491 @@ +project: 'Java API Client' +exclude: + - external-resources.md + - design/* + - local/README.md +cross_links: + - docs-content + - elasticsearch +toc: + - toc: reference + - toc: release-notes +subs: + ref: "https://www.elastic.co/guide/en/elasticsearch/reference/current" + ref-bare: "https://www.elastic.co/guide/en/elasticsearch/reference" + ref-8x: "https://www.elastic.co/guide/en/elasticsearch/reference/8.1" + ref-80: "https://www.elastic.co/guide/en/elasticsearch/reference/8.0" + ref-7x: "https://www.elastic.co/guide/en/elasticsearch/reference/7.17" + ref-70: "https://www.elastic.co/guide/en/elasticsearch/reference/7.0" + ref-60: "https://www.elastic.co/guide/en/elasticsearch/reference/6.0" + ref-64: "https://www.elastic.co/guide/en/elasticsearch/reference/6.4" + xpack-ref: "https://www.elastic.co/guide/en/x-pack/6.2" + logstash-ref: "https://www.elastic.co/guide/en/logstash/current" + kibana-ref: "https://www.elastic.co/guide/en/kibana/current" + kibana-ref-all: "https://www.elastic.co/guide/en/kibana" + beats-ref-root: "https://www.elastic.co/guide/en/beats" + beats-ref: "https://www.elastic.co/guide/en/beats/libbeat/current" + beats-ref-60: "https://www.elastic.co/guide/en/beats/libbeat/6.0" + beats-ref-63: "https://www.elastic.co/guide/en/beats/libbeat/6.3" + beats-devguide: "https://www.elastic.co/guide/en/beats/devguide/current" + auditbeat-ref: "https://www.elastic.co/guide/en/beats/auditbeat/current" + packetbeat-ref: "https://www.elastic.co/guide/en/beats/packetbeat/current" + metricbeat-ref: "https://www.elastic.co/guide/en/beats/metricbeat/current" + filebeat-ref: "https://www.elastic.co/guide/en/beats/filebeat/current" + functionbeat-ref: "https://www.elastic.co/guide/en/beats/functionbeat/current" + winlogbeat-ref: "https://www.elastic.co/guide/en/beats/winlogbeat/current" + heartbeat-ref: "https://www.elastic.co/guide/en/beats/heartbeat/current" + journalbeat-ref: "https://www.elastic.co/guide/en/beats/journalbeat/current" + ingest-guide: "https://www.elastic.co/guide/en/ingest/current" + fleet-guide: "https://www.elastic.co/guide/en/fleet/current" + apm-guide-ref: "https://www.elastic.co/guide/en/apm/guide/current" + apm-guide-7x: "https://www.elastic.co/guide/en/apm/guide/7.17" + apm-app-ref: "https://www.elastic.co/guide/en/kibana/current" + apm-agents-ref: "https://www.elastic.co/guide/en/apm/agent" + apm-android-ref: "https://www.elastic.co/guide/en/apm/agent/android/current" + apm-py-ref: "https://www.elastic.co/guide/en/apm/agent/python/current" + apm-py-ref-3x: "https://www.elastic.co/guide/en/apm/agent/python/3.x" + apm-node-ref-index: "https://www.elastic.co/guide/en/apm/agent/nodejs" + apm-node-ref: "https://www.elastic.co/guide/en/apm/agent/nodejs/current" + apm-node-ref-1x: "https://www.elastic.co/guide/en/apm/agent/nodejs/1.x" + apm-rum-ref: "https://www.elastic.co/guide/en/apm/agent/rum-js/current" + apm-ruby-ref: "https://www.elastic.co/guide/en/apm/agent/ruby/current" + apm-java-ref: "https://www.elastic.co/guide/en/apm/agent/java/current" + apm-go-ref: "https://www.elastic.co/guide/en/apm/agent/go/current" + apm-dotnet-ref: "https://www.elastic.co/guide/en/apm/agent/dotnet/current" + apm-php-ref: "https://www.elastic.co/guide/en/apm/agent/php/current" + apm-ios-ref: "https://www.elastic.co/guide/en/apm/agent/swift/current" + apm-lambda-ref: "https://www.elastic.co/guide/en/apm/lambda/current" + apm-attacher-ref: "https://www.elastic.co/guide/en/apm/attacher/current" + docker-logging-ref: "https://www.elastic.co/guide/en/beats/loggingplugin/current" + esf-ref: "https://www.elastic.co/guide/en/esf/current" + kinesis-firehose-ref: "https://www.elastic.co/guide/en/kinesis/{{kinesis_version}}" + estc-welcome-current: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions/current" + estc-welcome: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions/current" + estc-welcome-all: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions" + hadoop-ref: "https://www.elastic.co/guide/en/elasticsearch/hadoop/current" + stack-ref: "https://www.elastic.co/guide/en/elastic-stack/current" + stack-ref-67: "https://www.elastic.co/guide/en/elastic-stack/6.7" + stack-ref-68: "https://www.elastic.co/guide/en/elastic-stack/6.8" + stack-ref-70: "https://www.elastic.co/guide/en/elastic-stack/7.0" + stack-ref-80: "https://www.elastic.co/guide/en/elastic-stack/8.0" + stack-ov: "https://www.elastic.co/guide/en/elastic-stack-overview/current" + stack-gs: "https://www.elastic.co/guide/en/elastic-stack-get-started/current" + stack-gs-current: "https://www.elastic.co/guide/en/elastic-stack-get-started/current" + javaclient: "https://www.elastic.co/guide/en/elasticsearch/client/java-api/current" + java-api-client: "https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current" + java-rest: "https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current" + jsclient: "https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current" + jsclient-current: "https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current" + es-ruby-client: "https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/current" + es-dotnet-client: "https://www.elastic.co/guide/en/elasticsearch/client/net-api/current" + es-php-client: "https://www.elastic.co/guide/en/elasticsearch/client/php-api/current" + es-python-client: "https://www.elastic.co/guide/en/elasticsearch/client/python-api/current" + defguide: "https://www.elastic.co/guide/en/elasticsearch/guide/2.x" + painless: "https://www.elastic.co/guide/en/elasticsearch/painless/current" + plugins: "https://www.elastic.co/guide/en/elasticsearch/plugins/current" + plugins-8x: "https://www.elastic.co/guide/en/elasticsearch/plugins/8.1" + plugins-7x: "https://www.elastic.co/guide/en/elasticsearch/plugins/7.17" + plugins-6x: "https://www.elastic.co/guide/en/elasticsearch/plugins/6.8" + glossary: "https://www.elastic.co/guide/en/elastic-stack-glossary/current" + upgrade_guide: "https://www.elastic.co/products/upgrade_guide" + blog-ref: "https://www.elastic.co/blog/" + curator-ref: "https://www.elastic.co/guide/en/elasticsearch/client/curator/current" + curator-ref-current: "https://www.elastic.co/guide/en/elasticsearch/client/curator/current" + metrics-ref: "https://www.elastic.co/guide/en/metrics/current" + metrics-guide: "https://www.elastic.co/guide/en/metrics/guide/current" + logs-ref: "https://www.elastic.co/guide/en/logs/current" + logs-guide: "https://www.elastic.co/guide/en/logs/guide/current" + uptime-guide: "https://www.elastic.co/guide/en/uptime/current" + observability-guide: "https://www.elastic.co/guide/en/observability/current" + observability-guide-all: "https://www.elastic.co/guide/en/observability" + siem-guide: "https://www.elastic.co/guide/en/siem/guide/current" + security-guide: "https://www.elastic.co/guide/en/security/current" + security-guide-all: "https://www.elastic.co/guide/en/security" + endpoint-guide: "https://www.elastic.co/guide/en/endpoint/current" + sql-odbc: "https://www.elastic.co/guide/en/elasticsearch/sql-odbc/current" + ecs-ref: "https://www.elastic.co/guide/en/ecs/current" + ecs-logging-ref: "https://www.elastic.co/guide/en/ecs-logging/overview/current" + ecs-logging-go-logrus-ref: "https://www.elastic.co/guide/en/ecs-logging/go-logrus/current" + ecs-logging-go-zap-ref: "https://www.elastic.co/guide/en/ecs-logging/go-zap/current" + ecs-logging-go-zerolog-ref: "https://www.elastic.co/guide/en/ecs-logging/go-zap/current" + ecs-logging-java-ref: "https://www.elastic.co/guide/en/ecs-logging/java/current" + ecs-logging-dotnet-ref: "https://www.elastic.co/guide/en/ecs-logging/dotnet/current" + ecs-logging-nodejs-ref: "https://www.elastic.co/guide/en/ecs-logging/nodejs/current" + ecs-logging-php-ref: "https://www.elastic.co/guide/en/ecs-logging/php/current" + ecs-logging-python-ref: "https://www.elastic.co/guide/en/ecs-logging/python/current" + ecs-logging-ruby-ref: "https://www.elastic.co/guide/en/ecs-logging/ruby/current" + ml-docs: "https://www.elastic.co/guide/en/machine-learning/current" + eland-docs: "https://www.elastic.co/guide/en/elasticsearch/client/eland/current" + eql-ref: "https://eql.readthedocs.io/en/latest/query-guide" + extendtrial: "https://www.elastic.co/trialextension" + wikipedia: "https://en.wikipedia.org/wiki" + forum: "https://discuss.elastic.co/" + xpack-forum: "https://discuss.elastic.co/c/50-x-pack" + security-forum: "https://discuss.elastic.co/c/x-pack/shield" + watcher-forum: "https://discuss.elastic.co/c/x-pack/watcher" + monitoring-forum: "https://discuss.elastic.co/c/x-pack/marvel" + graph-forum: "https://discuss.elastic.co/c/x-pack/graph" + apm-forum: "https://discuss.elastic.co/c/apm" + enterprise-search-ref: "https://www.elastic.co/guide/en/enterprise-search/current" + app-search-ref: "https://www.elastic.co/guide/en/app-search/current" + workplace-search-ref: "https://www.elastic.co/guide/en/workplace-search/current" + enterprise-search-node-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/enterprise-search-node/current" + enterprise-search-php-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/php/current" + enterprise-search-python-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/python/current" + enterprise-search-ruby-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/ruby/current" + elastic-maps-service: "https://maps.elastic.co" + integrations-docs: "https://docs.elastic.co/en/integrations" + integrations-devguide: "https://www.elastic.co/guide/en/integrations-developer/current" + time-units: "https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#time-units" + byte-units: "https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#byte-units" + apm-py-ref-v: "https://www.elastic.co/guide/en/apm/agent/python/current" + apm-node-ref-v: "https://www.elastic.co/guide/en/apm/agent/nodejs/current" + apm-rum-ref-v: "https://www.elastic.co/guide/en/apm/agent/rum-js/current" + apm-ruby-ref-v: "https://www.elastic.co/guide/en/apm/agent/ruby/current" + apm-java-ref-v: "https://www.elastic.co/guide/en/apm/agent/java/current" + apm-go-ref-v: "https://www.elastic.co/guide/en/apm/agent/go/current" + apm-ios-ref-v: "https://www.elastic.co/guide/en/apm/agent/swift/current" + apm-dotnet-ref-v: "https://www.elastic.co/guide/en/apm/agent/dotnet/current" + apm-php-ref-v: "https://www.elastic.co/guide/en/apm/agent/php/current" + ecloud: "Elastic Cloud" + esf: "Elastic Serverless Forwarder" + ess: "Elasticsearch Service" + ece: "Elastic Cloud Enterprise" + eck: "Elastic Cloud on Kubernetes" + serverless-full: "Elastic Cloud Serverless" + serverless-short: "Serverless" + es-serverless: "Elasticsearch Serverless" + es3: "Elasticsearch Serverless" + obs-serverless: "Elastic Observability Serverless" + sec-serverless: "Elastic Security Serverless" + serverless-docs: "https://docs.elastic.co/serverless" + cloud: "https://www.elastic.co/guide/en/cloud/current" + ess-utm-params: "?page=docs&placement=docs-body" + ess-baymax: "?page=docs&placement=docs-body" + ess-trial: "https://cloud.elastic.co/registration?page=docs&placement=docs-body" + ess-product: "https://www.elastic.co/cloud/elasticsearch-service?page=docs&placement=docs-body" + ess-console: "https://cloud.elastic.co?page=docs&placement=docs-body" + ess-console-name: "Elasticsearch Service Console" + ess-deployments: "https://cloud.elastic.co/deployments?page=docs&placement=docs-body" + ece-ref: "https://www.elastic.co/guide/en/cloud-enterprise/current" + eck-ref: "https://www.elastic.co/guide/en/cloud-on-k8s/current" + ess-leadin: "You can run Elasticsearch on your own hardware or use our hosted Elasticsearch Service that is available on AWS, GCP, and Azure. https://cloud.elastic.co/registration{ess-utm-params}[Try the Elasticsearch Service for free]." + ess-leadin-short: "Our hosted Elasticsearch Service is available on AWS, GCP, and Azure, and you can https://cloud.elastic.co/registration{ess-utm-params}[try it for free]." + ess-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud.svg[link=\"https://cloud.elastic.co/registration{ess-utm-params}\", title=\"Supported on Elasticsearch Service\"]" + ece-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud_ece.svg[link=\"https://cloud.elastic.co/registration{ess-utm-params}\", title=\"Supported on Elastic Cloud Enterprise\"]" + cloud-only: "This feature is designed for indirect use by https://cloud.elastic.co/registration{ess-utm-params}[Elasticsearch Service], https://www.elastic.co/guide/en/cloud-enterprise/{ece-version-link}[Elastic Cloud Enterprise], and https://www.elastic.co/guide/en/cloud-on-k8s/current[Elastic Cloud on Kubernetes]. Direct use is not supported." + ess-setting-change: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud.svg[link=\"{ess-trial}\", title=\"Supported on {ess}\"] indicates a change to a supported https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html[user setting] for Elasticsearch Service." + ess-skip-section: "If you use Elasticsearch Service, skip this section. Elasticsearch Service handles these changes for you." + api-cloud: "https://www.elastic.co/docs/api/doc/cloud" + api-ece: "https://www.elastic.co/docs/api/doc/cloud-enterprise" + api-kibana-serverless: "https://www.elastic.co/docs/api/doc/serverless" + es-feature-flag: "This feature is in development and not yet available for use. This documentation is provided for informational purposes only." + es-ref-dir: "'{{elasticsearch-root}}/docs/reference'" + apm-app: "APM app" + uptime-app: "Uptime app" + synthetics-app: "Synthetics app" + logs-app: "Logs app" + metrics-app: "Metrics app" + infrastructure-app: "Infrastructure app" + siem-app: "SIEM app" + security-app: "Elastic Security app" + ml-app: "Machine Learning" + dev-tools-app: "Dev Tools" + ingest-manager-app: "Ingest Manager" + stack-manage-app: "Stack Management" + stack-monitor-app: "Stack Monitoring" + alerts-ui: "Alerts and Actions" + rules-ui: "Rules" + rac-ui: "Rules and Connectors" + connectors-ui: "Connectors" + connectors-feature: "Actions and Connectors" + stack-rules-feature: "Stack Rules" + user-experience: "User Experience" + ems: "Elastic Maps Service" + ems-init: "EMS" + hosted-ems: "Elastic Maps Server" + ipm-app: "Index Pattern Management" + ingest-pipelines: "ingest pipelines" + ingest-pipelines-app: "Ingest Pipelines" + ingest-pipelines-cap: "Ingest pipelines" + ls-pipelines: "Logstash pipelines" + ls-pipelines-app: "Logstash Pipelines" + maint-windows: "maintenance windows" + maint-windows-app: "Maintenance Windows" + maint-windows-cap: "Maintenance windows" + custom-roles-app: "Custom Roles" + data-source: "data view" + data-sources: "data views" + data-source-caps: "Data View" + data-sources-caps: "Data Views" + data-source-cap: "Data view" + data-sources-cap: "Data views" + project-settings: "Project settings" + manage-app: "Management" + index-manage-app: "Index Management" + data-views-app: "Data Views" + rules-app: "Rules" + saved-objects-app: "Saved Objects" + tags-app: "Tags" + api-keys-app: "API keys" + transforms-app: "Transforms" + connectors-app: "Connectors" + files-app: "Files" + reports-app: "Reports" + maps-app: "Maps" + alerts-app: "Alerts" + crawler: "Enterprise Search web crawler" + ents: "Enterprise Search" + app-search-crawler: "App Search web crawler" + agent: "Elastic Agent" + agents: "Elastic Agents" + fleet: "Fleet" + fleet-server: "Fleet Server" + integrations-server: "Integrations Server" + ingest-manager: "Ingest Manager" + ingest-management: "ingest management" + package-manager: "Elastic Package Manager" + integrations: "Integrations" + package-registry: "Elastic Package Registry" + artifact-registry: "Elastic Artifact Registry" + aws: "AWS" + stack: "Elastic Stack" + xpack: "X-Pack" + es: "Elasticsearch" + kib: "Kibana" + esms: "Elastic Stack Monitoring Service" + esms-init: "ESMS" + ls: "Logstash" + beats: "Beats" + auditbeat: "Auditbeat" + filebeat: "Filebeat" + heartbeat: "Heartbeat" + metricbeat: "Metricbeat" + packetbeat: "Packetbeat" + winlogbeat: "Winlogbeat" + functionbeat: "Functionbeat" + journalbeat: "Journalbeat" + es-sql: "Elasticsearch SQL" + esql: "ES|QL" + elastic-agent: "Elastic Agent" + k8s: "Kubernetes" + log-driver-long: "Elastic Logging Plugin for Docker" + security: "X-Pack security" + security-features: "security features" + operator-feature: "operator privileges feature" + es-security-features: "Elasticsearch security features" + stack-security-features: "Elastic Stack security features" + endpoint-sec: "Endpoint Security" + endpoint-cloud-sec: "Endpoint and Cloud Security" + elastic-defend: "Elastic Defend" + elastic-sec: "Elastic Security" + elastic-endpoint: "Elastic Endpoint" + swimlane: "Swimlane" + sn: "ServiceNow" + sn-itsm: "ServiceNow ITSM" + sn-itom: "ServiceNow ITOM" + sn-sir: "ServiceNow SecOps" + jira: "Jira" + ibm-r: "IBM Resilient" + webhook: "Webhook" + webhook-cm: "Webhook - Case Management" + opsgenie: "Opsgenie" + bedrock: "Amazon Bedrock" + gemini: "Google Gemini" + hive: "TheHive" + monitoring: "X-Pack monitoring" + monitor-features: "monitoring features" + stack-monitor-features: "Elastic Stack monitoring features" + watcher: "Watcher" + alert-features: "alerting features" + reporting: "X-Pack reporting" + report-features: "reporting features" + graph: "X-Pack graph" + graph-features: "graph analytics features" + searchprofiler: "Search Profiler" + xpackml: "X-Pack machine learning" + ml: "machine learning" + ml-cap: "Machine learning" + ml-init: "ML" + ml-features: "machine learning features" + stack-ml-features: "Elastic Stack machine learning features" + ccr: "cross-cluster replication" + ccr-cap: "Cross-cluster replication" + ccr-init: "CCR" + ccs: "cross-cluster search" + ccs-cap: "Cross-cluster search" + ccs-init: "CCS" + ilm: "index lifecycle management" + ilm-cap: "Index lifecycle management" + ilm-init: "ILM" + dlm: "data lifecycle management" + dlm-cap: "Data lifecycle management" + dlm-init: "DLM" + search-snap: "searchable snapshot" + search-snaps: "searchable snapshots" + search-snaps-cap: "Searchable snapshots" + slm: "snapshot lifecycle management" + slm-cap: "Snapshot lifecycle management" + slm-init: "SLM" + rollup-features: "data rollup features" + ipm: "index pattern management" + ipm-cap: "Index pattern" + rollup: "rollup" + rollup-cap: "Rollup" + rollups: "rollups" + rollups-cap: "Rollups" + rollup-job: "rollup job" + rollup-jobs: "rollup jobs" + rollup-jobs-cap: "Rollup jobs" + dfeed: "datafeed" + dfeeds: "datafeeds" + dfeed-cap: "Datafeed" + dfeeds-cap: "Datafeeds" + ml-jobs: "machine learning jobs" + ml-jobs-cap: "Machine learning jobs" + anomaly-detect: "anomaly detection" + anomaly-detect-cap: "Anomaly detection" + anomaly-job: "anomaly detection job" + anomaly-jobs: "anomaly detection jobs" + anomaly-jobs-cap: "Anomaly detection jobs" + dataframe: "data frame" + dataframes: "data frames" + dataframe-cap: "Data frame" + dataframes-cap: "Data frames" + watcher-transform: "payload transform" + watcher-transforms: "payload transforms" + watcher-transform-cap: "Payload transform" + watcher-transforms-cap: "Payload transforms" + transform: "transform" + transforms: "transforms" + transform-cap: "Transform" + transforms-cap: "Transforms" + dataframe-transform: "transform" + dataframe-transform-cap: "Transform" + dataframe-transforms: "transforms" + dataframe-transforms-cap: "Transforms" + dfanalytics-cap: "Data frame analytics" + dfanalytics: "data frame analytics" + dataframe-analytics-config: "'{dataframe} analytics config'" + dfanalytics-job: "'{dataframe} analytics job'" + dfanalytics-jobs: "'{dataframe} analytics jobs'" + dfanalytics-jobs-cap: "'{dataframe-cap} analytics jobs'" + cdataframe: "continuous data frame" + cdataframes: "continuous data frames" + cdataframe-cap: "Continuous data frame" + cdataframes-cap: "Continuous data frames" + cdataframe-transform: "continuous transform" + cdataframe-transforms: "continuous transforms" + cdataframe-transforms-cap: "Continuous transforms" + ctransform: "continuous transform" + ctransform-cap: "Continuous transform" + ctransforms: "continuous transforms" + ctransforms-cap: "Continuous transforms" + oldetection: "outlier detection" + oldetection-cap: "Outlier detection" + olscore: "outlier score" + olscores: "outlier scores" + fiscore: "feature influence score" + evaluatedf-api: "evaluate {dataframe} analytics API" + evaluatedf-api-cap: "Evaluate {dataframe} analytics API" + binarysc: "binary soft classification" + binarysc-cap: "Binary soft classification" + regression: "regression" + regression-cap: "Regression" + reganalysis: "regression analysis" + reganalysis-cap: "Regression analysis" + depvar: "dependent variable" + feature-var: "feature variable" + feature-vars: "feature variables" + feature-vars-cap: "Feature variables" + classification: "classification" + classification-cap: "Classification" + classanalysis: "classification analysis" + classanalysis-cap: "Classification analysis" + infer-cap: "Inference" + infer: "inference" + lang-ident-cap: "Language identification" + lang-ident: "language identification" + data-viz: "Data Visualizer" + file-data-viz: "File Data Visualizer" + feat-imp: "feature importance" + feat-imp-cap: "Feature importance" + nlp: "natural language processing" + nlp-cap: "Natural language processing" + apm-agent: "APM agent" + apm-go-agent: "Elastic APM Go agent" + apm-go-agents: "Elastic APM Go agents" + apm-ios-agent: "Elastic APM iOS agent" + apm-ios-agents: "Elastic APM iOS agents" + apm-java-agent: "Elastic APM Java agent" + apm-java-agents: "Elastic APM Java agents" + apm-dotnet-agent: "Elastic APM .NET agent" + apm-dotnet-agents: "Elastic APM .NET agents" + apm-node-agent: "Elastic APM Node.js agent" + apm-node-agents: "Elastic APM Node.js agents" + apm-php-agent: "Elastic APM PHP agent" + apm-php-agents: "Elastic APM PHP agents" + apm-py-agent: "Elastic APM Python agent" + apm-py-agents: "Elastic APM Python agents" + apm-ruby-agent: "Elastic APM Ruby agent" + apm-ruby-agents: "Elastic APM Ruby agents" + apm-rum-agent: "Elastic APM Real User Monitoring (RUM) JavaScript agent" + apm-rum-agents: "Elastic APM RUM JavaScript agents" + apm-lambda-ext: "Elastic APM AWS Lambda extension" + project-monitors: "project monitors" + project-monitors-cap: "Project monitors" + private-location: "Private Location" + private-locations: "Private Locations" + pwd: "YOUR_PASSWORD" + esh: "ES-Hadoop" + default-dist: "default distribution" + oss-dist: "OSS-only distribution" + observability: "Observability" + api-request-title: "Request" + api-prereq-title: "Prerequisites" + api-description-title: "Description" + api-path-parms-title: "Path parameters" + api-query-parms-title: "Query parameters" + api-request-body-title: "Request body" + api-response-codes-title: "Response codes" + api-response-body-title: "Response body" + api-example-title: "Example" + api-examples-title: "Examples" + api-definitions-title: "Properties" + multi-arg: "†footnoteref:[multi-arg,This parameter accepts multiple arguments.]" + multi-arg-ref: "†footnoteref:[multi-arg]" + yes-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png[Yes,20,15]" + no-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png[No,20,15]" + es-repo: "https://github.com/elastic/elasticsearch/" + es-issue: "https://github.com/elastic/elasticsearch/issues/" + es-pull: "https://github.com/elastic/elasticsearch/pull/" + es-commit: "https://github.com/elastic/elasticsearch/commit/" + kib-repo: "https://github.com/elastic/kibana/" + kib-issue: "https://github.com/elastic/kibana/issues/" + kibana-issue: "'{kib-repo}issues/'" + kib-pull: "https://github.com/elastic/kibana/pull/" + kibana-pull: "'{kib-repo}pull/'" + kib-commit: "https://github.com/elastic/kibana/commit/" + ml-repo: "https://github.com/elastic/ml-cpp/" + ml-issue: "https://github.com/elastic/ml-cpp/issues/" + ml-pull: "https://github.com/elastic/ml-cpp/pull/" + ml-commit: "https://github.com/elastic/ml-cpp/commit/" + apm-repo: "https://github.com/elastic/apm-server/" + apm-issue: "https://github.com/elastic/apm-server/issues/" + apm-pull: "https://github.com/elastic/apm-server/pull/" + kibana-blob: "https://github.com/elastic/kibana/blob/current/" + apm-get-started-ref: "https://www.elastic.co/guide/en/apm/get-started/current" + apm-server-ref: "https://www.elastic.co/guide/en/apm/server/current" + apm-server-ref-v: "https://www.elastic.co/guide/en/apm/server/current" + apm-server-ref-m: "https://www.elastic.co/guide/en/apm/server/master" + apm-server-ref-62: "https://www.elastic.co/guide/en/apm/server/6.2" + apm-server-ref-64: "https://www.elastic.co/guide/en/apm/server/6.4" + apm-server-ref-70: "https://www.elastic.co/guide/en/apm/server/7.0" + apm-overview-ref-v: "https://www.elastic.co/guide/en/apm/get-started/current" + apm-overview-ref-70: "https://www.elastic.co/guide/en/apm/get-started/7.0" + apm-overview-ref-m: "https://www.elastic.co/guide/en/apm/get-started/master" + infra-guide: "https://www.elastic.co/guide/en/infrastructure/guide/current" + a-data-source: "a data view" + icon-bug: "pass:[]" + icon-checkInCircleFilled: "pass:[]" + icon-warningFilled: "pass:[]" diff --git a/docs/external-resources.asciidoc b/docs/external-resources.asciidoc deleted file mode 100644 index 5a73c851c..000000000 --- a/docs/external-resources.asciidoc +++ /dev/null @@ -1,10 +0,0 @@ -[[external-resources]] -== External resources - -// Note: we keep the actual links in GitHub to ease maintenance and avoid publishing external -// links on the corporate website. This is particularly important as the documentation for -// older versions of the Elastic Stack may stop being updated on the website. - -There is some material related to the {java-client} available outside this website that provides additional information or different perspectives on the library. - -You can find a community-maintained list of external resources in the https://github.com/elastic/elasticsearch-java/tree/main/docs/external-resources.md[{java-client} GitHub repository]. diff --git a/docs/getting-started.asciidoc b/docs/getting-started.asciidoc deleted file mode 100644 index 337e2e3b1..000000000 --- a/docs/getting-started.asciidoc +++ /dev/null @@ -1,172 +0,0 @@ -[[getting-started-java]] -== Getting started - -This page guides you through the installation process of the Java client, shows -you how to instantiate the client, and how to perform basic Elasticsearch -operations with it. - -[discrete] -=== Requirements - -* Java 8 or later. -* A JSON object mapping library to allow seamless integration of -your application classes with the Elasticsearch API. The examples below -show usage with Jackson. - -[discrete] -=== Installation - -[discrete] -==== Installation in a Gradle project by using Jackson - -["source","groovy",subs="attributes+"] --------------------------------------------------- -dependencies { - implementation 'co.elastic.clients:elasticsearch-java:{version}' - implementation 'com.fasterxml.jackson.core:jackson-databind:2.17.0' -} --------------------------------------------------- - -[discrete] -==== Installation in a Maven project by using Jackson - -In the `pom.xml` of your project, add the following repository definition and -dependencies: - -["source","xml",subs="attributes+"] --------------------------------------------------- - - - - - co.elastic.clients - elasticsearch-java - {version} - - - - com.fasterxml.jackson.core - jackson-databind - 2.17.0 - - - - --------------------------------------------------- - - -Refer to the <> page to learn more. - - -[discrete] -=== Connecting - -You can connect to the Elastic Cloud using an API key and the Elasticsearch -endpoint. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/getting_started/ConnectingTest.java[create-client] --------------------------------------------------- - -Your Elasticsearch endpoint can be found on the **My deployment** page of your -deployment: - -image::images/es-endpoint.jpg[alt="Finding Elasticsearch endpoint",align="center"] - -You can generate an API key on the **Management** page under Security. - -image::images/create-api-key.png[alt="Create API key",align="center"] - -For other connection options, refer to the <> section. - - -[discrete] -=== Operations - -Time to use Elasticsearch! This section walks you through the basic, and most -important, operations of Elasticsearch. For more operations and more advanced -examples, refer to the <> page. - - -[discrete] -==== Creating an index - -This is how you create the `product` index: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingTest.java[create-products-index] --------------------------------------------------- - -[discrete] -==== Indexing documents - -This is a simple way of indexing a document, here a `Product` application object: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingTest.java[single-doc-dsl] --------------------------------------------------- - -[discrete] -==== Getting documents - -You can get documents by using the following code: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/ReadingTest.java[get-by-id] --------------------------------------------------- -<1> The get request, with the index name and identifier. -<2> The target class, here `Product`. - - -[discrete] -==== Searching documents - -This is how you can create a single match query with the Java client: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/SearchingTest.java[search-getting-started] --------------------------------------------------- - -[discrete] -==== Updating documents - -This is how you can update a document, for example to add a new field: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingTest.java[single-doc-update] --------------------------------------------------- - - - -[discrete] -==== Deleting documents - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingTest.java[single-doc-delete] --------------------------------------------------- - - -[discrete] -==== Deleting an index - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingTest.java[delete-products-index] --------------------------------------------------- - -[discrete] -== Examples - -The https://github.com/elastic/elasticsearch-java/tree/main/examples[examples] folder in the Github repository contains full working examples showing how to set up and use the client. - -[discrete] -== Further reading - -* Learn more about the <> of the Java client. diff --git a/docs/index-local.asciidoc b/docs/index-local.asciidoc deleted file mode 100644 index f7860b40d..000000000 --- a/docs/index-local.asciidoc +++ /dev/null @@ -1,5 +0,0 @@ -// Allow building docs locally without a checkout of the Elasticsearch repo -:elasticsearch-root: {docdir}/local/elasticsearch -:local-docs: true - -include::index.asciidoc[] diff --git a/docs/index.asciidoc b/docs/index.asciidoc deleted file mode 100644 index 29b4e6cd9..000000000 --- a/docs/index.asciidoc +++ /dev/null @@ -1,40 +0,0 @@ -= Elasticsearch Java API Client - -include::{asciidoc-dir}/../../shared/versions/stack/{source_branch}.asciidoc[] -include::{asciidoc-dir}/../../shared/attributes.asciidoc[] - -include::_releases.asciidoc[] - -:java-client: Java API Client -:doc-tests-src: {docdir}/../java-client/src/test/java/co/elastic/clients/documentation -:doc-tests-url: https://github.com/elastic/elasticsearch-java/tree/{branch}/java-client/src/test/java/co/elastic/clients/documentation -:doc-tests-blurb: The source code for the examples above can be found in the {doc-tests-url}[{java-client} tests]. - -:es-docs: https://www.elastic.co/guide/en/elasticsearch/reference/{branch} - -ifeval::["{release-state}"=="unreleased"] -:java-client-javadoc: https://snapshots.elastic.co/javadoc/co/elastic/clients/elasticsearch-java/{version}-SNAPSHOT -:rest-client-javadoc: https://snapshots.elastic.co/javadoc/org/elasticsearch/client/elasticsearch-rest-client/{version}-SNAPSHOT -:rest-client-sniffer-javadoc: https://snapshots.elastic.co/javadoc/org/elasticsearch/client/elasticsearch-rest-client-sniffer/{version}-SNAPSHOT -:version_qualified: {bare_version}-SNAPSHOT -endif::[] - -ifeval::["{release-state}"!="unreleased"] -:java-client-javadoc: https://artifacts.elastic.co/javadoc/co/elastic/clients/elasticsearch-java/{version} -:rest-client-javadoc: https://artifacts.elastic.co/javadoc/org/elasticsearch/client/elasticsearch-rest-client/{version} -:rest-client-sniffer-javadoc: https://artifacts.elastic.co/javadoc/org/elasticsearch/client/elasticsearch-rest-client-sniffer/{version} -:version_qualified: {bare_version} -endif::[] - -include::introduction.asciidoc[] -include::getting-started.asciidoc[] -include::setup/index.asciidoc[] -include::api-conventions/index.asciidoc[] - -include::usage/index.asciidoc[] - -include::troubleshooting/index.asciidoc[] -include::javadoc-and-source.asciidoc[] -include::release-notes/index.asciidoc[] -include::external-resources.asciidoc[] -include::{elasticsearch-root}/docs/java-rest/low-level/index.asciidoc[] diff --git a/docs/introduction.asciidoc b/docs/introduction.asciidoc deleted file mode 100644 index d283813d8..000000000 --- a/docs/introduction.asciidoc +++ /dev/null @@ -1,38 +0,0 @@ -[[introduction]] -== Introduction - -This is the documentation for the official Java API Client for {es}. The client -provides strongly typed requests and responses for all {es} APIs. - -[discrete] -=== Features - -* Strongly typed requests and responses for all {es} APIs. -* Blocking and asynchronous versions of all APIs. -* Use of fluent builders and functional patterns to allow writing concise yet - readable code when creating complex nested structures. -* Seamless integration of application classes by using an object mapper such as - Jackson or any JSON-B implementation. -* Delegates protocol handling to an http client such as the <> - that takes care of all transport-level concerns: HTTP connection pooling, - retries, node discovery, and so on. - -[discrete] -=== Elasticsearch server compatibility policy - -The {es} Java client is forward compatible; meaning that the client supports -communicating with greater or equal minor versions of {es} without breaking. It -does not mean that the client automatically supports new features of newer -{es} versions; it is only possible after a release of a new client version. For -example, a 8.12 client version won't automatically support the new features of -the 8.13 version of {es}, the 8.13 client version is required for that. {es} -language clients are only backwards compatible with default distributions and -without guarantees made. - -|=== -| Elasticsearch Version | Elasticsearch-Java Branch | Supported - -| main | main | -| 8.x | 8.x | 8.x -| 7.x | 7.x | 7.17 -|=== \ No newline at end of file diff --git a/docs/javadoc-and-source.asciidoc b/docs/javadoc-and-source.asciidoc deleted file mode 100644 index 8298f97c5..000000000 --- a/docs/javadoc-and-source.asciidoc +++ /dev/null @@ -1,6 +0,0 @@ -[[java-client-javadoc]] -== Javadoc and source code - -The javadoc for the {java-client} can be found at {java-client-javadoc}/index.html. - -The source code is at https://github.com/elastic/elasticsearch-java/ and is licensed under the Apache 2.0 License. diff --git a/docs/local/elasticsearch/docs/java-rest/low-level/index.asciidoc b/docs/local/elasticsearch/docs/java-rest/low-level/index.asciidoc deleted file mode 100644 index c7b8560f3..000000000 --- a/docs/local/elasticsearch/docs/java-rest/low-level/index.asciidoc +++ /dev/null @@ -1,4 +0,0 @@ -[[java-rest-low]] -== Java Low Level REST Client - -This is a stub for the Java Low Level REST Client. diff --git a/docs/reference/_basic_authentication.md b/docs/reference/_basic_authentication.md new file mode 100644 index 000000000..ff376ff33 --- /dev/null +++ b/docs/reference/_basic_authentication.md @@ -0,0 +1,51 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/_basic_authentication.html +--- + +# Basic authentication [_basic_authentication] + +Configuring basic authentication can be done by providing an `HttpClientConfigCallback` while building the `RestClient` through its builder. The interface has one method that receives an instance of [`org.apache.http.impl.nio.client.HttpAsyncClientBuilder`](https://hc.apache.org/httpcomponents-asyncclient-4.1.x/current/httpasyncclient/apidocs/org/apache/http/impl/nio/client/HttpAsyncClientBuilder.md) as an argument and has the same return type. The http client builder can be modified and then returned. In the following example we set a default credentials provider that requires basic authentication. + +```java +final CredentialsProvider credentialsProvider = + new BasicCredentialsProvider(); +credentialsProvider.setCredentials(AuthScope.ANY, + new UsernamePasswordCredentials("user", "test-user-password")); + +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200)) + .setHttpClientConfigCallback(new HttpClientConfigCallback() { + @Override + public HttpAsyncClientBuilder customizeHttpClient( + HttpAsyncClientBuilder httpClientBuilder) { + return httpClientBuilder + .setDefaultCredentialsProvider(credentialsProvider); + } + }); +``` + +Preemptive Authentication can be disabled, which means that every request will be sent without authorization headers to see if it is accepted and, upon receiving an HTTP 401 response, it will resend the exact same request with the basic authentication header. If you wish to do this, then you can do so by disabling it via the `HttpAsyncClientBuilder`: + +```java +final CredentialsProvider credentialsProvider = + new BasicCredentialsProvider(); +credentialsProvider.setCredentials(AuthScope.ANY, + new UsernamePasswordCredentials("user", "test-user-password")); + +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200)) + .setHttpClientConfigCallback(new HttpClientConfigCallback() { + @Override + public HttpAsyncClientBuilder customizeHttpClient( + HttpAsyncClientBuilder httpClientBuilder) { + httpClientBuilder.disableAuthCaching(); <1> + return httpClientBuilder + .setDefaultCredentialsProvider(credentialsProvider); + } + }); +``` + +1. Disable preemptive authentication + + diff --git a/docs/reference/_encrypted_communication.md b/docs/reference/_encrypted_communication.md new file mode 100644 index 000000000..4018f7c98 --- /dev/null +++ b/docs/reference/_encrypted_communication.md @@ -0,0 +1,95 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/_encrypted_communication.html +--- + +# Encrypted communication [_encrypted_communication] + +Encrypted communication using TLS can also be configured through the `HttpClientConfigCallback`. The [`org.apache.http.impl.nio.client.HttpAsyncClientBuilder`](https://hc.apache.org/httpcomponents-asyncclient-4.1.x/current/httpasyncclient/apidocs/org/apache/http/impl/nio/client/HttpAsyncClientBuilder.md) received as an argument exposes multiple methods to configure encrypted communication: `setSSLContext`, `setSSLSessionStrategy` and `setConnectionManager`, in order of precedence from the least important. + +When accessing an Elasticsearch cluster that is setup for TLS on the HTTP layer, the client needs to trust the certificate that Elasticsearch is using. The following is an example of setting up the client to trust the CA that has signed the certificate that Elasticsearch is using, when that CA certificate is available in a PKCS#12 keystore: + +```java +Path trustStorePath = Paths.get("/path/to/truststore.p12"); +KeyStore truststore = KeyStore.getInstance("pkcs12"); +try (InputStream is = Files.newInputStream(trustStorePath)) { + truststore.load(is, keyStorePass.toCharArray()); +} +SSLContextBuilder sslBuilder = SSLContexts.custom() + .loadTrustMaterial(truststore, null); +final SSLContext sslContext = sslBuilder.build(); +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200, "https")) + .setHttpClientConfigCallback(new HttpClientConfigCallback() { + @Override + public HttpAsyncClientBuilder customizeHttpClient( + HttpAsyncClientBuilder httpClientBuilder) { + return httpClientBuilder.setSSLContext(sslContext); + } + }); +``` + +The following is an example of setting up the client to trust the CA that has signed the certificate that Elasticsearch is using, when that CA certificate is available as a PEM encoded file. + +```java +Path caCertificatePath = Paths.get("/path/to/ca.crt"); +CertificateFactory factory = + CertificateFactory.getInstance("X.509"); +Certificate trustedCa; +try (InputStream is = Files.newInputStream(caCertificatePath)) { + trustedCa = factory.generateCertificate(is); +} +KeyStore trustStore = KeyStore.getInstance("pkcs12"); +trustStore.load(null, null); +trustStore.setCertificateEntry("ca", trustedCa); +SSLContextBuilder sslContextBuilder = SSLContexts.custom() + .loadTrustMaterial(trustStore, null); +final SSLContext sslContext = sslContextBuilder.build(); +RestClient.builder( + new HttpHost("localhost", 9200, "https")) + .setHttpClientConfigCallback(new HttpClientConfigCallback() { + @Override + public HttpAsyncClientBuilder customizeHttpClient( + HttpAsyncClientBuilder httpClientBuilder) { + return httpClientBuilder.setSSLContext(sslContext); + } + }); +``` + +When Elasticsearch is configured to require client TLS authentication, for example when a PKI realm is configured, the client needs to provide a client certificate during the TLS handshake in order to authenticate. The following is an example of setting up the client for TLS authentication with a certificate and a private key that are stored in a PKCS#12 keystore. + +```java +Path trustStorePath = Paths.get("/path/to/your/truststore.p12"); +Path keyStorePath = Paths.get("/path/to/your/keystore.p12"); +KeyStore trustStore = KeyStore.getInstance("pkcs12"); +KeyStore keyStore = KeyStore.getInstance("pkcs12"); +try (InputStream is = Files.newInputStream(trustStorePath)) { + trustStore.load(is, trustStorePass.toCharArray()); +} +try (InputStream is = Files.newInputStream(keyStorePath)) { + keyStore.load(is, keyStorePass.toCharArray()); +} +SSLContextBuilder sslBuilder = SSLContexts.custom() + .loadTrustMaterial(trustStore, null) + .loadKeyMaterial(keyStore, keyStorePass.toCharArray()); +final SSLContext sslContext = sslBuilder.build(); +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200, "https")) + .setHttpClientConfigCallback(new HttpClientConfigCallback() { + @Override + public HttpAsyncClientBuilder customizeHttpClient( + HttpAsyncClientBuilder httpClientBuilder) { + return httpClientBuilder.setSSLContext(sslContext); + } + }); +``` + +If the client certificate and key are not available in a keystore but rather as PEM encoded files, you cannot use them directly to build an SSLContext. You must rely on external libraries to parse the PEM key into a PrivateKey instance. Alternatively, you can use external tools to build a keystore from your PEM files, as shown in the following example: + +``` +openssl pkcs12 -export -in client.crt -inkey private_key.pem \ + -name "client" -out client.p12 +``` + +If no explicit configuration is provided, the [system default configuration](https://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/JSSERefGuide.md#CustomizingStores) will be used. + diff --git a/docs/reference/_license.md b/docs/reference/_license.md new file mode 100644 index 000000000..42b8bb6fc --- /dev/null +++ b/docs/reference/_license.md @@ -0,0 +1,16 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/_license.html +--- + +# License [_license] + +Copyright 2013-2019 Elasticsearch + +Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at + +``` +http://www.apache.org/licenses/LICENSE-2.0 +``` +Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. + diff --git a/docs/reference/_maven_repository.md b/docs/reference/_maven_repository.md new file mode 100644 index 000000000..2f5324972 --- /dev/null +++ b/docs/reference/_maven_repository.md @@ -0,0 +1,35 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/_maven_repository.html +--- + +# Maven Repository [_maven_repository] + +The REST client sniffer is subject to the same release cycle as Elasticsearch. Replace the version with the desired sniffer version, first released with `5.0.0-alpha4`. There is no relation between the sniffer version and the Elasticsearch version that the client can communicate with. Sniffer supports fetching the nodes list from Elasticsearch 2.x and onwards. + +If you are looking for a SNAPSHOT version, the Elastic Maven Snapshot repository is available at [https://snapshots.elastic.co/maven/](https://snapshots.elastic.co/maven/). + +## Maven configuration [_maven_configuration] + +Here is how you can configure the dependency using maven as a dependency manager. Add the following to your `pom.xml` file: + +```xml + + org.elasticsearch.client + elasticsearch-rest-client-sniffer + 9.0.0-beta1 + +``` + + +## Gradle configuration [_gradle_configuration] + +Here is how you can configure the dependency using gradle as a dependency manager. Add the following to your `build.gradle` file: + +```groovy +dependencies { + compile 'org.elasticsearch.client:elasticsearch-rest-client-sniffer:9.0.0-beta1' +} +``` + + diff --git a/docs/reference/_node_selector.md b/docs/reference/_node_selector.md new file mode 100644 index 000000000..1cd0f3408 --- /dev/null +++ b/docs/reference/_node_selector.md @@ -0,0 +1,50 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/_node_selector.html +--- + +# Node selector [_node_selector] + +The client sends each request to one of the configured nodes in round-robin fashion. Nodes can optionally be filtered through a node selector that needs to be provided when initializing the client. This is useful when sniffing is enabled, in case no dedicated master nodes should be hit by HTTP requests. For each request the client will run the eventually configured node selector to filter the node candidates, then select the next one in the list out of the remaining ones. + +```java +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200, "http")); +builder.setNodeSelector(new NodeSelector() { <1> + @Override + public void select(Iterable nodes) { + /* + * Prefer any node that belongs to rack_one. If none is around + * we will go to another rack till it's time to try and revive + * some of the nodes that belong to rack_one. + */ + boolean foundOne = false; + for (Node node : nodes) { + String rackId = node.getAttributes().get("rack_id").get(0); + if ("rack_one".equals(rackId)) { + foundOne = true; + break; + } + } + if (foundOne) { + Iterator nodesIt = nodes.iterator(); + while (nodesIt.hasNext()) { + Node node = nodesIt.next(); + String rackId = node.getAttributes().get("rack_id").get(0); + if ("rack_one".equals(rackId) == false) { + nodesIt.remove(); + } + } + } + } +}); +``` + +1. Set an allocation aware node selector that allows to pick a node in the local rack if any available, otherwise go to any other node in any rack. It acts as a preference rather than a strict requirement, given that it goes to another rack if none of the local nodes are available, rather than returning no nodes in such case which would make the client forcibly revive a local node whenever none of the nodes from the preferred rack is available. + + +::::{warning} +Node selectors that do not consistently select the same set of nodes will make round-robin behaviour unpredictable and possibly unfair. The preference example above is fine as it reasons about availability of nodes which already affects the predictability of round-robin. Node selection should not depend on other external factors or round-robin will not work properly. +:::: + + diff --git a/docs/reference/_number_of_threads.md b/docs/reference/_number_of_threads.md new file mode 100644 index 000000000..318c2d407 --- /dev/null +++ b/docs/reference/_number_of_threads.md @@ -0,0 +1,24 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/_number_of_threads.html +--- + +# Number of threads [_number_of_threads] + +The Apache Http Async Client starts by default one dispatcher thread, and a number of worker threads used by the connection manager, as many as the number of locally detected processors (depending on what `Runtime.getRuntime().availableProcessors()` returns). The number of threads can be modified as follows: + +```java +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200)) + .setHttpClientConfigCallback(new HttpClientConfigCallback() { + @Override + public HttpAsyncClientBuilder customizeHttpClient( + HttpAsyncClientBuilder httpClientBuilder) { + return httpClientBuilder.setDefaultIOReactorConfig( + IOReactorConfig.custom() + .setIoThreadCount(1) + .build()); + } + }); +``` + diff --git a/docs/reference/_other_authentication_methods.md b/docs/reference/_other_authentication_methods.md new file mode 100644 index 000000000..ef3e88fb0 --- /dev/null +++ b/docs/reference/_other_authentication_methods.md @@ -0,0 +1,41 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/_other_authentication_methods.html +--- + +# Other authentication methods [_other_authentication_methods] + +## Elasticsearch Token Service tokens [_elasticsearch_token_service_tokens] + +If you want the client to authenticate with an Elasticsearch access token, set the relevant HTTP request header. If the client makes requests on behalf of a single user only, you can set the necessary `Authorization` header as a default header as shown in the following example: + +```java +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200, "http")); +Header[] defaultHeaders = + new Header[]{new BasicHeader("Authorization", + "Bearer u6iuAxZ0RG1Kcm5jVFI4eU4tZU9aVFEwT2F3")}; +builder.setDefaultHeaders(defaultHeaders); +``` + + +## Elasticsearch API keys [_elasticsearch_api_keys] + +If you want the client to authenticate with an Elasticsearch API key, set the relevant HTTP request header. If the client makes requests on behalf of a single user only, you can set the necessary `Authorization` header as a default header as shown in the following example: + +```java +String apiKeyId = "uqlEyn8B_gQ_jlvwDIvM"; +String apiKeySecret = "HxHWk2m4RN-V_qg9cDpuX"; +String apiKeyAuth = + Base64.getEncoder().encodeToString( + (apiKeyId + ":" + apiKeySecret) + .getBytes(StandardCharsets.UTF_8)); +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200, "http")); +Header[] defaultHeaders = + new Header[]{new BasicHeader("Authorization", + "ApiKey " + apiKeyAuth)}; +builder.setDefaultHeaders(defaultHeaders); +``` + + diff --git a/docs/reference/_others.md b/docs/reference/_others.md new file mode 100644 index 000000000..6232fb867 --- /dev/null +++ b/docs/reference/_others.md @@ -0,0 +1,14 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/_others.html +--- + +# Others [_others] + +For any other required configuration needed, the Apache HttpAsyncClient docs should be consulted: [https://hc.apache.org/httpcomponents-asyncclient-4.1.x/](https://hc.apache.org/httpcomponents-asyncclient-4.1.x/) . + +::::{note} +If your application runs under the security manager you might be subject to the JVM default policies of caching positive hostname resolutions indefinitely and negative hostname resolutions for ten seconds. If the resolved addresses of the hosts to which you are connecting the client to vary with time then you might want to modify the default JVM behavior. These can be modified by adding [`networkaddress.cache.ttl=`](https://docs.oracle.com/javase/8/docs/technotes/guides/net/properties.md) and [`networkaddress.cache.negative.ttl=`](https://docs.oracle.com/javase/8/docs/technotes/guides/net/properties.md) to your [Java security policy](https://docs.oracle.com/javase/8/docs/technotes/guides/security/PolicyFiles.md). +:::: + + diff --git a/docs/reference/_timeouts.md b/docs/reference/_timeouts.md new file mode 100644 index 000000000..1e265ac80 --- /dev/null +++ b/docs/reference/_timeouts.md @@ -0,0 +1,36 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/_timeouts.html +--- + +# Timeouts [_timeouts] + +Configuring requests timeouts can be done by providing an instance of `RequestConfigCallback` while building the `RestClient` through its builder. The interface has one method that receives an instance of [`org.apache.http.client.config.RequestConfig.Builder`](https://hc.apache.org/httpcomponents-client-4.5.x/current/httpclient/apidocs/org/apache/http/client/config/RequestConfig.Builder.md) as an argument and has the same return type. The request config builder can be modified and then returned. In the following example we increase the connect timeout (defaults to 1 second) and the socket timeout (defaults to 30 seconds). + +```java +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200)) + .setRequestConfigCallback( + new RestClientBuilder.RequestConfigCallback() { + @Override + public RequestConfig.Builder customizeRequestConfig( + RequestConfig.Builder requestConfigBuilder) { + return requestConfigBuilder + .setConnectTimeout(5000) + .setSocketTimeout(60000); + } + }); +``` + +Timeouts also can be set per request with RequestOptions, which overrides RestClient customizeRequestConfig. + +```java +RequestConfig requestConfig = RequestConfig.custom() + .setConnectTimeout(5000) + .setSocketTimeout(60000) + .build(); +RequestOptions options = RequestOptions.DEFAULT.toBuilder() + .setRequestConfig(requestConfig) + .build(); +``` + diff --git a/docs/reference/_usage.md b/docs/reference/_usage.md new file mode 100644 index 000000000..d38511e87 --- /dev/null +++ b/docs/reference/_usage.md @@ -0,0 +1,100 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/_usage.html +--- + +# Usage [_usage] + +Once a `RestClient` instance has been created as shown in [Initialization](/reference/java-rest-low-usage-initialization.md), a `Sniffer` can be associated to it. The `Sniffer` will make use of the provided `RestClient` to periodically (every 5 minutes by default) fetch the list of current nodes from the cluster and update them by calling `RestClient#setNodes`. + +```java +RestClient restClient = RestClient.builder( + new HttpHost("localhost", 9200, "http")) + .build(); +Sniffer sniffer = Sniffer.builder(restClient).build(); +``` + +It is important to close the `Sniffer` so that its background thread gets properly shutdown and all of its resources are released. The `Sniffer` object should have the same lifecycle as the `RestClient` and get closed right before the client: + +```java +sniffer.close(); +restClient.close(); +``` + +The `Sniffer` updates the nodes by default every 5 minutes. This interval can be customized by providing it (in milliseconds) as follows: + +```java +RestClient restClient = RestClient.builder( + new HttpHost("localhost", 9200, "http")) + .build(); +Sniffer sniffer = Sniffer.builder(restClient) + .setSniffIntervalMillis(60000).build(); +``` + +It is also possible to enable sniffing on failure, meaning that after each failure the nodes list gets updated straightaway rather than at the following ordinary sniffing round. In this case a `SniffOnFailureListener` needs to be created at first and provided at `RestClient` creation. Also once the `Sniffer` is later created, it needs to be associated with that same `SniffOnFailureListener` instance, which will be notified at each failure and use the `Sniffer` to perform the additional sniffing round as described. + +```java +SniffOnFailureListener sniffOnFailureListener = + new SniffOnFailureListener(); +RestClient restClient = RestClient.builder( + new HttpHost("localhost", 9200)) + .setFailureListener(sniffOnFailureListener) <1> + .build(); +Sniffer sniffer = Sniffer.builder(restClient) + .setSniffAfterFailureDelayMillis(30000) <2> + .build(); +sniffOnFailureListener.setSniffer(sniffer); <3> +``` + +1. Set the failure listener to the `RestClient` instance +2. When sniffing on failure, not only do the nodes get updated after each failure, but an additional sniffing round is also scheduled sooner than usual, by default one minute after the failure, assuming that things will go back to normal and we want to detect that as soon as possible. Said interval can be customized at `Sniffer` creation time through the `setSniffAfterFailureDelayMillis` method. Note that this last configuration parameter has no effect in case sniffing on failure is not enabled like explained above. +3. Set the `Sniffer` instance to the failure listener + + +The Elasticsearch Nodes Info api doesn’t return the protocol to use when connecting to the nodes but only their `host:port` key-pair, hence `http` is used by default. In case `https` should be used instead, the `ElasticsearchNodesSniffer` instance has to be manually created and provided as follows: + +```java +RestClient restClient = RestClient.builder( + new HttpHost("localhost", 9200, "http")) + .build(); +NodesSniffer nodesSniffer = new ElasticsearchNodesSniffer( + restClient, + ElasticsearchNodesSniffer.DEFAULT_SNIFF_REQUEST_TIMEOUT, + ElasticsearchNodesSniffer.Scheme.HTTPS); +Sniffer sniffer = Sniffer.builder(restClient) + .setNodesSniffer(nodesSniffer).build(); +``` + +In the same way it is also possible to customize the `sniffRequestTimeout`, which defaults to one second. That is the `timeout` parameter provided as a query string parameter when calling the Nodes Info api, so that when the timeout expires on the server side, a valid response is still returned although it may contain only a subset of the nodes that are part of the cluster, the ones that have responded until then. + +```java +RestClient restClient = RestClient.builder( + new HttpHost("localhost", 9200, "http")) + .build(); +NodesSniffer nodesSniffer = new ElasticsearchNodesSniffer( + restClient, + TimeUnit.SECONDS.toMillis(5), + ElasticsearchNodesSniffer.Scheme.HTTP); +Sniffer sniffer = Sniffer.builder(restClient) + .setNodesSniffer(nodesSniffer).build(); +``` + +Also, a custom `NodesSniffer` implementation can be provided for advanced use cases that may require fetching the nodes from external sources rather than from Elasticsearch: + +```java +RestClient restClient = RestClient.builder( + new HttpHost("localhost", 9200, "http")) + .build(); +NodesSniffer nodesSniffer = new NodesSniffer() { + @Override + public List sniff() throws IOException { + return null; <1> + } + }; +Sniffer sniffer = Sniffer.builder(restClient) + .setNodesSniffer(nodesSniffer).build(); +``` + +1. Fetch the hosts from the external source + + diff --git a/docs/reference/aggregations.md b/docs/reference/aggregations.md new file mode 100644 index 000000000..3934f13ee --- /dev/null +++ b/docs/reference/aggregations.md @@ -0,0 +1,73 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/aggregations.html +--- + +# Aggregations [aggregations] + +An aggregation summarizes your data as metrics, statistics, or other analytics. + +::::{note} +See the [{{es}} documentation](elasticsearch://docs/reference/data-analysis/aggregations/index.md) for a full explanation of aggregations. +:::: + + + +## A simple aggregation [_a_simple_aggregation] + +In the example below we run an aggregation that creates a price histogram from a product index, for the products whose name match a user-provided text. To achieve this, we use a search request that has a query (explained in [Searching for documents](/reference/searching.md)) and an aggregation definition. + +This example is an analytics-type aggregation where we do not want to use the matching documents. A general pattern for search requests used for analytics is to set the result `size` to zero and the target class for search results to `Void`. + +If that same aggregation was used for to display products and the price histogram as drill-down facets, we would have set `size` to a non-zero value and used `Product` as the target class to process the results. + +```java +String searchText = "bike"; + +Query query = MatchQuery.of(m -> m + .field("name") + .query(searchText) +)._toQuery(); + +SearchResponse response = esClient.search(b -> b + .index("products") + .size(0) <1> + .query(query) <2> + .aggregations("price-histogram", a -> a <3> + .histogram(h -> h <4> + .field("price") + .interval(50.0) + ) + ), + Void.class <5> +); +``` + +1. Set the number of matching documents to zero as we only use the price histogram. +2. Set the query that fill filter the products on which to run the aggregation +3. Create an aggregation named "price-histogram". You can add as many named aggregations as needed. +4. Select the `histogram` aggregation variant. +5. We do not care about matches (`size` is set to zero), using `Void` will ignore any document in the response. + + +The response contains an aggregation result for each aggregation in the request. + +```java +List buckets = response.aggregations() + .get("price-histogram") <1> + .histogram() <2> + .buckets().array(); <3> + +for (HistogramBucket bucket: buckets) { + logger.info("There are " + bucket.docCount() + + " bikes under " + bucket.key()); +} +``` + +1. Get the results for the "price-histogram" aggregation. +2. Cast it down to the `histogram` variant results. This has to be consistent with the aggregation definition. +3. Buckets can be expressed as arrays or maps. This casts down to the array variant (the default). + + +The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation). + diff --git a/docs/reference/api-conventions.md b/docs/reference/api-conventions.md new file mode 100644 index 000000000..629a150cc --- /dev/null +++ b/docs/reference/api-conventions.md @@ -0,0 +1,28 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/api-conventions.html +--- + +# API conventions [api-conventions] + +The Java API Client uses a very consistent code structure, using modern code patterns that make complex requests easier to write and complex responses easier to process. The sections below explain these in details. + +* [Package structure and namespace clients](/reference/package-structure.md) +* [Method naming conventions](/reference/method-naming.md) +* [Blocking and asynchronous clients](/reference/blocking-async.md) +* [Building API objects](/reference/building-objects.md) +* [Lists and maps](/reference/lists-maps.md) +* [Variant types](/reference/variant-types.md) +* [Object life cycles and thread safety](/reference/object-lifecycles.md) +* [Creating API objects from JSON data](/reference/loading-json.md) +* [Exceptions](/reference/exception-conventions.md) + + + + + + + + + + diff --git a/docs/reference/blocking-async.md b/docs/reference/blocking-async.md new file mode 100644 index 000000000..d7456d3b3 --- /dev/null +++ b/docs/reference/blocking-async.md @@ -0,0 +1,40 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/blocking-and-async.html +--- + +# Blocking and asynchronous clients [blocking-and-async] + +API clients come in two flavors: blocking and asynchronous. All methods on asynchronous clients return a standard `CompletableFuture`. + +Both flavors can be used at the same time depending on your needs, sharing the same transport object: + +```java +ElasticsearchTransport transport = ... + +// Synchronous blocking client +ElasticsearchClient client = new ElasticsearchClient(transport); + +if (client.exists(b -> b.index("products").id("foo")).value()) { + logger.info("product exists"); +} + +// Asynchronous non-blocking client +ElasticsearchAsyncClient asyncClient = + new ElasticsearchAsyncClient(transport); + +asyncClient + .exists(b -> b.index("products").id("foo")) + .whenComplete((response, exception) -> { + if (exception != null) { + logger.error("Failed to index", exception); + } else { + logger.info("Product exists"); + } + }); +``` + +Although we won’t go in deeper details on asynchronous programming in Java, remember to handle failures of asynchronous tasks. It’s easy to overlook them and have errors go unnoticed. + +The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation). + diff --git a/docs/reference/breaking-changes-policy.md b/docs/reference/breaking-changes-policy.md new file mode 100644 index 000000000..91afce486 --- /dev/null +++ b/docs/reference/breaking-changes-policy.md @@ -0,0 +1,35 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/breaking-changes-policy.html +--- + +# Breaking changes policy [breaking-changes-policy] + +The Java API Client source code is generated from a [formal specification of the Elasticsearch API](https://github.com/elastic/elasticsearch-specification). This API specification is large, and although it is tested against hundreds of Elasticsearch test files, it may have discrepancies with the actual API that result in issues in the Java API Client. + +Fixing these discrepancies in the API specification results in code changes in the Java API Client, and some of these changes can require code updates in your applications. + +This section explains how these breaking changes are considered for inclusion in Java API Client releases. + + +## Breaking changes in patch releases [_breaking_changes_in_patch_releases] + +Some issues in the API specification are properties that have an incorrect type, such as a `long` that should be a `string`, or a required property that is actually optional. These issues can cause the Java API Client to not work properly or even throw exceptions. + +When a specification issue is discovered and resolved, it may require code updates in applications using the Java API Client. Such breaking changes are considered acceptable, *even in patch releases* (e.g. 7.17.0 → 7.17.1), as they introduce stability to APIs that may otherwise be unusable. + + +## Breaking changes in minor releases [_breaking_changes_in_minor_releases] + +Along with these bug fixes, the API specification is constantly refined, more precise type definitions are introduced to improve developer comfort and remove ambiguities. The specification of often-used APIs is fairly mature, so these changes happen generally on less often used APIs. These changes can also cause breaking changes requiring code updates which are considered *acceptable in minor releases* (e.g. 8.0 → 8.1). + + +## Breaking changes in major releases [_breaking_changes_in_major_releases] + +Major releases (e.g. 7.x → 8.x) can include larger refactorings of the API specification and the framework underlying the Java API Client. These refactorings are considered carefully and done only when they unlock new important features or new developments. + + +## Elasticsearch API stability guarantees [_elasticsearch_api_stability_guarantees] + +All Elasticsearch APIs have stability indicators, which imply potential changes. If an API is `stable` only additional non-breaking changes are added. In case of `experimental` APIs, breaking changes can be introduced any time, which means that these changes, will also be reflected in the Java API Client. + diff --git a/docs/reference/building-objects.md b/docs/reference/building-objects.md new file mode 100644 index 000000000..2c8f4d713 --- /dev/null +++ b/docs/reference/building-objects.md @@ -0,0 +1,103 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/building-objects.html +--- + +# Building API objects [building-objects] + + +## Builder objects [_builder_objects] + +All data types in the Java API Client are immutable. Object creation uses the [builder pattern](https://www.informit.com/articles/article.aspx?p=1216151&seqNum=2) that was popularized in **Effective Java** in 2008. + +```java +ElasticsearchClient client = ... +CreateIndexResponse createResponse = client.indices().create( + new CreateIndexRequest.Builder() + .index("my-index") + .aliases("foo", + new Alias.Builder().isWriteIndex(true).build() + ) + .build() +); +``` + +Note that a builder should not be reused after its `build()` method has been called. + + +## Builder lambda expressions [_builder_lambda_expressions] + +Although this works nicely, having to instantiate builder classes and call the `build()` method is a bit verbose. So every property setter in the Java API Client also accepts a lambda expression that takes a newly created builder as a parameter and returns a populated builder. The snippet above can also be written as: + +```java +ElasticsearchClient client = ... +CreateIndexResponse createResponse = client.indices() + .create(createIndexBuilder -> createIndexBuilder + .index("my-index") + .aliases("foo", aliasBuilder -> aliasBuilder + .isWriteIndex(true) + ) + ); +``` + +This approach allows for much more concise code, and also avoids importing classes (and even remembering their names) since types are inferred from the method parameter signature. + +Note in the above example that builder variables are only used to start a chain of property setters. The names of these variables are therefore unimportant and can be shortened to improve readability: + +```java +ElasticsearchClient client = ... +CreateIndexResponse createResponse = client.indices() + .create(c -> c + .index("my-index") + .aliases("foo", a -> a + .isWriteIndex(true) + ) + ); +``` + +Builder lambdas become particularly useful with complex nested queries like the one below, taken from the [intervals query API documentation](elasticsearch://docs/reference/query-languages/query-dsl-intervals-query.md). + +This example also highlights a useful naming convention for builder parameters in deeply nested structures. For lambda expressions with a single argument, Kotlin provides the implicit `it` parameter and Scala allows use of `_`. This can be approximated in Java by using an underscore or a single letter prefix followed by a number representing the depth level (i.e. `_0`, `_1`, or `b0`, `b1` and so on). Not only does this remove the need to create throw-away variable names, but it also improves code readability. Correct indentation also allows the structure of the query to stand out. + +```java +ElasticsearchClient client = ... +SearchResponse results = client + .search(b0 -> b0 + .query(b1 -> b1 + .intervals(b2 -> b2 + .field("my_text") + .allOf(b3 -> b3 + .ordered(true) + .intervals(b4 -> b4 + .match(b5 -> b5 + .query("my favorite food") + .maxGaps(0) + .ordered(true) + ) + ) + .intervals(b4 -> b4 + .anyOf(b5 -> b5 + .intervals(b6 -> b6 + .match(b7 -> b7 + .query("hot water") + ) + ) + .intervals(b6 -> b6 + .match(b7 -> b7 + .query("cold porridge") + ) + ) + ) + ) + ) + ) + ), + SomeApplicationData.class <1> +); +``` + +1. Search results will be mapped to `SomeApplicationData` instances to be readily available to the application. + + +The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation). + diff --git a/docs/reference/connecting.md b/docs/reference/connecting.md new file mode 100644 index 000000000..0974a6bd7 --- /dev/null +++ b/docs/reference/connecting.md @@ -0,0 +1,186 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/connecting.html +--- + +# Connecting [connecting] + +The Java API Client is structured around three main components: + +* **API client classes**. These provide strongly typed data structures and methods for {{es}} APIs. Since the {{es}} API is large, it is structured in feature groups (also called “namespaces”), each having its own client class. {{es}} core features are implemented in the `ElasticsearchClient` class. +* **A JSON object mapper**. This maps your application classes to JSON and seamlessly integrates them with the API client. +* **A transport layer implementation**. This is where all HTTP request handling takes place. + +This code snippet creates and wires together these three components: + +```java +// URL and API key +String serverUrl = "https://localhost:9200"; +String apiKey = "VnVhQ2ZHY0JDZGJrU..."; + +// Create the low-level client +RestClient restClient = RestClient + .builder(HttpHost.create(serverUrl)) + .setDefaultHeaders(new Header[]{ + new BasicHeader("Authorization", "ApiKey " + apiKey) + }) + .build(); + +// Create the transport with a Jackson mapper +ElasticsearchTransport transport = new RestClientTransport( + restClient, new JacksonJsonpMapper()); + +// And create the API client +ElasticsearchClient esClient = new ElasticsearchClient(transport); + +// Use the client... + +// Close the client, also closing the underlying transport object and network connections. +esClient.close(); +``` + +Authentication is managed by the [Java Low Level REST Client](/reference/java-low-level-rest-client.md). For further details on configuring authentication, refer to [its documentation](/reference/_basic_authentication.md). + + +## Your first request [_your_first_request] + +The code snippet below searches all items from a “product” index whose name matches “bicycle” and return them as instances of a `Product` application class. + +It illustrates the use of fluent functional builders to write search queries as concise DSL-like code. This pattern is explained in more detail in [*API conventions*](/reference/api-conventions.md). + +```java +SearchResponse search = esClient.search(s -> s + .index("products") + .query(q -> q + .term(t -> t + .field("name") + .value(v -> v.stringValue("bicycle")) + )), + Product.class); + +for (Hit hit: search.hits().hits()) { + processProduct(hit.source()); +} +``` + + +## Using a secure connection [using-a-secure-connection] + +The [Java Low Level REST Client](/reference/java-low-level-rest-client.md) documentation explains how to set up encrypted communications in detail. + +In self-managed installations, Elasticsearch will start with security features like authentication and TLS enabled. To connect to the Elasticsearch cluster you’ll need to configure the Java API Client to use HTTPS with the generated CA certificate in order to make requests successfully. + +When you start Elasticsearch for the first time you’ll see a distinct block like the one below in the output from Elasticsearch (you may have to scroll up if it’s been a while): + +```xml +-> Elasticsearch security features have been automatically configured! +-> Authentication is enabled and cluster connections are encrypted. + +-> Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`): + lhQpLELkjkrawaBoaz0Q + +-> HTTP CA certificate SHA-256 fingerprint: + a52dd93511e8c6045e21f16654b77c9ee0f34aea26d9f40320b531c474676228 +... +``` + +Note down the elastic user password and HTTP CA fingerprint for the next sections. In the examples below they will be stored in the variables `password` and `fingerprint` respectively. + +Depending on the context, you have two options for verifying the HTTPS connection: either verifying with the CA certificate itself or using the CA certificate fingerprint. For both cases, the Java API Client’s `TransportUtils` class provides convenience methods to easily create an `SSLContext`. + + +### Verifying HTTPS with a certificate fingerprint [_verifying_https_with_a_certificate_fingerprint] + +This method of verifying the HTTPS connection uses the certificate fingerprint value noted down earlier. + +```java +String fingerprint = ""; + +SSLContext sslContext = TransportUtils + .sslContextFromCaFingerprint(fingerprint); <1> + +BasicCredentialsProvider credsProv = new BasicCredentialsProvider(); <2> +credsProv.setCredentials( + AuthScope.ANY, new UsernamePasswordCredentials(login, password) +); + +RestClient restClient = RestClient + .builder(new HttpHost(host, port, "https")) <3> + .setHttpClientConfigCallback(hc -> hc + .setSSLContext(sslContext) <4> + .setDefaultCredentialsProvider(credsProv) + ) + .build(); + +// Create the transport and the API client +ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper()); +ElasticsearchClient esClient = new ElasticsearchClient(transport); + +// Use the client... + +// Close the client, also closing the underlying transport object and network connections. +esClient.close(); +``` + +1. Create an `SSLContext` with the certificate fingerprint. +2. Set up authentication. +3. Do not forget to set the protocol to `https`! +4. Configure the http client with the SSL and authentication configurations. + + +Note that the certificate fingerprint can also be calculated using `openssl x509` with the certificate file: + +```bash +openssl x509 -fingerprint -sha256 -noout -in /path/to/http_ca.crt +``` + +If you don’t have access to the generated CA file from Elasticsearch you can use the following script to output the root CA fingerprint of the Elasticsearch instance with `openssl s_client`: + +```bash +openssl s_client -connect localhost:9200 -servername localhost -showcerts /dev/null \ + | openssl x509 -fingerprint -sha256 -noout -in /dev/stdin +``` + + +### Verifying HTTPS with a CA certificate [_verifying_https_with_a_ca_certificate] + +The generated root CA certificate can be found in the `certs` directory in your Elasticsearch config location. If you’re running Elasticsearch in Docker there is [additional documentation](docs-content://deploy-manage/deploy/self-managed/install-elasticsearch-with-docker.md) for retrieving the CA certificate. + +Once you have made the `http_ca.crt` file available to your application, you can use it to set up the client: + +```java +File certFile = new File("/path/to/http_ca.crt"); + +SSLContext sslContext = TransportUtils + .sslContextFromHttpCaCrt(certFile); <1> + +BasicCredentialsProvider credsProv = new BasicCredentialsProvider(); <2> +credsProv.setCredentials( + AuthScope.ANY, new UsernamePasswordCredentials(login, password) +); + +RestClient restClient = RestClient + .builder(new HttpHost(host, port, "https")) <3> + .setHttpClientConfigCallback(hc -> hc + .setSSLContext(sslContext) <4> + .setDefaultCredentialsProvider(credsProv) + ) + .build(); + +// Create the transport and the API client +ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper()); +ElasticsearchClient esClient = new ElasticsearchClient(transport); + +// Use the client... + +// Close the client, also closing the underlying transport object and network connections. +esClient.close(); +``` + +1. Create an `SSLContext` with the `http_ca.crt` file. +2. Set up authentication. +3. Do not forget to set the protocol to `https`! +4. Configure the http client with the SSL and authentication configurations. + + +The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation). diff --git a/docs/reference/esql.md b/docs/reference/esql.md new file mode 100644 index 000000000..274aaf3af --- /dev/null +++ b/docs/reference/esql.md @@ -0,0 +1,94 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/esql.html +--- + +# ES|QL in the Java client [esql] + +This page helps you understand and use [ES|QL](docs-content://explore-analyze/query-filter/languages/esql.md) in the Java client. + +There are two ways to use ES|QL in the Java API Client: + +* Use the Elasticsearch [ES|QL API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-esql) directly: This is the most flexible approach, but it’s also the most complex because you must handle results in their raw form. You can choose the precise format of results, such as JSON, CSV, or text. +* Use ES|QL mapping helpers: These mappers take care of parsing the raw response into something readily usable by the application. Several mappers are available for different use cases, such as object mapping, cursor traversal of results, and dataframes. You can also define your own mapper for specific use cases. + + +## How to use the ES|QL API [esql-how-to] + +The [ES|QL query API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-esql) allows you to specify how results should be returned. You can choose a [response format](docs-content://explore-analyze/query-filter/languages/esql-rest.md#esql-rest-format) such as CSV, text, or JSON, then fine-tune it with parameters like column separators and locale. + +Because the response varies widely depending on the format, the Java API Client has a BinaryData object you can use according to the format specified in the request. + +The following example gets ES|QL results as CSV and parses them: + +```java +String queryAuthor = + """ + from books + | where author == "Isaac Asimov" + | sort year desc + | limit 10 + """; + +BinaryResponse response = client.esql().query(q -> q + .format("csv") + .delimiter(",") + .query(queryAuthor)); + +String result = new BufferedReader(new InputStreamReader(response.content())) + .lines().collect(Collectors.joining("\n")); +``` + + +## Consume ES|QL results [esql-consume-results] + +The previous example showed that although the raw ES|QL API offers maximum flexibility, additional work is required in order to make use of the result data. + +To simplify things, try working with these three main representations of ES|QL results (each with its own mapping helper): + +* **Objects**, where each row in the results is mapped to an object from your application domain. This is similar to what ORMs (object relational mappers) commonly do. + + ```java + List queryRes = (List) client.esql().query(ObjectsEsqlAdapter.of(Book.class), queryAuthor); + ``` + +* **Cursors**, where you scan the results row by row and access the data using column names. This is similar to database access libraries. + + ```java + ResultSet resultSet = client.esql().query(ResultSetEsqlAdapter.INSTANCE, queryAuthor); + ``` + + + +## Define your own mapping [esql-custom-mapping] + +Although the mappers provided by the Java API Client cover many use cases, your application might require a custom mapping. You can write your own mapper and use it in a similar way as the built-in ones. + +Note that mappers are meant to provide a more usable representation of ES|QL results—not to process the result data. Data processing should be based on the output of a result mapper. + +Here’s an example mapper that returns a simple column-oriented representation of the data: + +```java +public class CustomStringAdapter extends EsqlAdapterBase { + + public static final CustomStringAdapter INSTANCE = new CustomStringAdapter(); + + @Override + public String format() { + return "json"; + } + + @Override + public boolean columnar() { + return true; + } + + @Override + public String deserialize(ApiClient client, QueryRequest request, + BinaryResponse response) + throws IOException { + return new BufferedReader(new InputStreamReader(response.content())) + .lines().collect(Collectors.joining("\n")); + } +} +``` diff --git a/docs/reference/exception-conventions.md b/docs/reference/exception-conventions.md new file mode 100644 index 000000000..711d25400 --- /dev/null +++ b/docs/reference/exception-conventions.md @@ -0,0 +1,12 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/exception-conventions.html +--- + +# Exceptions [exception-conventions] + +Client methods can throw two kinds of exceptions: + +* Requests that were received by the {{es}} server but that were rejected (validation error, server internal timeout exceeded, etc) will produce an `ElasticsearchException`. This exception contains details about the error, provided by {{es}}. +* Requests that failed to reach the server (network error, server unavailable, etc) will produce a `TransportException`. That exception’s cause is the exception thrown by the lower-level implementation. In the case of the `RestClientTransport` it will be a `ResponseException` that contains the low level HTTP response. + diff --git a/docs/reference/external-resources.md b/docs/reference/external-resources.md new file mode 100644 index 000000000..86f059f50 --- /dev/null +++ b/docs/reference/external-resources.md @@ -0,0 +1,11 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/external-resources.html +--- + +# External resources [external-resources] + +There is some material related to the Java API Client available outside this website that provides additional information or different perspectives on the library. + +You can find a community-maintained list of external resources in the [Java API Client GitHub repository](https://github.com/elastic/elasticsearch-java/tree/main/docs/external-resources.md). + diff --git a/docs/reference/getting-started-serverless.md b/docs/reference/getting-started-serverless.md new file mode 100644 index 000000000..6a46c0860 --- /dev/null +++ b/docs/reference/getting-started-serverless.md @@ -0,0 +1,175 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/serverless/current/elasticsearch-java-client-getting-started.html +navigation_title: Getting started in {{serverless}} +--- + +# Getting started with the Java API client in {{serverless-full}}[elasticsearch-java-client-getting-started] + +This page guides you through the installation process of the Java client, shows you how to initialize the client, and how to perform basic {{es}} operations with it. + +See the [Java client](/reference/index.md) documentation for more detailed usage instructions. + +::::{note} +The same client is used for {{es3}}, on-premise and managed Elasticsearch. Some API endpoints are however not available in {{es3}}. + +:::: + + + +## Requirements [elasticsearch-java-client-getting-started-requirements] + +* Java 8 or later. +* A JSON object mapping library to allow seamless integration of your application classes with the {{es}} API. The examples below show usage with Jackson. + + +## Installation [elasticsearch-java-client-getting-started-installation] + +You can add the Java client to your Java project using either Gradle or Maven. + +Use the version with the highest version number found on [Maven Central](https://search.maven.org/artifact/co.elastic.clients/elasticsearch-java), like `8.16.1`. We refer to it as `elasticVersion` in the configuration examples below. + + +### Using Gradle [elasticsearch-java-client-getting-started-using-gradle] + +You can install the Java client as a Gradle dependency: + +```groovy +dependencies { + implementation "co.elastic.clients:elasticsearch-java:${elasticVersion}" + implementation "com.fasterxml.jackson.core:jackson-databind:2.17.0" +} +``` + + +### Using Maven [elasticsearch-java-client-getting-started-using-maven] + +You can install the Java client as a Maven dependency, add the following to the `pom.xml` of your project: + +```xml + + + + + co.elastic.clients + elasticsearch-java + ${elasticVersion} + + + + com.fasterxml.jackson.core + jackson-databind + 2.17.0 + + + + +``` + + +## Initialize the client [elasticsearch-java-client-getting-started-initialize-the-client] + +Initialize the client using your API key and {{es}} endpoint: + +```java +// URL and API key +String serverUrl = "https://...elastic.cloud"; +String apiKey = "VnVhQ2ZHY0JDZGJrU..."; + +// Create the low-level client +RestClient restClient = RestClient + .builder(HttpHost.create(serverUrl)) + .setDefaultHeaders(new Header[]{ + new BasicHeader("Authorization", "ApiKey " + apiKey) + }) + .build(); + +// Create the transport with a Jackson mapper +ElasticsearchTransport transport = new RestClientTransport( + restClient, new JacksonJsonpMapper()); + +// And create the API client +ElasticsearchClient esClient = new ElasticsearchClient(transport); +``` + +To get API keys for the {{es}} endpoint for a project, see [Get started](docs-content://solutions/search/get-started.md). + + +## Using the API [elasticsearch-java-client-getting-started-using-the-api] + +After you initialized the client, you can start ingesting documents. + + +### Creating an index and ingesting documents [elasticsearch-java-client-getting-started-creating-an-index-and-ingesting-documents] + +The following is an example of indexing a document, here a `Product` application object in the `products` index: + +```java +Product product = new Product("bk-1", "City bike", 123.0); + +IndexResponse response = esClient.index(i -> i + .index("products") + .id(product.getSku()) + .document(product) +); + +logger.info("Indexed with version " + response.version()); +``` + + +### Searching [elasticsearch-java-client-getting-started-searching] + +Now that some data is available, you can search your documents using the `search` API: + +```java +String searchText = "bike"; + +SearchResponse response = esClient.search(s -> s + .index("products") + .query(q -> q + .match(t -> t + .field("name") + .query(searchText) + ) + ), + Product.class +); +``` + +A few things to note in the above example: + +* The search query is built using a hierarchy of lambda expressions that closely follows the {{es}} HTTP API. Lambda expressions allows you to be guided by your IDE’s autocompletion, without having to import (or even know!) the actual classes representing a query. +* The last parameter `Product.class` instructs the client to return results as `Product` application objects instead of raw JSON. + + +### Updating [elasticsearch-java-client-getting-started-updating] + +You can update your documents using the `update` API: + +```java +Product product = new Product("bk-1", "City bike", 123.0); + +esClient.update(u -> u + .index("products") + .id("bk-1") + .upsert(product), + Product.class +); +``` + + +### Delete [elasticsearch-java-client-getting-started-delete] + +You can also delete documents: + +```java +esClient.delete(d -> d.index("products").id("bk-1")); +``` + + +### Deleting an index [elasticsearch-java-client-getting-started-deleting-an-index] + +```java +esClient.indices().delete(d -> d.index("products")); +``` + diff --git a/docs/reference/getting-started.md b/docs/reference/getting-started.md new file mode 100644 index 000000000..4559b6f2e --- /dev/null +++ b/docs/reference/getting-started.md @@ -0,0 +1,218 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/getting-started-java.html +--- + +# Getting started [getting-started-java] + +This page guides you through the installation process of the Java client, shows you how to instantiate the client, and how to perform basic Elasticsearch operations with it. + + +### Requirements [_requirements] + +* Java 8 or later. +* A JSON object mapping library to allow seamless integration of your application classes with the Elasticsearch API. The examples below show usage with Jackson. + + +### Installation [_installation] + + +#### Installation in a Gradle project by using Jackson [_installation_in_a_gradle_project_by_using_jackson] + +```groovy +dependencies { + implementation 'co.elastic.clients:elasticsearch-java:9.0.0-beta1' + implementation 'com.fasterxml.jackson.core:jackson-databind:2.17.0' +} +``` + + +#### Installation in a Maven project by using Jackson [_installation_in_a_maven_project_by_using_jackson] + +In the `pom.xml` of your project, add the following repository definition and dependencies: + +```xml + + + + + co.elastic.clients + elasticsearch-java + 9.0.0-beta1 + + + + com.fasterxml.jackson.core + jackson-databind + 2.17.0 + + + + +``` + +Refer to the [Installation](/reference/installation.md) page to learn more. + + +### Connecting [_connecting] + +You can connect to the Elastic Cloud using an API key and the Elasticsearch endpoint. + +```java +// URL and API key +String serverUrl = "https://localhost:9200"; +String apiKey = "VnVhQ2ZHY0JDZGJrU..."; + +// Create the low-level client +RestClient restClient = RestClient + .builder(HttpHost.create(serverUrl)) + .setDefaultHeaders(new Header[]{ + new BasicHeader("Authorization", "ApiKey " + apiKey) + }) + .build(); + +// Create the transport with a Jackson mapper +ElasticsearchTransport transport = new RestClientTransport( + restClient, new JacksonJsonpMapper()); + +// And create the API client +ElasticsearchClient esClient = new ElasticsearchClient(transport); + +// Use the client... + +// Close the client, also closing the underlying transport object and network connections. +esClient.close(); +``` + +Your Elasticsearch endpoint can be found on the **My deployment** page of your deployment: + +:::{image} ../images/es-endpoint.jpg +:alt: Finding Elasticsearch endpoint +::: + +You can generate an API key on the **Management** page under Security. + +:::{image} ../images/create-api-key.png +:alt: Create API key +::: + +For other connection options, refer to the [Connecting](/reference/connecting.md) section. + + +### Operations [_operations] + +Time to use Elasticsearch! This section walks you through the basic, and most important, operations of Elasticsearch. For more operations and more advanced examples, refer to the [*Using the Java API Client*](/reference/using-java-api-client.md) page. + + +#### Creating an index [_creating_an_index] + +This is how you create the `product` index: + +```java +esClient.indices().create(c -> c + .index("products") +); +``` + + +#### Indexing documents [_indexing_documents] + +This is a simple way of indexing a document, here a `Product` application object: + +```java +Product product = new Product("bk-1", "City bike", 123.0); + +IndexResponse response = esClient.index(i -> i + .index("products") + .id(product.getSku()) + .document(product) +); + +logger.info("Indexed with version " + response.version()); +``` + + +#### Getting documents [_getting_documents] + +You can get documents by using the following code: + +```java +GetResponse response = esClient.get(g -> g + .index("products") <1> + .id("bk-1"), + Product.class <2> +); + +if (response.found()) { + Product product = response.source(); + logger.info("Product name " + product.getName()); +} else { + logger.info ("Product not found"); +} +``` + +1. The get request, with the index name and identifier. +2. The target class, here `Product`. + + + +#### Searching documents [_searching_documents] + +This is how you can create a single match query with the Java client: + +```java +String searchText = "bike"; + +SearchResponse response = esClient.search(s -> s + .index("products") + .query(q -> q + .match(t -> t + .field("name") + .query(searchText) + ) + ), + Product.class +); +``` + + +#### Updating documents [_updating_documents] + +This is how you can update a document, for example to add a new field: + +```java +Product product = new Product("bk-1", "City bike", 123.0); + +esClient.update(u -> u + .index("products") + .id("bk-1") + .upsert(product), + Product.class +); +``` + + +#### Deleting documents [_deleting_documents] + +```java +esClient.delete(d -> d.index("products").id("bk-1")); +``` + + +#### Deleting an index [_deleting_an_index] + +```java +esClient.indices().delete(d -> d + .index("products") +); +``` + + +## Examples [_examples] + +The [examples](https://github.com/elastic/elasticsearch-java/tree/main/examples) folder in the Github repository contains full working examples showing how to set up and use the client. + + +## Further reading [_further_reading] + +* Learn more about the [*API conventions*](/reference/api-conventions.md) of the Java client. diff --git a/docs/reference/index.md b/docs/reference/index.md new file mode 100644 index 000000000..8f9fdc876 --- /dev/null +++ b/docs/reference/index.md @@ -0,0 +1,30 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/index.html + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/introduction.html +--- + +# Java [introduction] + +This is the documentation for the official Java API Client for {{es}}. The client provides strongly typed requests and responses for all {{es}} APIs. + + +## Features [_features] + +* Strongly typed requests and responses for all {{es}} APIs. +* Blocking and asynchronous versions of all APIs. +* Use of fluent builders and functional patterns to allow writing concise yet readable code when creating complex nested structures. +* Seamless integration of application classes by using an object mapper such as Jackson or any JSON-B implementation. +* Delegates protocol handling to an http client such as the [Java Low Level REST Client](/reference/java-low-level-rest-client.md) that takes care of all transport-level concerns: HTTP connection pooling, retries, node discovery, and so on. + + +## Elasticsearch server compatibility policy [_elasticsearch_server_compatibility_policy] + +The {{es}} Java client is forward compatible; meaning that the client supports communicating with greater or equal minor versions of {{es}} without breaking. It does not mean that the client automatically supports new features of newer {{es}} versions; it is only possible after a release of a new client version. For example, a 8.12 client version won’t automatically support the new features of the 8.13 version of {{es}}, the 8.13 client version is required for that. {{es}} language clients are only backwards compatible with default distributions and without guarantees made. + +| Elasticsearch Version | Elasticsearch-Java Branch | Supported | +| --- | --- | --- | +| main | main | | +| 8.x | 8.x | 8.x | +| 7.x | 7.x | 7.17 | + diff --git a/docs/reference/indexing-bulk.md b/docs/reference/indexing-bulk.md new file mode 100644 index 000000000..0b3eb882d --- /dev/null +++ b/docs/reference/indexing-bulk.md @@ -0,0 +1,199 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/indexing-bulk.html +--- + +# Bulk: indexing multiple documents [indexing-bulk] + +Bulk requests allow sending multiple document-related operations to {{es}} in one request. When you have multiple documents to ingest, this is more efficient than sending each document with a separate request. + +A bulk request can contain several kinds of operations: + +* create a document, indexing it after ensuring it doesn’t already exist, +* index a document, creating it if needed and replacing it if it exists, +* update a document that already exists in place, either with a script or a partial document, +* delete a document. + +::::{note} +See the [{{es}} API documentation](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) for a full explanation of bulk requests. +:::: + + + +## Indexing application objects [_indexing_application_objects] + +A `BulkRequest` contains a collection of operations, each operation being a [type with several variants](/reference/variant-types.md). To create this request, it is convenient to use a builder object for the main request, and the fluent DSL for each operation. + +The example below shows how to index a list or application objects. + +```java +List products = fetchProducts(); + +BulkRequest.Builder br = new BulkRequest.Builder(); + +for (Product product : products) { + br.operations(op -> op <1> + .index(idx -> idx <2> + .index("products") <3> + .id(product.getSku()) + .document(product) + ) + ); +} + +BulkResponse result = esClient.bulk(br.build()); + +// Log errors, if any +if (result.errors()) { + logger.error("Bulk had errors"); + for (BulkResponseItem item: result.items()) { + if (item.error() != null) { + logger.error(item.error().reason()); + } + } +} +``` + +1. Adds an operation (remember that [list properties are additive](/reference/lists-maps.md)). `op` is is a builder for `BulkOperation` which is a [variant type](/reference/variant-types.md). This type has `index`, `create`, `update` and `delete` variants. +2. Selects the `index` operation variant, `idx` is a builder for `IndexOperation`. +3. Sets the properties for the index operation, similar to [single document indexing](/reference/indexing.md): index name, identifier and document. + + + +## Indexing raw JSON data [indexing-raw-json-data] + +The `document` property of a bulk index request can be any object that can be serialized to JSON using your Elasticsearch client’s JSON mapper. However, data that is ingested in bulk is often available as JSON text (e.g. files on disk), and parsing this JSON just to re-serialize it to send the bulk request would be a waste of resources. So documents in bulk operations can also be of type `BinaryData` that are sent verbatim (without parsing) to the {{es}} server. + +In the example below we will use the Java API Client’s `BinaryData` to read json files from a log directory and send them in a bulk request. + +```java +// List json log files in the log directory +File[] logFiles = logDir.listFiles( + file -> file.getName().matches("log-.*\\.json") +); + +BulkRequest.Builder br = new BulkRequest.Builder(); + +for (File file: logFiles) { + FileInputStream input = new FileInputStream(file); + BinaryData data = BinaryData.of(IOUtils.toByteArray(input), ContentType.APPLICATION_JSON); + + br.operations(op -> op + .index(idx -> idx + .index("logs") + .document(data) + ) + ); +} +``` + + +## Streaming ingestion with the Bulk Ingester [_streaming_ingestion_with_the_bulk_ingester] + +The `BulkIngester` simplifies the usage of the Bulk API by providing a utility class that allows index/update/delete operations to be transparently grouped in bulk requests. You only have to `add()` bulk operations to the ingester and it will take care of grouping and sending them in bulk according to its configuration. + +The ingester will send a bulk request when one of the following criteria is met: + +* the number of operations exceeds a maximum (defaults to 1000) +* the bulk request size in bytes exceeds a maximum (defaults to 5 MiB) +* a delay since the last request has expired (periodic flush, no default) + +Additionally, you can define a maximum number of concurrent request waiting to be executed by {{es}} (defaults to 1). When that maximum is reached and the maximum number of operations have been collected, adding a new operation to the indexer will block. This is avoids overloading the {{es}} server by putting backpressure on the client application. + +```java +BulkIngester ingester = BulkIngester.of(b -> b + .client(esClient) <1> + .maxOperations(100) <2> + .flushInterval(1, TimeUnit.SECONDS) <3> +); + +for (File file: logFiles) { + FileInputStream input = new FileInputStream(file); + BinaryData data = BinaryData.of(IOUtils.toByteArray(input), ContentType.APPLICATION_JSON); + + ingester.add(op -> op <4> + .index(idx -> idx + .index("logs") + .document(data) + ) + ); +} + +ingester.close(); <5> +``` + +1. Sets the {{es}} client used to send bulk requests. +2. Sets the maximum number of operations to collect before sending a bulk request. +3. Sets the flush interval. +4. Adds a bulk operation to the ingester. +5. Closes the ingester to flush the pending operations and release resources. + + +Additionally, the bulk ingester accepts a listener so that your application can be notified of bulk requests that are sent and their result. To allow correlating bulk operations to application context, the `add()` method optionally accepts a `context` parameter. The type of this context parameter is used as the generic parameter of the `BulkIngester` object. You may have noticed the `Void` type in `BulkIngester` above: this is because we did not register a listener, and therefore did not care about context values. + +The following example shows how you can use context values to implement a bulk ingestion listener: as previously it sends JSON log files in bulk, but tracks bulk request errors and failed operations. When an operation fails, depending on the error type you may want to re-add it to the ingester. + +```java +BulkListener listener = new BulkListener() { <1> + @Override + public void beforeBulk(long executionId, BulkRequest request, List contexts) { + } + + @Override + public void afterBulk(long executionId, BulkRequest request, List contexts, BulkResponse response) { + // The request was accepted, but may contain failed items. + // The "context" list gives the file name for each bulk item. + logger.debug("Bulk request " + executionId + " completed"); + for (int i = 0; i < contexts.size(); i++) { + BulkResponseItem item = response.items().get(i); + if (item.error() != null) { + // Inspect the failure cause + logger.error("Failed to index file " + contexts.get(i) + " - " + item.error().reason()); + } + } + } + + @Override + public void afterBulk(long executionId, BulkRequest request, List contexts, Throwable failure) { + // The request could not be sent + logger.debug("Bulk request " + executionId + " failed", failure); + } +}; + +BulkIngester ingester = BulkIngester.of(b -> b + .client(esClient) + .maxOperations(100) + .flushInterval(1, TimeUnit.SECONDS) + .listener(listener) <2> +); + +for (File file: logFiles) { + FileInputStream input = new FileInputStream(file); + BinaryData data = BinaryData.of(IOUtils.toByteArray(input), ContentType.APPLICATION_JSON); + + ingester.add(op -> op + .index(idx -> idx + .index("logs") + .document(data) + ), + file.getName() <3> + ); +} + +ingester.close(); +``` + +1. Creates a listener where context values are strings for the ingested file name. +2. Registers the listener on the bulk ingester. +3. Sets the file name as the context value for a bulk operation. + + +The bulk ingest also exposes statistic information that allows monitoring the ingestion process and tune its configuration: + +* number of operations added, +* number of calls to `add()` that were blocked because the maximum number of concurrent requests was reached (contention), +* number of bulk requests sent, +* number of bulk requests that were blocked because the maximum number of concurrent requests was reached. + +The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation). + diff --git a/docs/reference/indexing.md b/docs/reference/indexing.md new file mode 100644 index 000000000..f95d91fac --- /dev/null +++ b/docs/reference/indexing.md @@ -0,0 +1,114 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/indexing.html +--- + +# Indexing single documents [indexing] + +The Java API Client offers several ways to index data: you can provide application objects that will be automatically mapped to JSON, or you can provide raw JSON data. Using application objects is more suited to applications with a well-defined domain model, whereas raw JSON is more suited to logging use cases with semi-structured data. + +In the examples below we use a `Product` domain object that has `sku`, `name` and `price` properties. + +::::{note} +See the [{{es}} API documentation](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-create) for a full explanation of index requests. +:::: + + + +## Using the fluent DSL [_using_the_fluent_dsl] + +The most direct way to build requests is using the fluent DSL. In the example below we index a product description in the `products` index, using the product’s SKU as the document identifier in the index. The `product` object will be mapped to JSON using the object mapper configured on the {{es}} client. + +```java +Product product = new Product("bk-1", "City bike", 123.0); + +IndexResponse response = esClient.index(i -> i + .index("products") + .id(product.getSku()) + .document(product) +); + +logger.info("Indexed with version " + response.version()); +``` + +You can also assign objects created with the DSL to variables. Java API Client classes have a static `of()` method for this, that creates an object with the DSL syntax. + +```java +Product product = new Product("bk-1", "City bike", 123.0); + +IndexRequest request = IndexRequest.of(i -> i + .index("products") + .id(product.getSku()) + .document(product) +); + +IndexResponse response = esClient.index(request); + +logger.info("Indexed with version " + response.version()); +``` + + +## Using classic builders [_using_classic_builders] + +If you’re more used to the classic builder pattern, it is also available. Builder objects are used under the hood by the fluent DSL syntax. + +```java +Product product = new Product("bk-1", "City bike", 123.0); + +IndexRequest.Builder indexReqBuilder = new IndexRequest.Builder<>(); +indexReqBuilder.index("product"); +indexReqBuilder.id(product.getSku()); +indexReqBuilder.document(product); + +IndexResponse response = esClient.index(indexReqBuilder.build()); + +logger.info("Indexed with version " + response.version()); +``` + + +## Using the asynchronous client [_using_the_asynchronous_client] + +The examples above used the synchronous {{es}} client. All {{es}} APIs are also available in the asynchronous client, using the same request and response types. See also [Blocking and asynchronous clients](/reference/blocking-async.md) for additional details. + +```java +ElasticsearchAsyncClient esAsyncClient = new ElasticsearchAsyncClient(transport); + +Product product = new Product("bk-1", "City bike", 123.0); + +esAsyncClient.index(i -> i + .index("products") + .id(product.getSku()) + .document(product) +).whenComplete((response, exception) -> { + if (exception != null) { + logger.error("Failed to index", exception); + } else { + logger.info("Indexed with version " + response.version()); + } +}); +``` + + +## Using raw JSON data [_using_raw_json_data] + +When the data you want to index comes from external sources, having to create domain objects may be cumbersome or outright impossible with semi-structured data. + +You can index data from an arbitrary source using `withJson()`. Using this method will read the source and use it for the index request’s `document` property. See [Creating API objects from JSON data](/reference/loading-json.md) for additional details. + +```java +Reader input = new StringReader( + "{'@timestamp': '2022-04-08T13:55:32Z', 'level': 'warn', 'message': 'Some log message'}" + .replace('\'', '"')); + +IndexRequest request = IndexRequest.of(i -> i + .index("logs") + .withJson(input) +); + +IndexResponse response = esClient.index(request); + +logger.info("Indexed with version " + response.version()); +``` + +The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation). + diff --git a/docs/reference/installation.md b/docs/reference/installation.md new file mode 100644 index 000000000..b174e9395 --- /dev/null +++ b/docs/reference/installation.md @@ -0,0 +1,88 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/installation.html +--- + +# Installation [installation] + +Requirements: + +* Java 8 or later. +* A JSON object mapping library to allow seamless integration of your application classes with the Elasticsearch API. The Java client has support for [Jackson](https://github.com/FasterXML/jackson) or a [JSON-B](https://github.com/eclipse-ee4j/jsonb-api) library like [Eclipse Yasson](https://github.com/eclipse-ee4j/yasson). + +Releases are hosted on [Maven Central](https://search.maven.org/search?q=g:co.elastic.clients). If you are looking for a SNAPSHOT version, the Elastic Maven Snapshot repository is available at [https://snapshots.elastic.co/maven/](https://snapshots.elastic.co/maven/). + + +## Installation in a Gradle project by using Jackson [gradle] + +```groovy +dependencies { + implementation 'co.elastic.clients:elasticsearch-java:9.0.0-beta1' + implementation 'com.fasterxml.jackson.core:jackson-databind:2.17.0' +} +``` + + +## Installation in a Maven project by using Jackson [maven] + +In the `pom.xml` of your project, add the following repository definition and dependencies: + +```xml + + + + + co.elastic.clients + elasticsearch-java + 9.0.0-beta1 + + + + com.fasterxml.jackson.core + jackson-databind + 2.17.0 + + + + +``` + + +## If you get ClassNotFoundException: jakarta.json.spi.JsonProvider [class-not-found-jsonprovider] + +It may happen that after setting up the dependencies, your application fails with `ClassNotFoundException: jakarta.json.spi.JsonProvider`. + +If this happens, you have to explicitly add the `jakarta.json:jakarta.json-api:2.0.1` dependency. + +```groovy +dependencies { + ... + implementation 'jakarta.json:jakarta.json-api:2.0.1' +} +``` + +```xml + + + ... + + jakarta.json + jakarta.json-api + 2.0.1 + + + + +``` + +**Why is this needed?** + +Some frameworks like Spring Boot or Helidon come with their Gradle and Maven plugins or their Maven BOM files to ease development and dependency management. These plugins and BOM define the versions to use for a number of well-known libraries. + +One of these libraries can be `jakarta.json:json-api` that defines the standard Java JSON API. In version `1.x` this library used the `javax.json` package, while in version `2.x` it uses the `jakarta.json` package after [the transition from JavaEE to JakartaEE](https://blogs.oracle.com/javamagazine/post/transition-from-java-ee-to-jakarta-ee). + +The Java API Client depends on version `2.0.1` of this library, in order to use the newer and future-proof `jakarta.json` package. But some build plugins and BOMs override the Java API Client’s dependency to use version `1.x` in the older `javax.json` namespace, resulting in `ClassNotFoundException: jakarta.json.spi.JsonProvider`. + +Adding the correct version as top-level project dependency solves the problem. + +If your application also requires `javax.json` you can add the `javax.json:javax.json-api:1.1.4` dependency, which is equivalent to `jakarta.json:jakarta.json-api:1.1.6`. diff --git a/docs/reference/java-low-level-rest-client.md b/docs/reference/java-low-level-rest-client.md new file mode 100644 index 000000000..05e7eeea2 --- /dev/null +++ b/docs/reference/java-low-level-rest-client.md @@ -0,0 +1,17 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-rest-low.html +--- + +# Java Low Level REST Client [java-rest-low] + +The low-level client’s features include: + +* minimal dependencies +* load balancing across all available nodes +* failover in case of node failures and upon specific response codes +* failed connection penalization (whether a failed node is retried depends on how many consecutive times it failed; the more failed attempts the longer the client will wait before trying that same node again) +* persistent connections +* trace logging of requests and responses +* optional automatic [discovery of cluster nodes](/reference/sniffer.md) + diff --git a/docs/reference/java-rest-low-config.md b/docs/reference/java-rest-low-config.md new file mode 100644 index 000000000..52574079d --- /dev/null +++ b/docs/reference/java-rest-low-config.md @@ -0,0 +1,16 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-rest-low-config.html +--- + +# Common configuration [java-rest-low-config] + +As explained in [Initialization](/reference/java-rest-low-usage-initialization.md), the `RestClientBuilder` supports providing both a `RequestConfigCallback` and an `HttpClientConfigCallback` which allow for any customization that the Apache Async Http Client exposes. Those callbacks make it possible to modify some specific behaviour of the client without overriding every other default configuration that the `RestClient` is initialized with. This section describes some common scenarios that require additional configuration for the low-level Java REST Client. + + + + + + + + diff --git a/docs/reference/java-rest-low-javadoc.md b/docs/reference/java-rest-low-javadoc.md new file mode 100644 index 000000000..e80307d97 --- /dev/null +++ b/docs/reference/java-rest-low-javadoc.md @@ -0,0 +1,9 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-rest-low-javadoc.html +--- + +# Javadoc [java-rest-low-javadoc] + +The javadoc for the low level REST client can be found at [https://snapshots.elastic.co/javadoc/org/elasticsearch/client/elasticsearch-rest-client/9.0.0-beta1-SNAPSHOT/index.html](https://snapshots.elastic.co/javadoc/org/elasticsearch/client/elasticsearch-rest-client/9.0.0-beta1-SNAPSHOT/index.md). + diff --git a/docs/reference/java-rest-low-usage-dependencies.md b/docs/reference/java-rest-low-usage-dependencies.md new file mode 100644 index 000000000..3bc944a11 --- /dev/null +++ b/docs/reference/java-rest-low-usage-dependencies.md @@ -0,0 +1,16 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-rest-low-usage-dependencies.html +--- + +# Dependencies [java-rest-low-usage-dependencies] + +The low-level Java REST client internally uses the [Apache Http Async Client](https://hc.apache.org/httpcomponents-asyncclient-4.1.x/) to send http requests. It depends on the following artifacts, namely the async http client and its own transitive dependencies: + +* org.apache.httpcomponents:httpasyncclient +* org.apache.httpcomponents:httpcore-nio +* org.apache.httpcomponents:httpclient +* org.apache.httpcomponents:httpcore +* commons-codec:commons-codec +* commons-logging:commons-logging + diff --git a/docs/reference/java-rest-low-usage-initialization.md b/docs/reference/java-rest-low-usage-initialization.md new file mode 100644 index 000000000..30e3bca66 --- /dev/null +++ b/docs/reference/java-rest-low-usage-initialization.md @@ -0,0 +1,88 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-rest-low-usage-initialization.html +--- + +# Initialization [java-rest-low-usage-initialization] + +A `RestClient` instance can be built through the corresponding `RestClientBuilder` class, created via `RestClient#builder(HttpHost...)` static method. The only required argument is one or more hosts that the client will communicate with, provided as instances of [HttpHost](https://hc.apache.org/httpcomponents-core-4.4.x/current/httpcore/apidocs/org/apache/http/HttpHost.md) as follows: + +```java +RestClient restClient = RestClient.builder( + new HttpHost("localhost", 9200, "http"), + new HttpHost("localhost", 9201, "http")).build(); +``` + +The `RestClient` class is thread-safe and ideally has the same lifecycle as the application that uses it. It is important that it gets closed when no longer needed so that all the resources used by it get properly released, as well as the underlying http client instance and its threads: + +```java +restClient.close(); +``` + +`RestClientBuilder` also allows to optionally set the following configuration parameters while building the `RestClient` instance: + +```java +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200, "http")); +Header[] defaultHeaders = new Header[]{new BasicHeader("header", "value")}; +builder.setDefaultHeaders(defaultHeaders); <1> +``` + +1. Set the default headers that need to be sent with each request, to prevent having to specify them with each single request + + +```java +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200, "http")); +builder.setFailureListener(new RestClient.FailureListener() { + @Override + public void onFailure(Node node) { + <1> + } +}); +``` + +1. Set a listener that gets notified every time a node fails, in case actions need to be taken. Used internally when sniffing on failure is enabled. + + +```java +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200, "http")); +builder.setNodeSelector(NodeSelector.SKIP_DEDICATED_MASTERS); <1> +``` + +1. Set the node selector to be used to filter the nodes the client will send requests to among the ones that are set to the client itself. This is useful for instance to prevent sending requests to dedicated master nodes when sniffing is enabled. By default the client sends requests to every configured node. + + +```java +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200, "http")); +builder.setRequestConfigCallback( + new RestClientBuilder.RequestConfigCallback() { + @Override + public RequestConfig.Builder customizeRequestConfig( + RequestConfig.Builder requestConfigBuilder) { + return requestConfigBuilder.setSocketTimeout(10000); <1> + } + }); +``` + +1. Set a callback that allows to modify the default request configuration (e.g. request timeouts, authentication, or anything that the [`org.apache.http.client.config.RequestConfig.Builder`](https://hc.apache.org/httpcomponents-client-4.5.x/current/httpclient/apidocs/org/apache/http/client/config/RequestConfig.Builder.md) allows to set) + + +```java +RestClientBuilder builder = RestClient.builder( + new HttpHost("localhost", 9200, "http")); +builder.setHttpClientConfigCallback(new HttpClientConfigCallback() { + @Override + public HttpAsyncClientBuilder customizeHttpClient( + HttpAsyncClientBuilder httpClientBuilder) { + return httpClientBuilder.setProxy( + new HttpHost("proxy", 9000, "http")); <1> + } + }); +``` + +1. Set a callback that allows to modify the http client configuration (e.g. encrypted communication over ssl, or anything that the [`org.apache.http.impl.nio.client.HttpAsyncClientBuilder`](https://hc.apache.org/httpcomponents-asyncclient-4.1.x/current/httpasyncclient/apidocs/org/apache/http/impl/nio/client/HttpAsyncClientBuilder.md) allows to set) + + diff --git a/docs/reference/java-rest-low-usage-logging.md b/docs/reference/java-rest-low-usage-logging.md new file mode 100644 index 000000000..9d2e44739 --- /dev/null +++ b/docs/reference/java-rest-low-usage-logging.md @@ -0,0 +1,59 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-rest-low-usage-logging.html +--- + +# Logging [java-rest-low-usage-logging] + +The Java REST client uses the same logging library that the Apache Async Http Client uses: [Apache Commons Logging](https://commons.apache.org/proper/commons-logging/), which comes with support for a number of popular logging implementations. The java packages to enable logging for are `org.elasticsearch.client` for the client itself and `org.elasticsearch.client.sniffer` for the sniffer. + +The request tracer logging can also be enabled to log every request and corresponding response in curl format. That comes handy when debugging, for instance in case a request needs to be manually executed to check whether it still yields the same response as it did. Enable trace logging for the `tracer` package to have such log lines printed out. Do note that this type of logging is expensive and should not be enabled at all times in production environments, but rather temporarily used only when needed. + +## Logback [_logback] + +### Trace Logs [_trace_logs] + +In order to enable trace logs for logback, we have to use [jcl-over-slf4j bridging module](https://www.slf4j.org/legacy.md#jclOverSLF4J). + +1. Add the following to your Gradle setting: + + ```groovy + dependencies { + implementation('org.slf4j:slf4j-api:1.8.0-beta2') + implementation('ch.qos.logback:logback-classic:1.3.0-alpha4') + implementation('org.slf4j:jcl-over-slf4j:1.8.0-beta2') + } + ``` + +2. Exclude `commons-logging.jar`: + + ```groovy + dependencies { + configurations.all { + exclude group: "commons-logging", module: "commons-logging" + } + } + ``` + +3. Add a tracer logger in Logback configuration: + + ```xml + + + + ``` + + + +### RestClient Debug Logs [_restclient_debug_logs] + +To enable debug logs for `RestClient` class, add the following to your Logback configuration: + +```xml + + + +``` + + + diff --git a/docs/reference/java-rest-low-usage-maven.md b/docs/reference/java-rest-low-usage-maven.md new file mode 100644 index 000000000..e6c97b5df --- /dev/null +++ b/docs/reference/java-rest-low-usage-maven.md @@ -0,0 +1,37 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-rest-low-usage-maven.html +--- + +# Maven Repository [java-rest-low-usage-maven] + +The low-level Java REST client is hosted on [Maven Central](https://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.elasticsearch.client%22). The minimum Java version required is `1.8`. + +The low-level REST client is subject to the same release cycle as Elasticsearch. Replace the version with the desired client version, first released with `5.0.0-alpha4`. There is no relation between the client version and the Elasticsearch version that the client can communicate with. The low-level REST client is compatible with all Elasticsearch versions. + +If you are looking for a SNAPSHOT version, the Elastic Maven Snapshot repository is available at [https://snapshots.elastic.co/maven/](https://snapshots.elastic.co/maven/). + +## Maven configuration [java-rest-low-usage-maven-maven] + +Here is how you can configure the dependency using maven as a dependency manager. Add the following to your `pom.xml` file: + +```xml + + org.elasticsearch.client + elasticsearch-rest-client + 9.0.0-beta1 + +``` + + +## Gradle configuration [java-rest-low-usage-maven-gradle] + +Here is how you can configure the dependency using gradle as a dependency manager. Add the following to your `build.gradle` file: + +```groovy +dependencies { + compile 'org.elasticsearch.client:elasticsearch-rest-client:9.0.0-beta1' +} +``` + + diff --git a/docs/reference/java-rest-low-usage-requests.md b/docs/reference/java-rest-low-usage-requests.md new file mode 100644 index 000000000..0bf474b6f --- /dev/null +++ b/docs/reference/java-rest-low-usage-requests.md @@ -0,0 +1,179 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-rest-low-usage-requests.html +--- + +# Performing requests [java-rest-low-usage-requests] + +Once the `RestClient` has been created, requests can be sent by calling either `performRequest` or `performRequestAsync`. `performRequest` is synchronous and will block the calling thread and return the `Response` when the request is successful or throw an exception if it fails. `performRequestAsync` is asynchronous and accepts a `ResponseListener` argument that it calls with a `Response` when the request is successful or with an `Exception` if it fails. + +This is synchronous: + +```java +Request request = new Request( + "GET", <1> + "/"); <2> +Response response = restClient.performRequest(request); +``` + +1. The HTTP method (`GET`, `POST`, `HEAD`, etc) +2. The endpoint on the server + + +And this is asynchronous: + +```java +Request request = new Request( + "GET", <1> + "/"); <2> +Cancellable cancellable = restClient.performRequestAsync(request, + new ResponseListener() { + @Override + public void onSuccess(Response response) { + <3> + } + + @Override + public void onFailure(Exception exception) { + <4> + } +}); +``` + +1. The HTTP method (`GET`, `POST`, `HEAD`, etc) +2. The endpoint on the server +3. Handle the response +4. Handle the failure + + +You can add request parameters to the request object: + +```java +request.addParameter("pretty", "true"); +``` + +You can set the body of the request to any `HttpEntity`: + +```java +request.setEntity(new NStringEntity( + "{\"json\":\"text\"}", + ContentType.APPLICATION_JSON)); +``` + +::::{important} +The `ContentType` specified for the `HttpEntity` is important because it will be used to set the `Content-Type` header so that Elasticsearch can properly parse the content. +:::: + + +You can also set it to a `String` which will default to a `ContentType` of `application/json`. + +```java +request.setJsonEntity("{\"json\":\"text\"}"); +``` + +## RequestOptions [java-rest-low-usage-request-options] + +The `RequestOptions` class holds parts of the request that should be shared between many requests in the same application. You can make a singleton instance and share it between all requests: + +```java +private static final RequestOptions COMMON_OPTIONS; +static { + RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder(); + builder.addHeader("Authorization", "Bearer " + TOKEN); <1> + builder.setHttpAsyncResponseConsumerFactory( <2> + new HttpAsyncResponseConsumerFactory + .HeapBufferedResponseConsumerFactory(30 * 1024 * 1024 * 1024)); + COMMON_OPTIONS = builder.build(); +} +``` + +1. Add any headers needed by all requests. +2. Customize the response consumer. + + +`addHeader` is for headers that are required for authorization or to work with a proxy in front of Elasticsearch. There is no need to set the `Content-Type` header because the client will automatically set that from the `HttpEntity` attached to the request. + +You can set the `NodeSelector` which controls which nodes will receive requests. `NodeSelector.SKIP_DEDICATED_MASTERS` is a good choice. + +You can also customize the response consumer used to buffer the asynchronous responses. The default consumer will buffer up to 100MB of response on the JVM heap. If the response is larger then the request will fail. You could, for example, lower the maximum size which might be useful if you are running in a heap constrained environment like the example above. + +Once you’ve created the singleton you can use it when making requests: + +```java +request.setOptions(COMMON_OPTIONS); +``` + +You can also customize these options on a per request basis. For example, this adds an extra header: + +```java +RequestOptions.Builder options = COMMON_OPTIONS.toBuilder(); +options.addHeader("cats", "knock things off of other things"); +request.setOptions(options); +``` + + +## Multiple parallel asynchronous actions [_multiple_parallel_asynchronous_actions] + +The client is quite happy to execute many actions in parallel. The following example indexes many documents in parallel. In a real world scenario you’d probably want to use the `_bulk` API instead, but the example is illustrative. + +```java +final CountDownLatch latch = new CountDownLatch(documents.length); +for (int i = 0; i < documents.length; i++) { + Request request = new Request("PUT", "/posts/doc/" + i); + //let's assume that the documents are stored in an HttpEntity array + request.setEntity(documents[i]); + restClient.performRequestAsync( + request, + new ResponseListener() { + @Override + public void onSuccess(Response response) { + <1> + latch.countDown(); + } + + @Override + public void onFailure(Exception exception) { + <2> + latch.countDown(); + } + } + ); +} +latch.await(); +``` + +1. Process the returned response +2. Handle the returned exception, due to communication error or a response with status code that indicates an error + + + +## Cancelling asynchronous requests [_cancelling_asynchronous_requests] + +The `performRequestAsync` method returns a `Cancellable` that exposes a single public method called `cancel`. Such method can be called to cancel the on-going request. Cancelling a request will result in aborting the http request through the underlying http client. On the server side, this does not automatically translate to the execution of that request being cancelled, which needs to be specifically implemented in the API itself. + +The use of the `Cancellable` instance is optional and you can safely ignore this if you don’t need it. A typical usecase for this would be using this together with frameworks like Rx Java or the Kotlin’s `suspendCancellableCoRoutine`. Cancelling no longer needed requests is a good way to avoid putting unnecessary load on Elasticsearch. + +```java +Request request = new Request("GET", "/posts/_search"); +Cancellable cancellable = restClient.performRequestAsync( + request, + new ResponseListener() { + @Override + public void onSuccess(Response response) { + <1> + } + + @Override + public void onFailure(Exception exception) { + <2> + } + } +); +cancellable.cancel(); +``` + +1. Process the returned response, in case it was ready before the request got cancelled +2. Handle the returned exception, which will most likely be a `CancellationException` as the request got cancelled + + + diff --git a/docs/reference/java-rest-low-usage-responses.md b/docs/reference/java-rest-low-usage-responses.md new file mode 100644 index 000000000..eb929cfd3 --- /dev/null +++ b/docs/reference/java-rest-low-usage-responses.md @@ -0,0 +1,41 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-rest-low-usage-responses.html +--- + +# Reading responses [java-rest-low-usage-responses] + +The `Response` object, either returned by the synchronous `performRequest` methods or received as an argument in `ResponseListener#onSuccess(Response)`, wraps the response object returned by the http client and exposes some additional information. + +```java +Response response = restClient.performRequest(new Request("GET", "/")); +RequestLine requestLine = response.getRequestLine(); <1> +HttpHost host = response.getHost(); <2> +int statusCode = response.getStatusLine().getStatusCode(); <3> +Header[] headers = response.getHeaders(); <4> +String responseBody = EntityUtils.toString(response.getEntity()); <5> +``` + +1. Information about the performed request +2. The host that returned the response +3. The response status line, from which you can for instance retrieve the status code +4. The response headers, which can also be retrieved by name though `getHeader(String)` +5. The response body enclosed in an [`org.apache.http.HttpEntity`](https://hc.apache.org/httpcomponents-core-4.4.x/current/httpcore/apidocs/org/apache/http/HttpEntity.md) object + + +When performing a request, an exception is thrown (or received as an argument in `ResponseListener#onFailure(Exception)` in the following scenarios: + +`IOException` +: communication problem (e.g. SocketTimeoutException) + +`ResponseException` +: a response was returned, but its status code indicated an error (not `2xx`). A `ResponseException` originates from a valid http response, hence it exposes its corresponding `Response` object which gives access to the returned response. + +::::{note} +A `ResponseException` is **not** thrown for `HEAD` requests that return a `404` status code because it is an expected `HEAD` response that simply denotes that the resource is not found. All other HTTP methods (e.g., `GET`) throw a `ResponseException` for `404` responses unless the `ignore` parameter contains `404`. `ignore` is a special client parameter that doesn’t get sent to Elasticsearch and contains a comma separated list of error status codes. It allows to control whether some error status code should be treated as an expected response rather than as an exception. This is useful for instance with the get api as it can return `404` when the document is missing, in which case the response body will not contain an error but rather the usual get api response, just without the document as it was not found. +:::: + + +Note that the low-level client doesn’t expose any helper for json marshalling and un-marshalling. Users are free to use the library that they prefer for that purpose. + +The underlying Apache Async Http Client ships with different [`org.apache.http.HttpEntity`](https://hc.apache.org/httpcomponents-core-4.4.x/current/httpcore/apidocs/org/apache/http/HttpEntity.md) implementations that allow to provide the request body in different formats (stream, byte array, string etc.). As for reading the response body, the `HttpEntity#getContent` method comes handy which returns an `InputStream` reading from the previously buffered response body. As an alternative, it is possible to provide a custom [`org.apache.http.nio.protocol.HttpAsyncResponseConsumer`](https://hc.apache.org/httpcomponents-core-4.4.x/current/httpcore-nio/apidocs/org/apache/http/nio/protocol/HttpAsyncResponseConsumer.md) that controls how bytes are read and buffered. diff --git a/docs/reference/java-rest-low-usage-shading.md b/docs/reference/java-rest-low-usage-shading.md new file mode 100644 index 000000000..766cb7a65 --- /dev/null +++ b/docs/reference/java-rest-low-usage-shading.md @@ -0,0 +1,68 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-rest-low-usage-shading.html +--- + +# Shading [java-rest-low-usage-shading] + +In order to avoid version conflicts, the dependencies can be shaded and packaged within the client in a single JAR file (sometimes called an "uber JAR" or "fat JAR"). Shading a dependency consists of taking its content (resources files and Java class files) and renaming some of its packages before putting them in the same JAR file as the low-level Java REST client. Shading a JAR can be accomplished by 3rd-party plugins for Gradle and Maven. + +Be advised that shading a JAR also has implications. Shading the Commons Logging layer, for instance, means that 3rd-party logging backends need to be shaded as well. + +## Maven configuration [java-rest-low-usage-shading-maven] + +Here is a configuration using the Maven [Shade](https://maven.apache.org/plugins/maven-shade-plugin/index.md) plugin. Add the following to your `pom.xml` file: + +```xml + + + + org.apache.maven.plugins + maven-shade-plugin + 3.1.0 + + + package + shade + + + + org.apache.http + hidden.org.apache.http + + + org.apache.logging + hidden.org.apache.logging + + + org.apache.commons.codec + hidden.org.apache.commons.codec + + + org.apache.commons.logging + hidden.org.apache.commons.logging + + + + + + + + +``` + + +## Gradle configuration [java-rest-low-usage-shading-gradle] + +Here is a configuration using the Gradle [ShadowJar](https://github.com/johnrengelman/shadow) plugin. Add the following to your `build.gradle` file: + +```groovy +shadowJar { + relocate 'org.apache.http', 'hidden.org.apache.http' + relocate 'org.apache.logging', 'hidden.org.apache.logging' + relocate 'org.apache.commons.codec', 'hidden.org.apache.commons.codec' + relocate 'org.apache.commons.logging', 'hidden.org.apache.commons.logging' +} +``` + + diff --git a/docs/reference/java-rest-low-usage.md b/docs/reference/java-rest-low-usage.md new file mode 100644 index 000000000..8c87b0d4e --- /dev/null +++ b/docs/reference/java-rest-low-usage.md @@ -0,0 +1,17 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-rest-low-usage.html +--- + +# Getting started [java-rest-low-usage] + +This section describes how to get started with the low-level REST client from getting the artifact to using it in an application. + + + + + + + + + diff --git a/docs/reference/java-rest-sniffer-javadoc.md b/docs/reference/java-rest-sniffer-javadoc.md new file mode 100644 index 000000000..a24c16f1c --- /dev/null +++ b/docs/reference/java-rest-sniffer-javadoc.md @@ -0,0 +1,9 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-rest-sniffer-javadoc.html +--- + +# Javadoc [java-rest-sniffer-javadoc] + +The javadoc for the REST client sniffer can be found at [https://snapshots.elastic.co/javadoc/org/elasticsearch/client/elasticsearch-rest-client-sniffer/9.0.0-beta1-SNAPSHOT/index.html](https://snapshots.elastic.co/javadoc/org/elasticsearch/client/elasticsearch-rest-client-sniffer/9.0.0-beta1-SNAPSHOT/index.md). + diff --git a/docs/reference/javadoc-source-code.md b/docs/reference/javadoc-source-code.md new file mode 100644 index 000000000..abcdd4183 --- /dev/null +++ b/docs/reference/javadoc-source-code.md @@ -0,0 +1,11 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-client-javadoc.html +--- + +# Javadoc and source code [java-client-javadoc] + +The javadoc for the Java API Client can be found at [https://snapshots.elastic.co/javadoc/co/elastic/clients/elasticsearch-java/9.0.0-beta1-SNAPSHOT/index.html](https://snapshots.elastic.co/javadoc/co/elastic/clients/elasticsearch-java/9.0.0-beta1-SNAPSHOT/index.md). + +The source code is at [https://github.com/elastic/elasticsearch-java/](https://github.com/elastic/elasticsearch-java/) and is licensed under the Apache 2.0 License. + diff --git a/docs/reference/lists-maps.md b/docs/reference/lists-maps.md new file mode 100644 index 000000000..b31308fd3 --- /dev/null +++ b/docs/reference/lists-maps.md @@ -0,0 +1,77 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/lists-and-maps.html +--- + +# Lists and maps [lists-and-maps] + + +## Additive builder setters [_additive_builder_setters] + +Properties of type `List` and `Map` are exposed by object builders as a set of overloaded additive-only methods that *update* the property value, by appending to lists and adding new entries to maps (or replacing existing ones). + +Object builders create immutable objects, and this also applies to list and map properties that are made immutable at object construction time. + +```java +// Prepare a list of index names +List names = Arrays.asList("idx-a", "idx-b", "idx-c"); + +// Prepare cardinality aggregations for fields "foo" and "bar" +Map cardinalities = new HashMap<>(); +cardinalities.put("foo-count", Aggregation.of(a -> a.cardinality(c -> c.field("foo")))); +cardinalities.put("bar-count", Aggregation.of(a -> a.cardinality(c -> c.field("bar")))); + +// Prepare an aggregation that computes the average of the "size" field +final Aggregation avgSize = Aggregation.of(a -> a.avg(v -> v.field("size"))); + +SearchRequest search = SearchRequest.of(r -> r + // Index list: + // - add all elements of a list + .index(names) + // - add a single element + .index("idx-d") + // - add a vararg list of elements + .index("idx-e", "idx-f", "idx-g") + + // Sort order list: add elements defined by builder lambdas + .sort(s -> s.field(f -> f.field("foo").order(SortOrder.Asc))) + .sort(s -> s.field(f -> f.field("bar").order(SortOrder.Desc))) + + // Aggregation map: + // - add all entries of an existing map + .aggregations(cardinalities) + // - add a key/value entry + .aggregations("avg-size", avgSize) + // - add a key/value defined by a builder lambda + .aggregations("price-histogram", + a -> a.histogram(h -> h.field("price"))) +); +``` + + +## List and map values are never `null` [_list_and_map_values_are_never_null] + +The {{es}} API has a lot of optional properties. For single-valued properties, the Java API Client represents missing optional values as `null`. Applications therefore have to null-check optional values before using them. + +For lists and maps however, applications often only care about whether they’re empty or not, or even just iterate on their content. Using `null` values is then cumbersome. To avoid this, Java API Client collection properties are never `null`, and missing optional collections are returned as an empty collection. + +If you ever need to distinguish between a missing (undefined) optional collection and an effectively-empty collection returned by {{es}}, the `ApiTypeHelper` class provides a utility method to distinguish them: + +```java +NodeStatistics stats = NodeStatistics.of(b -> b + .total(1) + .failed(0) + .successful(1) +); + +// The `failures` list was not provided. +// - it's not null +assertNotNull(stats.failures()); +// - it's empty +assertEquals(0, stats.failures().size()); +// - and if needed we can know it was actually not defined +assertFalse(ApiTypeHelper.isDefined(stats.failures())); +``` + +The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation). + diff --git a/docs/reference/loading-json.md b/docs/reference/loading-json.md new file mode 100644 index 000000000..4a4227528 --- /dev/null +++ b/docs/reference/loading-json.md @@ -0,0 +1,171 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/loading-json.html +--- + +# Creating API objects from JSON data [loading-json] + +A common workflow during application development with Elasticsearch is to use the Kibana Developer Console to interactively prepare and test queries, aggregations, index mappings and other complex API calls. This results in working JSON snippets that you may want to use in your application. + +As translating these JSON snippets to Java code can be time-consuming and error-prone, most of the data classes in the Java API Client can be loaded from JSON text: object builders have `withJson()` methods that populate the builder from raw JSON. This also allows you to combine dynamically loaded JSON with programmatic construction of objects. + +Under the hood, the `withJson()` methods call the object’s deserializer. The JSON text’s structure and value types therefore have to be correct for the target data structure. Using `withJson()` keeps the strong typing guarantees of the Java API Client. + + +## Examples [_examples_2] + + +### Loading an index definition from a resource file [_loading_an_index_definition_from_a_resource_file] + +Consider a resource file `some-index.json` containing an index definition: + +```json +{ + "mappings": { + "properties": { + "field1": { "type": "text" } + } + } +} +``` + +You can create an index from that definition as follows: + +```java +InputStream input = this.getClass() + .getResourceAsStream("some-index.json"); <1> + +CreateIndexRequest req = CreateIndexRequest.of(b -> b + .index("some-index") + .withJson(input) <2> +); + +boolean created = client.indices().create(req).acknowledged(); +``` + +1. open an input stream for the JSON resource file. +2. populate the index creation request with the resource file contents. + + + +### Ingesting documents from JSON files [_ingesting_documents_from_json_files] + +Similarly, you can read documents to be stored in {{es}} from data files: + +```java +FileReader file = new FileReader(new File(dataDir, "document-1.json")); + +IndexRequest req; <1> + +req = IndexRequest.of(b -> b + .index("some-index") + .withJson(file) +); + +client.index(req); +``` + +1. when calling `withJson()` on data structures that have generic type parameters, these generic types will be considered to be `JsonData`. + + + +### Creating a search request combining JSON and programmatic construction [_creating_a_search_request_combining_json_and_programmatic_construction] + +You can combine `withJson()` with regular calls to setter methods. The example below loads the query part of a search request from a `String` and programmatically adds an aggregation. + +```java +Reader queryJson = new StringReader( + "{" + + " \"query\": {" + + " \"range\": {" + + " \"@timestamp\": {" + + " \"gt\": \"now-1w\"" + + " }" + + " }" + + " }" + + "}"); + +SearchRequest aggRequest = SearchRequest.of(b -> b + .withJson(queryJson) <1> + .aggregations("max-cpu", a1 -> a1 <2> + .dateHistogram(h -> h + .field("@timestamp") + .calendarInterval(CalendarInterval.Hour) + ) + .aggregations("max", a2 -> a2 + .max(m -> m.field("host.cpu.usage")) + ) + ) + .size(0) +); + +Map aggs = client + .search(aggRequest, Void.class) <3> + .aggregations(); +``` + +1. loads the query from the JSON string. +2. adds the aggregation. +3. since this is an aggregation we don’t care about result documents and set their target class to `Void`, meaning they will just be ignored. Note that setting `size` to zero actually prevents any document from being returned. + + + +### Creating a search request from multiple JSON snippets [_creating_a_search_request_from_multiple_json_snippets] + +The `withJson()` methods are partial deserializers: the properties loaded from the JSON will set property values or replace the previous ones, but will not reset other properties not found in the JSON input. You can use this to combine multiple JSON snippets to build complex search requests. In the example below, we combine separate definitions of a query that selects some documents and an aggregation that is run on the results of this query. + +```java +Reader queryJson = new StringReader( + "{" + + " \"query\": {" + + " \"range\": {" + + " \"@timestamp\": {" + + " \"gt\": \"now-1w\"" + + " }" + + " }" + + " }," + + " \"size\": 100" + <1> + "}"); + +Reader aggregationJson = new StringReader( + "{" + + " \"size\": 0, " + <2> + " \"aggregations\": {" + + " \"hours\": {" + + " \"date_histogram\": {" + + " \"field\": \"@timestamp\"," + + " \"interval\": \"hour\"" + + " }," + + " \"aggregations\": {" + + " \"max-cpu\": {" + + " \"max\": {" + + " \"field\": \"host.cpu.usage\"" + + " }" + + " }" + + " }" + + " }" + + " }" + + "}"); + +SearchRequest aggRequest = SearchRequest.of(b -> b + .withJson(queryJson) <3> + .withJson(aggregationJson) <4> + .ignoreUnavailable(true) <5> +); + +Map aggs = client + .search(aggRequest, Void.class) + .aggregations(); +``` + +1. set max number of returned document to 100 for queries. +2. we do not want any matching document in aggregations. +3. loads the query part of the request. +4. loads the aggregation part of the request (overwrites `size` from the query). +5. additional request properties set programmatically. + + +Notice that order matters when the JSON snippets have some common properties: just as when setting property values programmatically, the last value that is set for a property overwrites the previous one. + +The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation). + diff --git a/docs/reference/method-naming.md b/docs/reference/method-naming.md new file mode 100644 index 000000000..77dfb4941 --- /dev/null +++ b/docs/reference/method-naming.md @@ -0,0 +1,12 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/method-naming.html +--- + +# Method naming conventions [method-naming] + +Classes in the Java API Client contain two kinds of methods and properties: + +* Methods and properties that are part of the API, such as `ElasticsearchClient.search()` or `SearchResponse.maxScore()`. They are derived from their respective names in the {{es}} JSON API using the standard Java `camelCaseNaming` convention. +* Methods and properties that are part of the framework on which the Java API Client is built, such as `Query._kind()`. These methods and properties are prefixed with an underscore to both avoid any naming conflicts with API names, and as an easy way to distinguish the API from the framework. + diff --git a/docs/reference/migrate-hlrc.md b/docs/reference/migrate-hlrc.md new file mode 100644 index 000000000..66ab070e7 --- /dev/null +++ b/docs/reference/migrate-hlrc.md @@ -0,0 +1,74 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/migrate-hlrc.html +--- + +# Migrating from the High Level Rest Client [migrate-hlrc] + +The {{es}} Java API Client is an entirely new client library that has no relation to the older High Level Rest Client (HLRC). This was a deliberate choice to provide a library that is independent from the {{es}} server code and that provides a very consistent and easier to use API for all {{es}} features. + +Migrating from the HLRC therefore requires some code rewrite in your application. This transition can however happen progressively as the two client libraries can coexist in a single application with no operational overhead. + + +## Compatibility mode: using a 7.17 client with {{es}} 8.x [_compatibility_mode_using_a_7_17_client_with_es_8_x] + +The HLRC version `7.17` can be used with {{es}} version `8.x` by enabling HLRC’s compatibility mode (see code sample below). In this mode HLRC sends additional headers that instruct {{es}} `8.x` to behave like a `7.x` server. + +The Java API Client doesn’t need this setting as compatibility mode is always enabled. + +You can use the HLRC version `7.x` with the Java API Client version `8.x`: + +```groovy +dependencies { + implementation 'co.elastic.clients:elasticsearch-java:9.0.0-beta1' + implementation 'org.elasticsearch.client:elasticsearch-rest-high-level-client:7.17.4' + // other dependencies <1> +} +``` + +1. See [Installation](/reference/installation.md) + + + +## Using the same http client with the HLRC and the Java API Client [_using_the_same_http_client_with_the_hlrc_and_the_java_api_client] + +To avoid any operational overhead during the transition phase where an application would use both the HLRC and the new Java API Client, both clients can share the same Low Level Rest Client, which is the network layer that manages all connections, round-robin strategies, node sniffing, and so on. + +The code below shows how to initialize both clients with the same HTTP client: + +```java +// Create the low-level client +RestClient httpClient = RestClient.builder( + new HttpHost("localhost", 9200) +).build(); + +// Create the HLRC +RestHighLevelClient hlrc = new RestHighLevelClientBuilder(httpClient) + .setApiCompatibilityMode(true) <1> + .build(); + +// Create the Java API Client with the same low level client +ElasticsearchTransport transport = new RestClientTransport( + httpClient, + new JacksonJsonpMapper() +); + +ElasticsearchClient esClient = new ElasticsearchClient(transport); + +// hlrc and esClient share the same httpClient +``` + +1. Enables compatibility mode that allows HLRC `7.17` to work with {{es}} `8.x`. + + + +## Transition strategies [_transition_strategies] + +There are different ways you can start transitioning away from the HLRC in your application code. + +For example: + +* keep the existing code as-is and use the new Java API Client for new features in your application, then later migrate the existing code, +* rewrite the parts of your application where the new Java API Client is much easier to use than that of the HLRC, like everything related to search, +* rewrite those parts where you need to map application objects to/from JSON, by leveraging the tight integration of the new Java API Client with JSON object mappers. + diff --git a/docs/reference/object-lifecycles.md b/docs/reference/object-lifecycles.md new file mode 100644 index 000000000..ca5554306 --- /dev/null +++ b/docs/reference/object-lifecycles.md @@ -0,0 +1,23 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/object-lifecycles.html +--- + +# Object life cycles and thread safety [object-lifecycles] + +There are five kinds of objects in the Java API Client with different life cycles: + +**Object mapper** +: Stateless and thread-safe, but can be costly to create. It’s usually a singleton that is created at application startup and used to create the transport. + +**Transport** +: Thread-safe, holds network resources through the underlying HTTP client. A transport object is associated with an {{es}} cluster and has to be explicitly closed to release the underlying resources such as network connections. + +**Clients** +: Immutable, stateless and thread-safe. These are very lightweight objects that just wrap a transport and provide API endpoints as methods. Closing a client closes the underlying transport. + +**Builders** +: Mutable, non thread-safe. Builders are transient objects that should not be reused after calling `build()`. + +**Requests & other API objects** +: Immutable, thread-safe. If your application uses the same request or same parts of a request over and over, these objects can be prepared in advance and reused across multiple calls over multiple clients with different transports. diff --git a/docs/reference/opentelemetry.md b/docs/reference/opentelemetry.md new file mode 100644 index 000000000..8f9368305 --- /dev/null +++ b/docs/reference/opentelemetry.md @@ -0,0 +1,111 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/opentelemetry.html +--- + +# Using OpenTelemetry [opentelemetry] + +You can use [OpenTelemetry](https://opentelemetry.io/) to monitor the performance and behavior of your {{es}} requests through the Java API Client. The Java API Client comes with built-in OpenTelemetry instrumentation that emits [distributed tracing spans](docs-content://solutions/observability/apps/traces-2.md) by default. With that, applications [instrumented with OpenTelemetry](https://opentelemetry.io/docs/instrumentation/java/manual/) or running the [OpenTelemetry Java Agent](https://opentelemetry.io/docs/instrumentation/java/automatic/) are inherently enriched with additional spans that contain insightful information about the execution of the {{es}} requests. + +The native instrumentation in the Java API Client follows the [OpenTelemetry Semantic Conventions for {{es}}](https://opentelemetry.io/docs/specs/semconv/database/elasticsearch/). In particular, the instrumentation in the client covers the logical layer of {{es}} requests. A single span per request is created that is processed by the service through the Java API Client. The following image shows a trace that records the handling of three different {{es}} requests: an `index`, `get` and a search `request`: + +:::{image} ../images/otel-waterfall-instrumented-without-http.jpg +:alt: Distributed trace with {{es}} spans +:class: screenshot +::: + +Usually, OpenTelemetry agents and auto-instrumentation modules come with instrumentation support for HTTP-level communication. In this case, in addition to the logical {{es}} client requests, spans will be captured for the physical HTTP requests emitted by the client. The following image shows a trace with both, {{es}} spans (in blue) and the corresponding HTTP-level spans (in red): + +:::{image} ../images/otel-waterfall-instrumented.jpg +:alt: Distributed trace with {{es}} and HTTP spans +:class: screenshot +::: + +Advanced Java API Client behavior such as nodes round-robin and request retries are revealed through the combination of logical {{es}} spans and the physical HTTP spans. The following example shows an `index` request in a scenario with two {{es}} nodes: + +:::{image} ../images/otel-waterfall-retries.jpg +:alt: Distributed trace with request retries +:class: screenshot +::: + +The first node is unavailable and results in an HTTP error, while the retry to the second node succeeds. Both HTTP requests are subsumed by the logical {{es}} request span (in blue). + + +## Setup the OpenTelemetry instrumentation [_setup_the_opentelemetry_instrumentation] + +When using the [OpenTelemetry Java SDK manually](https://opentelemetry.io/docs/instrumentation/java/manual) or using the [OpenTelemetry Java Agent](https://opentelemetry.io/docs/instrumentation/java/automatic/), the Java API Client’s OpenTelemetry instrumentation is enabled by default and uses the *globally* registered OpenTelemetry SDK instance (i.e. `GlobalOpenTelemetry`). So, if you don’t use a custom, local OpenTelemetry SDK instance, there is no explicit setup required to use the Java API Client’s OpenTelemetry instrumentation. + + +### Using a custom OpenTelemetry instance [_using_a_custom_opentelemetry_instance] + +In case you are using [manual OpenTelemetry instrumentation](https://opentelemetry.io/docs/instrumentation/java/manual/#example) with a custom OpenTelemetry SDK instance that is *not registered globally*, you can create the Java API Client using a custom OpenTelemetry instance. The following code snippet shows an example of using a custom OpenTelemetry instance. + +```java +// URL and API key +String serverUrl = "https://localhost:9200"; +String apiKey = "VnVhQ2ZHY0JDZGJrU..."; + +// Create the low-level client +RestClient restClient = RestClient + .builder(HttpHost.create(serverUrl)) + .setDefaultHeaders(new Header[]{ + new BasicHeader("Authorization", "ApiKey " + apiKey) + }) + .build(); +// Create and configure custom OpenTelemetry instance +OpenTelemetry customOtel = OpenTelemetrySdk.builder().build(); + +// Create Instrumentation instance using the custom OpenTelemetry instance +// Second constructor argument allows to enable/disable search body capturing +OpenTelemetryForElasticsearch esOtelInstrumentation = + new OpenTelemetryForElasticsearch(customOtel, false); + +// Create the transport with the custom Instrumentation instance +ElasticsearchTransport transport = new RestClientTransport( + restClient, new JacksonJsonpMapper(), null, esOtelInstrumentation +); + +// And create the API client +ElasticsearchClient esClient = new ElasticsearchClient(transport); + +// Use the client... + +// Close the client, also closing the underlying transport object and network connections. +esClient.close(); +``` + + +## Configuring the OpenTelemetry instrumentation [_configuring_the_opentelemetry_instrumentation] + +You can configure the OpenTelemetry instrumentation either through Java System properties or Environment Variables. The following configuration options are available. + + +### Enable / Disable the OpenTelemetry instrumentation [opentelemetry-config-enable] + +With this configuration option you can enable (default) or disable the built-in OpenTelemetry instrumentation. + +**Default:** `true` + +| | | +| --- | --- | +| Java System Property | `otel.instrumentation.elasticsearch.enabled` | +| Environment Variable | `OTEL_INSTRUMENTATION_ELASTICSEARCH_ENABLED` | + + +### Capture search request bodies [_capture_search_request_bodies] + +Per default, the built-in OpenTelemetry instrumentation does not capture request bodies due to data privacy considerations. You can use this option to enable capturing of search queries from the the request bodies of {{es}} search requests in case you wish to gather this information regardless. + +**Default:** `false` + +| | | +| --- | --- | +| Java System Property | `otel.instrumentation.elasticsearch.capture-search-query` | +| Environment Variable | `OTEL_INSTRUMENTATION_ELASTICSEARCH_CAPTURE_SEARCH_QUERY` | + + +## Overhead [_overhead] + +The OpenTelemetry instrumentation (as any other monitoring approach) may come with a little overhead on CPU, memory and/or latency. The overhead may only occur when the instrumentation is enabled (default) and an OpenTelemetry SDK (or an OpenTelemetry Agent) is active in the target application. In case that either the instrumentation is disabled or no OpenTelemetry SDK (or OpenTelemetry Agent) is active with the target application, there is no monitoring overhead expected when using the client. + +Even when the instrumentation is enabled and is being actively used (by an OpenTelemetry SDK), in the vast majority of cases the overhead is very small and negligible. In edge cases in which there is a noticable overhead the [instrumentation can be explicitly disabled](#opentelemetry-config-enable) to eliminate any potential overhead effect of the instrumentation. diff --git a/docs/reference/package-structure.md b/docs/reference/package-structure.md new file mode 100644 index 000000000..632415f89 --- /dev/null +++ b/docs/reference/package-structure.md @@ -0,0 +1,23 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/package-structure.html +--- + +# Package structure and namespace clients [package-structure] + +The {{es}} API is large and is organized into feature groups, as can be seen in the [{{es}} API documentation](elasticsearch://docs/reference/elasticsearch/rest-apis/index.md). + +The Java API Client follows this structure: feature groups are called “namespaces”, and each namespace is located in a subpackage of `co.elastic.clients.elasticsearch`. + +Each of the namespace clients can be accessed from the top level {{es}} client. The only exceptions are the “search” and “document” APIs which are located in the `core` subpackage and can be accessed on the main {{es}} client object. + +The snippet below shows how to use the indices namespace client to create an index (the lambda syntax is explained in [Building API objects](/reference/building-objects.md)): + +```java +// Create the "products" index +ElasticsearchClient client = ... +client.indices().create(c -> c.index("products")); +``` + +Namespace clients are very lightweight objects that can be created on the fly. + diff --git a/docs/reference/reading.md b/docs/reference/reading.md new file mode 100644 index 000000000..e6f27305b --- /dev/null +++ b/docs/reference/reading.md @@ -0,0 +1,71 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/reading.html +--- + +# Reading documents by id [reading] + +Elasticsearch is all about search, but you may also want to access documents directly, knowing their identifier. The "get" request is meant for this. + +::::{note} +See the [{{es}} API documentation](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-get) for a full explanation of get requests. +:::: + + + +## Reading a domain object [_reading_a_domain_object] + +The example below reads the document with identifier `bk-1` from the `products` index. + +The `get` request has two parameters: + +* the first parameter is the actual request, built below with the fluent DSL +* the second parameter is the class we want the document’s JSON to be mapped to. + +```java +GetResponse response = esClient.get(g -> g + .index("products") <1> + .id("bk-1"), + Product.class <2> +); + +if (response.found()) { + Product product = response.source(); + logger.info("Product name " + product.getName()); +} else { + logger.info ("Product not found"); +} +``` + +1. The get request, with the index name and identifier. +2. The target class, here `Product`. + + + +## Reading raw JSON [_reading_raw_json] + +When your index contains semi-structured data or if you don’t have a domain object definition, you can also read the document as raw JSON data. + +Raw JSON data is just another class that you can use as the result type for the get request. In the example below we use Jackson’s `ObjectNode`. We could also have used any JSON representation that can be deserialized by the JSON mapper associated to the `ElasticsearchClient`. + +```java +GetResponse response = esClient.get(g -> g + .index("products") + .id("bk-1"), + ObjectNode.class <1> +); + +if (response.found()) { + ObjectNode json = response.source(); + String name = json.get("name").asText(); + logger.info("Product name " + name); +} else { + logger.info("Product not found"); +} +``` + +1. The target class is a raw JSON object. + + +The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation). + diff --git a/docs/reference/release-highlights.md b/docs/reference/release-highlights.md new file mode 100644 index 000000000..645fcf83e --- /dev/null +++ b/docs/reference/release-highlights.md @@ -0,0 +1,11 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/release-highlights.html +--- + +# Release highlights [release-highlights] + +These are the important new features and changes in minor releases. Every release also updates the Java API Client to the latest [API specification](https://github.com/elastic/elasticsearch-specification). This includes new APIs and bug fixes in the specification of existing APIs. + +For a list of detailed changes, including bug fixes, please see the [GitHub project realease notes](https://github.com/elastic/elasticsearch-java/releases). + diff --git a/docs/reference/searching.md b/docs/reference/searching.md new file mode 100644 index 000000000..f2eb732d1 --- /dev/null +++ b/docs/reference/searching.md @@ -0,0 +1,158 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/searching.html +--- + +# Searching for documents [searching] + +Indexed documents are available for search in near real-time. + +::::{note} +See the {{es}} documentation for a full explanation of search requests: [search your data](docs-content://solutions/search/querying-for-search.md), [the query DSL](docs-content://solutions/search/querying-for-search.md), and [search APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-search). +:::: + + + +## Simple search query [_simple_search_query] + +There are many types of search queries that can be combined. We will start with the simple text match query, searching for bikes in the `products` index. + +The search result has a `hits` properties that contains the documents that matched the query along with information about the total number of matches that exist in the index. + +The total value comes with a relation that indicates if the total is exact (`eq` — equal) or approximate (`gte` — greater than or equal). + +Each returned document comes with its relevance score and additional information about its location in the index. + +```java +String searchText = "bike"; + +SearchResponse response = esClient.search(s -> s + .index("products") <1> + .query(q -> q <2> + .match(t -> t <3> + .field("name") <4> + .query(searchText) + ) + ), + Product.class <5> +); + +TotalHits total = response.hits().total(); +boolean isExactResult = total.relation() == TotalHitsRelation.Eq; + +if (isExactResult) { + logger.info("There are " + total.value() + " results"); +} else { + logger.info("There are more than " + total.value() + " results"); +} + +List> hits = response.hits().hits(); +for (Hit hit: hits) { + Product product = hit.source(); + logger.info("Found product " + product.getSku() + ", score " + hit.score()); +} +``` + +1. Name of the index we want to search. +2. The query part of the search request (a search request can also have other components like [aggregations](/reference/aggregations.md)). +3. Choose a query variant among the many available. We choose here the match query (full text search). +4. Configure the match query: we search for a term in the `name` field. +5. The target class for the matching documents. We use `Product` here, just like in [get request](/reference/reading.md) examples. + + +Similarly to [get](/reference/reading.md) operations, you can fetch documents matching your query as raw JSON by using a corresponding target class instead of `Product`, like JSON-P’s `JsonValue` or Jackson’s `ObjectNode`. + + +## Nested search queries [_nested_search_queries] + +{{es}} allows individual queries to be combined to build more complex search requests. In the example below we will search for bikes with a maximum price of 200. + +```java +String searchText = "bike"; +double maxPrice = 200.0; + +// Search by product name +Query byName = MatchQuery.of(m -> m <1> + .field("name") + .query(searchText) +)._toQuery(); <2> + +// Search by max price +Query byMaxPrice = RangeQuery.of(r -> r + .number(n -> n + .field("price") + .gte(maxPrice)) <3> +)._toQuery(); + +// Combine name and price queries to search the product index +SearchResponse response = esClient.search(s -> s + .index("products") + .query(q -> q + .bool(b -> b <4> + .must(byName) <5> + .must(byMaxPrice) + ) + ), + Product.class +); + +List> hits = response.hits().hits(); +for (Hit hit: hits) { + Product product = hit.source(); + logger.info("Found product " + product.getSku() + ", score " + hit.score()); +} +``` + +1. We’re creating the queries for individual criteria separately. +2. A `MatchQuery` is a query *variant* that we have to turn into the `Query` *union type*. See [variant types](/reference/variant-types.md) for additional details. +3. {{es}} range query accepts a large range of value types. We create here a JSON representation of the maximum price. +4. The search query is a boolean query that combines the text search and max price queries. +5. Both queries are added as `must` as we want results to match all criteria. + + + +## Templated search [_templated_search] + +A search template is a stored search that you can run with different variables. Search templates let you change your searches without modifying your application code. + +Before running a template search, you first have to create the template. This is a stored script that returns the search request body, and is usually defined as a Mustache template. This stored script can be created outside the application, and also with the Java API Client: + +```java +// Create a script +esClient.putScript(r -> r + .id("query-script") <1> + .script(s -> s + .lang("mustache") + .source("{\"query\":{\"match\":{\"{{field}}\":\"{{value}}\"}}}") + )); +``` + +1. Identifier of the template script to create. + + +To use the search template, use the `searchTemplate` method to refer to the script and provide values for its parameters: + +```java +SearchTemplateResponse response = esClient.searchTemplate(r -> r + .index("some-index") + .id("query-script") <1> + .params("field", JsonData.of("some-field")) <2> + .params("value", JsonData.of("some-data")), + Product.class +); + +List> hits = response.hits().hits(); +for (Hit hit: hits) { + Product product = hit.source(); + logger.info("Found product " + product.getSku() + ", score " + hit.score()); +} +``` + +1. Identifier of the template script to use. +2. Template parameter values. + + +For more in-depth information, see the [{{es}} search template documentation](docs-content://solutions/search/search-templates.md). + +The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation). + diff --git a/docs/reference/setup.md b/docs/reference/setup.md new file mode 100644 index 000000000..55bb77419 --- /dev/null +++ b/docs/reference/setup.md @@ -0,0 +1,17 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/_setup.html +--- + +# Setup [_setup] + +* [Installation](/reference/installation.md) +* [Connecting](/reference/connecting.md) +* [Migrating from the High Level Rest Client](/reference/migrate-hlrc.md) +* [Using OpenTelemetry](/reference/opentelemetry.md) +* [Java Low Level REST Client](/reference/java-low-level-rest-client.md) + + + + + diff --git a/docs/reference/sniffer.md b/docs/reference/sniffer.md new file mode 100644 index 000000000..cbb67a7f4 --- /dev/null +++ b/docs/reference/sniffer.md @@ -0,0 +1,14 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/sniffer.html +--- + +# Sniffer [sniffer] + +Minimal library that allows to automatically discover nodes from a running Elasticsearch cluster and set them to an existing `RestClient` instance. It retrieves by default the nodes that belong to the cluster using the Nodes Info api and uses jackson to parse the obtained json response. + +Compatible with Elasticsearch 2.x and onwards. + + + + diff --git a/docs/reference/toc.yml b/docs/reference/toc.yml new file mode 100644 index 000000000..939973ccf --- /dev/null +++ b/docs/reference/toc.yml @@ -0,0 +1,62 @@ +toc: + - file: index.md + # TO DO: Do we want these to be separate pages? + - file: getting-started.md + - file: getting-started-serverless.md + # - file: elasticsearch/serverless/getting-started-3.md + - file: setup.md + children: + - file: installation.md + - file: connecting.md + - file: migrate-hlrc.md + - file: opentelemetry.md + - file: api-conventions.md + children: + - file: package-structure.md + - file: method-naming.md + - file: blocking-async.md + - file: building-objects.md + - file: lists-maps.md + - file: variant-types.md + - file: object-lifecycles.md + - file: loading-json.md + - file: exception-conventions.md + - file: using-java-api-client.md + children: + - file: esql.md + - file: indexing.md + - file: indexing-bulk.md + - file: reading.md + - file: searching.md + - file: aggregations.md + - file: javadoc-source-code.md + - file: external-resources.md + - file: java-low-level-rest-client.md + children: + - file: java-rest-low-usage.md + children: + - file: java-rest-low-javadoc.md + - file: java-rest-low-usage-maven.md + - file: java-rest-low-usage-dependencies.md + - file: java-rest-low-usage-shading.md + - file: java-rest-low-usage-initialization.md + - file: java-rest-low-usage-requests.md + - file: java-rest-low-usage-responses.md + - file: java-rest-low-usage-logging.md + - file: java-rest-low-config.md + children: + - file: _timeouts.md + - file: _number_of_threads.md + - file: _basic_authentication.md + - file: _other_authentication_methods.md + - file: _encrypted_communication.md + - file: _others.md + - file: _node_selector.md + - file: sniffer.md + children: + - file: java-rest-sniffer-javadoc.md + - file: _maven_repository.md + - file: _usage.md + - file: breaking-changes-policy.md + - file: release-highlights.md + - file: _license.md \ No newline at end of file diff --git a/docs/reference/using-java-api-client.md b/docs/reference/using-java-api-client.md new file mode 100644 index 000000000..2cf7e63ab --- /dev/null +++ b/docs/reference/using-java-api-client.md @@ -0,0 +1,31 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/usage.html +--- + +# Using the Java API client [usage] + +The sections below provide tutorials on the most frequently used and some less obvious features of {{es}}. + +For a full reference, see the [Elasticsearch documentation](docs-content://get-started/index.md) and in particular the [REST APIs](elasticsearch://docs/reference/elasticsearch/rest-apis/index.md) section. The Java API Client follows closely the JSON structures described there, using the [Java API conventions](/reference/api-conventions.md). + +If you’re new to Elasticsearch, make sure also to read [Elasticsearch’s quick start](docs-content://solutions/search/get-started.md) that provides a good introduction. + +* [ES|QL in the Java client](/reference/esql.md) +* [Indexing single documents](/reference/indexing.md) +* [Bulk: indexing multiple documents](/reference/indexing-bulk.md) +* [Reading documents by id](/reference/reading.md) +* [Searching for documents](/reference/searching.md) +* [Aggregations](/reference/aggregations.md) + +::::{note} +This is still a work in progress, more sections will be added in the near future. +:::: + + + + + + + + diff --git a/docs/reference/variant-types.md b/docs/reference/variant-types.md new file mode 100644 index 000000000..9540ac463 --- /dev/null +++ b/docs/reference/variant-types.md @@ -0,0 +1,176 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/variant-types.html +--- + +# Variant types [variant-types] + +The {{es}} API has a lot of variant types: queries, aggregations, field mappings, analyzers, and so on. Finding the correct class name in such large collections can be challenging. + +The Java API Client builders make this easy: the builders for variant types, such as `Query`, have methods for each of the available implementations. We’ve seen this in action above with `intervals` (a kind of query) and `allOf`, `match` and `anyOf` (various kinds of intervals). + +This is because variant objects in the Java API Client are implementations of a “tagged union”: they contain the identifier (or tag) of the variant they hold and the value for that variant. For example, a `Query` object can contain an `IntervalsQuery` with tag `intervals`, a `TermQuery` with tag `term`, and so on. This approach allows writing fluent code where you can let the IDE completion features guide you to build and navigate complex nested structures: + +Variant builders have setter methods for every available implementation. They use the same conventions as regular properties and accept both a builder lambda expression and a ready-made object of the actual type of the variant. Here’s an example to build a term query: + +```java +Query query = new Query.Builder() + .term(t -> t <1> + .field("name") <2> + .value(v -> v.stringValue("foo")) + ) + .build(); <3> +``` + +1. Choose the `term` variant to build a term query. +2. Build the terms query with a builder lambda expression. +3. Build the `Query` that now holds a `TermQuery` object of kind `term`. + + +Variant objects have getter methods for every available implementation. These methods check that the object actually holds a variant of that kind and return the value downcasted to the correct type. They throw an `IllegalStateException` otherwise. This approach allows writing fluent code to traverse variants. + +```java +assertEquals("foo", query.term().value().stringValue()); +``` + +Variant objects also provide information on the variant kind they currently hold: + +* with `is` methods for each of the variant kinds: `isTerm()`, `isIntervals()`, `isFuzzy()`, etc. +* with a nested `Kind` enumeration that defines all variant kinds. + +This information can then be used to navigate down into specific variants after checking their actual kind: + +```java +if (query.isTerm()) { <1> + doSomething(query.term()); +} + +switch(query._kind()) { <2> + case Term: + doSomething(query.term()); + break; + case Intervals: + doSomething(query.intervals()); + break; + default: + doSomething(query._kind(), query._get()); <3> +} +``` + +1. Test if the variant is of a specific kind. +2. Test a larger set of variant kinds. +3. Get the kind and value held by the variant object. + + + +## Custom extensions provided by {{es}} plugins [variant-types-custom] + +{{es}} accepts plugins that can extend the available variants for a number of types. This includes queries, aggregations, text analyzers and tokenizers, ingest processors, etc. + +The Java API Client classes for these types accept a `_custom` variant in addition to the builtin ones. This allows you to use these plugin-defined extensions by providing arbitrary JSON in requests, and also receive arbitrary JSON produced by the plugins in responses. + +In the examples below we use a hypothetical plugin that adds a `sphere-distance` aggregation that groups documents containing 3D coordinates according to their distance to a reference location. + +To create a custom aggregation, use the `_custom()` aggregation type and provide its identifier, defined by the plugin, and parameters. The parameters can be any object or value that can be serialized to JSON. In the example below we use a simple map: + +```java +Map params = new HashMap<>(); <1> +params.put("interval", 10); +params.put("scale", "log"); +params.put("origin", new Double[]{145.0, 12.5, 1649.0}); + +SearchRequest request = SearchRequest.of(r -> r + .index("stars") + .aggregations("neighbors", agg -> agg + ._custom("sphere-distance", params) <2> + ) +); +``` + +1. Parameters for the custom aggregation. +2. Create a custom aggregation named `neighbors` of kind `sphere-distance` with its parameters. + + +The results of custom variants are returned as raw JSON represented by a `JsonData` object. You can then traverse the JSON tree to get the data. Since this is not always convenient, you can also define classes that represent that JSON data and deserialize them from the raw JSON. + +Traversing the JSON tree: + +```java +SearchResponse response = esClient.search(request, Void.class); <1> + +JsonData neighbors = response + .aggregations().get("neighbors") + ._custom(); <2> + +JsonArray buckets = neighbors.toJson() <3> + .asJsonObject() + .getJsonArray("buckets"); + +for (JsonValue item : buckets) { + JsonObject bucket = item.asJsonObject(); + double key = bucket.getJsonNumber("key").doubleValue(); + double docCount = bucket.getJsonNumber("doc_count").longValue(); + doSomething(key, docCount); +} +``` + +1. Use `Void` if you’re only interested in aggregation results, not search hits (see also [Aggregations](/reference/aggregations.md)). +2. Get the `neighbors` aggregation result as custom JSON result. +3. Traverse the JSON tree to extract the result data. + + +Using a class that represents the custom aggregation results: + +```java +SearchResponse response = esClient.search(request, Void.class); + +SphereDistanceAggregate neighbors = response + .aggregations().get("neighbors") + ._custom() + .to(SphereDistanceAggregate.class); <1> + +for (Bucket bucket : neighbors.buckets()) { + doSomething(bucket.key(), bucket.docCount()); +} +``` + +1. Deserialize the custom JSON to a dedicated `SphereDistanceAggregate` class. + + +Where `SphereDistanceAggregate` can be defined as follows: + +```java +public static class SphereDistanceAggregate { + private final List buckets; + @JsonCreator + public SphereDistanceAggregate( + @JsonProperty("buckets") List buckets + ) { + this.buckets = buckets; + } + public List buckets() { + return buckets; + }; +} + +public static class Bucket { + private final double key; + private final double docCount; + @JsonCreator + public Bucket( + @JsonProperty("key") double key, + @JsonProperty("doc_count") double docCount) { + this.key = key; + this.docCount = docCount; + } + public double key() { + return key; + } + public double docCount() { + return docCount; + } +} +``` + +The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation). + diff --git a/docs/release-notes/breaking-change-policy.asciidoc b/docs/release-notes/breaking-change-policy.asciidoc deleted file mode 100644 index 60974e598..000000000 --- a/docs/release-notes/breaking-change-policy.asciidoc +++ /dev/null @@ -1,30 +0,0 @@ -[[breaking-changes-policy]] -=== Breaking changes policy - -The {java-client} source code is generated from a https://github.com/elastic/elasticsearch-specification[formal specification of the Elasticsearch API]. This API specification is large, and although it is tested against hundreds of Elasticsearch test files, it may have discrepancies with the actual API that result in issues in the {java-client}. - -Fixing these discrepancies in the API specification results in code changes in the {java-client}, and some of these changes can require code updates in your applications. - -This section explains how these breaking changes are considered for inclusion in {java-client} releases. - -[discrete] -==== Breaking changes in patch releases - -Some issues in the API specification are properties that have an incorrect type, such as a `long` that should be a `string`, or a required property that is actually optional. These issues can cause the {java-client} to not work properly or even throw exceptions. - -When a specification issue is discovered and resolved, it may require code updates in applications using the {java-client}. Such breaking changes are considered acceptable, _even in patch releases_ (e.g. 7.17.0 -> 7.17.1), as they introduce stability to APIs that may otherwise be unusable. - -[discrete] -==== Breaking changes in minor releases - -Along with these bug fixes, the API specification is constantly refined, more precise type definitions are introduced to improve developer comfort and remove ambiguities. The specification of often-used APIs is fairly mature, so these changes happen generally on less often used APIs. These changes can also cause breaking changes requiring code updates which are considered _acceptable in minor releases_ (e.g. 8.0 -> 8.1). - -[discrete] -==== Breaking changes in major releases - -Major releases (e.g. 7.x -> 8.x) can include larger refactorings of the API specification and the framework underlying the {java-client}. These refactorings are considered carefully and done only when they unlock new important features or new developments. - -[discrete] -==== Elasticsearch API stability guarantees - -All Elasticsearch APIs have stability indicators, which imply potential changes. If an API is `stable` only additional non-breaking changes are added. In case of `experimental` APIs, breaking changes can be introduced any time, which means that these changes, will also be reflected in the {java-client}. diff --git a/docs/release-notes/breaking-changes.md b/docs/release-notes/breaking-changes.md new file mode 100644 index 000000000..357577c4d --- /dev/null +++ b/docs/release-notes/breaking-changes.md @@ -0,0 +1,51 @@ +--- +navigation_title: "Elasticsearch Java Client" +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/breaking-changes-policy.html +--- + +# Elasticsearch Java Client breaking changes [elasticsearch-java-client-breaking-changes] +Before you upgrade, carefully review the Elasticsearch Java Client breaking changes and take the necessary steps to mitigate any issues. + +To learn how to upgrade, check out . + +## Breaking changes policy [elasticsearch-java-client-breaking-changes-policy] +The Java API Client source code is generated from a [formal specification of the Elasticsearch API](https://github.com/elastic/elasticsearch-specification). This API specification is large, and although it is tested against hundreds of Elasticsearch test files, it may have discrepancies with the actual API that result in issues in the Java API Client. + +Fixing these discrepancies in the API specification results in code changes in the Java API Client, and some of these changes can require code updates in your applications. + +This section explains how these breaking changes are considered for inclusion in Java API Client releases. + +### Breaking changes in patch releases [elasticsearch-java-client-breaking-changes-patch-releases] +Some issues in the API specification are properties that have an incorrect type, such as a `long` that should be a `string`, or a required property that is actually optional. These issues can cause the Java API Client to not work properly or even throw exceptions. + +When a specification issue is discovered and resolved, it may require code updates in applications using the Java API Client. Such breaking changes are considered acceptable, *even in patch releases* (e.g. 7.17.0 → 7.17.1), as they introduce stability to APIs that may otherwise be unusable. + +### Breaking changes in minor releases [elasticsearch-java-client-breaking-changes-minor-releases] +Along with these bug fixes, the API specification is constantly refined, more precise type definitions are introduced to improve developer comfort and remove ambiguities. The specification of often-used APIs is fairly mature, so these changes happen generally on less often used APIs. These changes can also cause breaking changes requiring code updates which are considered *acceptable in minor releases* (e.g. 8.0 → 8.1). + +### Breaking changes in major releases [elasticsearch-java-client-breaking-changes-major-releases] +Major releases (e.g. 7.x → 8.x) can include larger refactorings of the API specification and the framework underlying the Java API Client. These refactorings are considered carefully and done only when they unlock new important features or new developments. + +### Elasticsearch API stability guarantees [elasticsearch-java-client-api-stability-guarantees] +All Elasticsearch APIs have stability indicators, which imply potential changes. If an API is `stable` only additional non-breaking changes are added. In case of `experimental` APIs, breaking changes can be introduced any time, which means that these changes, will also be reflected in the Java API Client. + +% ## Next version [elasticsearch-java-client-versionnext-breaking-changes] +% **Release date:** Month day, year + +% ::::{dropdown} Title of breaking change +% Description of the breaking change. +% For more information, check [PR #](PR link). +% **Impact**
Impact of the breaking change. +% **Action**
Steps for mitigating deprecation impact. +% :::: + +% ## 9.0.0 [elasticsearch-java-client-900-breaking-changes] +% **Release date:** March 25, 2025 + +% ::::{dropdown} Title of breaking change +% Description of the breaking change. +% For more information, check [PR #](PR link). +% **Impact**
Impact of the breaking change. +% **Action**
Steps for mitigating deprecation impact. +% :::: \ No newline at end of file diff --git a/docs/release-notes/deprecations.md b/docs/release-notes/deprecations.md new file mode 100644 index 000000000..0bbc2a202 --- /dev/null +++ b/docs/release-notes/deprecations.md @@ -0,0 +1,27 @@ +--- +navigation_title: "Elasticsearch Java Client" +--- +# Elasticsearch Java Client deprecations [elasticsearch-java-client-deprecations] +Review the deprecated functionality for your Elasticsearch Java Client version. While deprecations have no immediate impact, we strongly encourage you update your implementation after you upgrade. + +To learn how to upgrade, check out . + +% ## Next version +% **Release date:** Month day, year + +% ::::{dropdown} Deprecation title +% Description of the deprecation. +% For more information, check [PR #](PR link). +% **Impact**
Impact of deprecation. +% **Action**
Steps for mitigating deprecation impact. +% :::: + +% ## 9.0.0 [elasticsearch-java-client-900-deprecations] +% **Release date:** March 25, 2025 + +% ::::{dropdown} Deprecation title +% Description of the deprecation. +% For more information, check [PR #](PR link). +% **Impact**
Impact of deprecation. +% **Action**
Steps for mitigating deprecation impact. +% :::: \ No newline at end of file diff --git a/docs/release-notes/index.asciidoc b/docs/release-notes/index.asciidoc deleted file mode 100644 index 01cd8b2cb..000000000 --- a/docs/release-notes/index.asciidoc +++ /dev/null @@ -1,9 +0,0 @@ -[[release_notes]] -== Release notes - -* <> -* <> - -include::breaking-change-policy.asciidoc[] -include::release-highlights.asciidoc[] - diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md new file mode 100644 index 000000000..356616bc7 --- /dev/null +++ b/docs/release-notes/index.md @@ -0,0 +1,27 @@ +--- +navigation_title: "Elasticsearch Java Client" +--- + +# Elasticsearch Java Client release notes [elasticsearch-java-client-release-notes] + +Review the changes, fixes, and more in each version of Elasticsearch Java Client. + +To check for security updates, go to [Security announcements for the Elastic stack](https://discuss.elastic.co/c/announcements/security-announcements/31). + +% Release notes include only features, enhancements, and fixes. Add breaking changes, deprecations, and known issues to the applicable release notes sections. + +% ## version.next [felasticsearch-java-client-next-release-notes] +% **Release date:** Month day, year + +% ### Features and enhancements [elasticsearch-java-client-next-features-enhancements] +% * + +% ### Fixes [elasticsearch-java-client-next-fixes] +% * + +## 9.0.0 [elasticsearch-java-client-900-release-notes] +**Release date:** March 25, 2025 + +### Features and enhancements [elasticsearch-java-client-900-features-enhancements] + +### Fixes [elasticsearch-java-client-900-fixes] \ No newline at end of file diff --git a/docs/release-notes/known-issues.md b/docs/release-notes/known-issues.md new file mode 100644 index 000000000..deb84a55c --- /dev/null +++ b/docs/release-notes/known-issues.md @@ -0,0 +1,20 @@ +--- +navigation_title: "Elasticsearch Java Client" + +--- + +# Elasticsearch Java Client known issues [elasticsearch-java-client-known-issues] + +% Use the following template to add entries to this page. + +% :::{dropdown} Title of known issue +% **Details** +% On [Month/Day/Year], a known issue was discovered that [description of known issue]. + +% **Workaround** +% Workaround description. + +% **Resolved** +% On [Month/Day/Year], this issue was resolved. + +::: \ No newline at end of file diff --git a/docs/release-notes/release-highlights.asciidoc b/docs/release-notes/release-highlights.asciidoc deleted file mode 100644 index 628d6c753..000000000 --- a/docs/release-notes/release-highlights.asciidoc +++ /dev/null @@ -1,155 +0,0 @@ -[[release-highlights]] -=== Release highlights - -These are the important new features and changes in minor releases. Every release also updates the {java-client} to the latest https://github.com/elastic/elasticsearch-specification[API specification]. This includes new APIs and bug fixes in the specification of existing APIs. - -For a list of detailed changes, including bug fixes, please see the https://github.com/elastic/elasticsearch-java/releases[GitHub project realease notes]. - -[discrete] -==== Version 8.16 -* `ElasticsearchClient` is now `Closeable`. Closing a client object also closes the underlying transport - https://github.com/elastic/elasticsearch-java/pull/851[#851] -* Added option to make the response body available in case of deserialization error- https://github.com/elastic/elasticsearch-java/pull/886[#886]. - -** While it has always been possible to set the log level to `trace` and have the client print both the json bodies of the requests and responses, it's often not the best solution because of the large amount of information printed. -** To enable the feature: - - RestClientOptions options = new RestClientOptions(RequestOptions.DEFAULT, true); - ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper(), options); - ElasticsearchClient esClientWithOptions = new ElasticsearchClient(transport); - -** To retrieve the original body from the TransportException that gets thrown in case of deserialization errors: - - try{ - // some code that returns faulty json - } - catch (TransportException ex){ - try (RepeatableBodyResponse repeatableResponse = (RepeatableBodyResponse) ex.response()) { - BinaryData body = repeatableResponse.body(); - } - } - - -[discrete] -==== Version 8.15 - -* New https://www.elastic.co/guide/en/elasticsearch/reference/current/connector-apis.html[Connector API] available -* Fixed bug in BulkIngester - https://github.com/elastic/elasticsearch-java/pull/830[#830] -* Various bug fixes, among which are https://github.com/elastic/elasticsearch-java/pull/844[#844], https://github.com/elastic/elasticsearch-java/pull/847[#847] -* Changed builders for queries that use generic types, for example: - -** Old RangeQuery: -+ -[source,java] ----- -RangeQuery ra = RangeQuery.of(r -> r // no distinction between various types (date, term, number) - .field("day") - .gte(JsonData.of("2024-06-20")) // must use custom json for fields because there's no defined type - .lte(JsonData.of("2024-07-05")) - .boost(1.0F) - .format("yyyy-MM-dd") - .relation(RangeRelation.Contains)); ----- -+ -** New RangeQuery: -+ -[source,java] ----- -RangeQuery ra = RangeQuery.of(r -> r - .date(d -> d // choose query type before - .field("day") - .gte("2024-06-20") // field now only accepts valid type for the specified query - .lte("2024-07-05") - .boost(1.0F) - .format("yyyy-MM-dd") - .relation(RangeRelation.Contains))); ----- - -[discrete] -==== Version 8.14 -* No new feature. - -[discrete] -==== Version 8.13 - -* Add ES|QL helpers - https://github.com/elastic/elasticsearch-java/pull/763[#763] -* Add an example SpringBoot application - https://github.com/elastic/elasticsearch-java/pull/737[#737] - -[discrete] -==== Version 8.12 - -* No new feature. - -[discrete] -==== Version 8.11 - -* No new feature. - -[discrete] -==== Version 8.10 - -* Add instrumentation API and native OpenTelemetry implementation - https://github.com/elastic/elasticsearch-java/pull/588[#588], <> - -[discrete] -==== Version 8.9 - -* Refactor RestClientTransport to allow using other http client libraries - https://github.com/elastic/elasticsearch-java/pull/584[#584] - -[discrete] -==== Version 8.8 - -* Throw a TransportException when an error response cannot be parsed - https://github.com/elastic/elasticsearch-java/pull/579[#579] -* Speed up URL path encoding and remove dependency on httpclient - https://github.com/elastic/elasticsearch-java/pull/576[#576] -* Add buffered lookahead and JsonData implementation for Jackson - https://github.com/elastic/elasticsearch-java/pull/489[#489], https://github.com/elastic/elasticsearch-java/pull/567[#567] -* Expose additional BulkIngester metrics - https://github.com/elastic/elasticsearch-java/pull/513[#513] -* Allow unparsed binary data to be used for ingestion - https://github.com/elastic/elasticsearch-java/pull/508[#508] - -[discrete] -==== Version 8.7 - -* Add `BulkIngester` helper - https://github.com/elastic/elasticsearch-java/pull/474[#474], https://github.com/elastic/elasticsearch-java/pull/513[#513], <> -* Allow unparsed binary data to be used for ingestion - https://github.com/elastic/elasticsearch-java/pull/508[#508], <> -* Add convenience method `BooleanQuery.hasClauses()` - https://github.com/elastic/elasticsearch-java/pull/525[#525] - - -[discrete] -==== Version 8.6 - -* Allow using `java.lang.reflect.Type` in addition to `java.lang.Class` to define application types for deserialization - https://github.com/elastic/elasticsearch-java/pull/438[#438] -* Add buffered lookahead for Jackson to speed up union deserialization - https://github.com/elastic/elasticsearch-java/pull/489[#489] -* Cache the result of `JsonProvider.provider()` - https://github.com/elastic/elasticsearch-java/pull/485[#485] - -[discrete] -==== Version 8.5 - -* Add support for API endpoints that return binary content, such as the Vector tile API - https://github.com/elastic/elasticsearch-java/pull/434[#434] -* Add support for <> - https://github.com/elastic/elasticsearch-java/pull/370[#370], https://github.com/elastic/elasticsearch-java/pull/371[#371] -* Add SSL setup helper class and <> - https://github.com/elastic/elasticsearch-java/pull/371[#390] - -[discrete] -==== Version 8.4 - -* Add troubleshooting documentation on <> - https://github.com/elastic/elasticsearch-java/pull/301[#301] -* Allow <>. This is useful in scenarios where the Java application is used as a proxy and the responses are sent back to its client - https://github.com/elastic/elasticsearch-java/pull/316[#316] - -[discrete] -==== Version 8.3 - -* Add `toString()` implementation to all value classes - https://github.com/elastic/elasticsearch-java/pull/269[#269] - -[discrete] -==== Version 8.2 - -* Add <> to all builder classes - https://github.com/elastic/elasticsearch-java/pull/316[#200] -* Add troubleshooting docs for <> - https://github.com/elastic/elasticsearch-java/pull/215[#215] -* Improve JSON mapping errors by adding location and property path in the exception message - https://github.com/elastic/elasticsearch-java/pull/237[#237] - -[discrete] -==== Version 8.1 - -* Add documentation for <> with {es} 8.x - https://github.com/elastic/elasticsearch-java/pull/230[#230] - -[discrete] -==== Version 8.0 - -* Change the JSON-P implementation from Glassfish to Parsson - https://github.com/elastic/elasticsearch-java/pull/63[#63] -* Accept `null` values in lists - https://github.com/elastic/elasticsearch-java/pull/68[#68] diff --git a/docs/release-notes/toc.yml b/docs/release-notes/toc.yml new file mode 100644 index 000000000..a41006794 --- /dev/null +++ b/docs/release-notes/toc.yml @@ -0,0 +1,5 @@ +toc: + - file: index.md + - file: known-issues.md + - file: breaking-changes.md + - file: deprecations.md \ No newline at end of file diff --git a/docs/setup/connecting.asciidoc b/docs/setup/connecting.asciidoc deleted file mode 100644 index 71b13867b..000000000 --- a/docs/setup/connecting.asciidoc +++ /dev/null @@ -1,112 +0,0 @@ -[[connecting]] -=== Connecting - -The {java-client} is structured around three main components: - -* **API client classes**. These provide strongly typed data structures and -methods for {es} APIs. Since the {es} API is large, it is structured in feature -groups (also called “namespaces”), each having its own client class. {es} core -features are implemented in the `ElasticsearchClient` class. -* **A JSON object mapper**. This maps your application classes to JSON and -seamlessly integrates them with the API client. -* **A transport layer implementation**. This is where all HTTP request handling -takes place. - -This code snippet creates and wires together these three components: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/getting_started/ConnectingTest.java[create-client] --------------------------------------------------- - -Authentication is managed by the <>. For further details on -configuring authentication, refer to -{java-api-client}/_basic_authentication.html[its documentation]. - -[discrete] -==== Your first request - -The code snippet below searches all items from a “product” index whose name -matches “bicycle” and return them as instances of a `Product` application class. - -It illustrates the use of fluent functional builders to write search queries as -concise DSL-like code. This pattern is explained in more detail in -<>. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/getting_started/ConnectingTest.java[first-request] --------------------------------------------------- - -[discrete] -[[using-a-secure-connection]] -==== Using a secure connection - -The <> documentation explains how to set up encrypted communications in detail. - -In self-managed installations, Elasticsearch will start with security features like authentication and TLS enabled. To connect to the Elasticsearch cluster you’ll need to configure the {java-client} to use HTTPS with the generated CA certificate in order to make requests successfully. - -When you start Elasticsearch for the first time you’ll see a distinct block like the one below in the output from Elasticsearch (you may have to scroll up if it’s been a while): - -["source","xml"] ----------------------------------------------------------------- --> Elasticsearch security features have been automatically configured! --> Authentication is enabled and cluster connections are encrypted. - --> Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`): - lhQpLELkjkrawaBoaz0Q - --> HTTP CA certificate SHA-256 fingerprint: - a52dd93511e8c6045e21f16654b77c9ee0f34aea26d9f40320b531c474676228 -... ----------------------------------------------------------------- - -Note down the elastic user password and HTTP CA fingerprint for the next sections. In the examples below they will be stored in the variables `password` and `fingerprint` respectively. - -Depending on the context, you have two options for verifying the HTTPS connection: either verifying with the CA certificate itself or using the CA certificate fingerprint. For both cases, the {java-client}'s `TransportUtils` class provides convenience methods to easily create an `SSLContext`. - -[discrete] -===== Verifying HTTPS with a certificate fingerprint - -This method of verifying the HTTPS connection uses the certificate fingerprint value noted down earlier. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/getting_started/ConnectingTest.java[create-secure-client-fingerprint] --------------------------------------------------- -<1> Create an `SSLContext` with the certificate fingerprint. -<2> Set up authentication. -<3> Do not forget to set the protocol to `https`! -<4> Configure the http client with the SSL and authentication configurations. - -Note that the certificate fingerprint can also be calculated using `openssl x509` with the certificate file: -["source","bash"] --------------------------------------------------- -openssl x509 -fingerprint -sha256 -noout -in /path/to/http_ca.crt --------------------------------------------------- - -If you don’t have access to the generated CA file from Elasticsearch you can use the following script to output the root CA fingerprint of the Elasticsearch instance with `openssl s_client`: - -["source","bash"] --------------------------------------------------- -openssl s_client -connect localhost:9200 -servername localhost -showcerts /dev/null \ - | openssl x509 -fingerprint -sha256 -noout -in /dev/stdin --------------------------------------------------- - -[discrete] -===== Verifying HTTPS with a CA certificate - -The generated root CA certificate can be found in the `certs` directory in your Elasticsearch config location. If you’re running Elasticsearch in Docker there is {es-docs}/docker.html[additional documentation] for retrieving the CA certificate. - -Once you have made the `http_ca.crt` file available to your application, you can use it to set up the client: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/getting_started/ConnectingTest.java[create-secure-client-cert] --------------------------------------------------- -<1> Create an `SSLContext` with the `http_ca.crt` file. -<2> Set up authentication. -<3> Do not forget to set the protocol to `https`! -<4> Configure the http client with the SSL and authentication configurations. - -{doc-tests-blurb} diff --git a/docs/setup/index.asciidoc b/docs/setup/index.asciidoc deleted file mode 100644 index c7a47f6e0..000000000 --- a/docs/setup/index.asciidoc +++ /dev/null @@ -1,14 +0,0 @@ -== Setup - -* <> -* <> -* <> -* <> -* <> - -include::installation.asciidoc[] -include::connecting.asciidoc[] -include::migrate-hlrc.asciidoc[] -include::open-telemetry.asciidoc[] - -// include::low-level.asciidoc[leveloffset=+2] diff --git a/docs/setup/installation.asciidoc b/docs/setup/installation.asciidoc deleted file mode 100644 index 5fbd4460f..000000000 --- a/docs/setup/installation.asciidoc +++ /dev/null @@ -1,104 +0,0 @@ -[[installation]] -=== Installation - -Requirements: - -* Java 8 or later. -* A JSON object mapping library to allow seamless integration of - your application classes with the Elasticsearch API. The Java client has - support for https://github.com/FasterXML/jackson[Jackson] or a - https://github.com/eclipse-ee4j/jsonb-api[JSON-B] library like - https://github.com/eclipse-ee4j/yasson[Eclipse Yasson]. - - -Releases are hosted on -https://search.maven.org/search?q=g:co.elastic.clients[Maven Central]. If you -are looking for a SNAPSHOT version, the Elastic Maven Snapshot repository is -available at https://snapshots.elastic.co/maven/. - - -[discrete] -[[gradle]] -==== Installation in a Gradle project by using Jackson - -["source","groovy",subs="attributes+"] --------------------------------------------------- -dependencies { - implementation 'co.elastic.clients:elasticsearch-java:{version}' - implementation 'com.fasterxml.jackson.core:jackson-databind:2.17.0' -} --------------------------------------------------- - -[discrete] -[[maven]] -==== Installation in a Maven project by using Jackson - -In the `pom.xml` of your project, add the following repository definition and -dependencies: - -["source","xml",subs="attributes+"] --------------------------------------------------- - - - - - co.elastic.clients - elasticsearch-java - {version} - - - - com.fasterxml.jackson.core - jackson-databind - 2.17.0 - - - - --------------------------------------------------- - - -[discrete] -[[class-not-found-jsonprovider]] -==== If you get ClassNotFoundException: jakarta.json.spi.JsonProvider - -It may happen that after setting up the dependencies, your application fails with `ClassNotFoundException: jakarta.json.spi.JsonProvider`. - -If this happens, you have to explicitly add the `jakarta.json:jakarta.json-api:2.0.1` dependency. - -.Gradle: -["source","groovy",subs="attributes+"] --------------------------------------------------- -dependencies { - ... - implementation 'jakarta.json:jakarta.json-api:2.0.1' -} --------------------------------------------------- - -.Maven: -["source","xml",subs="attributes+"] --------------------------------------------------- - - - ... - - jakarta.json - jakarta.json-api - 2.0.1 - - - - --------------------------------------------------- - -**Why is this needed?** - -Some frameworks like Spring Boot or Helidon come with their Gradle and Maven plugins or their Maven BOM files to ease development and dependency management. These plugins and BOM define the versions to use for a number of well-known libraries. - -One of these libraries can be `jakarta.json:json-api` that defines the standard Java JSON API. In version `1.x` this library used the `javax.json` package, while in version `2.x` it uses the `jakarta.json` package after https://blogs.oracle.com/javamagazine/post/transition-from-java-ee-to-jakarta-ee[the transition from JavaEE to JakartaEE]. - -The {java-client} depends on version `2.0.1` of this library, in order to use the newer and future-proof `jakarta.json` package. But some build plugins and BOMs override the {java-client}'s dependency to use version `1.x` in the older `javax.json` namespace, resulting in `ClassNotFoundException: jakarta.json.spi.JsonProvider`. - -Adding the correct version as top-level project dependency solves the problem. - -If your application also requires `javax.json` you can add the `javax.json:javax.json-api:1.1.4` dependency, which is equivalent to `jakarta.json:jakarta.json-api:1.1.6`. diff --git a/docs/setup/migrate-hlrc.asciidoc b/docs/setup/migrate-hlrc.asciidoc deleted file mode 100644 index b37c65be1..000000000 --- a/docs/setup/migrate-hlrc.asciidoc +++ /dev/null @@ -1,65 +0,0 @@ -[[migrate-hlrc]] -=== Migrating from the High Level Rest Client - -The {es} Java API Client is an entirely new client library that has no relation -to the older High Level Rest Client (HLRC). This was a deliberate choice to -provide a library that is independent from the {es} server code and that -provides a very consistent and easier to use API for all {es} features. - -Migrating from the HLRC therefore requires some code rewrite in your -application. This transition can however happen progressively as the two client -libraries can coexist in a single application with no operational overhead. - -[discrete] -==== Compatibility mode: using a 7.17 client with {es} 8.x -The HLRC version `7.17` can be used with {es} version `8.x` by enabling -HLRC's compatibility mode (see code sample below). In this mode HLRC sends -additional headers that instruct {es} `8.x` to behave like a `7.x` server. - -The {java-client} doesn't need this setting as compatibility mode is always -enabled. - -You can use the HLRC version `7.x` with the {java-client} version `8.x`: - -["source","groovy",subs="attributes+"] --------------------------------------------------- -dependencies { - implementation 'co.elastic.clients:elasticsearch-java:{version}' - implementation 'org.elasticsearch.client:elasticsearch-rest-high-level-client:7.17.4' - // other dependencies <1> -} --------------------------------------------------- -<1> See <> - -[discrete] -==== Using the same http client with the HLRC and the Java API Client - -To avoid any operational overhead during the transition phase where an -application would use both the HLRC and the new Java API Client, both clients -can share the same Low Level Rest Client, which is the network layer that -manages all connections, round-robin strategies, node sniffing, and so on. - -The code below shows how to initialize both clients with the same HTTP client: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/getting_started/MigrateHlrcTest.java[migrate] --------------------------------------------------- -<1> Enables compatibility mode that allows HLRC `7.17` to work with {es} `8.x`. - -[discrete] -==== Transition strategies - -There are different ways you can start transitioning away from the HLRC in your -application code. - -For example: - -* keep the existing code as-is and use the new Java API Client for new features - in your application, then later migrate the existing code, -* rewrite the parts of your application where the new Java API Client is much - easier to use than that of the HLRC, like everything related to search, -* rewrite those parts where you need to map application objects to/from JSON, by - leveraging the tight integration of the new Java API Client with JSON object - mappers. - diff --git a/docs/setup/open-telemetry.asciidoc b/docs/setup/open-telemetry.asciidoc deleted file mode 100644 index 92a4d749d..000000000 --- a/docs/setup/open-telemetry.asciidoc +++ /dev/null @@ -1,75 +0,0 @@ -[[opentelemetry]] -=== Using OpenTelemetry - -You can use https://opentelemetry.io/[OpenTelemetry] to monitor the performance and behavior of your {es} requests through the Java API Client. -The Java API Client comes with built-in OpenTelemetry instrumentation that emits https://www.elastic.co/guide/en/apm/guide/current/apm-distributed-tracing.html[distributed tracing spans] by default. -With that, applications https://opentelemetry.io/docs/instrumentation/java/manual/[instrumented with OpenTelemetry] or running the https://opentelemetry.io/docs/instrumentation/java/automatic/[OpenTelemetry Java Agent] are inherently enriched with additional spans that contain insightful information about the execution of the {es} requests. - -The native instrumentation in the Java API Client follows the https://opentelemetry.io/docs/specs/semconv/database/elasticsearch/[OpenTelemetry Semantic Conventions for {es}]. In particular, the instrumentation in the client covers the logical layer of {es} requests. A single span per request is created that is processed by the service through the Java API Client. The following image shows a trace that records the handling of three different {es} requests: an `index`, `get` and a search `request`: - -[role="screenshot"] -image::images/otel-waterfall-instrumented-without-http.jpg[alt="Distributed trace with {es} spans",align="center"] - -Usually, OpenTelemetry agents and auto-instrumentation modules come with instrumentation support for HTTP-level communication. In this case, in addition to the logical {es} client requests, spans will be captured for the physical HTTP requests emitted by the client. The following image shows a trace with both, {es} spans (in blue) and the corresponding HTTP-level spans (in red): - -[role="screenshot"] -image::images/otel-waterfall-instrumented.jpg[alt="Distributed trace with {es} and HTTP spans",align="center"] - -Advanced Java API Client behavior such as nodes round-robin and request retries are revealed through the combination of logical {es} spans and the physical HTTP spans. The following example shows an `index` request in a scenario with two {es} nodes: - -[role="screenshot"] -image::images/otel-waterfall-retries.jpg[alt="Distributed trace with request retries",align="center"] - -The first node is unavailable and results in an HTTP error, while the retry to the second node succeeds. Both HTTP requests are subsumed by the logical {es} request span (in blue). - -[discrete] -==== Setup the OpenTelemetry instrumentation - -When using the https://opentelemetry.io/docs/instrumentation/java/manual[OpenTelemetry Java SDK manually] or using the https://opentelemetry.io/docs/instrumentation/java/automatic/[OpenTelemetry Java Agent], the Java API Client's OpenTelemetry instrumentation is enabled by default and uses the _globally_ registered OpenTelemetry SDK instance (i.e. `GlobalOpenTelemetry`). So, if you don't use a custom, local OpenTelemetry SDK instance, there is no explicit setup required to use the Java API Client's OpenTelemetry instrumentation. - -[discrete] -===== Using a custom OpenTelemetry instance -In case you are using https://opentelemetry.io/docs/instrumentation/java/manual/#example[manual OpenTelemetry instrumentation] with a custom OpenTelemetry SDK instance that is _not registered globally_, you can create the Java API Client using a custom OpenTelemetry instance. The following code snippet shows an example of using a custom OpenTelemetry instance. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/getting_started/ConnectingTest.java[create-client-otel] --------------------------------------------------- - -[discrete] -==== Configuring the OpenTelemetry instrumentation - -You can configure the OpenTelemetry instrumentation either through Java System properties or Environment Variables. -The following configuration options are available. - -[discrete] -[[opentelemetry-config-enable]] -===== Enable / Disable the OpenTelemetry instrumentation - -With this configuration option you can enable (default) or disable the built-in OpenTelemetry instrumentation. - -**Default:** `true` - -|============ -| Java System Property | `otel.instrumentation.elasticsearch.enabled` -| Environment Variable | `OTEL_INSTRUMENTATION_ELASTICSEARCH_ENABLED` -|============ - -[discrete] -===== Capture search request bodies - -Per default, the built-in OpenTelemetry instrumentation does not capture request bodies due to data privacy considerations. You can use this option to enable capturing of search queries from the the request bodies of {es} search requests in case you wish to gather this information regardless. - -**Default:** `false` - -|============ -| Java System Property | `otel.instrumentation.elasticsearch.capture-search-query` -| Environment Variable | `OTEL_INSTRUMENTATION_ELASTICSEARCH_CAPTURE_SEARCH_QUERY` -|============ - -[discrete] -==== Overhead - -The OpenTelemetry instrumentation (as any other monitoring approach) may come with a little overhead on CPU, memory and/or latency. The overhead may only occur when the instrumentation is enabled (default) and an OpenTelemetry SDK (or an OpenTelemetry Agent) is active in the target application. In case that either the instrumentation is disabled or no OpenTelemetry SDK (or OpenTelemetry Agent) is active with the target application, there is no monitoring overhead expected when using the client. - -Even when the instrumentation is enabled and is being actively used (by an OpenTelemetry SDK), in the vast majority of cases the overhead is very small and negligible. In edge cases in which there is a noticable overhead the <> to eliminate any potential overhead effect of the instrumentation. diff --git a/docs/troubleshooting/index.asciidoc b/docs/troubleshooting/index.asciidoc deleted file mode 100644 index 0fb11a41d..000000000 --- a/docs/troubleshooting/index.asciidoc +++ /dev/null @@ -1,29 +0,0 @@ -[[troubleshooting]] -== Troubleshooting - -// * <> -// * <> - -[discrete] -=== Exceptions - -* <> -* <> -* <> - -[discrete] -=== Miscellaneous - -* <> - -// [[debugging]] -// === Debugging - -//[[deprecation-warnings]] -// === Elasticsearch deprecation warnings - -include::missing-required-property.asciidoc[] -include::no-such-method-request-options.asciidoc[] -include::io-reactor-errors.asciidoc[] - -include::serialize-without-typed-keys.asciidoc[] diff --git a/docs/troubleshooting/io-reactor-errors.asciidoc b/docs/troubleshooting/io-reactor-errors.asciidoc deleted file mode 100644 index 8146646ff..000000000 --- a/docs/troubleshooting/io-reactor-errors.asciidoc +++ /dev/null @@ -1,33 +0,0 @@ -[[io-reactor-errors]] -=== Apache http-client I/O reactor errors - -Sending requests can sometimes fail with one of the following errors, coming from the Apache http-client library: - -* `Request cannot be executed; I/O reactor status: STOPPED` -* `I/O reactor terminated abnormally` -* `I/O reactor has been shut down` - -The I/O Reactor is the internal event loop in the http client library. It can terminate when an application callback throws an `Error`, like an `OutOfMemoryError` or a `StackOverflowError`. Remember that `Error` is different from a regular `Exception` and – https://docs.oracle.com/javase/8/docs/api/?java/lang/Error.html[quoting the Java documentation] – _indicates serious problems that a reasonable application should not try to catch_. - -In the context of the Elasticsearch Java clients, this can happen on two occasions: - -* the application calls the low level `RestClient` directly, using the asynchronous `performRequestAsync` method, and an `Error` is thrown in the `ResponseListener` provided by the application. -* an `OutOfMemoryError` happens while buffering the body of an http response. - -In the first case, it is the application's responsibility to catch `Error` in its `ResponseListener` and decide what to do when these errors happen. - -The second case is taken care of in the {java-client} since version 8.12: the error is wrapped in a `RuntimeException` that is reported to the application. - -In previous versions of the {java-client}, you can copy/paste the `SafeResponseConsumer` class in your project and initialize the `RestClientTransport` as follows: - -["source","java"] ------- -RestClient restClient = ... -JsonpMapper mapper = ... -RestClientOptions options = new RestClientOptions( - SafeResponseConsumer.DEFAULT_REQUEST_OPTIONS -); -RestClientTransport transport = new RestClientTransport( - restClient, mapper, options -); ------- diff --git a/docs/troubleshooting/missing-required-property.asciidoc b/docs/troubleshooting/missing-required-property.asciidoc deleted file mode 100644 index a338b35d1..000000000 --- a/docs/troubleshooting/missing-required-property.asciidoc +++ /dev/null @@ -1,38 +0,0 @@ -[[missing-required-property]] -=== `MissingRequiredPropertyException` in a response - -The {java-client} distinguishes optional and required properties. Optional properties are marked with the `@Nullable` annotation. - -When an API object is built and a required property hasn't been set, a `MissingRequiredPropertyException` is thrown. This applies both to request object built by your application and to response objects returned by Elasticsearch, so that you can be assured that a property that does not have the `@Nullable` annotation will never be `null`. - -However, there may be bugs in the https://github.com/elastic/elasticsearch-specification[Elasticsearch API specification] where a response object's property is incorrectly required, leading to a `MissingRequiredPropertyException` when deserializing a response. If this happens, here's how you can work around it: - -* Make sure you use the latest release of the {java-client}. The issue may already have been fixed. -* If the issue is still present on the latest version, https://github.com/elastic/elasticsearch-java/issues/new/choose[open an issue] so that we can fix it in the next release. Please help us to improve the {java-client}. -* Temporarily disable required property checks for the offending request: - -WARNING: This is a workaround. Do not consider this as a permanent solution, and please https://github.com/elastic/elasticsearch-java/issues/new/choose[open an issue] so that the problem can be fixed in a future release. - -["source","java"] --------------------------------------------------- - ApiTypeHelper.DANGEROUS_disableRequiredPropertiesCheck(true); - SomeRequest request = SomeRequest.of(...); - SomeResponse response = esClient.someApi(request); - ApiTypeHelper.DANGEROUS_disableRequiredPropertiesCheck(false); - // Do something with response -} --------------------------------------------------- - -The `DANGEROUS_disableRequiredPropertiesCheck` method disables required property checks on the current thread, and for response deserialization in asynchronous requests. As its name implies, it is dangerous as it removes the guarantees of properties that are not `@Nullable`. This is a temporary workaround until the issue is fixed. - -Note that the result of this method is an `AutoCloseable` object that resets required property checks to its previous setting. You can therefore use it in a try-with-resource block as follows: - -["source","java"] --------------------------------------------------- -try (ApiTypeHelper.DisabledChecksHandle h = - ApiTypeHelper.DANGEROUS_disableRequiredPropertiesCheck(true)) { - SomeRequest request = SomeRequest.of(...); - SomeResponse response = esClient.someApi(request); - // Do something with response -} --------------------------------------------------- diff --git a/docs/troubleshooting/no-such-method-request-options.asciidoc b/docs/troubleshooting/no-such-method-request-options.asciidoc deleted file mode 100644 index 942f79588..000000000 --- a/docs/troubleshooting/no-such-method-request-options.asciidoc +++ /dev/null @@ -1,36 +0,0 @@ - -[[no-such-method-request-options]] -=== `NoSuchMethodError RequestOptions$Builder.removeHeader` when creating a client - -In certain contexts you may encounter an error when creating the `ElasticsearchClient` saying that the method `RequestOptions$Builder.removeHeader` does not exist: - -["source","java"] --------------------------------------------------- -java.lang.NoSuchMethodError: 'org.elasticsearch.client.RequestOptions$Builder org.elasticsearch.client.RequestOptions$Builder.removeHeader(java.lang.String)' --------------------------------------------------- - -This method was introduced in `elasticsearch-rest-client` version 7.16.0. The error happens because your project is using an older version of this dependency. - -This happens in particular when the project is using the https://docs.spring.io/spring-boot/docs/current/maven-plugin/reference/htmlsingle/[Spring Boot Maven Plugin], as this plugin https://github.com/spring-projects/spring-boot/blob/main/spring-boot-project/spring-boot-dependencies/build.gradle[defines versions for commonly used libraries], including `elasticsearch-rest-client`. Depending on the version of Spring Boot used in the project, that version may be outdated. - -To solve this issue, you have to add the `elasticsearch-rest-client` dependency explicitly in your project, with the same version as `elasticsearch-java` (see also <>). - -Using Gradle: - -["source","groovy",subs="attributes+"] --------------------------------------------------- -implementation 'org.elasticsearch.client:elasticsearch-rest-client:{version}' --------------------------------------------------- - - -Using Maven: - -["source","xml",subs="attributes+"] --------------------------------------------------- - - org.elasticsearch.client - elasticsearch-rest-client - {version} - --------------------------------------------------- - diff --git a/docs/troubleshooting/serialize-without-typed-keys.asciidoc b/docs/troubleshooting/serialize-without-typed-keys.asciidoc deleted file mode 100644 index 0d39c178d..000000000 --- a/docs/troubleshooting/serialize-without-typed-keys.asciidoc +++ /dev/null @@ -1,24 +0,0 @@ -[[serialize-without-typed-keys]] -=== Serializing aggregations and suggestions without typed keys - -{es} search requests accept a `typed_key` parameter that allow returning type information along with the name in aggregation and suggestion results (see the {es-docs}/search-aggregations.html#return-agg-type[aggregations documentation] for additional details). - -The {java-client} always adds this parameter to search requests, as type information is needed to know the concrete class that should be used to deserialize aggregation and suggestion results. - -Symmetrically, the {java-client} also serializes aggregation and suggestion results using this `typed_keys` format, so that it can correctly deserialize the results of its own serialization. - -["source","java"] --------------------------------------------------- -ElasticsearchClient esClient = ... -include-tagged::{doc-tests-src}/troubleshooting/TroubleShootingTests.java[aggregation-typed-keys] --------------------------------------------------- - -However, in some use cases serializing objects in the `typed_keys` format may not be desirable, for example when the {java-client} is used in an application that acts as a front-end to other services that expect the default format for aggregations and suggestions. - -You can disable `typed_keys` serialization by setting the `JsonpMapperFeatures.SERIALIZE_TYPED_KEYS` attribute to `false` on your mapper object: - -["source","java"] --------------------------------------------------- -ElasticsearchClient esClient = ... -include-tagged::{doc-tests-src}/troubleshooting/TroubleShootingTests.java[aggregation-no-typed-keys] --------------------------------------------------- diff --git a/docs/usage/aggregations.asciidoc b/docs/usage/aggregations.asciidoc deleted file mode 100644 index 51c64b341..000000000 --- a/docs/usage/aggregations.asciidoc +++ /dev/null @@ -1,40 +0,0 @@ -[[aggregations]] -=== Aggregations - -An aggregation summarizes your data as metrics, statistics, or other analytics. - -NOTE: See the {es-docs}/search-aggregations.html[{es} documentation] for a full explanation of aggregations. - -[discrete] -==== A simple aggregation - -In the example below we run an aggregation that creates a price histogram from a product index, for the products whose name match a user-provided text. To achieve this, we use a search request that has a query (explained in <>) and an aggregation definition. - -This example is an analytics-type aggregation where we do not want to use the matching documents. A general pattern for search requests used for analytics is to set the result `size` to zero and the target class for search results to `Void`. - -If that same aggregation was used for to display products and the price histogram as drill-down facets, we would have set `size` to a non-zero value and used `Product` as the target class to process the results. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/AggregationsTest.java[price-histo-request] --------------------------------------------------- -<1> Set the number of matching documents to zero as we only use the price histogram. -<2> Set the query that fill filter the products on which to run the aggregation -<3> Create an aggregation named "price-histogram". You can add as many named aggregations as needed. -<4> Select the `histogram` aggregation variant. -<5> We do not care about matches (`size` is set to zero), using `Void` will ignore any document in the response. - -The response contains an aggregation result for each aggregation in the request. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/AggregationsTest.java[price-histo-response] --------------------------------------------------- -<1> Get the results for the "price-histogram" aggregation. -<2> Cast it down to the `histogram` variant results. This has to be consistent with the aggregation definition. -<3> Buckets can be expressed as arrays or maps. This casts down to the array variant (the default). - - -// TODO: nested aggregations - -{doc-tests-blurb} diff --git a/docs/usage/esql.asciidoc b/docs/usage/esql.asciidoc deleted file mode 100644 index 865eba69a..000000000 --- a/docs/usage/esql.asciidoc +++ /dev/null @@ -1,131 +0,0 @@ -[[esql]] -=== ES|QL in the Java client - -This page helps you understand and use {ref}/esql.html[ES|QL] in the -Java client. - -There are two ways to use ES|QL in the {java-client}: - -* Use the Elasticsearch {es-docs}/esql-apis.html[ES|QL API] directly: This -is the most flexible approach, but it's also the most complex because you must handle -results in their raw form. You can choose the precise format of results, -such as JSON, CSV, or text. -* Use ES|QL mapping helpers: These mappers take care of parsing the raw -response into something readily usable by the application. Several mappers are -available for different use cases, such as object mapping, cursor -traversal of results, and dataframes. You can also define your own mapper for specific -use cases. - - - -[discrete] -[[esql-how-to]] -==== How to use the ES|QL API - -The {es-docs}/esql-query-api.html[ES|QL query API] allows you to specify how -results should be returned. You can choose a -{es-docs}/esql-rest.html#esql-rest-format[response format] such as CSV, text, or -JSON, then fine-tune it with parameters like column separators -and locale. - -Because the response varies widely depending on the format, the -{java-client} has a BinaryData object you can use according to the -format specified in the request. - -The following example gets ES|QL results as CSV and parses them: - -[source,java] ------------------------------------- -String queryAuthor = - """ - from books - | where author == "Isaac Asimov" - | sort year desc - | limit 10 - """; - -BinaryResponse response = client.esql().query(q -> q - .format("csv") - .delimiter(",") - .query(queryAuthor)); - -String result = new BufferedReader(new InputStreamReader(response.content())) - .lines().collect(Collectors.joining("\n")); ------------------------------------- - - -[discrete] -[[esql-consume-results]] -==== Consume ES|QL results - -The previous example showed that although the raw ES|QL API offers maximum -flexibility, additional work is required in order to make use of the -result data. - -To simplify things, try working with these three main representations of ES|QL -results (each with its own mapping helper): - -* **Objects**, where each row in the results is mapped to an object from your -application domain. This is similar to what ORMs (object relational mappers) -commonly do. -+ --- - -[source,java] ------------------------------------- -List queryRes = (List) client.esql().query(ObjectsEsqlAdapter.of(Book.class), queryAuthor); - ------------------------------------- --- -* **Cursors**, where you scan the results row by row and access the data using -column names. This is similar to database access libraries. -+ --- -[source,java] ------------------------------------- -ResultSet resultSet = client.esql().query(ResultSetEsqlAdapter.INSTANCE, queryAuthor); ------------------------------------- --- - - -[discrete] -[[esql-custom-mapping]] -==== Define your own mapping - -Although the mappers provided by the {java-client} cover many use cases, your -application might require a custom mapping. -You can write your own mapper and use it in a similar way as the -built-in ones. - -Note that mappers are meant to provide a more usable representation of ES|QL -results—not to process the result data. Data processing should be based on -the output of a result mapper. - -Here's an example mapper that returns a simple column-oriented -representation of the data: - -[source,java] ------------------------------------- -public class CustomStringAdapter extends EsqlAdapterBase { - - public static final CustomStringAdapter INSTANCE = new CustomStringAdapter(); - - @Override - public String format() { - return "json"; - } - - @Override - public boolean columnar() { - return true; - } - - @Override - public String deserialize(ApiClient client, QueryRequest request, - BinaryResponse response) - throws IOException { - return new BufferedReader(new InputStreamReader(response.content())) - .lines().collect(Collectors.joining("\n")); - } -} ------------------------------------- diff --git a/docs/usage/index.asciidoc b/docs/usage/index.asciidoc deleted file mode 100644 index 854ef8ab4..000000000 --- a/docs/usage/index.asciidoc +++ /dev/null @@ -1,33 +0,0 @@ -[[usage]] -== Using the Java API Client - -The sections below provide tutorials on the most frequently used and some less obvious features of {es}. - -For a full reference, see the {es-docs}/[Elasticsearch documentation] and in particular the {es-docs}/rest-apis.html[REST APIs] section. The {java-client} follows closely the JSON structures described there, using the <>. - -If you're new to Elasticsearch, make sure also to read {es-docs}/getting-started.html[Elasticsearch's quick start] that provides a good introduction. - -* <> -* <> -* <> -* <> -* <> -* <> - -//// -* <> -//// - -NOTE: This is still a work in progress, more sections will be added in the near future. - -include::esql.asciidoc[] -include::indexing.asciidoc[] -include::indexing-bulk.asciidoc[] -include::reading.asciidoc[] -include::searching.asciidoc[] -include::aggregations.asciidoc[] - -//// -include::indices.asciidoc[] -//// - diff --git a/docs/usage/indexing-bulk.asciidoc b/docs/usage/indexing-bulk.asciidoc deleted file mode 100644 index b289ed6d0..000000000 --- a/docs/usage/indexing-bulk.asciidoc +++ /dev/null @@ -1,91 +0,0 @@ -[[indexing-bulk]] -=== Bulk: indexing multiple documents - -Bulk requests allow sending multiple document-related operations to {es} in one request. When you have multiple documents to ingest, this is more efficient than sending each document with a separate request. - -A bulk request can contain several kinds of operations: - -* create a document, indexing it after ensuring it doesn't already exist, -* index a document, creating it if needed and replacing it if it exists, -* update a document that already exists in place, either with a script or a partial document, -* delete a document. - -NOTE: See the {es-docs}/docs-bulk.html[{es} API documentation] for a full explanation of bulk requests. - -[discrete] -==== Indexing application objects - -A `BulkRequest` contains a collection of operations, each operation being a <>. To create this request, it is convenient to use a builder object for the main request, and the fluent DSL for each operation. - -The example below shows how to index a list or application objects. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingBulkTest.java[bulk-objects] --------------------------------------------------- -<1> Adds an operation (remember that <>). `op` is is a builder for `BulkOperation` which is a <>. This type has `index`, `create`, `update` and `delete` variants. -<2> Selects the `index` operation variant, `idx` is a builder for `IndexOperation`. -<3> Sets the properties for the index operation, similar to <>: index name, identifier and document. - -[discrete] -[[indexing-raw-json-data]] -==== Indexing raw JSON data - -The `document` property of a bulk index request can be any object that can be serialized to JSON using your Elasticsearch client's JSON mapper. However, data that is ingested in bulk is often available as JSON text (e.g. files on disk), and parsing this JSON just to re-serialize it to send the bulk request would be a waste of resources. So documents in bulk operations can also be of type `BinaryData` that are sent verbatim (without parsing) to the {es} server. - -In the example below we will use the {java-client}'s `BinaryData` to read json files from a log directory and send them in a bulk request. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingBulkTest.java[bulk-json] --------------------------------------------------- - -[discrete] -==== Streaming ingestion with the Bulk Ingester - -The `BulkIngester` simplifies the usage of the Bulk API by providing a utility class that allows index/update/delete operations to be transparently grouped in bulk requests. You only have to `add()` bulk operations to the ingester and -it will take care of grouping and sending them in bulk according to its configuration. - -The ingester will send a bulk request when one of the following criteria is met: - -- the number of operations exceeds a maximum (defaults to 1000) -- the bulk request size in bytes exceeds a maximum (defaults to 5 MiB) -- a delay since the last request has expired (periodic flush, no default) - -Additionally, you can define a maximum number of concurrent request waiting to be executed by {es} (defaults to 1). When that maximum is reached and the maximum number of operations have been collected, adding a new operation to the indexer will block. This is avoids overloading the {es} server by putting backpressure on the client application. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingBulkTest.java[bulk-ingester-setup] --------------------------------------------------- -<1> Sets the {es} client used to send bulk requests. -<2> Sets the maximum number of operations to collect before sending a bulk request. -<3> Sets the flush interval. -<4> Adds a bulk operation to the ingester. -<5> Closes the ingester to flush the pending operations and release resources. - -Additionally, the bulk ingester accepts a listener so that your application can be notified of bulk requests that are -sent and their result. To allow correlating bulk operations to application context, the `add()` method optionally -accepts a `context` parameter. The type of this context parameter is used as the generic parameter of the `BulkIngester` -object. You may have noticed the `Void` type in `BulkIngester` above: this is because we did not register a listener, -and therefore did not care about context values. - -The following example shows how you can use context values to implement a bulk ingestion listener: as previously it -sends JSON log files in bulk, but tracks bulk request errors and failed operations. When an operation fails, depending on the error type you may want to re-add it to the ingester. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingBulkTest.java[bulk-ingester-context] --------------------------------------------------- -<1> Creates a listener where context values are strings for the ingested file name. -<2> Registers the listener on the bulk ingester. -<3> Sets the file name as the context value for a bulk operation. - -The bulk ingest also exposes statistic information that allows monitoring the ingestion process and tune its configuration: - -- number of operations added, -- number of calls to `add()` that were blocked because the maximum number of concurrent requests was reached (contention), -- number of bulk requests sent, -- number of bulk requests that were blocked because the maximum number of concurrent requests was reached. - -{doc-tests-blurb} diff --git a/docs/usage/indexing.asciidoc b/docs/usage/indexing.asciidoc deleted file mode 100644 index f6bc7446a..000000000 --- a/docs/usage/indexing.asciidoc +++ /dev/null @@ -1,61 +0,0 @@ -[[indexing]] -=== Indexing single documents - -The {java-client} offers several ways to index data: you can provide application objects that will be automatically mapped to JSON, or you can provide raw JSON data. Using application objects is more suited to applications with a well-defined domain model, whereas raw JSON is more suited to logging use cases with semi-structured data. - -In the examples below we use a `Product` domain object that has `sku`, `name` and `price` properties. - -NOTE: See the {es-docs}/docs-index_.html[{es} API documentation] for a full explanation of index requests. - -[discrete] -==== Using the fluent DSL - -The most direct way to build requests is using the fluent DSL. In the example below we index a product description in the `products` index, using the product's SKU as the document identifier in the index. The `product` object will be mapped to JSON using the object mapper configured on the {es} client. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingTest.java[single-doc-dsl] --------------------------------------------------- - -You can also assign objects created with the DSL to variables. {java-client} classes have a static `of()` method for this, that creates an object with the DSL syntax. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingTest.java[single-doc-dsl-of] --------------------------------------------------- - -[discrete] -==== Using classic builders - -If you're more used to the classic builder pattern, it is also available. Builder objects are used under the hood by the fluent DSL syntax. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingTest.java[single-doc-builder] --------------------------------------------------- - -[discrete] -==== Using the asynchronous client - -The examples above used the synchronous {es} client. All {es} APIs are also available in the asynchronous client, using the same request and response types. See also <> for additional details. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingTest.java[single-doc-dsl-async] --------------------------------------------------- - -[discrete] -==== Using raw JSON data - -When the data you want to index comes from external sources, having to create domain objects may be cumbersome or outright impossible with semi-structured data. - -You can index data from an arbitrary source using `withJson()`. Using this method will read the source and use it for the index request's `document` property. See <> for additional details. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/IndexingTest.java[single-doc-json] --------------------------------------------------- - -// TODO: Add an example with version conflict detection - -{doc-tests-blurb} diff --git a/docs/usage/indices.asciidoc b/docs/usage/indices.asciidoc deleted file mode 100644 index f5416f7e9..000000000 --- a/docs/usage/indices.asciidoc +++ /dev/null @@ -1,4 +0,0 @@ -[[indices]] -=== Creating and managing indices - -TBD diff --git a/docs/usage/reading.asciidoc b/docs/usage/reading.asciidoc deleted file mode 100644 index ab47662f3..000000000 --- a/docs/usage/reading.asciidoc +++ /dev/null @@ -1,40 +0,0 @@ -[[reading]] -=== Reading documents by id - -Elasticsearch is all about search, but you may also want to access documents directly, knowing their identifier. The "get" request is meant for this. - -NOTE: See the {es-docs}/docs-get.html[{es} API documentation] for a full explanation of get requests. - -[discrete] -==== Reading a domain object - -The example below reads the document with identifier `bk-1` from the `products` index. - -The `get` request has two parameters: - -* the first parameter is the actual request, built below with the fluent DSL -* the second parameter is the class we want the document's JSON to be mapped to. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/ReadingTest.java[get-by-id] --------------------------------------------------- -<1> The get request, with the index name and identifier. -<2> The target class, here `Product`. - -[discrete] -==== Reading raw JSON - -When your index contains semi-structured data or if you don't have a domain object definition, you can also read the document as raw JSON data. - -Raw JSON data is just another class that you can use as the result type for the get request. In the example below we use Jackson's `ObjectNode`. We could also have used any JSON representation that can be deserialized by the JSON mapper associated to the `ElasticsearchClient`. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/ReadingTest.java[get-by-id-json] --------------------------------------------------- -<1> The target class is a raw JSON object. - -{doc-tests-blurb} - - diff --git a/docs/usage/searching.asciidoc b/docs/usage/searching.asciidoc deleted file mode 100644 index c87ff3e75..000000000 --- a/docs/usage/searching.asciidoc +++ /dev/null @@ -1,71 +0,0 @@ -[[searching]] -=== Searching for documents - -Indexed documents are available for search in near real-time. - -NOTE: See the {es} documentation for a full explanation of search requests: {es-docs}/search-your-data.html[search your data], {es-docs}/search-your-data.html[the query DSL], and {es-docs}/search.html[search APIs]. - -[discrete] -==== Simple search query - -There are many types of search queries that can be combined. We will start with the simple text match query, searching for bikes in the `products` index. - -The search result has a `hits` properties that contains the documents that matched the query along with information about the total number of matches that exist in the index. - -The total value comes with a relation that indicates if the total is exact (`eq` -- equal) or approximate (`gte` -- greater than or equal). - -Each returned document comes with its relevance score and additional information about its location in the index. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/SearchingTest.java[search-simple] --------------------------------------------------- -<1> Name of the index we want to search. -<2> The query part of the search request (a search request can also have other components like <>). -<3> Choose a query variant among the many available. We choose here the match query (full text search). -<4> Configure the match query: we search for a term in the `name` field. -<5> The target class for the matching documents. We use `Product` here, just like in <> examples. - -Similarly to <> operations, you can fetch documents matching your query as raw JSON by using a corresponding target class instead of `Product`, like JSON-P's `JsonValue` or Jackson's `ObjectNode`. - -[discrete] -==== Nested search queries - -{es} allows individual queries to be combined to build more complex search requests. In the example below we will search for bikes with a maximum price of 200. - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/SearchingTest.java[search-nested] --------------------------------------------------- -<1> We're creating the queries for individual criteria separately. -<2> A `MatchQuery` is a query _variant_ that we have to turn into the `Query` _union type_. See <> for additional details. -<3> {es} range query accepts a large range of value types. We create here a JSON representation of the maximum price. -<4> The search query is a boolean query that combines the text search and max price queries. -<5> Both queries are added as `must` as we want results to match all criteria. - -[discrete] -==== Templated search - -A search template is a stored search that you can run with different variables. Search templates let you change your searches without modifying your application code. - -Before running a template search, you first have to create the template. This is a stored script that returns the search request body, and is usually defined as a Mustache template. This stored script can be created outside the application, and also with the {java-client}: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/SearchingTest.java[search-template-script] --------------------------------------------------- -<1> Identifier of the template script to create. - -To use the search template, use the `searchTemplate` method to refer to the script and provide values for its parameters: - -["source","java"] --------------------------------------------------- -include-tagged::{doc-tests-src}/usage/SearchingTest.java[search-template-query] --------------------------------------------------- -<1> Identifier of the template script to use. -<2> Template parameter values. - -For more in-depth information, see the {es-docs}/search-template.html[{es} search template documentation]. - - -{doc-tests-blurb}