From 3cbde5d21d2e478500d7e82212f8bb05655bd5a2 Mon Sep 17 00:00:00 2001 From: Florian Bernd Date: Mon, 26 May 2025 16:41:58 +0200 Subject: [PATCH] Update documentation (#8537) --- README.md | 30 +- ..._options_on_elasticsearchclientsettings.md | 13 +- docs/reference/aggregations.md | 155 +++++----- docs/reference/configuration.md | 2 - docs/reference/connecting.md | 14 +- docs/reference/examples.md | 32 +- docs/reference/getting-started.md | 31 +- docs/reference/index.md | 25 +- docs/reference/installation.md | 29 +- docs/reference/mappings.md | 25 +- docs/reference/migration-guide.md | 222 -------------- docs/reference/query.md | 50 ++-- docs/reference/recommendations.md | 3 - docs/reference/serialization.md | 2 - docs/reference/source-serialization.md | 278 ++++++------------ docs/reference/toc.yml | 1 - docs/reference/using-net-client.md | 4 +- 17 files changed, 264 insertions(+), 652 deletions(-) delete mode 100644 docs/reference/migration-guide.md diff --git a/README.md b/README.md index db52376d46..f783ecb779 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,6 @@ Repository for **Elastic.Clients.Elasticsearch** the official .NET client for [Elasticsearch](https://github.com/elastic/elasticsearch). -*Older branches include both previous clients, **NEST** and **Elasticsearch.Net**.* **[Download the latest version of Elasticsearch](https://www.elastic.co/downloads/elasticsearch)** or @@ -61,40 +60,13 @@ for comprehensive information on installation, configuration and usage. The API reference documentation is available [here](https://elastic.github.io/elasticsearch-net). -## Versions - -### Elasticsearch 8.x Clusters - -We have released the next generation of the .NET client for Elasticsearch, which -aligns with v8 of Elasticsearch. We have renamed this library -`Elastic.Clients.Elasticsearch`, and the packages are published on -[NuGet](https://www.nuget.org/packages/Elastic.Clients.Elasticsearch/). The -8.0.x versions do not offer complete feature parity with the existing `NEST` -client. We therefore recommend you thoroughly review our -[release notes and migration guidance](https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/release-notes-8.0.0.html) -before attempting to migrate existing applications to the -`Elastic.Clients.Elasticsearch` library. - -Until the new client supports all endpoints and features your application -requires, you may continue to use the latest `7.17.x` client to communicate with -Elasticsearch v8 servers. Please review -[our documentation](https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html), -which describes how to enable compatibility mode and secure communications with -a v8 cluster. - -### Elasticsearch 7.x Clusters - -We recommend using the latest `7.17.x` -[NEST client](https://www.nuget.org/packages/Nest) to communicate with -Elasticsearch v7 servers. - ## Contributing See [CONTRIBUTING.md](./CONTRIBUTING.md) ## Copyright and License -This software is Copyright (c) 2014-2022 by Elasticsearch BV. +This software is Copyright (c) 2014-2025 by Elasticsearch BV. This is free software, licensed under [The Apache License Version 2.0](https://github.com/elastic/elasticsearch-net/blob/main/LICENSE.txt). \ No newline at end of file diff --git a/docs/reference/_options_on_elasticsearchclientsettings.md b/docs/reference/_options_on_elasticsearchclientsettings.md index 49d5f904bd..d4d01ef8ac 100644 --- a/docs/reference/_options_on_elasticsearchclientsettings.md +++ b/docs/reference/_options_on_elasticsearchclientsettings.md @@ -20,7 +20,6 @@ The following is a list of available connection configuration options on `Elasti `Base64ApiKey` for Elastic Cloud style encoded api keys ``` - `ClientCertificate` : Use the following certificates to authenticate all HTTP requests. You can also set them on individual request using `ClientCertificates`. @@ -34,7 +33,6 @@ The following is a list of available connection configuration options on `Elasti For Core CLR, this setting applies to the `MaxConnectionsPerServer` property on the `HttpClientHandler` instances used by the `HttpClient` inside the default `IConnection` implementation. - `DeadTimeout` : The time to put dead nodes out of rotation (this will be multiplied by the number of times they’ve been dead). @@ -43,13 +41,11 @@ The following is a list of available connection configuration options on `Elasti The client by default will use the value of a property named `Id` on a CLR type as the `_id` to send to {{es}}. Adding a type will disable this behaviour for that CLR type. If `Id` inference should be disabled for all CLR types, use `DefaultDisableIdInference`. - `DefaultFieldNameInferrer` : Specifies how field names are inferred from CLR property names. By default, the client camel cases property names. For example, CLR property `EmailAddress` will be inferred as "emailAddress" {{es}} document field name. - `DefaultIndex` : The default index to use for a request when no index has been explicitly specified and no default indices are specified for the given CLR type specified for the request. @@ -82,7 +78,6 @@ The following is a list of available connection configuration options on `Elasti For Desktop CLR, sets `ServicePointManager`.`SetTcpKeepAlive`. - `EnableTcpStats` : Enable statistics about TCP connections to be collected when making a request. @@ -110,8 +105,8 @@ The following is a list of available connection configuration options on `Elasti `OnRequestCompleted` : Allows you to register a callback every time a an API call is returned. -`OnRequestDataCreated` -: An action to run when the `RequestData` for a request has been created. +`OnBeforeRequest` +: An action to run before a request is made. `PingTimeout` : The timeout in milliseconds to use for ping requests, which are issued to determine whether a node is alive. @@ -121,7 +116,6 @@ The following is a list of available connection configuration options on `Elasti Note: this is not a guarantee you will always get prettified json. - `Proxy` : If your connection has to go through proxy, use this method to specify the proxy url. @@ -148,7 +142,6 @@ The following is a list of available connection configuration options on `Elasti Reasons for such exceptions could be search parser errors, index missing exceptions, and so on. - `TransferEncodingChunked` : Whether the request should be sent with chunked Transfer-Encoding. @@ -171,5 +164,3 @@ var settings= new ElasticsearchClientSettings() var client = new ElasticsearchClient(settings); ``` - - diff --git a/docs/reference/aggregations.md b/docs/reference/aggregations.md index 8a3ca8e832..0717ddd3a6 100644 --- a/docs/reference/aggregations.md +++ b/docs/reference/aggregations.md @@ -7,49 +7,50 @@ mapped_pages: This page demonstrates how to use aggregations. - ## Top-level aggreggation [_top_level_aggreggation] - ### Fluent API [_fluent_api] ```csharp -var response = await client - .SearchAsync(search => search - .Index("persons") - .Query(query => query - .MatchAll(_ => {}) - ) - .Aggregations(aggregations => aggregations - .Add("agg_name", aggregation => aggregation - .Max(max => max - .Field(x => x.Age) - ) - ) - ) - .Size(10) - ); +var response = await client.SearchAsync(search => search + .Indices("persons") + .Query(query => query + .MatchAll() + ) + .Aggregations(aggregations => aggregations + .Add("agg_name", aggregation => aggregation + .Max(max => max + .Field(x => x.Age) + ) + ) + ) + .Size(10) +); ``` - ### Object initializer API [_object_initializer_api] ```csharp var response = await client.SearchAsync(new SearchRequest("persons") { - Query = Query.MatchAll(new MatchAllQuery()), - Aggregations = new Dictionary - { - { "agg_name", Aggregation.Max(new MaxAggregation - { - Field = Infer.Field(x => x.Age) - })} - }, - Size = 10 + Query = new Query + { + MatchAll = new MatchAllQuery() + }, + Aggregations = new Dictionary + { + { "agg_name", new Aggregation + { + Max = new MaxAggregation + { + Field = Infer.Field(x => x.Age) + } + }} + }, + Size = 10 }); ``` - ### Consume the response [_consume_the_response] ```csharp @@ -57,74 +58,76 @@ var max = response.Aggregations!.GetMax("agg_name")!; Console.WriteLine(max.Value); ``` - ## Sub-aggregation [_sub_aggregation] - ### Fluent API [_fluent_api_2] ```csharp -var response = await client - .SearchAsync(search => search - .Index("persons") - .Query(query => query - .MatchAll(_ => {}) - ) - .Aggregations(aggregations => aggregations - .Add("firstnames", aggregation => aggregation - .Terms(terms => terms - .Field(x => x.FirstName) - ) - .Aggregations(aggregations => aggregations - .Add("avg_age", aggregation => aggregation - .Max(avg => avg - .Field(x => x.Age) - ) - ) - ) - ) - ) - .Size(10) - ); +var response = await client.SearchAsync(search => search + .Indices("persons") + .Query(query => query + .MatchAll(_ => {}) + ) + .Aggregations(aggregations => aggregations + .Add("firstnames", aggregation => aggregation <1> + .Terms(terms => terms + .Field(x => x.FirstName) + ) + .Aggregations(aggregations => aggregations + .Add("avg_age", aggregation => aggregation <2> + .Max(avg => avg + .Field(x => x.Age) + ) + ) + ) + ) + ) + .Size(10) +); ``` +1. The top level `Terms` aggregation with name `firstnames`. +2. Nested aggregation of type `Max` with name `avg_age`. ### Object initializer API [_object_initializer_api_2] ```csharp -var topLevelAggregation = Aggregation.Terms(new TermsAggregation -{ - Field = Infer.Field(x => x.FirstName) -}); - -topLevelAggregation.Aggregations = new Dictionary +var response = await client.SearchAsync(new SearchRequest { - { "avg_age", new MaxAggregation - { - Field = Infer.Field(x => x.Age) - }} -}; - -var response = await client.SearchAsync(new SearchRequest("persons") -{ - Query = Query.MatchAll(new MatchAllQuery()), - Aggregations = new Dictionary - { - { "firstnames", topLevelAggregation} - }, - Size = 10 + Query = new Query + { + MatchAll = new MatchAllQuery() + }, + Aggregations = new Dictionary + { + { "firstnames", new Aggregation + { + Terms = new TermsAggregation + { + Field = Infer.Field(x => x.FirstName) + }, + Aggregations = new Dictionary + { + { "avg_age", new Aggregation + { + Max = new MaxAggregation + { + Field = Infer.Field(x => x.Age) + } + }} + } + }} + } }); ``` - ### Consume the response [_consume_the_response_2] ```csharp var firstnames = response.Aggregations!.GetStringTerms("firstnames")!; foreach (var bucket in firstnames.Buckets) { - var avg = bucket.Aggregations.GetAverage("avg_age")!; - Console.WriteLine($"The average age for persons named '{bucket.Key}' is {avg}"); + var avg = bucket.Aggregations.GetAverage("avg_age")!; + Console.WriteLine($"The average age for persons named '{bucket.Key}' is {avg}"); } ``` - diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 0577876926..a52f233366 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -6,5 +6,3 @@ mapped_pages: # Configuration [configuration] Connecting to {{es}} with the client is easy, but it’s possible that you’d like to change the default connection behaviour. There are a number of configuration options available on `ElasticsearchClientSettings` that can be used to control how the client interact with {{es}}. - - diff --git a/docs/reference/connecting.md b/docs/reference/connecting.md index 5c7fc326ca..02bde82d9e 100644 --- a/docs/reference/connecting.md +++ b/docs/reference/connecting.md @@ -10,10 +10,10 @@ This page contains the information you need to create an instance of the .NET Cl It’s possible to connect to your {{es}} cluster via a single node, or by specifying multiple nodes using a node pool. Using a node pool has a few advantages over a single node, such as load balancing and cluster failover support. The client provides convenient configuration options to connect to an Elastic Cloud deployment. ::::{important} -Client applications should create a single instance of `ElasticsearchClient` that is used throughout your application for its entire lifetime. Internally the client manages and maintains HTTP connections to nodes, reusing them to optimize performance. If you use a dependency injection container for your application, the client instance should be registered with a singleton lifetime. -:::: +Client applications should create a single instance of `ElasticsearchClient` that is used throughout your application for its entire lifetime. Internally the client manages and maintains HTTP connections to nodes, reusing them to optimize performance. If you use a dependency injection container for your application, the client instance should be registered with a singleton lifetime. +:::: ## Connecting to a cloud deployment [cloud-deployment] @@ -36,8 +36,6 @@ var client = new ElasticsearchClient("", new ApiKey("")); <1> 1. Replace the placeholder string values above with your cloud ID and the API key configured for your application to access your deployment. - - ## Connecting to a single node [single-node] Single node configuration is best suited to connections to a multi-node cluster running behind a load balancer or reverse proxy, which is exposed via a single URL. It may also be convenient to use a single node during local application development. If the URL represents a single {{es}} node, be aware that this offers no resiliency should the server be unreachable or unresponsive. @@ -92,7 +90,6 @@ var client = new ElasticsearchClient(settings); The preceding snippet demonstrates configuring the client to authenticate by providing a username and password with basic authentication. If preferred, you may also use `ApiKey` authentication as shown in the cloud connection example. - ## Connecting to multiple nodes using a node pool [multiple-nodes] To provide resiliency, you should configure multiple nodes for your cluster to which the client attempts to communicate. By default, the client cycles through nodes for each request in a round robin fashion. The client also tracks unhealthy nodes and avoids sending requests to them until they become healthy. @@ -107,9 +104,9 @@ using Elastic.Transport; var nodes = new Uri[] { - new Uri("https://myserver1:9200"), - new Uri("https://myserver2:9200"), - new Uri("https://myserver3:9200") + new Uri("https://myserver1:9200"), + new Uri("https://myserver2:9200"), + new Uri("https://myserver3:9200") }; var pool = new StaticNodePool(nodes); @@ -120,4 +117,3 @@ var settings = new ElasticsearchClientSettings(pool) var client = new ElasticsearchClient(settings); ``` - diff --git a/docs/reference/examples.md b/docs/reference/examples.md index 67cc8034ae..354143d0d6 100644 --- a/docs/reference/examples.md +++ b/docs/reference/examples.md @@ -10,15 +10,13 @@ This page helps you to understand how to perform various basic {{es}} CRUD (crea These examples assume you have an instance of the `ElasticsearchClient` accessible via a local variable named `client` and several using directives in your C# file. ```csharp -using System; using Elastic.Clients.Elasticsearch; -using Elastic.Clients.Elasticsearch.QueryDsl; + var client = new ElasticsearchClient(); <1> ``` 1. The default constructor, assumes an unsecured {{es}} server is running and exposed on *http://localhost:9200*. See [connecting](/reference/connecting.md) for examples of connecting to secured servers and [Elastic Cloud](https://www.elastic.co/cloud) deployments. - The examples operate on data representing tweets. Tweets are modelled in the client application using a C# class named *Tweet* containing several properties that map to the document structure being stored in {{es}}. ```csharp @@ -33,8 +31,6 @@ public class Tweet 1. By default, the .NET client will try to find a property called `Id` on the class. When such a property is present it will index the document into {{es}} using the ID specified by the value of this property. - - ## Indexing a document [indexing-net] Documents can be indexed by creating an instance representing a tweet and indexing it via the client. In these examples, we will work with an index named *my-tweet-index*. @@ -48,7 +44,7 @@ var tweet = new Tweet <1> Message = "Trying out the client, so far so good?" }; -var response = await client.IndexAsync(tweet, "my-tweet-index"); <2> +var response = await client.IndexAsync(tweet, x => x.Index("my-tweet-index")); <2> if (response.IsValidResponse) <3> { @@ -61,12 +57,10 @@ if (response.IsValidResponse) <3> 3. Check the `IsValid` property on the response to confirm that the request and operation succeeded. 4. Access the `IndexResponse` properties, such as the ID, if necessary. - - ## Getting a document [getting-net] ```csharp -var response = await client.GetAsync(1, idx => idx.Index("my-tweet-index")); <1> +var response = await client.GetAsync(1, x => x.Index("my-tweet-index")); <1> if (response.IsValidResponse) { @@ -77,19 +71,20 @@ if (response.IsValidResponse) 1. The `GetResponse` is mapped 1-to-1 with the Elasticsearch JSON response. 2. The original document is deserialized as an instance of the Tweet class, accessible on the response via the `Source` property. - - ## Searching for documents [searching-net] The client exposes a fluent interface and a powerful query DSL for searching. ```csharp var response = await client.SearchAsync(s => s <1> - .Index("my-tweet-index") <2> + .Indices("my-tweet-index") <2> .From(0) .Size(10) .Query(q => q - .Term(t => t.User, "stevejgordon") <3> + .Term(t => t <3> + .Field(x => x.User) + .Value("stevejgordon") + ) ) ); @@ -104,7 +99,6 @@ if (response.IsValidResponse) 3. Execute a term query against the `user` field, searching for tweets authored by the user *stevejgordon*. 4. Documents matched by the query are accessible via the `Documents` collection property on the `SearchResponse`. - You may prefer using the object initializer syntax for requests if lambdas aren’t your thing. ```csharp @@ -112,7 +106,7 @@ var request = new SearchRequest("my-tweet-index") <1> { From = 0, Size = 10, - Query = new TermQuery("user") { Value = "stevejgordon" } + Query = new Query { Term = new TermQuery { Field = "user", Value = "stevejgordon" } } }; var response = await client.SearchAsync(request); <2> @@ -126,8 +120,6 @@ if (response.IsValidResponse) 1. Create an instance of `SearchRequest`, setting properties to control the search operation. 2. Pass the request to the `SearchAsync` method on the client. - - ## Updating documents [updating-net] Documents can be updated in several ways, including by providing a complete replacement for an existing document ID. @@ -136,7 +128,8 @@ Documents can be updated in several ways, including by providing a complete repl tweet.Message = "This is a new message"; <1> var response = await client.UpdateAsync("my-tweet-index", 1, u => u - .Doc(tweet)); <2> + .Doc(tweet) +); <2> if (response.IsValidResponse) { @@ -147,8 +140,6 @@ if (response.IsValidResponse) 1. Update a property on the existing tweet instance. 2. Send the updated tweet object in the update request. - - ## Deleting documents [deleting-net] Documents can be deleted by providing the ID of the document to remove. @@ -161,4 +152,3 @@ if (response.IsValidResponse) Console.WriteLine("Delete document succeeded."); } ``` - diff --git a/docs/reference/getting-started.md b/docs/reference/getting-started.md index 63b5d34776..7291920dc7 100644 --- a/docs/reference/getting-started.md +++ b/docs/reference/getting-started.md @@ -8,12 +8,10 @@ mapped_pages: This page guides you through the installation process of the .NET client, shows you how to instantiate the client, and how to perform basic Elasticsearch operations with it. - ### Requirements [_requirements] * .NET Core, .NET 5+ or .NET Framework (4.6.1 and higher). - ### Installation [_installation] To install the latest version of the client for SDK style projects, run the following command: @@ -24,7 +22,6 @@ dotnet add package Elastic.Clients.Elasticsearch Refer to the [*Installation*](/reference/installation.md) page to learn more. - ### Connecting [_connecting] You can connect to the Elastic Cloud using an API key and the Elasticsearch endpoint. @@ -47,7 +44,6 @@ You can generate an API key on the **Management** page under Security. For other connection options, refer to the [*Connecting*](/reference/connecting.md) section. - ### Operations [_operations] Time to use Elasticsearch! This section walks you through the basic, and most important, operations of Elasticsearch. For more operations and more advanced examples, refer to the [*CRUD usage examples*](/reference/examples.md) page. @@ -61,7 +57,6 @@ This is how you create the `my_index` index: var response = await client.Indices.CreateAsync("my_index"); ``` - #### Indexing documents [_indexing_documents] This is a simple way of indexing a document: @@ -74,16 +69,15 @@ var doc = new MyDoc Message = "Trying out the client, so far so good?" }; -var response = await client.IndexAsync(doc, "my_index"); +var response = await client.IndexAsync(doc, x => x.Index("my_index")); ``` - #### Getting documents [_getting_documents] You can get documents by using the following code: ```csharp -var response = await client.GetAsync(id, idx => idx.Index("my_index")); +var response = await client.GetAsync("my-index", id); if (response.IsValidResponse) { @@ -91,18 +85,20 @@ if (response.IsValidResponse) } ``` - #### Searching documents [_searching_documents] This is how you can create a single match query with the .NET client: ```csharp -var response = await client.SearchAsync(s => s - .Index("my_index") +var response = await client.SearchAsync(s => s + .Indices("my_index") .From(0) .Size(10) .Query(q => q - .Term(t => t.User, "flobernd") + .Term(t => t + .Field(x => x.User) + .Value("flobernd") + ) ) ); @@ -112,7 +108,6 @@ if (response.IsValidResponse) } ``` - #### Updating documents [_updating_documents] This is how you can update a document, for example to add a new field: @@ -120,25 +115,23 @@ This is how you can update a document, for example to add a new field: ```csharp doc.Message = "This is a new message"; -var response = await client.UpdateAsync("my_index", 1, u => u - .Doc(doc)); +var response = await client.UpdateAsync("my_index", id, u => u + .Doc(doc) +); ``` - #### Deleting documents [_deleting_documents] ```csharp -var response = await client.DeleteAsync("my_index", 1); +var response = await client.DeleteAsync("my_index", id); ``` - #### Deleting an index [_deleting_an_index] ```csharp var response = await client.Indices.DeleteAsync("my_index"); ``` - ## Further reading [_further_reading] * Refer to the [*Usage recommendations*](/reference/recommendations.md) page to learn more about how to use the client the most efficiently. diff --git a/docs/reference/index.md b/docs/reference/index.md index 90ef20d616..d2170791cb 100644 --- a/docs/reference/index.md +++ b/docs/reference/index.md @@ -10,10 +10,7 @@ mapped_pages: Designed for .NET application developers, the .NET language client library provides a strongly typed API and query DSL for interacting with {{es}}. The .NET client includes higher-level abstractions, such as helpers for coordinating bulk indexing and update operations. It also comes with built-in, configurable cluster failover retry mechanisms. -The {{es}} .NET client is available as a [NuGet](https://www.nuget.org/packages/Elastic.Clients.Elasticsearch) package for use with .NET Core, .NET 5+, and .NET Framework (4.6.1 and later) applications. - -*NOTE: This documentation covers the v8 .NET client for {{es}}, for use with {{es}} 8.x versions. To develop applications targeting {{es}} v7, use the [v7 (NEST) client](https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17).* - +The {{es}} .NET client is available as a [NuGet](https://www.nuget.org/packages/Elastic.Clients.Elasticsearch) package ## Features [features] @@ -30,15 +27,29 @@ The .NET {{es}} client is built on the Elastic Transport library, which provides * Connection management and load balancing across all available nodes. * Request retries and dead connections handling. - ## {{es}} version compatibility [_es_version_compatibility] -Language clients are forward compatible: clients support communicating with current and later minor versions of {{es}}. {{es}} language clients are backward compatible with default distributions only and without guarantees. +Language clients are **forward compatible**: + +Given a constant major version of the client, each related minor version is compatible with its equivalent- and all later {{es}} minor versions of the **same or next higher** major version. + +For example: +| Client Version | Compatible with {{es}} `8.x` | Compatible with {{es}} `9.x` | Compatible with {{es}} `10.x` | +| ---: | :-- | :-- | :-- | +| 9.x | ❌ no | ✅ yes | ✅ yes | +| 8.x | ✅ yes | ✅ yes | ❌ no | + +Language clients are also **backward compatible** across minor versions within the **same** major version (without strong guarantees), but **never** backward compatible with earlier {{es}} major versions. + +:::{note} + +Compatibility does not imply feature parity. For example, an `8.12` client is compatible with `8.13`, but does not support any of the new features introduced in {{es}} `8.13`. + +::: ## Questions, bugs, comments, feature requests [_questions_bugs_comments_feature_requests] To submit a bug report or feature request, use [GitHub issues](https://github.com/elastic/elasticsearch-net/issues). For more general questions and comments, try the community forum on [discuss.elastic.co](https://discuss.elastic.co/c/elasticsearch). Mention `.NET` in the title to indicate the discussion topic. - diff --git a/docs/reference/installation.md b/docs/reference/installation.md index 8fbb2dcd2e..bcdad1b991 100644 --- a/docs/reference/installation.md +++ b/docs/reference/installation.md @@ -7,12 +7,6 @@ mapped_pages: This page shows you how to install the .NET client for {{es}}. -::::{important} -The v8 client for .NET does not have complete feature parity with the v7 `NEST` client. It may not be suitable for for all applications until additional endpoints and features are supported. We therefore recommend you thoroughly review our [release notes](/release-notes/index.md) before attempting to migrate existing applications to the `Elastic.Clients.Elasticsearch` library. Until the new client supports all endpoints and features your application requires, you may continue to use the 7.17.x [NEST](https://www.nuget.org/packages/NEST) client to communicate with v8 Elasticsearch servers using compatibility mode. Refer to the [Connecting to Elasticsearch v8.x using the v7.17.x client documentation](https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html) for guidance on configuring the 7.17.x client. -:::: - - - ## Installing the .NET client [dot-net-client] For SDK style projects, you can install the {{es}} client by running the following .NET CLI command in your terminal: @@ -46,22 +40,23 @@ To learn how to connect the {{es}} client, refer to the [Connecting](/reference/ The {{es}} client is compatible with currently maintained .NET runtime versions. Compatibility with End of Life (EOL) .NET runtimes is not guaranteed or supported. -Language clients are forward compatible; meaning that the clients support communicating with greater or equal minor versions of {{es}} without breaking. It does not mean that the clients automatically support new features of newer {{es}} versions; it is only possible after a release of a new client version. For example, a 8.12 client version won’t automatically support the new features of the 8.13 version of {{es}}, the 8.13 client version is required for that. {{es}} language clients are only backwards compatible with default distributions and without guarantees made. +Language clients are **forward compatible**: -| Elasticsearch Version | Elasticsearch-NET Branch | Supported | -| --- | --- | --- | -| main | main | | -| 8.x | 8.x | 8.x | -| 7.x | 7.x | 7.17 | +Given a constant major version of the client, each related minor version is compatible with its equivalent- and all later {{es}} minor versions of the **same or next higher** major version. -Refer to the [end-of-life policy](https://www.elastic.co/support/eol) for more information. +For example: +| Client Version | Compatible with {{es}} `8.x` | Compatible with {{es}} `9.x` | Compatible with {{es}} `10.x` | +| ---: | :-- | :-- | :-- | +| 9.x | ❌ no | ✅ yes | ✅ yes | +| 8.x | ✅ yes | ✅ yes | ❌ no | -## CI feed [ci-feed] +Language clients are also **backward compatible** across minor versions within the **same** major version (without strong guarantees), but **never** backward compatible with earlier {{es}} major versions. -We publish CI builds of our client packages, including the latest unreleased features. If you want to experiment with the latest bits, you can add the CI feed to your list of NuGet package sources. +:::{note} -Feed URL: [https://f.feedz.io/elastic/all/nuget/index.json](https://f.feedz.io/elastic/all/nuget/index.json) +Compatibility does not imply feature parity. For example, an `8.12` client is compatible with `8.13`, but does not support any of the new features introduced in {{es}} `8.13`. -We do not recommend using CI builds for production applications as they are not formally supported until they are released. +::: +Refer to the [end-of-life policy](https://www.elastic.co/support/eol) for more information. diff --git a/docs/reference/mappings.md b/docs/reference/mappings.md index b28b8ba4a5..24c35a31cd 100644 --- a/docs/reference/mappings.md +++ b/docs/reference/mappings.md @@ -7,18 +7,17 @@ mapped_pages: This page demonstrates how to configure custom mappings on an index. - ## Configure mappings during index creation [_configure_mappings_during_index_creation] ```csharp await client.Indices.CreateAsync(index => index - .Index("index") - .Mappings(mappings => mappings - .Properties(properties => properties - .IntegerNumber(x => x.Age!) - .Keyword(x => x.FirstName!, keyword => keyword.Index(false)) - ) - ) + .Index("index") + .Mappings(mappings => mappings + .Properties(properties => properties + .IntegerNumber(x => x.Age!) + .Keyword(x => x.FirstName!, keyword => keyword.Index(false)) + ) + ) ); ``` @@ -27,11 +26,11 @@ await client.Indices.CreateAsync(index => index ```csharp await client.Indices.PutMappingAsync(mappings => mappings - .Indices("index") - .Properties(properties => properties - .IntegerNumber(x => x.Age!) - .Keyword(x => x.FirstName!, keyword => keyword.Index(false)) - ) + .Indices("index") + .Properties(properties => properties + .IntegerNumber(x => x.Age!) + .Keyword(x => x.FirstName!, keyword => keyword.Index(false)) + ) ); ``` diff --git a/docs/reference/migration-guide.md b/docs/reference/migration-guide.md deleted file mode 100644 index 43a4bcec67..0000000000 --- a/docs/reference/migration-guide.md +++ /dev/null @@ -1,222 +0,0 @@ ---- -mapped_pages: - - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/migration-guide.html ---- - -# Migration guide: From NEST v7 to .NET Client v8 [migration-guide] - -The following migration guide explains the current state of the client, missing features, breaking changes and our rationale for some of the design choices we have introduced. - - -## Version 8 is a refresh [_version_8_is_a_refresh] - -::::{important} -It is important to highlight that v8 of the Elasticsearch .NET Client represents a new start for the client design. It is important to review how this may affect your code and usage. - -:::: - - -Mature code becomes increasingly hard to maintain over time. Major releases allow us to simplify and better align our language clients with each other in terms of design. It is crucial to find the right balance between uniformity across programming languages and the idiomatic concerns of each language. For .NET, we typically compare and contrast with [Java](https://github.com/elastic/elasticsearch-java) and [Go](https://github.com/elastic/go-elasticsearch) to make sure that our approach is equivalent for each of these. We also take heavy inspiration from Microsoft framework design guidelines and the conventions of the wider .NET community. - - -### New Elastic.Clients.Elasticsearch NuGet package [_new_elastic_clients_elasticsearch_nuget_package] - -We have shipped the new code-generated client as a [NuGet package](https://www.nuget.org/packages/Elastic.Clients.Elasticsearch/) with a new root namespace, `Elastic.Clients.Elasticsearch`. The v8 client is built upon the foundations of the v7 `NEST` client, but there are changes. By shipping as a new package, the expectation is that migration can be managed with a phased approach. - -While this is a new package, we have aligned the major version (v8.x.x) with the supported {{es}} server version to clearly indicate the client/server compatibility. The v8 client is designed to work with version 8 of {{es}}. - -The v7 `NEST` client continues to be supported but will not gain new features or support for new {{es}} endpoints. It should be considered deprecated in favour of the new client. - - -### Limited feature set [_limited_feature_set] - -::::{warning} -The version 8 Elasticsearch .NET Client does not have feature parity with the previous v7 `NEST` high-level client. - -:::: - - -If a feature you depend on is missing (and not explicitly documented below as a feature that we do not plan to reintroduce), open [an issue](https://github.com/elastic/elasticsearch-net/issues/new/choose) or comment on a relevant existing issue to highlight your need to us. This will help us prioritise our roadmap. - - -## Code generation [_code_generation] - -Given the size of the {{es}} API surface today, it is no longer practical to maintain thousands of types (requests, responses, queries, aggregations, etc.) by hand. To ensure consistent, accurate, and timely alignment between language clients and {{es}}, the 8.x clients, and many of the associated types are now automatically code-generated from a [shared specification](https://github.com/elastic/elasticsearch-specification). This is a common solution to maintaining alignment between client and server among SDKs and libraries, such as those for Azure, AWS and the Google Cloud Platform. - -Code-generation from a specification has inevitably led to some differences between the existing v7 `NEST` types and those available in the new v7 Elasticsearch .NET Client. For version 8, we generate strictly from the specification, special casing a few areas to improve usability or to align with language idioms. - -The base type hierarchy for concepts such as `Properties`, `Aggregations` and `Queries` is no longer present in generated code, as these arbitrary groupings do not align with concrete concepts of the public server API. These considerations do not preclude adding syntactic sugar and usability enhancements to types in future releases on a case-by-case basis. - - -## Elastic.Transport [_elastic_transport] - -The .NET client includes a transport layer responsible for abstracting HTTP concepts and to provide functionality such as our request pipeline. This supports round-robin load-balancing of requests to nodes, pinging failed nodes and sniffing the cluster for node roles. - -In v7, this layer shipped as `Elasticsearch.Net` and was considered our low-level client which could be used to send and receive raw JSON bytes between the client and server. - -As part of the work for 8.0.0, we have moved the transport layer out into a [new dedicated package](https://www.nuget.org/packages/Elastic.Transport) and [repository](https://github.com/elastic/elastic-transport-net), named `Elastic.Transport`. This supports reuse across future clients and allows consumers with extremely high-performance requirements to build upon this foundation. - - -## System.Text.Json for serialization [_system_text_json_for_serialization] - -The v7 `NEST` high-level client used an internalized and modified version of [Utf8Json](https://github.com/neuecc/Utf8Json) for request and response serialization. This was introduced for its performance improvements over [Json.NET](https://www.newtonsoft.com/json), the more common JSON framework at the time. - -While Utf8Json provides good value, we have identified minor bugs and performance issues that have required maintenance over time. Some of these are hard to change without more significant effort. This library is no longer maintained, and any such changes cannot easily be contributed back to the original project. - -With .NET Core 3.0, Microsoft shipped new [System.Text.Json APIs](https://devblogs.microsoft.com/dotnet/try-the-new-system-text-json-apis) that are included in-the-box with current versions of .NET. We have adopted `System.Text.Json` for all serialization. Consumers can still define and register their own `Serializer` implementation for their document types should they prefer to use a different serialization library. - -By adopting `System.Text.Json`, we now depend on a well-maintained and supported library from Microsoft. `System.Text.Json` is designed from the ground up to support the latest performance optimizations in .NET and, as a result, provides both fast and low-allocation serialization. - - -## Mockability of ElasticsearchClient [_mockability_of_elasticsearchclient] - -Testing code is an important part of software development. We recommend that consumers prefer introducing an abstraction for their use of the Elasticsearch .NET Client as the prefered way to decouple consuming code from client types and support unit testing. - -To support user testing scenarios, we have unsealed the `ElasticsearchClient` type and made its methods virtual. This supports mocking the type directly for unit testing. This is an improvement over the original `IElasticClient` interface from `NEST` (v7) which only supported mocking of top-level client methods. - -We have also introduced a `TestableResponseFactory` in `Elastic.Transport` to make it easier to create response instances with specific status codes and validity that can be used during unit testing. - -These changes are in addition to our existing support for testing with an `InMemoryConnection`, virtualized clusters and with our [`Elastic.Elasticsearch.Managed`](https://github.com/elastic/elasticsearch-net-abstractions/blob/master/src/Elastic.Elasticsearch.Managed) library for integration testing against real {{es}} instances. - - -## Migrating to Elastic.Clients.Elasticsearch [_migrating_to_elastic_clients_elasticsearch] - -::::{warning} -The version 8 client does not currently have full-feature parity with `NEST`. The client primary use case is for application developers communicating with {{es}}. - -:::: - - -The version 8 client focuses on core endpoints, more specifically for common CRUD scenarios. The intention is to reduce the feature gap in subsequent versions. Review this documentation carefully to learn about the missing features and reduced API surface details before migrating from the v7 `NEST` client! - -The choice to code-generate a new evolution of the Elasticsearch .NET Client introduces some significant breaking changes. - -The v8 client is shipped as a new [NuGet package](https://www.nuget.org/packages/Elastic.Clients.Elasticsearch/) which can be installed alongside v7 `NEST`. Some consumers may prefer a phased migration with both packages side-by-side for a short period of time to manage complex migrations. In addition, `NEST` 7.17.x can continue to be used in [compatibility mode](https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html) with {{es}} 8.x servers until the v8 Elasticsearch .NET Client features align with application requirements. - - -## Breaking Changes [_breaking_changes] - -::::{warning} -As a result of code-generating a majority of the client types, version 8 of the client includes multiple breaking changes. - -:::: - - -We have strived to keep the core foundation reasonably similar, but types emitted through code-generation are subject to change between `NEST` (v7) and the new `Elastic.Clients.Elasticsearch` (v8) package. - - -### Namespaces [_namespaces] - -The package and top-level namespace for the v8 client have been renamed to `Elastic.Clients.Elasticsearch`. All types belong to this namespace. When necessary, to avoid potential conflicts, types are generated into suitable sub-namespaces based on the [{{es}} specification](https://github.com/elastic/elasticsearch-specification). Additional `using` directives may be required to access such types when using the Elasticsearch .NET Client. - -Transport layer concepts have moved to the new `Elastic.Transport` NuGet package and related types are defined under its namespace. Some configuration and low-level transport functionality may require a `using` directive for the `Elastic.Transport` namespace. - - -### Type names [_type_names] - -Type names may have changed from previous versions. These are not listed explicitly due to the potentially vast number of subtle differences. Type names will now more closely align to those used in the JSON and as documented in the {{es}} documentation. - - -### Class members [_class_members] - -Types may include renamed properties based on the {{es}} specification, which differ from the original `NEST` property names. The types used for properties may also have changed due to code-generation. If you identify missing or incorrectly-typed properties, please open [an issue](https://github.com/elastic/elasticsearch-net/issues/new/choose) to alert us. - - -### Sealing classes [_sealing_classes] - -Opinions on "sealing by default" within the .NET ecosystem tend to be quite polarized. Microsoft seal all internal types for potential performance gains and we see a benefit in starting with that approach for the Elasticsearch .NET Client, even for our public API surface. - -While it prevents inheritance and, therefore, may inhibit a few consumer scenarios, sealing by default is intended to avoid the unexpected or invalid extension of types that could inadvertently be broken in the future. - - -### Removed features [_removed_features] - -As part of the clean-slate redesign of the new client, certain features are removed from the v8.0 client. These are listed below: - - -#### Attribute mappings [_attribute_mappings] - -In previous versions of the `NEST` client, attributes could be used to configure the mapping behaviour and inference for user types. It is recommended that mapping be completed via the fluent API when configuring client instances. `System.Text.Json` attributes may be used to rename and ignore properties during source serialization. - - -#### CAT APIs [_cat_apis] - -The [CAT APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-cat) of {{es}} are intended for human-readable usage and will no longer be supported via the v8 Elasticsearch .NET Client. - - -#### Interface removal [_interface_removal] - -Several interfaces are removed to simplify the library and avoid interfaces where only a single implementation of that interface is expected to exist, such as `IElasticClient` in `NEST`. Abstract base classes are preferred over interfaces across the library, as this makes it easier to add enhancements without introducing breaking changes for derived types. - - -### Missing features [_missing_features] - -The following are some of the main features which have not been re-implemented for the v8 client. These might be reviewed and prioritized for inclusion in future releases. - -* Query DSL operators for combining queries. -* Scroll Helper. -* Fluent API for union types. -* `AutoMap` for field datatype inference. -* Visitor pattern support for types such as `Properties`. -* Support for `JoinField` which affects `ChildrenAggregation`. -* Conditionless queries. -* DiagnosticSources have been removed in `Elastic.Transport` to provide a clean-slate for an improved diagnostics story. The Elasticsearch .NET Client emits [OpenTelemetry](https://opentelemetry.io/) compatible `Activity` spans which can be consumed by APM agents such as the [Elastic APM Agent for .NET](apm-agent-dotnet://reference/index.md). -* Documentation is a work in progress, and we will expand on the documented scenarios in future releases. - - -## Reduced API surface [_reduced_api_surface] - -In the current versions of the code-generated .NET client, supporting commonly used endpoints is critical. Some specific queries and aggregations need further work to generate code correctly, hence they are not included yet. Ensure that the features you are using are currently supported before migrating. - -An up to date list of all supported and unsupported endpoints can be found on [GitHub](https://github.com/elastic/elasticsearch-net/issues/7890). - - -## Workarounds for missing features [_workarounds_for_missing_features] - -If you encounter a missing feature with the v8 client, there are several ways to temporarily work around this issue until we officially reintroduce the feature. - -`NEST` 7.17.x can continue to be used in [compatibility mode](https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html) with {{es}} 8.x servers until the v8 Elasticsearch .NET Client features align with application requirements. - -As a last resort, the low-level client `Elastic.Transport` can be used to create any desired request by hand: - -```csharp -public class MyRequestParameters : RequestParameters -{ - public bool Pretty - { - get => Q("pretty"); - init => Q("pretty", value); - } -} - -// ... - -var body = """ - { - "name": "my-api-key", - "expiration": "1d", - "...": "..." - } - """; - -MyRequestParameters requestParameters = new() -{ - Pretty = true -}; - -var pathAndQuery = requestParameters.CreatePathWithQueryStrings("/_security/api_key", - client.ElasticsearchClientSettings); -var endpointPath = new EndpointPath(Elastic.Transport.HttpMethod.POST, pathAndQuery); - -// Or, if the path does not contain query parameters: -// new EndpointPath(Elastic.Transport.HttpMethod.POST, "my_path") - -var response = await client.Transport - .RequestAsync( - endpointPath, - PostData.String(body), - null, - null, - cancellationToken: default) - .ConfigureAwait(false); -``` diff --git a/docs/reference/query.md b/docs/reference/query.md index 11f0e1d145..e45650dad0 100644 --- a/docs/reference/query.md +++ b/docs/reference/query.md @@ -7,45 +7,45 @@ mapped_pages: This page demonstrates how to perform a search request. - ## Fluent API [_fluent_api_3] ```csharp -var response = await client - .SearchAsync(search => search - .Index("persons") - .Query(query => query - .Term(term => term - .Field(x => x.FirstName) - .Value("Florian") - ) - ) - .Size(10) - ); +var response = await client.SearchAsync(search => search + .Index("persons") + .Query(query => query + .Term(term => term + .Field(x => x.FirstName) + .Value("Florian") + ) + ) + .Size(10) +); ``` - ## Object initializer API [_object_initializer_api_3] ```csharp -var response = await client - .SearchAsync(new SearchRequest("persons") - { - Query = Query.Term(new TermQuery(Infer.Field(x => x.FirstName)) - { - Value = "Florian" - }), - Size = 10 - }); +var response = await client.SearchAsync( + new SearchRequest("persons") + { + Query = new Query + { + Term = new TermQuery + { + Field = Infer.Field(x => x.FirstName), + Value = "Florian" + } + }, + Size = 10 + } +); ``` - ## Consume the response [_consume_the_response_3] ```csharp foreach (var person in response.Documents) { - Console.WriteLine(person.FirstName); + Console.WriteLine(person.FirstName); } ``` - diff --git a/docs/reference/recommendations.md b/docs/reference/recommendations.md index 91a2654462..8dcb61c415 100644 --- a/docs/reference/recommendations.md +++ b/docs/reference/recommendations.md @@ -7,7 +7,6 @@ mapped_pages: To achieve the most efficient use of the Elasticsearch .NET Client, we recommend following the guidance defined in this article. - ## Reuse the same client instance [_reuse_the_same_client_instance] When working with the Elasticsearch .NET Client we recommend that consumers reuse a single instance of `ElasticsearchClient` for the entire lifetime of the application. When reusing the same instance: @@ -18,8 +17,6 @@ When working with the Elasticsearch .NET Client we recommend that consumers reus The `ElasticsearchClient` type is thread-safe and can be shared and reused across multiple threads in consuming applications. Client reuse can be achieved by creating a singleton static instance or by registering the type with a singleton lifetime when using dependency injection containers. - ## Prefer asynchronous methods [_prefer_asynchronous_methods] The Elasticsearch .NET Client exposes synchronous and asynchronous methods on the `ElasticsearchClient`. We recommend always preferring the asynchronous methods, which have the `Async` suffix. Using the Elasticsearch .NET Client requires sending HTTP requests to {{es}} servers. Access to {{es}} is sometimes slow or delayed, and some complex queries may take several seconds to return. If such operations are blocked by calling the synchronous methods, the thread must wait until the HTTP request is complete. In high-load scenarios, this can cause significant thread usage, potentially affecting the throughput and performance of consuming applications. By preferring the asynchronous methods, application threads can continue with other work that doesn’t depend on the web resource until the potentially blocking task completes. - diff --git a/docs/reference/serialization.md b/docs/reference/serialization.md index 70b9bef886..9eee4e1c3e 100644 --- a/docs/reference/serialization.md +++ b/docs/reference/serialization.md @@ -8,5 +8,3 @@ mapped_pages: By default, the .NET client for {{es}} uses the Microsoft System.Text.Json library for serialization. The client understands how to serialize and deserialize the request and response types correctly. It also handles (de)serialization of user POCO types representing documents read or written to {{es}}. The client has two distinct serialization responsibilities - serialization of the types owned by the `Elastic.Clients.Elasticsearch` library and serialization of source documents, modeled in application code. The first responsibility is entirely internal; the second is configurable. - - diff --git a/docs/reference/source-serialization.md b/docs/reference/source-serialization.md index e6b110b039..7ef85f16cf 100644 --- a/docs/reference/source-serialization.md +++ b/docs/reference/source-serialization.md @@ -7,8 +7,9 @@ mapped_pages: Source serialization refers to the process of (de)serializing POCO types in consumer applications as source documents indexed and retrieved from {{es}}. A source serializer implementation handles serialization, with the default implementation using the `System.Text.Json` library. As a result, you may use `System.Text.Json` attributes and converters to control the serialization behavior. -* [Modelling documents with types](#modeling-documents-with-types) -* [Customizing source serialization](#customizing-source-serialization) +- [Modelling documents with types](#modeling-documents-with-types) +- [Customizing source serialization](#customizing-source-serialization) +- [Native AOT](#native-aot) ## Modeling documents with types [modeling-documents-with-types] @@ -32,7 +33,6 @@ public class MyDocument We can then index the an instance of the document into {{es}}. ```csharp -using System.Threading.Tasks; using Elastic.Clients.Elasticsearch; var document = new MyDocument @@ -40,8 +40,7 @@ var document = new MyDocument StringProperty = "value" }; -var indexResponse = await Client - .IndexAsync(document, "my-index-name"); +var indexResponse = await Client.IndexAsync(document); ``` The index request is serialized, with the source serializer handling the `MyDocument` type, serializing the POCO property named `StringProperty` to the JSON object member named `stringProperty`. @@ -52,19 +51,17 @@ The index request is serialized, with the source serializer handling the `MyDocu } ``` - - ## Customizing source serialization [customizing-source-serialization] The built-in source serializer handles most POCO document models correctly. Sometimes, you may need further control over how your types are serialized. ::::{note} -The built-in source serializer uses the [Microsoft `System.Text.Json` library](https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/overview) internally. You can apply `System.Text.Json` attributes and converters to control the serialization of your document types. -:::: +The built-in source serializer uses the [Microsoft `System.Text.Json` library](https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/overview) internally. You can apply `System.Text.Json` attributes and converters to control the serialization of your document types. +:::: -#### Using `System.Text.Json` attributes [system-text-json-attributes] +### Using `System.Text.Json` attributes [system-text-json-attributes] `System.Text.Json` includes attributes that can be applied to types and properties to control their serialization. These can be applied to your POCO document types to perform actions such as controlling the name of a property or ignoring a property entirely. Visit the [Microsoft documentation for further examples](https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/overview). @@ -86,20 +83,11 @@ public class Person 1. The `JsonPropertyName` attribute ensures the `FirstName` property uses the JSON name `forename` when serialized. 2. The `JsonIgnore` attribute prevents the `Age` property from appearing in the serialized JSON. - We can then index an instance of the document into {{es}}. ```csharp -using System; -using System.Text.Json; -using System.Text.Json.Serialization; -using System.Threading.Tasks; -using Elastic.Transport; -using Elastic.Clients.Elasticsearch; -using Elastic.Clients.Elasticsearch.Serialization; - var person = new Person { FirstName = "Steve", Age = 35 }; -var indexResponse = await Client.IndexAsync(person, "my-index-name"); +var indexResponse = await Client.IndexAsync(person); ``` The index request is serialized, with the source serializer handling the `Person` type, serializing the POCO property named `FirstName` to the JSON object member named `forename`. The `Age` property is ignored and does not appear in the JSON. @@ -110,15 +98,14 @@ The index request is serialized, with the source serializer handling the `Person } ``` - -#### Configuring custom `JsonSerializerOptions` [configuring-custom-jsonserializeroptions] +### Configuring custom `JsonSerializerOptions` [configuring-custom-jsonserializeroptions] The default source serializer applies a set of standard `JsonSerializerOptions` when serializing source document types. In some circumstances, you may need to override some of our defaults. This is achievable by creating an instance of `DefaultSourceSerializer` and passing an `Action`, which is applied after our defaults have been set. This mechanism allows you to apply additional settings or change the value of our defaults. The `DefaultSourceSerializer` includes a constructor that accepts the current `IElasticsearchClientSettings` and a `configureOptions` `Action`. ```csharp -public DefaultSourceSerializer(IElasticsearchClientSettings settings, Action configureOptions); +public DefaultSourceSerializer(IElasticsearchClientSettings settings, Action? configureOptions = null); ``` Our application defines the following `Person` class, which models a document we will index to {{es}}. @@ -133,16 +120,15 @@ public class Person We want to serialize our source document using Pascal Casing for the JSON properties. Since the options applied in the `DefaultSouceSerializer` set the `PropertyNamingPolicy` to `JsonNamingPolicy.CamelCase`, we must override this setting. After configuring the `ElasticsearchClientSettings`, we index our document to {{es}}. ```csharp -using System; using System.Text.Json; -using System.Text.Json.Serialization; -using System.Threading.Tasks; using Elastic.Transport; using Elastic.Clients.Elasticsearch; using Elastic.Clients.Elasticsearch.Serialization; -static void ConfigureOptions(JsonSerializerOptions o) => <1> +static void ConfigureOptions(JsonSerializerOptions o) <1> +{ o.PropertyNamingPolicy = null; +} var nodePool = new SingleNodePool(new Uri("http://localhost:9200")); var settings = new ElasticsearchClientSettings( @@ -152,13 +138,12 @@ var settings = new ElasticsearchClientSettings( var client = new ElasticsearchClient(settings); var person = new Person { FirstName = "Steve" }; -var indexResponse = await client.IndexAsync(person, "my-index-name"); +var indexResponse = await client.IndexAsync(person); ``` 1. A local function can be defined, accepting a `JsonSerializerOptions` parameter. Here, we set `PropertyNamingPolicy` to `null`. This returns to the default behavior for `System.Text.Json`, which uses Pascal Case. 2. When creating the `ElasticsearchClientSettings`, we supply a `SourceSerializerFactory` using a lambda. The factory function creates a new instance of `DefaultSourceSerializer`, passing in the `settings` and our `ConfigureOptions` local function. We have now configured the settings with a custom instance of the source serializer. - The `Person` instance is serialized, with the source serializer serializing the POCO property named `FirstName` using Pascal Case. ```javascript @@ -170,11 +155,13 @@ The `Person` instance is serialized, with the source serializer serializing the As an alternative to using a local function, we could store an `Action` into a variable instead, which can be passed to the `DefaultSouceSerializer` constructor. ```csharp -Action configureOptions = o => o.PropertyNamingPolicy = null; +Action configureOptions = o => +{ + o.PropertyNamingPolicy = null; +} ``` - -#### Registering custom `System.Text.Json` converters [registering-custom-converters] +### Registering custom `System.Text.Json` converters [registering-custom-converters] In certain more advanced situations, you may have types which require further customization during serialization than is possible using `System.Text.Json` property attributes. In these cases, the recommendation from Microsoft is to leverage a custom `JsonConverter`. Source document types serialized using the `DefaultSourceSerializer` can leverage the power of custom converters. @@ -201,7 +188,6 @@ public enum CustomerType 1. The `JsonConverter` attribute signals to `System.Text.Json` that it should use a converter of type `CustomerConverter` when serializing instances of this class. - When serializing this class, rather than include a string value representing the value of the `CustomerType` property, we must send a boolean property named `isStandard`. This requirement can be achieved with a custom JsonConverter implementation. ```csharp @@ -285,24 +271,16 @@ public class CustomerConverter : JsonConverter 1. When reading, this converter reads the `isStandard` boolean and translate this to the correct `CustomerType` enum value. 2. When writing, this converter translates the `CustomerType` enum value to an `isStandard` boolean property. - We can then index a customer document into {{es}}. ```csharp -using System; -using System.Text.Json; -using System.Text.Json.Serialization; -using System.Threading.Tasks; -using Elastic.Transport; -using Elastic.Clients.Elasticsearch; -using Elastic.Clients.Elasticsearch.Serialization; - var customer = new Customer { CustomerName = "Customer Ltd", CustomerType = CustomerType.Enhanced }; -var indexResponse = await Client.IndexAsync(customer, "my-index-name"); + +var indexResponse = await Client.IndexAsync(customer); ``` The `Customer` instance is serialized using the custom converter, creating the following JSON document. @@ -314,184 +292,103 @@ The `Customer` instance is serialized using the custom converter, creating the f } ``` +### Creating a custom `Serializer` [creating-custom-serializers] -#### Creating a custom `SystemTextJsonSerializer` [creating-custom-system-text-json-serializer] - -The built-in `DefaultSourceSerializer` includes the registration of `JsonConverter` instances which apply during source serialization. In most cases, these provide the proper behavior for serializing source documents, including those which use `Elastic.Clients.Elasticsearch` types on their properties. - -An example of a situation where you may require more control over the converter registration order is for serializing `enum` types. The `DefaultSourceSerializer` registers the `System.Text.Json.Serialization.JsonStringEnumConverter`, so enum values are serialized using their string representation. Generally, this is the preferred option for types used to index documents to {{es}}. - -In some scenarios, you may need to control the string value sent for an enumeration value. That is not directly supported in `System.Text.Json` but can be achieved by creating a custom `JsonConverter` for the `enum` type you wish to customize. In this situation, it is not sufficient to use the `JsonConverterAttribute` on the `enum` type to register the converter. `System.Text.Json` will prefer the converters added to the `Converters` collection on the `JsonSerializerOptions` over an attribute applied to an `enum` type. It is, therefore, necessary to either remove the `JsonStringEnumConverter` from the `Converters` collection or register a specialized converter for your `enum` type before the `JsonStringEnumConverter`. - -The latter is possible via several techniques. When using the {{es}} .NET library, we can achieve this by deriving from the abstract `SystemTextJsonSerializer` class. - -Here we have a POCO which uses the `CustomerType` enum as the type for a property. - -```csharp -using System.Text.Json.Serialization; - -public class Customer -{ - public string CustomerName { get; set; } - public CustomerType CustomerType { get; set; } -} - -public enum CustomerType -{ - Standard, - Enhanced -} -``` +Suppose you prefer using an alternative JSON serialization library for your source types. In that case, you can inject an isolated serializer only to be called for the serialization of `_source`, `_fields`, or wherever a user-provided value is expected to be written and returned. -To customize the strings used during serialization of the `CustomerType`, we define a custom `JsonConverter` specific to our `enum` type. +Implementing `Elastic.Transport.Serializer` is technically enough to create a custom source serializer. ```csharp -using System.Text.Json.Serialization; +using Elastic.Transport; -public class CustomerTypeConverter : JsonConverter +public class VanillaSerializer : Serializer { - public override CustomerType Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) - { - return reader.GetString() switch <1> - { - "basic" => CustomerType.Standard, - "premium" => CustomerType.Enhanced, - _ => throw new JsonException( - $"Unknown value read when deserializing {nameof(CustomerType)}."), - }; - } - - public override void Write(Utf8JsonWriter writer, CustomerType value, JsonSerializerOptions options) - { - switch (value) <2> - { - case CustomerType.Standard: - writer.WriteStringValue("basic"); - return; - case CustomerType.Enhanced: - writer.WriteStringValue("premium"); - return; - } - - writer.WriteNullValue(); - } -} -``` - -1. When reading, this converter translates the string used in the JSON to the matching enum value. -2. When writing, this converter translates the `CustomerType` enum value to a custom string value written to the JSON. - + public override object? Deserialize(Type type, Stream stream) => + throw new NotImplementedException(); -We create a serializer derived from `SystemTextJsonSerializer` to give us complete control of converter registration order. + public override T Deserialize(Stream stream) => + throw new NotImplementedException(); -```csharp -using System.Text.Json; -using Elastic.Clients.Elasticsearch.Serialization; + public override ValueTask DeserializeAsync(Type type, Stream stream, CancellationToken cancellationToken = default) => + throw new NotImplementedException(); -public class MyCustomSerializer : SystemTextJsonSerializer <1> -{ - private readonly JsonSerializerOptions _options; + public override ValueTask DeserializeAsync(Stream stream, CancellationToken cancellationToken = default) => + throw new NotImplementedException(); - public MyCustomSerializer(IElasticsearchClientSettings settings) : base(settings) - { - var options = DefaultSourceSerializer.CreateDefaultJsonSerializerOptions(false); <2> + public override void Serialize(object? data, Type type, Stream stream, SerializationFormatting formatting = SerializationFormatting.None, + CancellationToken cancellationToken = default) => + throw new NotImplementedException(); - options.Converters.Add(new CustomerTypeConverter()); <3> + public override void Serialize(T data, Stream stream, SerializationFormatting formatting = SerializationFormatting.None) => + throw new NotImplementedException(); - _options = DefaultSourceSerializer.AddDefaultConverters(options); <4> - } + public override Task SerializeAsync(object? data, Type type, Stream stream, SerializationFormatting formatting = SerializationFormatting.None, + CancellationToken cancellationToken = default) => + throw new NotImplementedException(); - protected override JsonSerializerOptions CreateJsonSerializerOptions() => _options; <5> + public override Task SerializeAsync(T data, Stream stream, SerializationFormatting formatting = SerializationFormatting.None, + CancellationToken cancellationToken = default) => + throw new NotImplementedException(); } ``` -1. Inherit from `SystemTextJsonSerializer`. -2. In the constructor, use the factory method `DefaultSourceSerializer.CreateDefaultJsonSerializerOptions` to create default options for serialization. No default converters are registered at this stage because we pass `false` as an argument. -3. Register our `CustomerTypeConverter` as the first converter. -4. To apply any default converters, call the `DefaultSourceSerializer.AddDefaultConverters` helper method, passing the options to modify. -5. Implement the `CreateJsonSerializerOptions` method returning the stored `JsonSerializerOptions`. - - -Because we have registered our `CustomerTypeConverter` before the default converters (which include the `JsonStringEnumConverter`), our converter takes precedence when serializing `CustomerType` instances on source documents. - -The base `SystemTextJsonSerializer` class handles the implementation details of binding, which is required to ensure that the built-in converters can access the `IElasticsearchClientSettings` where needed. - -We can then index a customer document into {{es}}. +Registering up the serializer is performed in the `ConnectionSettings` constructor. ```csharp -using System; -using System.Text.Json; -using System.Text.Json.Serialization; -using System.Threading.Tasks; using Elastic.Transport; using Elastic.Clients.Elasticsearch; -using Elastic.Clients.Elasticsearch.Serialization; - -var customer = new Customer -{ - CustomerName = "Customer Ltd", - CustomerType = CustomerType.Enhanced -}; -var indexResponse = await client.IndexAsync(customer, "my-index-name"); +var nodePool = new SingleNodePool(new Uri("http://localhost:9200")); +var settings = new ElasticsearchClientSettings( + nodePool, + sourceSerializer: (defaultSerializer, settings) => + new VanillaSerializer()); <1> +var client = new ElasticsearchClient(settings); ``` -The `Customer` instance is serialized using the custom `enum` converter, creating the following JSON document. +1. If implementing `Serializer` is enough, why must we provide an instance wrapped in a factory `Func`? -```javascript +There are various cases where you might have a POCO type that contains an `Elastic.Clients.Elasticsearch` type as one of its properties. The `SourceSerializerFactory` delegate provides access to the default built-in serializer so you can access it when necessary. For example, consider if you want to use percolation; you need to store {{es}} queries as part of the `_source` of your document, which means you need to have a POCO that looks like this. + +```csharp +using Elastic.Clients.Elasticsearch.QueryDsl; + +public class MyPercolationDocument { - "customerName": "Customer Ltd", - "customerType": "premium" <1> + public Query Query { get; set; } + public string Category { get; set; } } ``` -1. The string value applied during serialization is provided by our custom converter. +A custom serializer would not know how to serialize `Query` or other `Elastic.Clients.Elasticsearch` types that could appear as part of the `_source` of a document. Therefore, your custom `Serializer` would need to store a reference to our built-in serializer and delegate serialization of Elastic types back to it. +::::{note} +Depending on the use-case, it might be easier to instead derive from `SystemTextJsonSerializer` and/or create a custom `IJsonSerializerOptionsProvider` implementation. -#### Creating a custom `Serializer` [creating-custom-serializers] +:::: -Suppose you prefer using an alternative JSON serialization library for your source types. In that case, you can inject an isolated serializer only to be called for the serialization of `_source`, `_fields`, or wherever a user-provided value is expected to be written and returned. +## Native AOT [native-aot] -Implementing `Elastic.Transport.Serializer` is technically enough to create a custom source serializer. +To use the Elasticsearch client in a [native AOT](https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot) application, `System.Text.Json` must be configured to use [source generation](https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/source-generation). + +Source generation is always for internal `Elastic.Clients.Elasticsearch` types, but additional steps are required to also enable source generation for user defined types. ```csharp -using System; -using System.IO; -using System.Threading; -using System.Threading.Tasks; -using Elastic.Transport; +using System.Text.Json.Serialization; -public class VanillaSerializer : Serializer +[JsonSerializable(typeof(Person), GenerationMode = JsonSourceGenerationMode.Default)] <2> +[JsonSerializable(typeof(...), GenerationMode = JsonSourceGenerationMode.Default)] +public sealed partial class UserTypeSerializerContext : + JsonSerializerContext <1> { - public override object Deserialize(Type type, Stream stream) => - throw new NotImplementedException(); - - public override T Deserialize(Stream stream) => - throw new NotImplementedException(); - - public override ValueTask DeserializeAsync(Type type, Stream stream, CancellationToken cancellationToken = default) => - throw new NotImplementedException(); - - public override ValueTask DeserializeAsync(Stream stream, CancellationToken cancellationToken = default) => - throw new NotImplementedException(); - - public override void Serialize(T data, Stream stream, SerializationFormatting formatting = SerializationFormatting.None) => - throw new NotImplementedException(); - - public override Task SerializeAsync(T data, Stream stream, - SerializationFormatting formatting = SerializationFormatting.None, CancellationToken cancellationToken = default) => - throw new NotImplementedException(); } ``` -Registering up the serializer is performed in the `ConnectionSettings` constructor. +1. Create a new `partial` class that derives from `JsonSerializerContext`. +2. Annotate the serializer context with `JsonSerializable` attributes for all source types that are (de-)serialized using the Elasticsearch client. ```csharp -using System; -using System.Text.Json; -using System.Text.Json.Serialization; -using System.Threading.Tasks; using Elastic.Transport; using Elastic.Clients.Elasticsearch; using Elastic.Clients.Elasticsearch.Serialization; @@ -499,26 +396,21 @@ using Elastic.Clients.Elasticsearch.Serialization; var nodePool = new SingleNodePool(new Uri("http://localhost:9200")); var settings = new ElasticsearchClientSettings( nodePool, - sourceSerializer: (defaultSerializer, settings) => - new VanillaSerializer()); <1> + sourceSerializer: (_, settings) => + new DefaultSourceSerializer(settings, UserTypeSerializerContext.Default)); <1> var client = new ElasticsearchClient(settings); -``` -1. If implementing `Serializer` is enough, why must we provide an instance wrapped in a factory `Func`? +var person = new Person { FirstName = "Steve" }; +var indexResponse = await client.IndexAsync(person); +``` +1. When creating the `ElasticsearchClientSettings`, we supply a `SourceSerializerFactory` using a lambda. The factory function creates a new instance of `DefaultSourceSerializer`, passing in the `settings` and our `UserTypeSerializerContext`. We have now configured the settings with a custom instance of the source serializer. -There are various cases where you might have a POCO type that contains an `Elastic.Clients.Elasticsearch` type as one of its properties. The `SourceSerializerFactory` delegate provides access to the default built-in serializer so you can access it when necessary. For example, consider if you want to use percolation; you need to store {{es}} queries as part of the `_source` of your document, which means you need to have a POCO that looks like this. +As an alternative, the `UserTypeSerializerContext` can as well be set when configuring the `JsonSerializerOptions` as described in [Configuring custom `JsonSerializerOptions`](#configuring-custom-jsonserializeroptions): ```csharp -using Elastic.Clients.Elasticsearch.QueryDsl; - -public class MyPercolationDocument +static void ConfigureOptions(JsonSerializerOptions o) { - public Query Query { get; set; } - public string Category { get; set; } + o.TypeInfoResolver = UserTypeSerializerContext.Default; } ``` - -A custom serializer would not know how to serialize `Query` or other `Elastic.Clients.Elasticsearch` types that could appear as part of the `_source` of a document. Therefore, your custom `Serializer` would need to store a reference to our built-in serializer and delegate serialization of Elastic types back to it. - - diff --git a/docs/reference/toc.yml b/docs/reference/toc.yml index 828400800c..a6cb9a9650 100644 --- a/docs/reference/toc.yml +++ b/docs/reference/toc.yml @@ -20,7 +20,6 @@ toc: - file: query.md - file: recommendations.md - file: transport.md - - file: migration-guide.md - file: troubleshoot/index.md children: - file: troubleshoot/logging.md diff --git a/docs/reference/using-net-client.md b/docs/reference/using-net-client.md index ace92b95d1..e64080e82d 100644 --- a/docs/reference/using-net-client.md +++ b/docs/reference/using-net-client.md @@ -18,7 +18,7 @@ If you’re new to {{es}}, make sure also to read [Elasticsearch’s quick start * [Using ES|QL](/reference/esql.md) ::::{note} -This is still a work in progress, more sections will be added in the near future. -:::: +This is still a work in progress, more sections will be added in the near future. +::::