diff --git a/docs/reference/images/elasticsearch-client-net-api-capture-requests-localhost.png b/docs/reference/images/elasticsearch-client-net-api-capture-requests-localhost.png new file mode 100644 index 00000000000..3730681e5ed Binary files /dev/null and b/docs/reference/images/elasticsearch-client-net-api-capture-requests-localhost.png differ diff --git a/docs/reference/images/elasticsearch-client-net-api-capture-requests-remotehost.png b/docs/reference/images/elasticsearch-client-net-api-capture-requests-remotehost.png new file mode 100644 index 00000000000..26a913d2cbe Binary files /dev/null and b/docs/reference/images/elasticsearch-client-net-api-capture-requests-remotehost.png differ diff --git a/docs/reference/images/elasticsearch-client-net-api-inspect-requests.png b/docs/reference/images/elasticsearch-client-net-api-inspect-requests.png new file mode 100644 index 00000000000..8b4a79b4093 Binary files /dev/null and b/docs/reference/images/elasticsearch-client-net-api-inspect-requests.png differ diff --git a/docs/reference/toc.yml b/docs/reference/toc.yml index e8e6d33012e..828400800c0 100644 --- a/docs/reference/toc.yml +++ b/docs/reference/toc.yml @@ -20,4 +20,15 @@ toc: - file: query.md - file: recommendations.md - file: transport.md - - file: migration-guide.md \ No newline at end of file + - file: migration-guide.md + - file: troubleshoot/index.md + children: + - file: troubleshoot/logging.md + children: + - file: troubleshoot/logging-with-onrequestcompleted.md + - file: troubleshoot/logging-with-fiddler.md + - file: troubleshoot/debugging.md + children: + - file: troubleshoot/audit-trail.md + - file: troubleshoot/debug-information.md + - file: troubleshoot/debug-mode.md \ No newline at end of file diff --git a/docs/reference/troubleshoot/audit-trail.md b/docs/reference/troubleshoot/audit-trail.md new file mode 100644 index 00000000000..96d3008aaf2 --- /dev/null +++ b/docs/reference/troubleshoot/audit-trail.md @@ -0,0 +1,66 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/audit-trail.html +--- + +# Audit trail [audit-trail] + +Elasticsearch.Net and NEST provide an audit trail for the events within the request pipeline that occur when a request is made. This audit trail is available on the response as demonstrated in the following example. + +We’ll use a Sniffing connection pool here since it sniffs on startup and pings before first usage, so we can get an audit trail with a few events out + +```csharp +var pool = new SniffingConnectionPool(new []{ TestConnectionSettings.CreateUri() }); +var connectionSettings = new ConnectionSettings(pool) + .DefaultMappingFor(i => i + .IndexName("project") + ); + +var client = new ElasticClient(connectionSettings); +``` + +After issuing the following request + +```csharp +var response = client.Search(s => s + .MatchAll() +); +``` + +The audit trail is provided in the [Debug information](debug-information.md) in a human readable fashion, similar to + +``` +Valid NEST response built from a successful low level call on POST: /project/doc/_search +# Audit trail of this API call: + - [1] SniffOnStartup: Took: 00:00:00.0360264 + - [2] SniffSuccess: Node: http://localhost:9200/ Took: 00:00:00.0310228 + - [3] PingSuccess: Node: http://127.0.0.1:9200/ Took: 00:00:00.0115074 + - [4] HealthyResponse: Node: http://127.0.0.1:9200/ Took: 00:00:00.1477640 +# Request: + +# Response: + +``` +to help with troubleshootin + +```csharp +var debug = response.DebugInformation; +``` + +But can also be accessed manually: + +```csharp +response.ApiCall.AuditTrail.Count.Should().Be(4, "{0}", debug); +response.ApiCall.AuditTrail[0].Event.Should().Be(SniffOnStartup, "{0}", debug); +response.ApiCall.AuditTrail[1].Event.Should().Be(SniffSuccess, "{0}", debug); +response.ApiCall.AuditTrail[2].Event.Should().Be(PingSuccess, "{0}", debug); +response.ApiCall.AuditTrail[3].Event.Should().Be(HealthyResponse, "{0}", debug); +``` + +Each audit has a started and ended `DateTime` on it that will provide some understanding of how long it took + +```csharp +response.ApiCall.AuditTrail + .Should().OnlyContain(a => a.Ended - a.Started >= TimeSpan.Zero); +``` + diff --git a/docs/reference/troubleshoot/debug-information.md b/docs/reference/troubleshoot/debug-information.md new file mode 100644 index 00000000000..f682179a044 --- /dev/null +++ b/docs/reference/troubleshoot/debug-information.md @@ -0,0 +1,165 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/debug-information.html +--- + +# Debug information [debug-information] + +Every response from Elasticsearch.Net and NEST contains a `DebugInformation` property that provides a human readable description of what happened during the request for both successful and failed requests + +```csharp +var response = client.Search(s => s + .Query(q => q + .MatchAll() + ) +); + +response.DebugInformation.Should().Contain("Valid NEST response"); +``` + +This can be useful in tracking down numerous problems and can also be useful when filing an [issue](https://github.com/elastic/elasticsearch-net/issues) on the GitHub repository. + +## Request and response bytes [_request_and_response_bytes] + +By default, the request and response bytes are not available within the debug information, but can be enabled globally on Connection Settings by setting `DisableDirectStreaming`. This disables direct streaming of + +1. the serialized request type to the request stream +2. the response stream to a deserialized response type + +```csharp +var connectionPool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); + +var settings = new ConnectionSettings(connectionPool) + .DisableDirectStreaming(); <1> + +var client = new ElasticClient(settings); +``` + +1. disable direct streaming for **all** requests + + +or on a *per request* basis + +```csharp +var response = client.Search(s => s + .RequestConfiguration(r => r + .DisableDirectStreaming() <1> + ) + .Query(q => q + .MatchAll() + ) +); +``` + +1. disable direct streaming for **this** request only + + +Configuring `DisableDirectStreaming` on an individual request takes precedence over any global configuration. + +There is typically a performance and allocation cost associated with disabling direct streaming since both the request and response bytes must be buffered in memory, to allow them to be exposed on the response call details. + + +## TCP statistics [_tcp_statistics] + +It can often be useful to see the statistics for active TCP connections, particularly when trying to diagnose issues with the client. The client can collect the states of active TCP connections just before making a request, and expose these on the response and in the debug information. + +Similarly to `DisableDirectStreaming`, TCP statistics can be collected for every request by configuring on `ConnectionSettings` + +```csharp +var connectionPool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); + +var settings = new ConnectionSettings(connectionPool) + .EnableTcpStats(); <1> + +var client = new ElasticClient(settings); +``` + +1. collect TCP statistics for **all** requests + + +or on a *per request* basis + +```csharp +var response = client.Search(s => s + .RequestConfiguration(r => r + .EnableTcpStats() <1> + ) + .Query(q => q + .MatchAll() + ) +); + +var debugInformation = response.DebugInformation; +``` + +1. collect TCP statistics for **this** request only + + +With `EnableTcpStats` set, the states of active TCP connections will now be included on the response and in the debug information. + +The client includes a `TcpStats` class to help with retrieving more detail about active TCP connections should it be required + +```csharp +var tcpStatistics = TcpStats.GetActiveTcpConnections(); <1> +var ipv4Stats = TcpStats.GetTcpStatistics(NetworkInterfaceComponent.IPv4); <2> +var ipv6Stats = TcpStats.GetTcpStatistics(NetworkInterfaceComponent.IPv6); <3> + +var response = client.Search(s => s + .Query(q => q + .MatchAll() + ) +); +``` + +1. Retrieve details about active TCP connections, including local and remote addresses and ports +2. Retrieve statistics about IPv4 +3. Retrieve statistics about IPv6 + + +::::{note} +Collecting TCP statistics may not be accessible in all environments, for example, Azure App Services. When this is the case, `TcpStats.GetActiveTcpConnections()` returns `null`. + +:::: + + + +## ThreadPool statistics [_threadpool_statistics] + +It can often be useful to see the statistics for thread pool threads, particularly when trying to diagnose issues with the client. The client can collect statistics for both worker threads and asynchronous I/O threads, and expose these on the response and in debug information. + +Similar to collecting TCP statistics, ThreadPool statistics can be collected for all requests by configuring `EnableThreadPoolStats` on `ConnectionSettings` + +```csharp +var connectionPool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); + +var settings = new ConnectionSettings(connectionPool) + .EnableThreadPoolStats(); <1> + +var client = new ElasticClient(settings); +``` + +1. collect thread pool statistics for **all** requests + + +or on a *per request* basis + +```csharp +var response = client.Search(s => s + .RequestConfiguration(r => r + .EnableThreadPoolStats() <1> + ) + .Query(q => q + .MatchAll() + ) + ); + +var debugInformation = response.DebugInformation; <2> +``` + +1. collect thread pool statistics for **this** request only +2. contains thread pool statistics + + +With `EnableThreadPoolStats` set, the statistics of thread pool threads will now be included on the response and in the debug information. + + diff --git a/docs/reference/troubleshoot/debug-mode.md b/docs/reference/troubleshoot/debug-mode.md new file mode 100644 index 00000000000..fce96b34be4 --- /dev/null +++ b/docs/reference/troubleshoot/debug-mode.md @@ -0,0 +1,50 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/debug-mode.html +--- + +# Debug mode [debug-mode] + +The [Debug information](debug-information.md) explains that every response from Elasticsearch.Net and NEST contains a `DebugInformation` property, and properties on `ConnectionSettings` and `RequestConfiguration` can control which additional information is included in debug information, for all requests or on a per request basis, respectively. + +During development, it can be useful to enable the most verbose debug information, to help identify and troubleshoot problems, or simply ensure that the client is behaving as expected. The `EnableDebugMode` setting on `ConnectionSettings` is a convenient shorthand for enabling verbose debug information, configuring a number of settings like + +* disabling direct streaming to capture request and response bytes +* prettyfying JSON responses from Elasticsearch +* collecting TCP statistics when a request is made +* collecting thread pool statistics when a request is made +* including the Elasticsearch stack trace in the response if there is a an error on the server side + +```csharp +IConnectionPool pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); + +var settings = new ConnectionSettings(pool) + .EnableDebugMode(); <1> + +var client = new ElasticClient(settings); + +var response = client.Search(s => s + .Query(q => q + .MatchAll() + ) +); + +var debugInformation = response.DebugInformation; <2> +``` + +1. configure debug mode +2. verbose debug information + + +In addition to exposing debug information on the response, debug mode will also cause the debug information to be written to the trace listeners in the `System.Diagnostics.Debug.Listeners` collection by default, when the request has completed. A delegate can be passed when enabling debug mode to perform a different action when a request has completed, using [`OnRequestCompleted`](logging-with-onrequestcompleted.md) + +```csharp +var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); +var client = new ElasticClient(new ConnectionSettings(pool) + .EnableDebugMode(apiCallDetails => + { + // do something with the call details e.g. send with logging framework + }) +); +``` + diff --git a/docs/reference/troubleshoot/debugging.md b/docs/reference/troubleshoot/debugging.md new file mode 100644 index 00000000000..2514e064b48 --- /dev/null +++ b/docs/reference/troubleshoot/debugging.md @@ -0,0 +1,14 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/debugging.html +--- + +# Debugging [debugging] + +Elasticsearch.Net and NEST provide an audit trail and debug information to help you resolve issues: + +* [](audit-trail.md) +* [](debug-information.md) +* [](debug-mode.md) + + diff --git a/docs/reference/troubleshoot/index.md b/docs/reference/troubleshoot/index.md new file mode 100644 index 00000000000..b1163f89ecc --- /dev/null +++ b/docs/reference/troubleshoot/index.md @@ -0,0 +1,13 @@ +--- +navigation_title: Troubleshoot +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/troubleshooting.html +--- + +# Troubleshoot: {{es}} .NET client [troubleshooting] + +The client can provide rich details about what occurred in the request pipeline during the process of making a request, and can also provide the raw request and response JSON. + +* [Logging](logging.md) +* [Debugging](debugging.md) + diff --git a/docs/reference/troubleshoot/logging-with-fiddler.md b/docs/reference/troubleshoot/logging-with-fiddler.md new file mode 100644 index 00000000000..184819c9be1 --- /dev/null +++ b/docs/reference/troubleshoot/logging-with-fiddler.md @@ -0,0 +1,50 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/logging-with-fiddler.html +--- + +# Logging with Fiddler [logging-with-fiddler] + +A web debugging proxy such as [Fiddler](http://www.telerik.com/fiddler) is a useful way to capture HTTP traffic from a machine, particularly whilst developing against a local Elasticsearch cluster. + +## Capturing traffic to a remote cluster [_capturing_traffic_to_a_remote_cluster] + +To capture traffic against a remote cluster is as simple as launching Fiddler! You may want to also filter traffic to only show requests to the remote cluster by using the filters tab + +:::{image} ../images/elasticsearch-client-net-api-capture-requests-remotehost.png +:alt: Capturing requests to a remote host +::: + + +## Capturing traffic to a local cluster [_capturing_traffic_to_a_local_cluster] + +The .NET Framework is hardcoded not to send requests for `localhost` through any proxies and as a proxy Fiddler will not receive such traffic. + +This is easily circumvented by using `ipv4.fiddler` as the hostname instead of `localhost` + +```csharp +var isFiddlerRunning = Process.GetProcessesByName("fiddler").Any(); +var host = isFiddlerRunning ? "ipv4.fiddler" : "localhost"; + +var connectionSettings = new ConnectionSettings(new Uri($"http://{host}:9200")) + .PrettyJson(); <1> + +var client = new ElasticClient(connectionSettings); +``` + +1. prettify json requests and responses to make them easier to read in Fiddler + + +With Fiddler running, the requests and responses will now be captured and can be inspected in the Inspectors tab + +:::{image} ../images/elasticsearch-client-net-api-inspect-requests.png +:alt: Inspecting requests and responses +::: + +As before, you may also want to filter traffic to only show requests to `ipv4.fiddler` on the port on which you are running Elasticsearch. + +:::{image} ../images/elasticsearch-client-net-api-capture-requests-localhost.png +:alt: Capturing requests to localhost +::: + + diff --git a/docs/reference/troubleshoot/logging-with-onrequestcompleted.md b/docs/reference/troubleshoot/logging-with-onrequestcompleted.md new file mode 100644 index 00000000000..f1c365ffaf6 --- /dev/null +++ b/docs/reference/troubleshoot/logging-with-onrequestcompleted.md @@ -0,0 +1,209 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/logging-with-on-request-completed.html +--- + +# Logging with OnRequestCompleted [logging-with-on-request-completed] + +When constructing the connection settings to pass to the client, you can pass a callback of type `Action` to the `OnRequestCompleted` method that can eavesdrop every time a response(good or bad) is received. + +If you have complex logging needs this is a good place to add that in since you have access to both the request and response details. + +In this example, we’ll use `OnRequestCompleted` on connection settings to increment a counter each time it’s called. + +```csharp +var counter = 0; +var client = new ElasticClient(new AlwaysInMemoryConnectionSettings().OnRequestCompleted(r => counter++)); <1> + +client.RootNodeInfo(); <2> +counter.Should().Be(1); + +await client.RootNodeInfoAsync(); <3> +counter.Should().Be(2); +``` + +1. Construct a client +2. Make a synchronous call and assert the counter is incremented +3. Make an asynchronous call and assert the counter is incremented + + +`OnRequestCompleted` is called even when an exception is thrown, so it can be used even if the client is configured to throw exceptions + +```csharp +var counter = 0; +var client = FixedResponseClient.Create( <1> + new { }, + 500, + connectionSettings => connectionSettings + .ThrowExceptions() <2> + .OnRequestCompleted(r => counter++) +); + +Assert.Throws(() => client.RootNodeInfo()); <3> +counter.Should().Be(1); + +await Assert.ThrowsAsync(async () => await client.RootNodeInfoAsync()); +counter.Should().Be(2); +``` + +1. Configure a client with a connection that **always returns a HTTP 500 response +2. Always throw exceptions when a call results in an exception +3. Assert an exception is thrown and the counter is incremented + + +Here’s an example using `OnRequestCompleted()` for more complex logging + +::::{note} +By default, the client writes directly to the request stream and deserializes directly from the response stream. + +If you would also like to capture the request and/or response bytes, you also need to set `.DisableDirectStreaming()` to `true`. + +:::: + + +```csharp +var list = new List(); +var connectionPool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); + +var settings = new ConnectionSettings(connectionPool, new InMemoryConnection()) <1> + .DefaultIndex("default-index") + .DisableDirectStreaming() <2> + .OnRequestCompleted(apiCallDetails => <3> + { + // log out the request and the request body, if one exists for the type of request + if (apiCallDetails.RequestBodyInBytes != null) + { + list.Add( + $"{apiCallDetails.HttpMethod} {apiCallDetails.Uri} " + + $"{Encoding.UTF8.GetString(apiCallDetails.RequestBodyInBytes)}"); + } + else + { + list.Add($"{apiCallDetails.HttpMethod} {apiCallDetails.Uri}"); + } + + // log out the response and the response body, if one exists for the type of response + if (apiCallDetails.ResponseBodyInBytes != null) + { + list.Add($"Status: {apiCallDetails.HttpStatusCode}" + + $"{Encoding.UTF8.GetString(apiCallDetails.ResponseBodyInBytes)}"); + } + else + { + list.Add($"Status: {apiCallDetails.HttpStatusCode}"); + } + }); + +var client = new ElasticClient(settings); + +var syncResponse = client.Search(s => s <4> + .AllIndices() + .Scroll("2m") + .Sort(ss => ss + .Ascending(SortSpecialField.DocumentIndexOrder) + ) +); + +list.Count.Should().Be(2); + +var asyncResponse = await client.SearchAsync(s => s <5> + .AllIndices() + .Scroll("10m") + .Sort(ss => ss + .Ascending(SortSpecialField.DocumentIndexOrder) + ) +); + +list.Count.Should().Be(4); +list.Should().BeEquivalentTo(new[] <6> +{ + @"POST http://localhost:9200/_all/_search?typed_keys=true&scroll=2m {""sort"":[{""_doc"":{""order"":""asc""}}]}", + @"Status: 200", + @"POST http://localhost:9200/_all/_search?typed_keys=true&scroll=10m {""sort"":[{""_doc"":{""order"":""asc""}}]}", + @"Status: 200" +}); +``` + +1. Here we use `InMemoryConnection` but in a real application, you’d use an `IConnection` that *actually* sends the request, such as `HttpConnection` +2. Disable direct streaming so we can capture the request and response bytes +3. Perform some action when a request completes. Here, we’re just adding to a list, but in your application you may be logging to a file. +4. Make a synchronous call +5. Make an asynchronous call +6. Assert the list contains the contents written in the delegate passed to `OnRequestCompleted` + + +When running an application in production, you probably don’t want to disable direct streaming for *all* requests, since doing so will incur a performance overhead, due to buffering request and response bytes in memory. It can however be useful to capture requests and responses in an ad-hoc fashion, perhaps to troubleshoot an issue in production. + +`DisableDirectStreaming` can be enabled on a *per-request* basis for this purpose. In using this feature, it is possible to configure a general logging mechanism in `OnRequestCompleted` and log out request and responses only when necessary + +```csharp +var list = new List(); +var connectionPool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); + +var settings = new ConnectionSettings(connectionPool, new InMemoryConnection()) + .DefaultIndex("default-index") + .OnRequestCompleted(apiCallDetails => + { + // log out the request and the request body, if one exists for the type of request + if (apiCallDetails.RequestBodyInBytes != null) + { + list.Add( + $"{apiCallDetails.HttpMethod} {apiCallDetails.Uri} " + + $"{Encoding.UTF8.GetString(apiCallDetails.RequestBodyInBytes)}"); + } + else + { + list.Add($"{apiCallDetails.HttpMethod} {apiCallDetails.Uri}"); + } + + // log out the response and the response body, if one exists for the type of response + if (apiCallDetails.ResponseBodyInBytes != null) + { + list.Add($"Status: {apiCallDetails.HttpStatusCode}" + + $"{Encoding.UTF8.GetString(apiCallDetails.ResponseBodyInBytes)}"); + } + else + { + list.Add($"Status: {apiCallDetails.HttpStatusCode}"); + } + }); + +var client = new ElasticClient(settings); + +var syncResponse = client.Search(s => s <1> + .AllIndices() + .Scroll("2m") + .Sort(ss => ss + .Ascending(SortSpecialField.DocumentIndexOrder) + ) +); + +list.Count.Should().Be(2); + +var asyncResponse = await client.SearchAsync(s => s <2> + .RequestConfiguration(r => r + .DisableDirectStreaming() + ) + .AllIndices() + .Scroll("10m") + .Sort(ss => ss + .Ascending(SortSpecialField.DocumentIndexOrder) + ) +); + +list.Count.Should().Be(4); +list.Should().BeEquivalentTo(new[] +{ + @"POST http://localhost:9200/_all/_search?typed_keys=true&scroll=2m", <3> + @"Status: 200", + @"POST http://localhost:9200/_all/_search?typed_keys=true&scroll=10m {""sort"":[{""_doc"":{""order"":""asc""}}]}", <4> + @"Status: 200" +}); +``` + +1. Make a synchronous call where the request and response bytes will not be buffered +2. Make an asynchronous call where `DisableDirectStreaming()` is enabled +3. Only the method and url for the first request is captured +4. the body of the second request is captured + + diff --git a/docs/reference/troubleshoot/logging.md b/docs/reference/troubleshoot/logging.md new file mode 100644 index 00000000000..8efdbc997d7 --- /dev/null +++ b/docs/reference/troubleshoot/logging.md @@ -0,0 +1,14 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/logging.html +--- + +# Logging [logging] + +While developing with Elasticsearch using NEST, it can be extremely valuable to see the requests that NEST generates and sends to Elasticsearch, as well as the responses returned. + +* [](logging-with-onrequestcompleted.md) +* [](logging-with-fiddler.md) + + +