diff --git a/elasticsearch/_async/client/__init__.py b/elasticsearch/_async/client/__init__.py
index 25f832f5d..b5a5d9d71 100644
--- a/elasticsearch/_async/client/__init__.py
+++ b/elasticsearch/_async/client/__init__.py
@@ -628,6 +628,7 @@ async def bulk(
error_trace: t.Optional[bool] = None,
filter_path: t.Optional[t.Union[str, t.Sequence[str]]] = None,
human: t.Optional[bool] = None,
+ include_source_on_error: t.Optional[bool] = None,
list_executed_pipelines: t.Optional[bool] = None,
pipeline: t.Optional[str] = None,
pretty: t.Optional[bool] = None,
@@ -730,11 +731,13 @@ async def bulk(
The other two shards that make up the index do not participate in the _bulk
request at all.
Count search results. Get the number of documents matching a query.
-The query can either be provided using a simple query string as a parameter or using the Query DSL defined within the request body.
- The latter must be nested in a query
key, which is the same as the search API.
The query can be provided either by using a simple query string as a parameter, or by defining Query DSL within the request body.
+ The query is optional. When no query is provided, the API uses match_all
to count all the documents.
The count API supports multi-target syntax. You can run a single count API search across multiple data streams and indices.
The operation is broadcast across all shards. For each shard ID group, a replica is chosen and the search is run against it. This means that replicas increase the scalability of the count.
- `_shards
section of the API response reveals the number of shard copies on which replication succeeded and failed.
- `A document's source is not available if it is disabled in the mapping.
- `Explain a document match result. - Returns information about why a specific document matches, or doesn’t match, a query.
+ Get information about why a specific document matches, or doesn't match, a query. + It computes a score explanation for a query and a specific document. - `keyword
family.
- `Get a list of supported script contexts and their methods.
- `Get a list of available script types, languages, and contexts.
- `The kNN search API supports restricting the search using a filter. The search will return the top k documents that also match the filter query.
+A kNN search response has the exact same structure as a search API response. + However, certain sections have a meaning specific to kNN search:
+_score
is determined by the similarity between the query and document vector.hits.total
object contains the total number of nearest neighbor candidates considered, which is num_candidates * num_shards
. The hits.total.relation
will always be eq
, indicating an exact value.Get multiple JSON documents by ID from one or more indices. If you specify an index in the request URI, you only need to specify the document IDs in the request body. To ensure fast responses, this multi get (mget) API responds with partial results if one or more shards fail.
+Filter source fields
+By default, the _source
field is returned for every document (if stored).
+ Use the _source
and _source_include
or source_exclude
attributes to filter what fields are returned for a particular document.
+ You can include the _source
, _source_includes
, and _source_excludes
query parameters in the request URI to specify the defaults to use when there are no per-document instructions.
Get stored fields
+Use the stored_fields
attribute to specify the set of stored fields you want to retrieve.
+ Any requested fields that are not stored are ignored.
+ You can include the stored_fields
query parameter in the request URI to specify the defaults to use when there are no per-document instructions.
Content-Type
header should be set to application/x-ndjson
.
- `Run multiple templated searches.
+Run multiple templated searches with a single request.
+ If you are providing a text file or text input to curl
, use the --data-binary
flag instead of -d
to preserve newlines.
+ For example:
$ cat requests
+ { "index": "my-index" }
+ { "id": "my-search-template", "params": { "query_string": "hello world", "from": 0, "size": 10 }}
+ { "index": "my-other-index" }
+ { "id": "my-other-search-template", "params": { "query_type": "match_all" }}
+
+ $ curl -H "Content-Type: application/x-ndjson" -XGET localhost:9200/_msearch/template --data-binary "@requests"; echo
+
- `Get multiple term vectors.
-You can specify existing documents by index and ID or provide artificial documents in the body of the request. +
Get multiple term vectors with a single request.
+ You can specify existing documents by index and ID or provide artificial documents in the body of the request.
You can specify the index in the request body or request URI.
The response contains a docs
array with all the fetched termvectors.
Each element has the structure provided by the termvectors API.
Artificial documents
+You can also use mtermvectors
to generate term vectors for artificial documents provided in the body of the request.
+ The mapping used is determined by the specified _index
.
Evaluate the quality of ranked search results over a set of typical search queries.
- `Render a search template as a search request body.
- `Run a script. - Runs a script and returns a result.
+Run a script.
+Runs a script and returns a result. + Use this API to build and test scripts, such as when defining a script for a runtime field. + This API requires very few dependencies and is especially useful if you don't have permissions to write documents on a cluster.
+The API uses several contexts, which control how scripts are run, what variables are available at runtime, and what the return type is.
+Each context requires a script, but additional parameters depend on the context you're using for that script.
- `IMPORTANT: Results from a scrolling search reflect the state of the index at the time of the initial search request. Subsequent indexing or document changes only affect later search and scroll requests.
- `Search a vector tile.
-Search a vector tile for geospatial values.
- - - `Search a vector tile for geospatial values. + Before using this API, you should be familiar with the Mapbox vector tile specification. + The API returns results as a binary mapbox vector tile.
+Internally, Elasticsearch translates a vector tile search API request into a search containing:
+geo_bounding_box
query on the <field>
. The query uses the <zoom>/<x>/<y>
tile as a bounding box.geotile_grid
or geohex_grid
aggregation on the <field>
. The grid_agg
parameter determines the aggregation type. The aggregation uses the <zoom>/<x>/<y>
tile as a bounding box.geo_bounds
aggregation on the <field>
. The search only includes this aggregation if the exact_bounds
parameter is true
.with_labels
is true
, the internal search will include a dynamic runtime field that calls the getLabelPosition
function of the geometry doc value. This enables the generation of new point features containing suggested geometry labels, so that, for example, multi-polygons will have only one label.For example, Elasticsearch may translate a vector tile search API request with a grid_agg
argument of geotile
and an exact_bounds
argument of true
into the following search
GET my-index/_search
+ {
+ "size": 10000,
+ "query": {
+ "geo_bounding_box": {
+ "my-geo-field": {
+ "top_left": {
+ "lat": -40.979898069620134,
+ "lon": -45
+ },
+ "bottom_right": {
+ "lat": -66.51326044311186,
+ "lon": 0
+ }
+ }
+ }
+ },
+ "aggregations": {
+ "grid": {
+ "geotile_grid": {
+ "field": "my-geo-field",
+ "precision": 11,
+ "size": 65536,
+ "bounds": {
+ "top_left": {
+ "lat": -40.979898069620134,
+ "lon": -45
+ },
+ "bottom_right": {
+ "lat": -66.51326044311186,
+ "lon": 0
+ }
+ }
+ }
+ },
+ "bounds": {
+ "geo_bounds": {
+ "field": "my-geo-field",
+ "wrap_longitude": false
+ }
+ }
+ }
+ }
+
+ The API returns results as a binary Mapbox vector tile. + Mapbox vector tiles are encoded as Google Protobufs (PBF). By default, the tile contains three layers:
+hits
layer containing a feature for each <field>
value matching the geo_bounding_box
query.aggs
layer containing a feature for each cell of the geotile_grid
or geohex_grid
. The layer only contains features for cells with matching data.geotile_grid
or geohex_grid
.The API only returns features that can display at its zoom level. + For example, if a polygon feature has no area at its zoom level, the API omits it. + The API returns errors as UTF-8 encoded JSON.
+IMPORTANT: You can specify several options for this API as either a query parameter or request body parameter. + If you specify both parameters, the query parameter takes precedence.
+Grid precision for geotile
+For a grid_agg
of geotile
, you can use cells in the aggs
layer as tiles for lower zoom levels.
+ grid_precision
represents the additional zoom levels available through these cells. The final precision is computed by as follows: <zoom> + grid_precision
.
+ For example, if <zoom>
is 7 and grid_precision
is 8, then the geotile_grid
aggregation will use a precision of 15.
+ The maximum final precision is 29.
+ The grid_precision
also determines the number of cells for the grid as follows: (2^grid_precision) x (2^grid_precision)
.
+ For example, a value of 8 divides the tile into a grid of 256 x 256 cells.
+ The aggs
layer only contains features for cells with matching data.
Grid precision for geohex
+For a grid_agg
of geohex
, Elasticsearch uses <zoom>
and grid_precision
to calculate a final precision as follows: <zoom> + grid_precision
.
This precision determines the H3 resolution of the hexagonal cells produced by the geohex
aggregation.
+ The following table maps the H3 resolution for each precision.
+ For example, if <zoom>
is 3 and grid_precision
is 3, the precision is 6.
+ At a precision of 6, hexagonal cells have an H3 resolution of 2.
+ If <zoom>
is 3 and grid_precision
is 4, the precision is 7.
+ At a precision of 7, hexagonal cells have an H3 resolution of 3.
Precision | +Unique tile bins | +H3 resolution | +Unique hex bins | +Ratio | +
---|---|---|---|---|
1 | +4 | +0 | +122 | +30.5 | +
2 | +16 | +0 | +122 | +7.625 | +
3 | +64 | +1 | +842 | +13.15625 | +
4 | +256 | +1 | +842 | +3.2890625 | +
5 | +1024 | +2 | +5882 | +5.744140625 | +
6 | +4096 | +2 | +5882 | +1.436035156 | +
7 | +16384 | +3 | +41162 | +2.512329102 | +
8 | +65536 | +3 | +41162 | +0.6280822754 | +
9 | +262144 | +4 | +288122 | +1.099098206 | +
10 | +1048576 | +4 | +288122 | +0.2747745514 | +
11 | +4194304 | +5 | +2016842 | +0.4808526039 | +
12 | +16777216 | +6 | +14117882 | +0.8414913416 | +
13 | +67108864 | +6 | +14117882 | +0.2103728354 | +
14 | +268435456 | +7 | +98825162 | +0.3681524172 | +
15 | +1073741824 | +8 | +691776122 | +0.644266719 | +
16 | +4294967296 | +8 | +691776122 | +0.1610666797 | +
17 | +17179869184 | +9 | +4842432842 | +0.2818666889 | +
18 | +68719476736 | +10 | +33897029882 | +0.4932667053 | +
19 | +274877906944 | +11 | +237279209162 | +0.8632167343 | +
20 | +1099511627776 | +11 | +237279209162 | +0.2158041836 | +
21 | +4398046511104 | +12 | +1660954464122 | +0.3776573213 | +
22 | +17592186044416 | +13 | +11626681248842 | +0.6609003122 | +
23 | +70368744177664 | +13 | +11626681248842 | +0.165225078 | +
24 | +281474976710656 | +14 | +81386768741882 | +0.2891438866 | +
25 | +1125899906842620 | +15 | +569707381193162 | +0.5060018015 | +
26 | +4503599627370500 | +15 | +569707381193162 | +0.1265004504 | +
27 | +18014398509482000 | +15 | +569707381193162 | +0.03162511259 | +
28 | +72057594037927900 | +15 | +569707381193162 | +0.007906278149 | +
29 | +288230376151712000 | +15 | +569707381193162 | +0.001976569537 | +
Hexagonal cells don't align perfectly on a vector tile. + Some cells may intersect more than one vector tile. + To compute the H3 resolution for each precision, Elasticsearch compares the average density of hexagonal bins at each resolution with the average density of tile bins at each zoom level. + Elasticsearch uses the H3 resolution that is closest to the corresponding geotile density.
+ + + `Get the search shards.
Get the indices and shards that a search request would be run against. This information can be useful for working out issues or planning optimizations with routing and shard preferences. - When filtered aliases are used, the filter is returned as part of the indices section.
+ When filtered aliases are used, the filter is returned as part of theindices
section.
+ If the Elasticsearch security features are enabled, you must have the view_index_metadata
or manage
index privilege for the target data stream, index, or alias.
Run a search with a search template.
- `Get terms in an index.
Discover terms that match a partial string in an index. - This "terms enum" API is designed for low-latency look-ups used in auto-complete scenarios.
-If the complete
property in the response is false, the returned terms set may be incomplete and should be treated as approximate.
- This can occur due to a few reasons, such as a request timeout or a node error.
NOTE: The terms enum API may return terms from deleted documents. Deleted documents are initially only marked as deleted. It is not until their segments are merged that documents are actually deleted. Until that happens, the terms enum API will return terms from these documents.
+ This API is designed for low-latency look-ups used in auto-complete scenarios. ++- `info + The terms enum API may return terms from deleted documents. Deleted documents are initially only marked as deleted. It is not until their segments are merged that documents are actually deleted. Until that happens, the terms enum API will return terms from these documents.
+
Get term vector information.
Get information and statistics about terms in the fields of a particular document.
+You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request.
+ You can specify the fields you are interested in through the fields
parameter or by adding the fields to the request body.
+ For example:
GET /my-index-000001/_termvectors/1?fields=message
+
+ Fields can be specified using wildcards, similar to the multi match query.
+Term vectors are real-time by default, not near real-time.
+ This can be changed by setting realtime
parameter to false
.
You can request three types of values: term information, term statistics, and field statistics. + By default, all term information and field statistics are returned for all fields but term statistics are excluded.
+Term information
+positions: true
)offsets: true
)payloads: true
), as base64 encoded bytesIf the requested information wasn't stored in the index, it will be computed on the fly if possible. + Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.
+++warn + Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.
+
Behaviour
+The term and field statistics are not accurate.
+ Deleted documents are not taken into account.
+ The information is only retrieved for the shard the requested document resides in.
+ The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context.
+ By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected.
+ Use routing
only to hit a particular shard.
_source
, you can access the following variables through the ctx
map: _index
, _type
, _id
, _version
, _routing
, and _now
(the current timestamp).
- `Update documents. Updates documents that match the specified query. If no query is specified, performs an update on every document in the data stream or index without modifying the source, which is useful for picking up mapping changes.
+If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or alias:
+read
index
or write
You can specify the query criteria in the request URI or the request body using the same syntax as the search API.
+When you submit an update by query request, Elasticsearch gets a snapshot of the data stream or index when it begins processing the request and updates matching documents using internal versioning.
+ When the versions match, the document is updated and the version number is incremented.
+ If a document changes between the time that the snapshot is taken and the update operation is processed, it results in a version conflict and the operation fails.
+ You can opt to count version conflicts instead of halting and returning by setting conflicts
to proceed
.
+ Note that if you opt to count version conflicts, the operation could attempt to update more documents from the source than max_docs
until it has successfully updated max_docs
documents or it has gone through every document in the source query.
NOTE: Documents with a version equal to 0 cannot be updated using update by query because internal versioning does not support 0 as a valid version number.
+While processing an update by query request, Elasticsearch performs multiple search requests sequentially to find all of the matching documents. + A bulk update request is performed for each batch of matching documents. + Any query or update failures cause the update by query request to fail and the failures are shown in the response. + Any update requests that completed successfully still stick, they are not rolled back.
+Throttling update requests
+To control the rate at which update by query issues batches of update operations, you can set requests_per_second
to any positive decimal number.
+ This pads each batch with a wait time to throttle the rate.
+ Set requests_per_second
to -1
to turn off throttling.
Throttling uses a wait time between batches so that the internal scroll requests can be given a timeout that takes the request padding into account.
+ The padding time is the difference between the batch size divided by the requests_per_second
and the time spent writing.
+ By default the batch size is 1000, so if requests_per_second
is set to 500
:
target_time = 1000 / 500 per second = 2 seconds
+ wait_time = target_time - write_time = 2 seconds - .5 seconds = 1.5 seconds
+
+ Since the batch is issued as a single _bulk request, large batch sizes cause Elasticsearch to create many requests and wait before starting the next set. + This is "bursty" instead of "smooth".
+Slicing
+Update by query supports sliced scroll to parallelize the update process. + This can improve efficiency and provide a convenient way to break the request down into smaller parts.
+Setting slices
to auto
chooses a reasonable number for most data streams and indices.
+ This setting will use one slice per shard, up to a certain limit.
+ If there are multiple source data streams or indices, it will choose the number of slices based on the index or backing index with the smallest number of shards.
Adding slices
to _update_by_query
just automates the manual process of creating sub-requests, which means it has some quirks:
slices
only contains the status of completed slices.slices
will rethrottle the unfinished sub-request proportionally.requests_per_second
and max_docs
on a request with slices are distributed proportionally to each sub-request. Combine that with the point above about distribution being uneven and you should conclude that using max_docs
with slices
might not result in exactly max_docs
documents being updated.If you're slicing manually or otherwise tuning automatic slicing, keep in mind that:
+Whether query or update performance dominates the runtime depends on the documents being reindexed and cluster resources.
+Update the document source
+Update by query supports scripts to update the document source.
+ As with the update API, you can set ctx.op
to change the operation that is performed.
Set ctx.op = "noop"
if your script decides that it doesn't have to make any changes.
+ The update by query operation skips updating the document and increments the noop
counter.
Set ctx.op = "delete"
if your script decides that the document should be deleted.
+ The update by query operation deletes the document and increments the deleted
counter.
Update by query supports only index
, noop
, and delete
.
+ Setting ctx.op
to anything else is an error.
+ Setting any other field in ctx
is an error.
+ This API enables you to only modify the source of matching documents; you cannot move them.
cancel_task
cluster privilege.
- `Get the async search status.
Get the status of a previously submitted async search request given its identifier, without retrieving search results.
- If the Elasticsearch security features are enabled, use of this API is restricted to the monitoring_user
role.
monitor
cluster privilege or greater privileges.search.max_async_search_response_size
cluster level setting.
- `NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
- `NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
- `NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
- `IMPORTANT: CAT APIs are only intended for human consumption using the command line or the Kibana console. They are not intended for use by applications. For application consumption, use the aliases API.
- `IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications.
- `Get help for the CAT APIs.
- `IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.
- `Delete auto-follow patterns. - Delete a collection of cross-cluster replication auto-follow patterns.
+Delete auto-follow patterns.
+Delete a collection of cross-cluster replication auto-follow patterns.
- `Get follower information. - Get information about all cross-cluster replication follower indices. +
Get follower information.
+Get information about all cross-cluster replication follower indices. For example, the results include follower index names, leader index names, replication options, and whether the follower indices are active or paused.
- `Get follower stats. - Get cross-cluster replication follower stats. +
Get follower stats.
+Get cross-cluster replication follower stats. The API returns shard-level stats about the "following tasks" associated with each shard for the specified indices.
- `Get auto-follow patterns. - Get cross-cluster replication auto-follow patterns.
+Get auto-follow patterns.
+Get cross-cluster replication auto-follow patterns.
- `Pause an auto-follow pattern. - Pause a cross-cluster replication auto-follow pattern. +
Pause an auto-follow pattern.
+Pause a cross-cluster replication auto-follow pattern. When the API returns, the auto-follow pattern is inactive. New indices that are created on the remote cluster and match the auto-follow patterns are ignored.
You can resume auto-following with the resume auto-follow pattern API. @@ -498,11 +505,13 @@ async def pause_auto_follow_pattern( Remote indices that were created while the pattern was paused will also be followed, unless they have been deleted or closed in the interim.
- `Pause a follower. - Pause a cross-cluster replication follower index. +
Pause a follower.
+Pause a cross-cluster replication follower index. The follower index will not fetch any additional operations from the leader index. You can resume following with the resume follower API. You can pause and resume a follower index to change the configuration of the following task.
- `Resume an auto-follow pattern. - Resume a cross-cluster replication auto-follow pattern that was paused. +
Resume an auto-follow pattern.
+Resume a cross-cluster replication auto-follow pattern that was paused. The auto-follow pattern will resume configuring following indices for newly created indices that match its patterns on the remote cluster. Remote indices created while the pattern was paused will also be followed unless they have been deleted or closed in the interim.
- `Get cross-cluster replication stats. - This API returns stats about auto-following and the same shard-level stats as the get follower stats API.
+Get cross-cluster replication stats.
+This API returns stats about auto-following and the same shard-level stats as the get follower stats API.
- `Unfollow an index. - Convert a cross-cluster replication follower index to a regular index. +
Unfollow an index.
+Convert a cross-cluster replication follower index to a regular index. The API stops the following task associated with a follower index and removes index metadata and settings associated with cross-cluster replication. The follower index must be paused and closed before you call the unfollow API.
-NOTE: Currently cross-cluster replication does not support converting an existing regular index to a follower index. Converting a follower index to a regular index is an irreversible operation.
++- `info + Currently cross-cluster replication does not support converting an existing regular index to a follower index. Converting a follower index to a regular index is an irreversible operation.
+
composed_of
list.
- `Get remote cluster information. - Get all of the configured remote cluster information. - This API returns connection and endpoint information keyed by the configured remote cluster alias.
+Get remote cluster information.
+Get information about configured remote clusters. + The API returns connection and endpoint information keyed by the configured remote cluster alias.
++- `info + This API returns information that reflects current state on the local cluster. + The
+connected
field does not necessarily reflect whether a remote cluster is down or unavailable, only whether there is currently an open connection to it. + Elasticsearch does not spontaneously try to reconnect to a disconnected remote cluster. + To trigger a reconnection, attempt a cross-cluster search, ES|QL cross-cluster search, or try the resolve cluster endpoint.
Once the problem has been corrected, allocation can be manually retried by calling the reroute API with the ?retry_failed
URI query parameter, which will attempt a single retry round for these shards.
Update the last_seen
field in the connector and set it to the current timestamp.
Get the details about a connector.
- `Get information about all connectors.
- `Create or update a connector.
- `Get a connector sync job.
- `Get information about all stored connector sync jobs listed by their creation date in ascending order.
- `