15.4.1 breaking change - unrecognized parameter: _sourceInclude, _sourceExclude

[onehorsetown]

Not an issue, just a heads up to others that might search for this error.

[illegal_argument_exception] request [...] contains unrecognized parameter: [_sourceInclude] -> did you mean any of [_source_include, _source_includes, _source_exclude, _source_excludes]

[illegal_argument_exception] request [...] contains unrecognized parameters: [_sourceExclude] -> did you mean any of [_source_exclude, _source_excludes, _source_include, _source_includes]?, [_sourceInclude] -> did you mean any of [_source_include, _source_includes, _source_exclude, _source_excludes]?

Upon upgrading to version 15.4.1, my code broke because the breaking change documented here:

https://www.elastic.co/guide/en/elasticsearch/reference/6.6/breaking-changes-6.6.html

Since the TypeScript types (https://www.npmjs.com/package/@types/elasticsearch) have not been updated to support the renamed properties, the easiest solution to explicitly set the apiVersion when constructing the Elasticsearch Client object:

new Client({  apiVersion: '6.5', ... }

otherwise the library will use client version 6.6 and your existing code will break.

[austince]

As a generally related question, what is the reasoning behind releasing breaking changes as patch releases, as was done in 15.4.1?

[delvedor]

Hello!
It is not a breaking change since the parameters will continue to work as expected.
In Elasticsearch 6.6 you will get a deprecation warning, while in Elasticsearch they will not work anymore.

The main issue here is that we generate the APIs from a given spec and the client offers both snake_case and camelCase query parameters. If you pass a query parameter in camelCase internally it will be converted to snake_case, unless it's an unknown parameter, in which case it will be sent as is.
I'll think about how this can be solved, thanks!

[devagirir]

Hi Tomas, I gave query in snake case, following the format which I sent to client. {"index":"sampleindex","size":1000,"sort":"doc.ddid.keyword:asc","search_after":"00D1I000003nRMaUAM-11e9-bd8f-4f1e395f9c1c"} Let me know if I can pass in any other format to make it work. Thanks, Ravindra From: Tomas Della Vedova <notifications@github.com> Sent: 20 February 2019 15:52 To: elastic/elasticsearch-js <elasticsearch-js@noreply.github.com> Cc: Subscribed <subscribed@noreply.github.com> Subject: Re: [elastic/elasticsearch-js] 15.4.1 breaking change - unrecognized parameter: _sourceInclude, _sourceExclude (#765) Hello! It is not a breaking change since the parameters will continue to work as expected. In Elasticsearch 6.6 you will get a deprecation warning, while in Elasticsearch they will not work anymore. The main issue here is that we generate the APIs from a given spec and the client offers both snake_case and camelCase query parameters. If you pass a query parameter in camelCase internally it will be converted to snake_case, unless it's an unknown parameter, in which case it will be sent as is. I'll think about how this can be solved, thanks! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#765 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/As0uRvu0Gb2x6-WZlzRO7Ry2CqFAzUIFks5vPSGugaJpZM4an2-4>.

[delvedor]

@devagirir it seems that yours is a different situation, can you please open a new issue with an example to reproduce?

[austince]

@delvedor, perhaps this instance isn't a breaking change, but other breaking changes are released in patch releases, such as dropping the 5.x line in 15.4.1. Happy to open up another issue about this if you think that is best - just want to get an idea of the methodology you use for versioning.

[peterdemartini]

I disagree, this IS a breaking change since upgrading it caused our code to break, without changing anything else. This change caused the client to fail before it even tried to send the request to the server, so the depreciation warning you speak of never happened. In order to fix this, we had to explicitly set the apiVersion to 6.5.

[onehorsetown]

Yes, it was a breaking change, but at the same time I bear some responsibility in that I never explicitly set the apiVersion value.

new Client({ apiVersion: '6.5', ... }

Perhaps this should be a required parameter in the next major release.

[delvedor]

I'm sorry for the inconvenience. Unfortunately we can't fix right now.
In any case, I highly recommend always to set the apiVersion.

Perhaps this should be a required parameter in the next major release.

We are working on a new client, you can find it in the next branch. We will no longer ship all the versions of the API, but the version of the package will also be the version of Elasticsearch.
The new package will be released as @elastic/elasticsearch in npm.

This means that if you install version 6.5 of the client, you will have the version 6.5 of the API and so on.

We are very close to finishing it, I'll let you know more as soon as I can :)

[austince]

Even if you set the apiVersion explicitly, version 15.4.1 dropped support for anything below 5.6, which is breaking. In the short lifespan this versioning schema has left, can breaking changes be released as major releases?

[delvedor]

With the current client, it is difficult to support all the version of the API at once because of the package size (and as said, the inconvenient of the camelCase vs snake_case), with the next one every breaking change will be released as a major release.

[austince]

@delvedor - a potential problem occured to me while working on a project that connects to multiple different elasticsearch clusters, all running different versions: with the new versioning scheme, where package version 6.3 maps directly to API version 6.3, how will one be able to handle multiple different connections? Will we really have to alias every version we need to use?

[stale]

We understand that this might be important for you, but this issue has been automatically marked as stale because it has not had recent activity either from our end or yours.
It will be closed if no further activity occurs, please write a comment if you would like to keep this going.

Note: in the past months we have built a new client, that has just landed in master. If you want to open an issue or a pr for the legacy client, you should do that in https://github.com/elastic/elasticsearch-js-legacy