Logging implementation needs to be better

[djMax]

I'm not yet sure how exactly to make it better, but the current logging implementation prevents me from associating a request context with the log messages. e.g. in a distributed system I assign a correlation id to all requests at the highest level, and all log messages should contain that id. But the signature of the ES logging infra prevents me from doing that. ONE option would be some sort of "shadow" client instance that reuses the connection of the root instance but has it's own logging implementation. There may be better options that I can't see without being more familiar with the ES code.

[delvedor]

Hi! Thank you for the ideas!
We are currently in the process of redesigning the client (I'll publish some code soon) and the logging will be completely changed.
The idea is to have a debug logger for development purposes and some events which you can listen to that will give you all the data you need.

Example:

client.on('request', requestObject => {
  console.log(requestObject)
})

client.on('response', (requestObject, responseObject) => {
  console.log(requestObject)
  console.log(responseObject)
})

client.on('error', error => {
  console.log(error)
})

In this way, you can implement your own logging solution and logic.

What do you think? :)

[djMax]

Who makes that request object? If it's the es client, the problem still is I have no way at the API level to provide context. Either there needs to be some way to pass context to ALL api functions (that makes its way to these events), or there needs to be a way to get a client that shares everything but the logger. Know what I mean?

[djMax]

Basically every object would need some sort of "resource-conserving" clone method.

[delvedor]

Can you make a code example of an ideal solution for you?

[djMax]

The ES node module is fairly complex and what I would call "old school" Javascript, so it's very hard to fit my desire into that model. BUT, the key is that those events (request/response/error) have some way to tie back to some element of the first entry into the API. So if the choice was to modify all APIs to take some sort of extra param or option called "context" (or whatever), the events would include that argument in their arguments. If, on the other hand, we chose to allow a client to be "easily copied" such that it shared connections and such, then it would be something like "client.cloneWithContext(context)" and then again, the log events would include that context.

[delvedor]

The ES node module is fairly complex and what I would call "old school" Javascript, so it's very hard to fit my desire into that model.

As said above, we are working on reimplementing the client, so this is the best moment to ask for features request!
You can find the initial prototype here.

So if the choice was to modify all APIs to take some sort of extra param or option called "context" (or whatever), the events would include that argument in their arguments.

If I've understood correctly you need something like the following:

client.on('response', (requestObject, responseObject) => {
  console.log(requestObject.context.id) // 1234
})

client.search({
  index: 'my-index',
  type: 'doc',
  query: {
    match: { foo: 'bar' }
  },
  context: {
    id: 1234
  }
}, (err, body) => { ... })

Would this solve your issue?

[djMax]

It would definitely solve it.

[delvedor]

That's something that we could arrange; we should find a good name that not clash with Elasticsearch's options.

[delvedor]

Closed in #834.