Skip to content

X-Opaque-ID header can cause "No colon found" in Hive #1872

Closed
@masseyke

Description

@masseyke

As of #1770 we are putting the hadoop job's name into the request as the X-Opaque-ID header for the sake of troubleshooting. Unfortunately Hive job names often contain a newline. When put into a header this is interpreted as a second header. And since there is no : you get an error like

2022-01-19T10:10:34.398-0600 [ERROR] [system.err] Caused by: org.elasticsearch.hadoop.rest.EsHadoopInvalidRequest: org.elasticsearch.hadoop.rest.EsHadoopRemoteException: illegal_argument_exception: No colon found
2022-01-19T10:10:34.398-0600 [ERROR] [system.err] null
2022-01-19T10:10:34.398-0600 [ERROR] [system.err]       at org.elasticsearch.hadoop.rest.RestClient.checkResponse(RestClient.java:487)
2022-01-19T10:10:34.398-0600 [ERROR] [system.err]       at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:444)
2022-01-19T10:10:34.398-0600 [ERROR] [system.err]       at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:438)
2022-01-19T10:10:34.398-0600 [ERROR] [system.err]       at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:406)
2022-01-19T10:10:34.398-0600 [ERROR] [system.err]       at org.elasticsearch.hadoop.rest.RestClient.mainInfo(RestClient.java:755)
2022-01-19T10:10:34.398-0600 [ERROR] [system.err]       at org.elasticsearch.hadoop.hive.EsStorageHandler.configureJobConf(EsStorageHandler.java:163)
2022-01-19T10:10:34.399-0600 [ERROR] [system.err]       ... 25 more

Hive queries could also have non-ASCII characters, which are invalid for http headers. This could also cause problems (unverified).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions