Skip to content

Add Redshift dialect, handle square brackets properly #471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 4, 2022

Conversation

mskrzypkows
Copy link
Contributor

We need to detect [ or " for Redshift quotes around indentifier and at the same time exclude
treating JSON paths as indentifer

We need to detect `[` or `"` for Redshift quotes around indentifier and at the same time exclude
treating JSON paths as indentifer
@mskrzypkows
Copy link
Contributor Author

Hi @alamb, could you give me some feedback? May the PR be reviewed?

@alamb alamb changed the title Redshift square bracket handling Add Redshift dialect, handle square brackets properly May 4, 2022
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @mskrzypkows for the contribution.

I am sorry for the delay.

I am not super familiar with the Redshift SQL dialect, nor have I used it and I don't have access to such a system. I didn't find any mention of using [ and ] as quote characters in the redshift docs. Here is where I looked: https://docs.aws.amazon.com/redshift/latest/dg/r_names.html

Can you maybe point me at some examples or reference documentation with this syntax?

I think in general this PR is looking good to merge because:

  1. It is a new Dialect (and thus existing dialects are unaffected)
  2. It has good test coverate

I left some small style / documentation suggestions

}

fn is_identifier_start(&self, ch: char) -> bool {
PostgreSqlDialect {}.is_identifier_start(ch)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

/// It's needed to distinguish treating square brackets as quotes from
/// treating them as json path. If there is identifier then we assume
/// there is no json path.
fn is_proper_identifier_inside_quotes(&self, mut _chars: Peekable<Chars<'_>>) -> bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
fn is_proper_identifier_inside_quotes(&self, mut _chars: Peekable<Chars<'_>>) -> bool {
fn is_proper_identifier_inside_quotes(&self, mut chars: Peekable<Chars<'_>>) -> bool {

I think typically variables that are not used get prefixed with_ while those that are used do not have the _ prefix.

@coveralls
Copy link

coveralls commented May 4, 2022

Pull Request Test Coverage Report for Build 2270327359

  • 62 of 65 (95.38%) changed or added relevant lines in 5 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.03%) to 90.426%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/dialect/mod.rs 1 2 50.0%
src/dialect/redshift.rs 9 10 90.0%
src/tokenizer.rs 4 5 80.0%
Totals Coverage Status
Change from base Build 2260629052: 0.03%
Covered Lines: 8151
Relevant Lines: 9014

💛 - Coveralls

@mskrzypkows
Copy link
Contributor Author

Can you maybe point me at some examples or reference documentation with this syntax?

You're right, it's hard to find some documentation mentioning support for [ and ]. I've tried to find something, but failed. I know it works from tests on Redshift DB. Furthermore, I don't know why it's not present in the documentation, while DB support it. I filled the "Documentation feedback form" so hopefully Amazon will mention support for square brackets soon.
I suppose that Redshift mimics this behaviour from MSSQL: https://docs.microsoft.com/en-us/sql/relational-databases/databases/database-identifiers?view=sql-server-ver15

@alamb
Copy link
Contributor

alamb commented May 4, 2022

Thanks again @mskrzypkows !

@alamb alamb merged commit 7fc6361 into apache:main May 4, 2022
@mskrzypkows
Copy link
Contributor Author

My pleasure. Glad that it's merged :-)

@alamb
Copy link
Contributor

alamb commented May 4, 2022 via email

ovr pushed a commit to cube-js/sqlparser-rs that referenced this pull request Sep 20, 2022
* Redshift square bracket handling

We need to detect `[` or `"` for Redshift quotes around indentifier and at the same time exclude
treating JSON paths as indentifer

* RedshiftSqlDialect documentation update

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* Renamed _chars to chars

* Fixed warnings

* Missing license

Co-authored-by: Maciej Skrzypkowski <maciej.skrzypkowski@satoricyber.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
ovr pushed a commit to cube-js/sqlparser-rs that referenced this pull request Sep 20, 2022
* Redshift square bracket handling

We need to detect `[` or `"` for Redshift quotes around indentifier and at the same time exclude
treating JSON paths as indentifer

* RedshiftSqlDialect documentation update

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* Renamed _chars to chars

* Fixed warnings

* Missing license

Co-authored-by: Maciej Skrzypkowski <maciej.skrzypkowski@satoricyber.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants