Resolve parsing error for CHARACTER SET and COLLATE in MySQL ALTER TABLE (issue 2027)

[minleejae]

Problem

When attempting to parse ALTER TABLE statements in MySQL that include CHARACTER SET and COLLATE specifications for column types such as text, tinytext, mediumtext, and longtext, a ParseException is thrown. This issue is currently present in JSqlParser version 5.0.
#2027

Solution

This PR introduces updates to the parsing rules within JSqlParser, allowing for correct handling of CHARACTER SET and COLLATE clauses in MySQL ALTER TABLE statements. The changes involve modifying the token recognition patterns and parser rules to ensure that these keywords are properly interpreted when following the mentioned text data types.

Tests

The following test cases have been added to verify the correct parsing of modified statements:

• Simple ALTER TABLE with text
• ALTER TABLE with text, including CHARACTER SET and COLLATE
• The same checks for tinytext, mediumtext, and longtext.

These tests ensure that the parser remains stable and performs as expected across these common use cases in MySQL.

I look forward to feedback and further suggestions!

[manticore-projects]

fixes #2027

[minleejae]

Hi, @manticore-projects
In my development environment, the NestedBracketsPerformanceTest passes successfully, but I'm not sure why the Gradle build is failing.

[manticore-projects]

Hi, @manticore-projects In my development environment, the NestedBracketsPerformanceTest passes successfully, but I'm not sure why the Gradle build is failing.

Greetings!

Likely because the GitHub build environment is slower (cpu clock/speed wise) than your local environment -- meaning that your change still slows the parser down, which is not a welcome development.

[minleejae]

@manticore-projects
I understood the cause of the problem.
I don't think I've changed the code much enough to have a big impact on the overall performance, but what should I do to fix this?

[manticore-projects]

I don't think I've changed the code much enough to have a big impact on the overall performance, but what should I do to fix this?

Any change in the Grammar can have potentially big impact.
Also, the impact does not need to be big, likely just something like the "straw that broke the camel's back".

I would still merge it (and fix this performance exception manually by adding timeout). However, please address my concern on the TEXT token first -- I need it as a data type token.

[minleejae]

@manticore-projects
I understand the potential impact of changes to the grammar and will consider ways to further enhance performance. Additionally, I have updated the code to include "TEXT" as a data type token as requested. d8fc0d8

[manticore-projects]

Something is wrong. In the Github Maven task I see:

Error:  COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
Error:  /home/runner/work/JSqlParser/JSqlParser/target/generated-sources/javacc/net/sf/jsqlparser/parser/CCJSqlParser.java:[22081,7] unreachable statement

However, when pulling your PR and compile locally everything seems to work. This confuses the hell out me right now.

[manticore-projects]

I have merged it and will test/fix whatever problem directly.
Thank you for your contribution. Cheers!

[manticore-projects]

Your changes cause real breakage:

    case K_TEXT_LITERAL:{
      tk = jj_consume_token(K_TEXT_LITERAL);
type = tk.image;
            return new ColDataType(type, precision, scale);
      break;
      }

Although I don't understand why gradle check works but Maven does not.

[minleejae]

@manticore-projects
I apologize for not catching this issue in my environment. It seems like it has been resolved in the main branch, so thank you for fixing it.

[manticore-projects]

@manticore-projects I apologize for not catching this issue in my environment. It seems like it has been resolved in the main branch, so thank you for fixing it.

No problem at all, that's what team work is for. Thank you again for your help and contribution!
Looking forward seeing more MySQL related improvements from you. Cheers!