Skip to content

Patch float with trailing zero for JSON and add useFloatWithTrailingZero flag #1038

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

chenhuiyeh
Copy link

@chenhuiyeh chenhuiyeh commented May 21, 2025

This pull request enables support for MySQL 8+ for retaining trailing zeros in floating-point numbers in JSON format. This is achieved through the addition of the useFloatWithTrailingZero setting option within the go-mysql library. Note this only works with MySQL version 8 onwards.

A patch addressing this feature was initially applied in ghostferry PR #368. This current PR seeks to integrate these updates into the upstream codebase to extend functionality for users and projects relying on this library.

This change is part of the broader initiative referenced in ghostferry PR #384.

@chenhuiyeh chenhuiyeh changed the title batch float with trailing zero and add a useFloatWithTrailingZero fla… patch float with trailing zero and add useFloatWithTrailingZero flag May 21, 2025
@dveeden
Copy link
Collaborator

dveeden commented May 21, 2025

It would be good to add some tests and a description or an issue link.

@chenhuiyeh chenhuiyeh changed the title patch float with trailing zero and add useFloatWithTrailingZero flag Patch float with trailing zero for JSON and add useFloatWithTrailingZero flag May 21, 2025
@chenhuiyeh chenhuiyeh force-pushed the float_with_trailing_zero branch from 814aa96 to 9d46fb3 Compare May 21, 2025 19:29
@lance6716
Copy link
Collaborator

Hi, please fix the CI. And I'll take a look after you remove the draft status of PR.

@lance6716
Copy link
Collaborator

Hi, do you need some help?

@chenhuiyeh
Copy link
Author

chenhuiyeh commented Jun 4, 2025 via email

@lance6716

This comment was marked as resolved.

@@ -0,0 +1,403 @@
package replication
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided to pull the unit tests out specifically for this feature/fix

@chenhuiyeh chenhuiyeh marked this pull request as ready for review June 5, 2025 19:20
Copy link
Collaborator

@lance6716 lance6716 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest lgtm. some linter errors

Error: replication/json_binary_test.go:372:14: unusedwrite: unused write to field tableIDSize (govet)
  		tableIDSize:              6,
  		           ^
  Error: replication/json_binary_test.go:374:10: unusedwrite: unused write to field Version (govet)
  		Version:                  2,
  		       ^
  Error: replication/json_binary_test.go:381:14: unusedwrite: unused write to field tableIDSize (govet)
  		tableIDSize:              6,
  		           ^
  Error: replication/json_binary_test.go:383:10: unusedwrite: unused write to field Version (govet)
  		Version:                  2,
  		       ^
  Error: replication/json_binary_test.go:297:[36](https://github.com/go-mysql-org/go-mysql/actions/runs/15475222894/job/43585097889?pr=1038#step:4:38): SA4026: in Go, the floating-point literal '-0.0' is the same as '0.0', it does not produce a negative zero (staticcheck)
  			input:    FloatWithTrailingZero(-0.0),
  			                                ^

},
}

// 5.0 as IEEE 754 double precision in little endian
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nits: I think you can use https://pkg.go.dev/math#Float64bits and binary.BigEndian.PutUint64. This line with comment is also good enough.

useFloatWithTrailingZero: true,
}

// Mock a simple JSON binary that would contain a double value
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the codes are not aligned with these comments. Maybe delete this test?

require.False(t, parser.useFloatWithTrailingZero)
}

func TestBinlogSyncerConfig_UseFloatWithTrailingZero(t *testing.T) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think this test is also a bit trivial, we don't need it.

},
{
name: "negative zero",
input: FloatWithTrailingZero(-0.0),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you really want to test -0.0, but it can't be expressed by golang literal. I think you can manually get the IEEE 754 byte representation and check it in other test cases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants