Skip to content

Reduce copying during parsing #60

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 19, 2023
Merged

Conversation

MichaReiser
Copy link
Contributor

@MichaReiser MichaReiser commented May 18, 2023

This removes the need for copying statements and other nodes during parsing.

Performance

This gives me a 4% performance improvement in ruff's parser benchmark (parses a couple of files and traverses the AST)

group                        main                                   stmt
-----                        ----                                   ----
parser/large/dataset.py      1.04      2.6±0.01ms    15.4 MB/sec    1.00      2.5±0.02ms    16.0 MB/sec
parser/numpy/ctypeslib.py    1.04    499.8±3.36µs    33.3 MB/sec    1.00    478.8±4.59µs    34.8 MB/sec
parser/numpy/globals.py      1.04     51.2±0.76µs    57.6 MB/sec    1.00     49.2±0.45µs    60.0 MB/sec
parser/pydantic/types.py     1.04  1100.1±11.99µs    23.2 MB/sec    1.00   1061.7±9.17µs    24.0 MB/sec

head
},

// The first compound statement
<s:CompoundStatement> => vec![s],
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old grammar created a vec for every statement. The new grammar simply appends statements at the end of the vec.

Copy link
Member

@youknowone youknowone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 awesome

MichaReiser and others added 2 commits May 18, 2023 22:32
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants