Skip to content

BUG: column names with degree sign make query fail #42826

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Aug 4, 2021
+13 −0
Merged

BUG: column names with degree sign make query fail #42826

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e7fd1c0
Update parsing.py
fdion Jul 30, 2021
1032161
Merge pull request #1 from fdion/handle-common-degree-symbol
fdion Jul 30, 2021
53f9b69
adding test
fdion Jul 31, 2021
1135217
adding whatsnew
fdion Jul 31, 2021
b52f8dd
addressing feedback
fdion Aug 3, 2021
0aa3c62
addressing feedback
fdion Aug 4, 2021
5e10cd3
resolve conflict
fdion Aug 4, 2021
d4f1f7a
Merge branch 'master' into master
fdion Aug 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions doc/source/whatsnew/v1.4.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,8 @@ Indexing
- Bug in indexing on a :class:`MultiIndex` failing to drop scalar levels when the indexer is a tuple containing a datetime-like string (:issue:`42476`)
- Bug in :meth:`DataFrame.sort_values` and :meth:`Series.sort_values` when passing an ascending value, failed to raise or incorrectly raising ``ValueError`` (:issue:`41634`)
- Bug in updating values of :class:`pandas.Series` using boolean index, created by using :meth:`pandas.DataFrame.pop` (:issue:`42530`)
- Bug in :meth:`DataFrame.query` did not handle the degree sign in a backticked column name, such as \`Temp(°C)\`, used in an expression to query a dataframe (:issue:`42826`)
-

Missing
^^^^^^^
Expand Down
1 change: 1 addition & 0 deletions pandas/core/computation/parsing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ def create_valid_python_identifier(name: str) -> str:
"!": "_EXCLAMATIONMARK_",
"$": "_DOLLARSIGN_",
"€": "_EUROSIGN_",
"°": "_DEGREESIGN_",
# Including quotes works, but there are exceptions.
"'": "_SINGLEQUOTE_",
'"': "_DOUBLEQUOTE_",
Expand Down
10 changes: 10 additions & 0 deletions pandas/tests/computation/test_eval.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2035,6 +2035,16 @@ def test_truediv_deprecated(engine, parser):
assert match in str(m[0].message)


@pytest.mark.parametrize("column", ["Temp(°C)", "Capacitance(μF)"])
def test_query_token(engine, column):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add a comment referencing this pr here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and done

# See: https://github.com/pandas-dev/pandas/pull/42826
df = DataFrame(np.random.randn(5, 2), columns=[column, "b"])
expected = df[df[column] > 5]
query_string = f"`{column}` > 5"
result = df.query(query_string, engine=engine)
tm.assert_frame_equal(result, expected)


def test_negate_lt_eq_le(engine, parser):
df = DataFrame([[0, 10], [1, 20]], columns=["cat", "count"])
expected = df[~(df.cat > 0)]
Expand Down