ujson Py2 compat removal #26199
ujson Py2 compat removal #26199
Conversation
| #define PY_ARRAY_UNIQUE_SYMBOL UJSON_NUMPY | ||
| #define NO_IMPORT_ARRAY | ||
| #include <numpy/arrayobject.h> // NOLINT(build/include_order) | ||
| #include <ultrajson.h> // NOLINT(build/include_order) | ||
| #define PY_SSIZE_T_CLEAN |
There was a problem hiding this comment.
This definition is suggested by the Python developers:
https://docs.python.org/3/c-api/intro.html#include-files
I noticed we haven't used anywhere else so could probably be added as a follow up in other places
| #include <numpy/arrayscalars.h> // NOLINT(build/include_order) | ||
| #include <numpy/ndarraytypes.h> // NOLINT(build/include_order) | ||
| #include <numpy/npy_math.h> // NOLINT(build/include_order) | ||
| #include <stdio.h> // NOLINT(build/include_order) |
There was a problem hiding this comment.
#include <Python.h> implies this so no need to state again
Codecov Report
@@ Coverage Diff @@
## master #26199 +/- ##
==========================================
- Coverage 91.98% 91.98% -0.01%
==========================================
Files 175 175
Lines 52372 52372
==========================================
- Hits 48176 48172 -4
- Misses 4196 4200 +4
Continue to review full report at Codecov.
|
Codecov Report
@@ Coverage Diff @@
## master #26199 +/- ##
==========================================
- Coverage 91.98% 91.98% -0.01%
==========================================
Files 175 175
Lines 52372 52372
==========================================
- Hits 48177 48173 -4
- Misses 4195 4199 +4
Continue to review full report at Codecov.
|
| @@ -530,7 +528,7 @@ PyObject *JSONToObj(PyObject *self, PyObject *args, PyObject *kwargs) { | |||
| decoder->preciseFloat = 1; | |||
| } | |||
|
|
|||
| if (PyString_Check(arg)) { | |||
| if (PyBytes_Check(arg)) { | |||
There was a problem hiding this comment.
so if this is correct then the error message is wrong? should be bytes or unicode? is there something that hits this in testing?
There was a problem hiding this comment.
Updated the error message and added a test to test_ujson. FWIW I don't think anything user-facing hits this as doing:
pd.read_json(None)Would yield:
>>> pd.read_json(None)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/williamayd/clones/pandas/pandas/io/json/json.py", line 437, in read_json
path_or_buf, encoding=encoding, compression=compression,
File "/Users/williamayd/clones/pandas/pandas/io/common.py", line 207, in get_filepath_or_buffer
raise ValueError(msg.format(_type=type(filepath_or_buffer)))
ValueError: Invalid file path or buffer object type: <class 'NoneType'>You could hit it directly internally if you did something like:
import pandas._libs.json as ujson
ujson.loads(None)which now yields:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Expected 'str' or 'bytes'
|
thanks @WillAyd |
There was a header file in the ujson source specifically meant to bridge compatibility between Py2 and Py3. With the Py2 support drop this could be removed and the source code simplified.
Most of these changes are simply verbatim, though there are probably a few follow ups on context to simplify further that could be made