BUG: handle none values in json.normalize()
#53773
Conversation
punndcoder28
force-pushed
the
fix-json-recursive-extract
branch
from
72c9816 to
b4cab55
Compare
punndcoder28
changed the title
json.normalize()
punndcoder28
changed the title
json.normalize()json.normalize()
| @@ -490,6 +490,8 @@ def _pull_records(js: dict[str, Any], spec: list | str) -> list: | |||
| meta_keys = [sep.join(val) for val in _meta] | |||
|
|
|||
| def _recursive_extract(data, path, seen_meta, level: int = 0) -> None: | |||
| if data is None: | |||
There was a problem hiding this comment.
Isn't this generally an issue if data is not list like?
There was a problem hiding this comment.
While running the tests I noticed while passing scalar values must be done in a list as is the expected behavior. But if I have a row of data in my json and one of the field values is null then the normalize() fails by throwing an exception when I instead expect it to ignore the entire row, which this change should do.
There was a problem hiding this comment.
Could you add a test where a nested json value is a scalar like 2 as assert the result?
There was a problem hiding this comment.
Do you mean something like this as my expected value?
expected = DataFrame(
{
"nested_field_id": 1,
}
)
There was a problem hiding this comment.
hi all, what is the status on this one? Any idea when it could be merged? (original request: #53719)
There was a problem hiding this comment.
@mroeschke Hi. In regard to the previous comment by me, is that what is required in the tests?
There was a problem hiding this comment.
A test where the input has something like "nested_field": [{"nested_field_id": [1]}],
|
This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this. |
|
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen. |
This PR fixes the error which is thrown when a field value in
jsonisnullwhilejson_normalizeit to aDataFrame. Implemented tests to validate the changedoc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.