Skip to content

Fix problem with objects with invalid repr #1899

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 8, 2023

Conversation

tamarinvs19
Copy link
Collaborator

@tamarinvs19 tamarinvs19 commented Mar 6, 2023

Description

Fix problem with serialization of "unserializable" objects (for example, logging.Logger or socket.socket). We cannot serialize it so cannot generate tests but in previous version we generated tests with string representation of these objects (<Logger x (NOSET)> or <socket.socket ... >).

In future we can use mocks at these situations.

Fixes #1822

How to test

Manual tests

See #1822 (comment)
Expected: no one test have been generated

Self-check list

Check off the item if the statement is true. Hint: [x] is a marked item.

Please do not delete the list or its items.

  • I've set the proper labels for my PR (at least, for category and component).
  • PR title and description are clear and intelligible.
  • I've added enough comments to my code, particularly in hard-to-understand areas.
  • The functionality I've repaired, changed or added is covered with automated tests.
  • Manual tests have been provided optionally.
  • The documentation for the functionality I've been working on is up-to-date.

@tamarinvs19 tamarinvs19 added lang-python Issue is related to python support ctg-bug-fix PR is fixing a bug labels Mar 6, 2023
@tamarinvs19 tamarinvs19 self-assigned this Mar 6, 2023
@@ -1,4 +1,4 @@
mypy==1.0.0
coverage
utbot-executor==1.1.27
utbot-executor==1.1.31
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked this version of utbot_executor and I don't like the way you check whether an object has correct repr:

if not isinstance(py_object, str) and r.startswith('<') and r.endswith('>')

One can override __repr__ method of their (unserializable) object so that it will pass this check.

From CPython sources we see that there is a finite number of types that are not serialized with __reduce__ method: https://github.com/python/cpython/blob/main/Lib/pickle.py#L734

Maybe we should just list them?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't find socket.socket and logging.Logger in this file... For example, logging.Logger has method __reduce__ but it sometimes raises error pickle.PicklingError

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can call eval and check that we got the same type (but not the same value because object can be serializable and incomparable)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But socket.socket and logging.Logger can never be serialized by pickle.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eval check may be tricky because of modules that are needed to be imported

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But socket.socket and logging.Logger can never be serialized by pickle.

I can pickle logging.Logger:

import logging
import pickle

pickle.dumps(logging.getLogger())

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eval check may be tricky because of modules that are needed to be imported

I agree, it can be a problem

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is interesting...

image

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added list of repr-able objects from pickle source code (tamarinvs19/utbot_executor@e6dc858)

* add list of `repr`-able types
* add attempt to call `repr . evel . repr` and compare with `repr`
tochilinak
tochilinak previously approved these changes Mar 8, 2023
Copy link
Member

@tochilinak tochilinak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is a problem with None.

image

@tochilinak tochilinak dismissed their stale review March 8, 2023 07:40

Had a closer look

@tamarinvs19
Copy link
Collaborator Author

tamarinvs19 commented Mar 8, 2023

I think there is a problem with None.

image

Fixed tamarinvs19/utbot_executor@62e618b

@tamarinvs19 tamarinvs19 merged commit 9367b5e into main Mar 8, 2023
@tamarinvs19 tamarinvs19 deleted the tamarinvs19/python-fix-invalid-repr branch March 8, 2023 11:34
@alisevych alisevych added this to the 2023.03 Release milestone Mar 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ctg-bug-fix PR is fixing a bug lang-python Issue is related to python support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot generate test for Python generics: No seed candidates generated
3 participants