Skip to content

Pin Faker to Prevent Asteroid Crash and Fix CI #300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 6, 2021

Conversation

jpulec
Copy link
Contributor

@jpulec jpulec commented Jan 6, 2021

There's some sort of issue with version 5.0.2 of Faker which updates a
lot of code to use the pathlib library instead of os utils. This causes
the sys.path_importer_cache to get filled with PosixPath objects instead
of strings as it's keys, which causes an asteroid crash.

For now, we can just pin Faker to a version that still works until the
upstream issue is resolved.

There's some sort of issue with version 5.0.2 of Faker which updates a
lot of code to use the pathlib library instead of os utils. This causes
the sys.path_importer_cache to get filled with PosixPath objects instead
of strings as it's keys, which causes an asteroid crash.

For now, we can just pin Faker to a version that still works until the
upstream issue is resolved.
@coveralls
Copy link

coveralls commented Jan 6, 2021

Pull Request Test Coverage Report for Build 1334

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 86.867%

Totals Coverage Status
Change from base Build 1266: 0.0%
Covered Lines: 721
Relevant Lines: 830

💛 - Coveralls

It's be EOL'd since Sept 2020, so time to ditch it.
@jpulec jpulec requested a review from atodorov January 6, 2021 08:04
@atodorov
Copy link
Contributor

atodorov commented Jan 6, 2021

There's some sort of issue with version 5.0.2 of Faker which updates a
lot of code to use the pathlib library instead of os utils. This causes
the sys.path_importer_cache to get filled with PosixPath objects instead
of strings as it's keys, which causes an asteroid crash.

I'm not immediately finding Faker as a dependency in astroid code base. If you can share some pointers files/methods you looked at that would be helpful.

For now, we can just pin Faker to a version that still works until the
upstream issue is resolved.

Seems to me this isn't reported upstream so I want to gather as much info as possible before looking more into it.

@atodorov atodorov merged commit d15f20c into pylint-dev:master Jan 6, 2021
@jpulec
Copy link
Contributor Author

jpulec commented Jan 6, 2021

I'm not immediately finding Faker as a dependency in astroid code base. If you can share some pointers files/methods you looked at that would be helpful.

Sure. Asteroid doesn't depend on Faker, but due to some global import state it causes this issue. If you check out this build log from the first time the travis builds failed, you'll see the actual error is:

AttributeError: 'PosixPath' object has no attribute 'rstrip'

The first astroid (and not importlib) stack frame where this happens is in astroid/interpreter/_import/spec.py:281.

Here's the surrounding relevant code:

def _search_zip(modpath, pic):
    for filepath, importer in list(pic.items()):
        if importer is not None:
            found = importer.find_module(modpath[0])

If you look further up the stack, you'll find that ZipFinder is the class causing issues. In the find_module method, which passes self._zipimporters for the pic parameter of _search_zip.

self._zipimporters is populated by _precache_zipimporters, which references sys.path_importer_cache.

I'm not exactly sure how/why, but somehow Faker v5.0.2+ is adding entries to sys.path_importer_cache which are pathlib.PosixPath objects instead of actual strs. Eventually this results in FileFinder instance trying to call find_module() with a PosixPath instead of a str which is what yields the initial error.

Hopefully that can provide a little more color. Let me know if any of that is unclear.

carlio added a commit that referenced this pull request Jan 8, 2021
atodorov pushed a commit that referenced this pull request Jan 10, 2021
@michael-k michael-k mentioned this pull request Apr 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants