Wiring not done correctly with PyInstaller

[saledemon]

Hi, first off, I'd like to say thank you for this amazing library. So easy to use and so efficient.

Now, the problem I have is that my project runs perfectly when I simply run the python script, but when I bundle it into an exe with PyInstaller, it raises this error :

TypeError: 'Provide' object is not subscriptable

I have tested the following :

class Main:
  def __init__(self):
      self.container = Container()
      with open(CONFIG_FILE_PATH) as jsonfile:
          self.container.config.from_dict(json.load(jsonfile))
      self.container.wire(packages=[photovisual, gui])
      print('config : ' + str(self.container.config()))

  @inject
  def start(self, config: dict = Provide[Container.config]):
      print("config : "+str(config))
      root = Tk()
      configwin = ConfigDialog(root)
      ses = SessionWindow(root)

      configwin.onGo(ses.setupWindow)
      configwin.pack()
      root.mainloop()

The first print works fine, it prints out the config dictionnary. The second one in method start when I use @Inject doesn't. It prints the following :

config : <dependency_injector.wiring.Provide object at 0x0000013EFB974070>

It looks a lot like [Issue #328] (#328), though I must say it again, my script does actually work, the error only appears when I try to run the executable file created by PyInstaller. All my packages have __init__.py and I'm not making any declaration during declaative phase.

So my guess is that somehow, bundling the project with PyInstaller breaks the wiring. Has someone ever use this library with PyInstaller ?

Prior to this error, I also had a ModuleNotFoundError when building the executable. The module dependency_injector.errors could not be found, so I added it in the hidden imports. These two issues may or may not be related...

Here is my Container class :

class Container(containers.DeclarativeContainer):

  config = providers.Configuration()
  drive_service = providers.Singleton(drive.DriveUploader, credentials=config.creds)

[rmk135]

Hi @saledemon,

Could you, please, check what version of Python you run when application is packed by PyInstaller?
I fixed a similar issue some time ago. It was related to a specific Python version. May be there
is something specific with PyInstaller or its Python version. It should be easy-fixable, but
we need to localize the issue first. Thanks for bring up the issue!

Best,
Roman

[saledemon]

My Python version is 3.9.2 and PyInstaller is 4.2

I read somewhere that some problems arise if you download PyInstaller with pip and that sometimes it's better to download it some other way... I'll look around for this solution, maybe the problem is there somehow

[veotani]

Hi, having the same issue. Python 3.8.8; PyInstaller 4.3; dependency-injector 4.35.2. I too have project running when I execute it as a python script, but getting this Provide issue because of PyInstaller.

[rmk135]

@veotani thanks for the report. Could you provide any information on how to reproduce the issue?

[veotani]

@rmk135 yes, I've created simple fastapi project with the same structure that I use in my real project. There I have an API layer and domain. Here I have an injection example: I have UseCase injected into endpoint of the application. In the README I described how to run the project and see that behavior is different when you run the project as a script and as an executable built with PyInstaller.

https://github.com/veotani/fastapi-example-bundle

Please, let me know if:

1. This is my own mistake and I'm not doing wiring properly;
2. Instructions are not clear and you want me to be more verbose on them;
3. This is too much for an example and I should try to come up with a simpler example (I will actually try to do it tomorrow, so let me know, if you need it).

Any other directions on how to make your assistance easier for you are also welcome.

Thank you!

[veotani]

Hi again.

I made a simpler example demonstrating the issue.

https://github.com/veotani/di-pyinstaller-bug

[veotani]

I've just tested if this behavior is relevant when you wire modules and the answer is "no". If you need to package your application with PyInstaller and wiring packages ends up raising the exception from the OP post, then you probably should use module wiring at the moment.

In other words, if you change two lines starting at this line to:

import printer.hello_world
container.wire(modules=[printer.hello_world])

the script will run.

[rmk135]

@veotani , thanks for the example. I have successfully reproduced the issue with https://github.com/veotani/di-pyinstaller-bug
I'll debug what's going on and get back to you shortly.

[veotani]

@rmk135, as I see, package wiring is implemented with paths of these packages. When you use bundled app, you have no paths and modules are treated as frozen modules. If only package could contain links to modules contained within it, solution could be simple. According to PEP 451:

Package
The concept does not change, nor does the term. However, the distinction between modules and packages is mostly superficial. Packages are modules. They simply have a __path__ attribute and import may add attributes bound to submodules. The typically perceived difference is a source of confusion. This proposal explicitly de-emphasizes the distinction between packages and modules where it makes sense to do so.

I wonder if this

import may add attributes bound to submodules

can help, but haven't found anything yet.

Seems like it's necessary to see, how PyInstaller stores modules.

I decided to share it, hope it will help. Going to see, if I can find anything interesting about Python module system. But if you find a solution, I'd love to see it!

Thanks!

[rmk135]

@veotani, thanks for the input. I also found this section: https://pyinstaller.readthedocs.io/en/stable/when-things-go-wrong.html#listing-hidden-imports. Seems like it's quite related.

I didn't find any quick solution. Will take deeper look later

[rmk135]

Hi @veotani. Do you consider upgrading PyInstaller to the latest version? FYI your example appears to work on version 4.5.1.

[veotani]

Hi @rmk135, yes, updating to latest version helps to avoid this issue. There was a reason for me to stick with 4.3, will describe it at the end of my message.

To conclude this conversation: for everyone who is having this problem they might either use module wiring for specific versions of PyInstaller or update PyInstaller.

I've tested latest versions of PyInstaller and here is what I've got:

• In version 4.3 you will end up with wiring error.
• In version 4.4 you will have startswith first arg must be str or a tuple of str, not PureWindowsPath error.
• In versions 4.5 and 4.5.1 everything is ok.

Last time I've been working on my project was in July, after that I had my vacation. In July latest PyInstaller version was 4.4 with PureWindowsPath error. In the middle of the August I've googled it a little and there was a solution to rollback to 4.3. This is how I ended having 4.3, completely missing there already was a 4.5 and even 4.5.1.

Thank you very much for the help! This problem certainly has nothing to do with dependency-injector. IMO issue can now be closed.

[rmk135]

Hi @veotani. Got it, good that everything works on your end. I'm also going to take a look if I could fix "startswith first arg" issue from 4.4 to support more versions.