[SR-15126] Swift Package Manager (SPM) on Windows gets stuck after displaying "...\swiftxml-manifest.exe -handle ..."

[swift-ci]

Previous ID	SR-15126
Radar	None
Original Reporter	Stefan Springer (JIRA User)
Type	Bug
Status	Resolved
Resolution	Done

Environment

Swift 5.4.2 on Windows Home or Pro 64 Bit with Visual Studio 2019 or 2022 and with different versions of Git.

Additional Detail from JIRA

Votes	0
Component/s	libdispatch
Labels	Bug, Windows
Assignee	@compnerd
Priority	Medium

md5: 9d72cdb96d96f67bcc8dd6e21450caa1

Issue Description:

As explained in the Swift forums, my SPM always gets stuck after displaying e.g.

C:\Users\stefan\AppData\Local\Temp\TemporaryDirectory.eK9Mr0\swiftxml-manifest.exe -handle 414

At that moment that directory does not exist any more, so maybe the problem is at some later step.

(Tested with that repository of commit 4c3f8ffa. After that commit, I changed the tools version to 5.3 again to see if- with all other correction in the installation – it then works, but it then wrongly displays "error: Dependencies could not be resolved because 'SwiftXML' >= 0.1.12 contains incompatible tools version (5.3.0) and root depends on 'SwiftXML' 0.1.14..<1.0.0".)

Note: In the Swift forums it is noted that there are some upcoming changes concerning the manifest loading.

[swift-ci]

Comment by Stefan Springer (JIRA)

Still the same error with Swift 5.4.3 and 5.5 (Development): Doing nothing after displaying

C:\Users\stefan\AppData\Local\Temp\TemporaryDirectory.kRH0ol\swiftxml-manifest.exe -handle 428

(or with another temporary directory or handle) when the directory

C:\Users\stefan\AppData\Local\Temp\TemporaryDirectory.kRH0ol

does not exist any more.

Could SPM maybe at least output more information (with the "-v" option) after displaying "...\swiftxml-manifest.exe -handle ..." so one could see what goes wrong?

[compnerd]

I'm not sure if this belongs in the Windows category if you intend this to be about "more information". That is not a Windows specific path, and I don't see what Windows could do differently there.

[swift-ci]

Comment by Stefan Springer (JIRA)

I'm not sure if this belongs in the Windows category

• With the same projects tested, everything works fine on macOS and CentOS 7.

• The above error occurs with those projects on different instances of Windows.

So the assumption that it has something to do with running on Windows seems to be sensible assumption. One other user also reported a problem with on of the projects on Windows. There is nothing special about my Windows environments (trying with fresh installed OSs) except that they are running in a Parallels VM.

[compnerd]

Could you share some information about your VM configuration? How many vCPUs are you assigning the VM?

[swift-ci]

Comment by Stefan Springer (JIRA)

Could you share some information about your VM configuration? How many vCPUs are you assigning the VM?

Two CPU cores (of four) for the VM, 12 GB RAM out of 16 GB, high res display. No other special configuration (standard configuration via quick install with desktop shared etc., user automatically logged in without password). On an iMac a few years old with i5 chip. Newest Parallels version (I think it was the version before the current one). Newest macOS (was Big Sur at that time).

[compnerd]

> Two CPU cores (of four) for the VM,

VERY interesting. Can you try an alternate configuration? Increasing the vCPU count is the important thing, I think that the memory is more than plenty (this is a test - we need to fix the issue, but I want to see if what you are seeing is the same issue). I have reason to believe that this is a "bug" in libdispatch.

[swift-ci]

Comment by Stefan Springer (JIRA)

Can you try an alternate configuration? Increasing the vCPU count is the important thing

Great that you have an idea where the problem could be. (I will be working on another machine soon and maybe there is no problem, but resolving the problem on the old machine would be a sensible thing to do, at least one other person also reported a problem.)

Hmm, on that 4-cores-machine I could only go to 3 cores for the VM, and Parallels warned me that Windows does not function correctly with an uneven number of cores (I thought 1 is uneven?...). I tried anyway and SPM (-v) gets stuck with the following message, but that maybe does not mean anything.

git.exe -C C:\Users\stefan\Projekte\SwiftXMLDemo.build\repositories\SwiftXML-6088ac03 cat-file -p 2ce69c7fc09d5b517190a9aef4d797282be18acd

UPDATE: The above error is reproducable. I then switched back to 2 cores for the VM and got back the situation where SPM gets stuck with "...\swiftxml-manifest.exe -handle 498". I think that reinforces your assumption.

[compnerd]

Okay; this is likely the inability to overcommit from libdispatch; that is something needs to be fixed but is a bit more intrusive and I need a bit of time to work through that.

[compnerd]

Stefan Springer (JIRA User) one more request if you don't mind. Could you please try https://artprodeus21.artifacts.visualstudio.com/A8fd008a0-56bc-482c-ba46-67f9425510be/3133d6ab-80a8-4996-ac4f-03df25cd3224/_apis/artifact/cGlwZWxpbmVhcnRpZmFjdDovL2NvbXBuZXJkL3Byb2plY3RJZC8zMTMzZDZhYi04MGE4LTQ5OTYtYWM0Zi0wM2RmMjVjZDMyMjQvYnVpbGRJZC81NTM4OC9hcnRpZmFjdE5hbWUvaW5zdGFsbGVyLmV4ZQ2/content?format=file&subPath=%2Finstaller.exe and see if it helps? This is a random snapshot off of main with #585 that should hopefully address the underlying libdispatch issue. There are some other changes that may also help this that are being worked upon, but this should hopefully show if this is the same issue or not.

[compnerd]

Ugh; sorry, seems like that won't work. Could you please just replace dispatch.dll in C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin with https://artprodeus21.artifacts.visualstudio.com/A8fd008a0-56bc-482c-ba46-67f9425510be/3133d6ab-80a8-4996-ac4f-03df25cd3224/_apis/artifact/cGlwZWxpbmVhcnRpZmFjdDovL2NvbXBuZXJkL3Byb2plY3RJZC8zMTMzZDZhYi04MGE4LTQ5OTYtYWM0Zi0wM2RmMjVjZDMyMjQvYnVpbGRJZC81NTM4OC9hcnRpZmFjdE5hbWUvc2RrLXdpbmRvd3MteDY00/content?format=file&subPath=%2FLibrary%2FDeveloper%2FPlatforms%2FWindows.platform%2FDeveloper%2FSDKs%2FWindows.sdk%2Fusr%2Fbin%2Fdispatch.dll as a test and see if it resolves the issue.

[swift-ci]

Comment by Stefan Springer (JIRA)

Yes! After replacing that DLL the Build completed. What I do not understand: After replacing the DLL again with the original one (each time I removed .swiftpm in user dir., .build in the project dir., and both org.swift.foundation.URLCache and org.swift.swiftpm in %TEMP%) it still functions. Is this possible?

UPDATE: After a Windows restart, it again did not work with the old DLL. (Any "DLL caching" involved here? Strange.) But then, even after restarting Windows, it also did not work with the new DLL. But then after a few (two or three?) times it again worked with the new DLL. I do not understand this situation; intuitively I would say the new DLL is better than the old one (a move into the right direction?) but still not perfect. Could this be?

UPDATE 2: Then, with the new DLL, it worked several times in a row (each time with the mentioned folders removed), but then, it worked only in the third try (without removing folders) with on the first try the message "error: the manifest is missing a Swift tools version specification" and on the second "https://github.com/stefanspringer1/SwiftXML.git @ 0.1.14: error: /\Package.swift has no Package.swift manifest for version 0.1.14 in https://github.com/stefanspringer1/SwiftXML.git". But yes, with the new DLL I have hopes that at least after a few tries it works, whereas with the old one, I just gave up on the project with that form.

[compnerd]

Its plausible; there are other changes being made to t-s-c and Foundation that will also help this. However, the dispatch issue is definitely the underlying cause for the issue that you are seeing then. I'm surprised that it didn't work with the new dll, though, I think that there are two copies of libdispatch and the one that gets mapped in is important. But it working with the new DLL a couple of times leads me to believe that this is the issue, and that once it is merged and integrated properly, it should resolve the issue.

Thanks for reporting the issue and the testing!

[swift-ci]

Comment by Stefan Springer (JIRA)

Thanks for reporting the issue and the testing!

Thanks to you! When will we see the improvement(s) in a new release?

[compnerd]

I've just merged the changes to dispatch on main, so hopefully the next snapshot should resolve the issue.