Process: do not inherit all file descriptors to child processes #2485

23:28:08 2019-09-03 22:28:08.364 SourceKitLSPPackageTests.xctest[82487:dcff9700] manifest parse error(s):
23:28:08 
/home/buildnode/jenkins/workspace/swift-corelibs-foundation-PR-Linux/tmp/TemporaryDirectory.I33WuL/pkg/Package.swift:3:5: error: type annotation missing in pattern
23:28:08 let pack
23:28:08     ^
23:28:08 Test Case 'SwiftPMWorkspaceTests.testUnparsablePackage' passed (0.094 seconds)
23:28:08 Test Suite 'SwiftPMWorkspaceTests' passed at 2019-09-03 22:28:08.365

retrying

weissi · 2019-09-04T07:07:17Z

@swift-ci please test

weissi · 2019-09-04T16:32:20Z

@aciidb0mb3r tests are now failing with

09:57:47 2019-09-04 08:57:47.646 SourceKitLSPPackageTests.xctest[16659:f4ff9700] manifest parse error(s):
09:57:47 
/home/buildnode/jenkins/workspace/swift-corelibs-foundation-PR-Linux/tmp/TemporaryDirectory.5Pe7PM/pkg/Package.swift:3:5: error: type annotation missing in pattern
09:57:47 let pack
09:57:47     ^
09:57:47 Test Case 'SwiftPMWorkspaceTests.testUnparsablePackage' passed (0.093 seconds)
09:57:47 Test Suite 'SwiftPMWorkspaceTests' passed at 2019-09-04 08:57:47.647

any ideas? I'm just going to retry this again :)

weissi · 2019-09-04T16:32:25Z

@swift-ci please test

aciidgh · 2019-09-04T16:41:49Z

It looks like those are just logs about manifest parse error, not a test failure.

09:57:47 Test Case 'SwiftPMWorkspaceTests.testUnparsablePackage' passed (0.093 seconds)

weissi · 2019-09-04T16:52:15Z

It looks like those are just logs about manifest parse error, not a test failure.

09:57:47 Test Case 'SwiftPMWorkspaceTests.testUnparsablePackage' passed (0.093 seconds)

oh, you're right. @millenomi any ideas what that Package.swift could be?

weissi · 2019-09-04T19:02:55Z

now I've got this

19:03:36 PASS: lldb-Suite :: lang/swift/protocol_optional/TestSwiftProtocolOptional.py (245 of 292)
19:03:36 UNRESOLVED: lldb-Suite :: lang/swift/parseable_interfaces/shared/TestSwiftInterfaceNoDebugInfo.py (246 of 292)
19:03:36 ******************** TEST 'lldb-Suite :: lang/swift/parseable_interfaces/shared/TestSwiftInterfaceNoDebugInfo.py' FAILED ********************
19:03:36 Unable to find 'RESULT: PASSED' in dotest output:
19:03:36

weissi · 2019-09-04T19:03:04Z

@swift-ci please test

weissi · 2019-09-05T07:43:41Z

hmm, this one again

22:08:28 Test Case 'SwiftPMWorkspaceTests.testUnparsablePackage' started at 2019-09-04 21:08:28.913
22:08:29 2019-09-04 21:08:29.011 SourceKitLSPPackageTests.xctest[130421:4effd700] manifest parse error(s):
22:08:29 
/home/buildnode/jenkins/workspace/swift-corelibs-foundation-PR-Linux/tmp/TemporaryDirectory.RlekeW/pkg/Package.swift:3:5: error: type annotation missing in pattern
22:08:29 let pack
22:08:29     ^

@millenomi any ideas?

benlangmuir · 2019-09-05T18:10:28Z

I've never seen this timeout before and without a stacktrace it's hard to know what went wrong. This test does use Foundation.Process as part of its infrastructure to run commands and capture output over pipes, so may actually be affected by the PR.

weissi · 2019-09-10T11:43:39Z

ok, I repro locally:

(lldb) bt
* thread #1, name = 'SourceKitLSPPac', stop reason = signal SIGSTOP
  * frame #0: 0x00007fb379a60811 libc.so.6`__GI_ppoll(fds=0x00007fff33e63f70, nfds=1, timeout=<unavailable>, sigmask=0x0000000000000000) at ppoll.c:50
    frame #1: 0x00007fb37b41e805 libFoundation.so`__CFRunLoopServiceFileDescriptors + 277
    frame #2: 0x00007fb37b41a6f1 libFoundation.so`__CFRunLoopRun + 1153
    frame #3: 0x00007fb37b419fd9 libFoundation.so`CFRunLoopRunSpecific + 473
    frame #4: 0x00007fb37b36aa9b libFoundation.so`Foundation.RunLoop.run(mode: Foundation.RunLoop.Mode, before: Foundation.Date) -> Swift.Bool + 283
    frame #5: 0x00007fb37b33cbb5 libFoundation.so`Foundation.Process.waitUntilExit() -> () + 181
    frame #6: 0x000056100b92b2a6 SourceKitLSPPackageTests.xctest`static Process.tibs_checkNonZeroExit(arguments=2 values, environment=nil, self=Process) at Process.swift:48:7
    frame #7: 0x000056100b94d27d SourceKitLSPPackageTests.xctest`findTool(name="ninja") at TibsToolchain.swift:196:33
    frame #8: 0x000056100be6c1d9 SourceKitLSPPackageTests.xctest`TibsToolchain.init(sktc=<unavailable>) at SKTibsTestWorkspace.swift:152:15
    frame #9: 0x000056100be6be1c SourceKitLSPPackageTests.xctest`SKTibsTestWorkspace.init(immutableProjectDir=<unavailable>, persistentBuildDir=<unavailable>, tmpDir=<unavailable>, toolchain=<unavailable>, self=<unavailable>) at SKTibsTestWorkspace.swift:46:18
    frame #10: 0x000056100be6bc73 SourceKitLSPPackageTests.xctest`SKTibsTestWorkspace.__allocating_init(immutableProjectDir:persistentBuildDir:tmpDir:toolchain:) at SKTibsTestWorkspace.swift:0
    frame #11: 0x000056100be6df63 SourceKitLSPPackageTests.xctest`XCTestCase.staticSourceKitTibsWorkspace(name="CodeCompleteSingleModule", testFile="/home/jweiss/devel/sourcekit-lsp/Tests/SourceKitTests/SourceKitTests.swift", self=0x000056100d086080) at SKTibsTestWorkspace.swift:96:25
    frame #12: 0x000056100bf7a4c1 SourceKitLSPPackageTests.xctest`SKTests.testCodeCompleteSwiftTibs(self=<unavailable>) at SourceKitTests.swift:98:24
    frame #13: 0x000056100bf92379 SourceKitLSPPackageTests.xctest`partial apply for SKTests.testCodeCompleteSwiftTibs() at <compiler-generated>:0
    frame #14: 0x000056100bf9080f SourceKitLSPPackageTests.xctest`thunk for @escaping @callee_guaranteed () -> (@error @owned Error) at <compiler-generated>:0
    frame #15: 0x000056100bf92364 SourceKitLSPPackageTests.xctest`thunk for @escaping @callee_guaranteed () -> (@error @owned Error)partial apply at <compiler-generated>:0
    frame #16: 0x00007fb37c65e831 libXCTest.so`partial apply forwarder for reabstraction thunk helper from @escaping @callee_guaranteed () -> (@out (), @error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@error @owned Swift.Error) + 17
    frame #17: 0x00007fb37c65e63e libXCTest.so`partial apply forwarder for closure #1 (XCTest.XCTestCase) throws -> () in XCTest.(test in _3BE257A46ADB477C7BF2D39968B39F9D)<A where A: XCTest.XCTestCase>((A) -> () throws -> ()) -> (XCTest.XCTestCase) throws -> () + 78
    frame #18: 0x00007fb37c65e5b4 libXCTest.so`partial apply forwarder for reabstraction thunk helper from @escaping @callee_guaranteed (@guaranteed XCTest.XCTestCase) -> (@error @owned Swift.Error) to @escaping @callee_guaranteed (@in_guaranteed XCTest.XCTestCase) -> (@out (), @error @owned Swift.Error) + 20
    frame #19: 0x00007fb37c65e8b9 libXCTest.so`reabstraction thunk helper from @escaping @callee_guaranteed (@guaranteed XCTest.XCTestCase) -> (@error @owned Swift.Error) to @escaping @callee_guaranteed (@in_guaranteed XCTest.XCTestCase) -> (@out (), @error @owned Swift.Error)partial apply forwarder with unmangled suffix ".16" + 9
    frame #20: 0x00007fb37c651627 libXCTest.so`partial apply forwarder for reabstraction thunk helper from @escaping @callee_guaranteed (@in_guaranteed XCTest.XCTestCase) -> (@out (), @error @owned Swift.Error) to @escaping @callee_guaranteed (@guaranteed XCTest.XCTestCase) -> (@error @owned Swift.Error) + 39
    frame #21: 0x00007fb37c65d02f libXCTest.so`XCTest.XCTestCase.invokeTest() -> () + 63
    frame #22: 0x00007fb37c65ce9c libXCTest.so`XCTest.XCTestCase.perform(XCTest.XCTestRun) -> () + 172
    frame #23: 0x00007fb37c660ebf libXCTest.so`XCTest.XCTest.run() -> () + 191
    frame #24: 0x00007fb37c65ec18 libXCTest.so`XCTest.XCTestSuite.perform(XCTest.XCTestRun) -> () + 232
    frame #25: 0x00007fb37c660ebf libXCTest.so`XCTest.XCTest.run() -> () + 191
    frame #26: 0x00007fb37c65ec18 libXCTest.so`XCTest.XCTestSuite.perform(XCTest.XCTestRun) -> () + 232
    frame #27: 0x00007fb37c660ebf libXCTest.so`XCTest.XCTest.run() -> () + 191
    frame #28: 0x00007fb37c65ec30 libXCTest.so`XCTest.XCTestSuite.perform(XCTest.XCTestRun) -> () + 256
    frame #29: 0x00007fb37c660ebf libXCTest.so`XCTest.XCTest.run() -> () + 191
    frame #30: 0x00007fb37c65be23 libXCTest.so`XCTest.XCTMain(Swift.Array<(testCaseClass: XCTest.XCTestCase.Type, allTests: Swift.Array<(Swift.String, (XCTest.XCTestCase) throws -> ())>)>) -> Swift.Never + 1091
    frame #31: 0x000056100bf26ff3 SourceKitLSPPackageTests.xctest`main at LinuxMain.swift:18:1
    frame #32: 0x00007fb379985830 libc.so.6`__libc_start_main(main=(SourceKitLSPPackageTests.xctest`main at LinuxMain.swift), argc=1, argv=0x00007fff33e65aa8, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=0x00007fff33e65a98) at libc-start.c:291
    frame #33: 0x000056100b88e5f9 SourceKitLSPPackageTests.xctest`_start + 41

So we're hanging in waitUntilExit. I think it might be that we close the taskSocketPair[1] too early.

weissi · 2019-09-10T11:44:52Z

@swift-ci please test

weissi · 2019-09-10T11:44:55Z

@swift-ci please test

weissi · 2019-09-10T11:45:06Z

@swift-ci please test

weissi · 2019-09-10T14:41:08Z

@swift-ci please test

weissi · 2019-09-10T14:41:14Z

@swift-ci please test

weissi · 2019-09-10T14:41:19Z

@swift-ci please test

weissi · 2019-09-10T14:41:29Z

@swift-ci please test

weissi · 2019-09-10T18:59:41Z

@swift-ci please test

weissi · 2019-09-10T19:01:42Z

@swift-ci please test

weissi · 2019-09-10T21:01:32Z

@swift-ci please test

weissi · 2019-09-11T10:01:42Z

@swift-ci please test

weissi · 2019-09-11T10:31:02Z

@swift-ci please test

weissi · 2019-09-11T16:34:03Z

@swift-ci please test

weissi · 2019-09-11T16:38:40Z

@swift-ci please test

- POSIX_SPAWN_CLOEXEC_DEFAULT is a constant offered by the Darwin module. Import it conditionally that way. - Do not close Pipe/FileHandle-owned fds.

weissi · 2019-09-20T13:07:31Z

@swift-ci please test

weissi · 2019-09-20T17:40:07Z

@swift-ci please test

weissi · 2019-09-24T11:35:01Z

ping @millenomi , is this good to go?

weissi · 2019-09-28T10:44:32Z

friendly ping @millenomi

millenomi reviewed Aug 22, 2019

View reviewed changes

TestFoundation/xdgTestHelper/main.swift Show resolved Hide resolved

millenomi reviewed Aug 22, 2019

View reviewed changes

TestFoundation/TestProcess.swift Show resolved Hide resolved

weissi marked this pull request as ready for review September 11, 2019 20:40

weissi requested a review from millenomi September 11, 2019 20:40

weissi changed the title ~~do not inherit fds~~ Process: do not inherit all file descriptors to child processes Sep 11, 2019

weissi and others added 3 commits September 20, 2019 14:05

do not inherit fds

f4eb86b

Cleanup:

eb23f1e

- POSIX_SPAWN_CLOEXEC_DEFAULT is a constant offered by the Darwin module. Import it conditionally that way. - Do not close Pipe/FileHandle-owned fds.

more cleanup

c6da1a0

millenomi merged commit a33ed41 into swiftlang:master Oct 15, 2019

shahmishal mentioned this pull request Oct 16, 2019

Revert "Process: do not inherit all file descriptors to child processes" #2539

Merged

weissi deleted the jw-dont-inherit-fds branch October 16, 2019 20:35

weissi mentioned this pull request Aug 22, 2019

[SR-11354] Foundation.Process inherits file descriptors into the child process #3291

Closed

Process: do not inherit all file descriptors to child processes #2485

Process: do not inherit all file descriptors to child processes #2485

Uh oh!

Conversation

weissi commented Aug 22, 2019

Uh oh!

millenomi commented Aug 22, 2019

Uh oh!

weissi commented Aug 22, 2019

Uh oh!

millenomi commented Aug 22, 2019

Uh oh!

Uh oh!

millenomi commented Aug 22, 2019

Uh oh!

Uh oh!

millenomi commented Aug 23, 2019

Uh oh!

weissi commented Aug 30, 2019

Uh oh!

weissi commented Aug 30, 2019

Uh oh!

weissi commented Aug 30, 2019

Uh oh!

weissi commented Sep 2, 2019

Uh oh!

weissi commented Sep 2, 2019

Uh oh!

weissi commented Sep 3, 2019

Uh oh!

weissi commented Sep 3, 2019

Uh oh!

weissi commented Sep 3, 2019

Uh oh!

weissi commented Sep 3, 2019

Uh oh!

weissi commented Sep 3, 2019

Uh oh!

weissi commented Sep 3, 2019

Uh oh!

weissi commented Sep 3, 2019

Uh oh!

weissi commented Sep 3, 2019

Uh oh!

weissi commented Sep 3, 2019

Uh oh!

weissi commented Sep 4, 2019

Uh oh!

weissi commented Sep 4, 2019

Uh oh!

weissi commented Sep 4, 2019

Uh oh!

weissi commented Sep 4, 2019

Uh oh!

aciidgh commented Sep 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

weissi commented Sep 4, 2019

Uh oh!

weissi commented Sep 4, 2019

Uh oh!

weissi commented Sep 4, 2019

Uh oh!

weissi commented Sep 5, 2019

Uh oh!

benlangmuir commented Sep 5, 2019

Uh oh!

weissi commented Sep 10, 2019

Uh oh!

weissi commented Sep 10, 2019

Uh oh!

weissi commented Sep 10, 2019

Uh oh!

weissi commented Sep 10, 2019

Uh oh!

weissi commented Sep 10, 2019

Uh oh!

weissi commented Sep 10, 2019

Uh oh!

weissi commented Sep 10, 2019

aciidgh commented Sep 4, 2019 •

edited

Loading