remove trailing period after "N seconds"

[tiagomartinho]

• remove trailing period in PrintObserver to match Apple XCTest as stated
  in the FIXME comment
• update functional test suite accordingly

[modocache]

You are my hero! 😍 Thanks!

[modocache]

@swift-ci please test

[tiagomartinho]

Welcome 😅 I'm happy to contribute

[modocache]

Hmm, strange, looks like some of the macOS tests failed. At first glance, it's not clear whether this pull request is at fault. On the one hand, I can see the following output:

There were more lines expected to appear than there were lines in the actual input. Unmet expectation: "Test Suite 'All tests' started at \\d+:\\d+:\\d+\\.\\d+"

On the other hand, the following is being output to stderr. I'm not sure it's relevant output, and as far as I know it's ignored because we currently only make expectations on stdout:

2017-01-08 09:19:13.111 Asynchronous-Notifications-Handler[15929:29214] Assertions enabled
dyld: lazy symbol binding failed: Symbol not found: __os_log_set_nscf_formatter
  Referenced from: /Users/buildnode/jenkins/workspace/swift-corelibs-xctest-PR-osx/Ninja-DebugAssert/xctest-macosx-x86_64/Debug/SwiftFoundation.framework/Versions/A/SwiftFoundation
  Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: __os_log_set_nscf_formatter
  Referenced from: /Users/buildnode/jenkins/workspace/swift-corelibs-xctest-PR-osx/Ninja-DebugAssert/xctest-macosx-x86_64/Debug/SwiftFoundation.framework/Versions/A/SwiftFoundation
  Expected in: /usr/lib/libSystem.B.dylib

@tiagomartinho, did you test this locally? If so, did you use macOS or Linux? If macOS, could you try executing the same command as CI does, to see if the failure occurs? CI is running this:

swift/utils/build-script \
    --test --validation-test \
    --lit-args=-v --compiler-vendor=apple \
    --xctest \
    -- \
    --reconfigure \
    --verbose-build \
    --build-ninja --build-swift-stdlib-unittest-extra \
    --skip-test-cmark --skip-test-swift --skip-test-libdispatch --skip-test-foundation

[tiagomartinho]

@modocache yes I did try it locally using the "SwiftXCTestFunctionalTests" scheme in Xcode. I use macOS 10.12.2 and the latest Swift Toolchain (2017-01-05).

I will run the same command as the CI does and check the results.

[tiagomartinho]

It works on my machine 😞

/Users/tiagomartinho/Development/swift-source/swift-corelibs-xctest/../llvm/utils/lit/lit.py -sv --no-progress-bar Tests/Functional
-- Testing: 19 tests, 8 threads --
Testing Time: 13.31s
Expected Passes : 19
** BUILD SUCCEEDED **
--- Finished tests for xctest ---

I however removed the option --build-ninja, the command I ran was:

../swift/utils/build-script \ --test --validation-test \ --lit-args=-v --compiler-vendor=apple \ --xctest \ -- \ --reconfigure \ --verbose-build \ --build-swift-stdlib-unittest-extra \ --skip-test-cmark --skip-test-swift --skip-test-libdispatch --skip-test-foundation

[larryonoff]

I have the same issue in PR #176, see the comment from @ikesyo. This issue only happens on macOS, not on Linux.

[tiagomartinho]

@larryonoff do you think it's a problem with the changes made to XCTest?

[larryonoff]

@tiagomartinho I don't think that your changes caused this issue. I assume that something doesn't match with the current Xcode project settings in 'master' branch and CI requirements, or as @ikesyo mentioned that swift-corelibs-foundation doesn't work well on macOS 10.11.

I thought that my changes caused the issue, because I made serious changes in project settings. But now it doesn't look so, since you have the same issue and you didn't change any project settings.

PS. Moreover I have an assumption now that if someone will trigger CI for the current 'master' branch it will fail on macOS.

[modocache]

Yes, that makes sense. Looks like the build is broken. First person to submit a pull request to fix it wins! Might have to make a change to swift-corelibs-foundation, or maybe apple/swift.

[tiagomartinho]

How would you advise to work on it @modocache?

I could not reproduce the problem on my machine.

[modocache]

Good point. In #176 @ikesyo mentioned that the build fails on macOS 10.11 because of swiftlang/swift-corelibs-foundation#709, so it might require that version of the OS in order to test properly...

Unfortunately swiftlang/swift-corelibs-foundation#709 is a large pull request, so it'd be difficult, but one option would be looking through that to see if there's any specific line that is causing problems.

The quicker solution would probably be to email swift-dev or swift-corelibs-dev and explain the problem. Maybe the ideal solution is to:

1. Have someone on the Swift CI team update the CI to macOS Sierra
2. Have build-script print out the current macOS version, for future reference

In the meantime, I'm going to go ahead and merge this pull request. It works on Linux, and I'm confident it'd work on macOS, too. Sorry for the complications!

[ikesyo]

I've submitted a pull request: swiftlang/swift-corelibs-foundation#784

[larryonoff]

BTW it still would be great to know the CI environment (e.g. macOS version, Xcode version, what else ?) when it tests PR. Should I write into swift-dev or swift-corelibs-dev? or create issue in Jira ?

[modocache]

I think create a pull request to add it to apple/swift/utils/build-script, that way everyone will be able to find out whenever they want by checking the logs from someone's "swift-ci please test".