[SR-2856] libdispatch dispatch_timer_set_time test is flaky

[swift-ci]

Previous ID	SR-2856
Radar	rdar://problem/28598500
Original Reporter	erg (JIRA User)
Type	Bug
Status	Resolved
Resolution	Done

Additional Detail from JIRA

Votes	0
Component/s	libdispatch
Labels	Bug
Assignee	dgrove-oss (JIRA)
Priority	Medium

md5: 5383403a9de42fe43ff508c2c62f0e0b

Issue Description:

This fails sometimes in the Ubuntu CI.

https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-15_10/8010/console

```
PASS: dispatch_starfish
PASS: dispatch_data
PASS: dispatch_io_net
PASS: dispatch_select

libdispatch 1.3: tests/test-suite.log

1. TOTAL: 22

2. PASS: 21

3. SKIP: 0

4. XFAIL: 0

5. FAIL: 1

6. XPASS: 0

7. ERROR: 0

.. contents:: :depth: 2

FAIL: dispatch_timer_set_time

==================================================
[TEST] Dispatch Update Timer
[PID] 41980

[BEGIN] dispatch_source_create
Actual: 0x947180
Expected: 0x947180
[PASS] dispatch_source_create

[BEGIN] dispatch_source_timer_create
Actual: 0x947180
Expected: 0x947180
[PASS] dispatch_source_timer_create
1
2
3
4
5
6
7

[BEGIN] total duration
Actual: 3
Expected: <3
[FAIL] total duration (dispatch_timer_set_time.c:56)
dispatch_timer_set_time.c:56

[BEGIN] Process exited

Actual: 0
Expected: 0
[PASS] Process exited
[PERF] wall time: 0.000113
[PERF] user time: 0.304000
[PERF] system time: 1.084000
[PERF] max resident set size: 19684
[PERF] page faults: 0
[PERF] swaps: 0
[PERF] voluntary context switches: 319840
[PERF] involuntary context switches: 11
FAIL dispatch_timer_set_time (exit status: 1)

============================================================================
Testsuite summary for libdispatch 1.3

1. TOTAL: 22

2. PASS: 21

3. SKIP: 0

4. XFAIL: 0

5. FAIL: 1

6. XPASS: 0

7. ERROR: 0
   ```

[swift-ci]

Comment by Daniel A. Steffen (JIRA)

[BEGIN] total duration
Actual: 3
Expected: <3
[FAIL] total duration (dispatch_timer_set_time.c:56)
dispatch_timer_set_time.c:56

looks like a spurious time-related failure (due to load on the CI?)

dgrove-oss (JIRA User) have you seen this before ?

it seem that the time constants here could be increased to make this more robust (but make the test take longer of course) ?

[swift-ci]

Comment by David Grove (JIRA)

I think we've seen this test fail a handful of times in the CI. I can take a look at weakening the success criteria to make it less susceptible to noise on the CI test machine.

[swift-ci]

Comment by David Grove (JIRA)

Submitted #183 to adjust the number of fast ticks and the timeout to reduce sensitivity to load on CI machine.

[swift-ci]

Comment by David Grove (JIRA)

PR merged.

[swift-ci]

Comment by Doug Coleman (JIRA)

Would this change also be appropriate to apply to ``FAIL: dispatch_suspend_timer``?

``dispatch_suspend_timer`` fails here twice:

https://ci.swift.org/view/swift-3.0-branch/job/oss-swift-3.0-incremental-RA-linux-ubuntu-15_10/247/console
https://ci.swift.org/view/swift-3.0-branch/job/oss-swift-3.0-incremental-RA-linux-ubuntu-15_10/249/console

Perhaps anywhere in the test suite where the code is similar?

[swift-ci]

Comment by Doug Coleman (JIRA)

And also we should get this into swift-3.0-branch.

[swift-ci]

Comment by David Grove (JIRA)

I took a quick look at the other timer tests. The exact same fix doesn't apply, but the other tests do suffer from the same underlying problem: they assume a relatively unloaded system so that tasks/timers execute with smallish delays.

[swift-ci]

Comment by Doug Coleman (JIRA)

I went ahead and opened another issue for the remaining test issues.

https://bugs.swift.org/browse/SR-2931

This particular `dispatch_timer_set_time` fix got into swift-3.0-branch, so let's continue the discussion/work on the new issue. Thank you for looking at it!