[Flaky CI]: Node (16) Unit Tests `@sentry/core` `offline.test.ts`

[AbhiPrasad]

Flakiness Type

Timeout

Name of Job

Node (16) Unit Tests

Name of Test

@sentry/core test/lib/transports/offline.test.ts

Link to Test Run

https://github.com/getsentry/sentry-javascript/actions/runs/4115185807/jobs/7104054722

Details

> @sentry/core:test

$ jest
 PASS  test/lib/base.test.ts (9.164 s)
 PASS  test/lib/transports/base.test.ts
 PASS  test/lib/integration.test.ts
 PASS  test/lib/integrations/inboundfilters.test.ts
 FAIL  test/lib/transports/offline.test.ts (17.526 s)
  ● makeOfflineTransport › Follows the Retry-After header

    expect(received).toEqual(expected) // deep equality

    Expected: 2
    Received: 1

      310 |     await delay(4_000);
      311 |
    > 312 |     expect(getSendCount()).toEqual(2);
          |                            ^
      313 |     expect(queuedCount).toEqual(0);
      314 |     expect(getCalls()).toEqual(['pop', 'pop']);
      315 |   }, 7_000);

      at Object.<anonymous> (test/lib/transports/offline.test.ts:312:28)
      at fulfilled (node_modules/tslib/tslib.js:112:62)

[timfish]

This is still occasionally reoccurring.

[timfish]

Looks like this fails around 10% of the time in CI and much less than 1% on my machine.

Since I can get the tests passing by adding a few console.log throughout the code, I suspect this could be caused by a race in the implementation rather than a poorly designed test.

[AbhiPrasad]

Maybe there is an issue from popping from the store inside the timeout? That causes a stale retry header somehow?

[timfish]

I've been left a little stumped by this. It's not helped by the fact that it only really reproduces occasionally in CI.

I played CI whack a mole on and off for most of yesterday and it looks like the 2nd envelope is always popped from the store but it appears sometimes it's not counted at sent.