Description
Description
We just upgraded redis from 4.7.0 to 5.1.0. In our CI we have a job to verify that all types are still correct (using tsc --noEmit). Since upgrading redis this job now runs out of memory:
CI log
> tsc --noEmit -p tsconfig.test.json
<--- Last few GCs --->
[47:0x7d3f4fbaf000] 79561 ms: Mark-Compact 2043.8 (2085.9) -> 2042.7 (2086.9) MB, pooled: 0 MB, 2499.84 / 0.00 ms (average mu = 0.093, current mu = 0.008) allocation failure; scavenge might not succeed
[47:0x7d3f4fbaf000] 82274 ms: Mark-Compact 2044.7 (2086.9) -> 2043.6 (2087.9) MB, pooled: 0 MB, 2692.69 / 0.00 ms (average mu = 0.049, current mu = 0.007) allocation failure; scavenge might not succeed
<--- JS stacktrace --->
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
ELIFECYCLE Command failed.
Aborted (core dumped)
I can reproduce this locally using NODE_OPTIONS="--max-old-space-size=2048" pnpm tsc --noEmit -p tsconfig.test.json (this does also include our application code, so the max size is probably different for a small reproduction).
I analyzed the output on my local machine (Macbook M3 Pro, 36GB RAM) using @typescript/analyze-trace, which gave the following output:
Hot Spots
└─ Check file [35m/project/test/test-helpers/[36mmocks.ts[39m[35m[39m (2747ms)
└─ Check expression from (line 21, char 5) to (line 23, char 7) (2734ms)
└─ Check expression from (line 21, char 18) to (line 23, char 7) (2717ms)
└─ Determine variance of type 335076 (2638ms)
└─ Compare types 689415 and 689406 (1366ms)
└─ Compare types 689415 and 689398 (1366ms)
└─ Compare types 689407 and 689398 (1330ms)
├─ {"id":689407,"kind":"GenericTypeAlias","name":"WithCommands","aliasTypeArguments":[335026,335036,335037,335038,47,335040],"location":{"path":"[35m/project/node_modules/[36m.pnpm[39m[35m/@redis+client@5.1.0/node_modules/[36m@redis/client[39m[35m/dist/lib/client/multi-command.d.ts[39m","line":9,"char":1}}
│ ├─ {"id":335026,"kind":"TypeParameter","name":"REPLIES","location":{"path":"[35m/project/node_modules/[36m.pnpm[39m[35m/@redis+client@5.1.0/node_modules/[36m@redis/client[39m[35m/dist/lib/client/multi-command.d.ts[39m","line":25,"char":41}}
│ ├─ {"id":335036,"kind":"TypeParameter","name":"M","location":{"path":"[35m/project/node_modules/[36m.pnpm[39m[35m/@redis+client@5.1.0/node_modules/[36m@redis/client[39m[35m/dist/lib/client/multi-command.d.ts[39m","line":25,"char":69}}
│ ├─ {"id":335037,"kind":"TypeParameter","name":"F","location":{"path":"[35m/project/node_modules/[36m.pnpm[39m[35m/@redis+client@5.1.0/node_modules/[36m@redis/client[39m[35m/dist/lib/client/multi-command.d.ts[39m","line":25,"char":93}}
│ ├─ {"id":335038,"kind":"TypeParameter","name":"S","location":{"path":"[35m/project/node_modules/[36m.pnpm[39m[35m/@redis+client@5.1.0/node_modules/[36m@redis/client[39m[35m/dist/lib/client/multi-command.d.ts[39m","line":25,"char":119}}
│ ├─ {"id":47,"kind":"TypeParameter"}
│ └─ {"id":335040,"kind":"TypeParameter","name":"TYPE_MAPPING","location":{"path":"[35m/project/node_modules/[36m.pnpm[39m[35m/@redis+client@5.1.0/node_modules/[36m@redis/client[39m[35m/dist/lib/client/multi-command.d.ts[39m","line":25,"char":170}}
└─ {"id":689398,"kind":"GenericTypeAlias","name":"WithCommands","aliasTypeArguments":[335026,335036,335037,335038,46,335040],"location":{"path":"[35m/project/node_modules/[36m.pnpm[39m[35m/@redis+client@5.1.0/node_modules/[36m@redis/client[39m[35m/dist/lib/client/multi-command.d.ts[39m","line":9,"char":1}}
├─ {"id":335026,"kind":"TypeParameter","name":"REPLIES","location":{"path":"[35m/project/node_modules/[36m.pnpm[39m[35m/@redis+client@5.1.0/node_modules/[36m@redis/client[39m[35m/dist/lib/client/multi-command.d.ts[39m","line":25,"char":41}}
├─ {"id":335036,"kind":"TypeParameter","name":"M","location":{"path":"[35m/project/node_modules/[36m.pnpm[39m[35m/@redis+client@5.1.0/node_modules/[36m@redis/client[39m[35m/dist/lib/client/multi-command.d.ts[39m","line":25,"char":69}}
├─ {"id":335037,"kind":"TypeParameter","name":"F","location":{"path":"[35m/project/node_modules/[36m.pnpm[39m[35m/@redis+client@5.1.0/node_modules/[36m@redis/client[39m[35m/dist/lib/client/multi-command.d.ts[39m","line":25,"char":93}}
├─ {"id":335038,"kind":"TypeParameter","name":"S","location":{"path":"[35m/project/node_modules/[36m.pnpm[39m[35m/@redis+client@5.1.0/node_modules/[36m@redis/client[39m[35m/dist/lib/client/multi-command.d.ts[39m","line":25,"char":119}}
├─ {"id":46,"kind":"TypeParameter"}
└─ {"id":335040,"kind":"TypeParameter","name":"TYPE_MAPPING","location":{"path":"[35m/project/node_modules/[36m.pnpm[39m[35m/@redis+client@5.1.0/node_modules/[36m@redis/client[39m[35m/dist/lib/client/multi-command.d.ts[39m","line":25,"char":170}}
This led me to a file used in our tests:
Affected code (unrelated code omitted for brevity)
// test/test-helpers/mocks.ts
import { createClient, type RedisClientType } from 'redis';
export class RedisTestContainer {
private redis: RedisClientType | undefined;
public async init() {
this.redis = createClient({ /* config */ });
await this.redis.connect();
return this.redis;
}
public async stop() {
await this.redis?.stop();
}
}It seems that type checking RedisClientType is taking 2.7s locally, most likely taking up a bunch of memory as it does so. From the output it seems the culprit is somewhere in multi-command.d.ts.
Replacing RedisClientType with { connect: () => Promise<unknown>; quit: () => Promise<unknown> } fixes the issue for us, since those are the only two methods we use anyway.
This fixes the issue for us, as running NODE_OPTIONS="--max-old-space-size=2048" pnpm tsc --noEmit -p tsconfig.test.json now finishes without errors. Our CI also no longer runs out of memory.
Node.js Version
22.14.0
Redis Server Version
Node Redis Version
5.1.0
Platform
MacOS, Linux