fix: skip failing benchmarks

[hoeck]

fixes #1611

Description

Skip failing benchmarks to not block other benchmarks from running properly.

This is a quick fix to keep libraries in the list that do not support older node versions like stnl that only seems to work for node 23.

One potential downside / danger is that we have to look out for dependabot changes that update library versions that later fail to run. So this fix is only an intermediate solution. We should replace ts-node with something more modern in the near future.

Testing

Check that even though stnl errors, the benchmark for valita still runs fine with node 22: nvm use node 22; npm run start stnl valita.

Checklist

• Conducted a self-review of the code changes: of course 😁
• Updated documentation, if necessary: added inline comments
• Added tests to validate the functionality or fix: atm we don't have tests at all for benchmark infrastructure code

[scarf005]

Suggested change

	console.error('Skipped "%s" benchmark due to an error', c);
	console.error('Skipped "%s" benchmark due to an error', c, e);

might be better to print error messages for easier debugging.

[hoeck]

Good catch but this only silences the exception thrown by childProcess.execFileSync. The original error in the subprocess is still printed to stdout or stderr:

erik@t460:~/src/typescript-runtime-type-benchmarks (fix-skip-errors-during-benchmark)$ nvm use 16
Now using node v16.20.1 (npm v8.19.4)
erik@t460:~/src/typescript-runtime-type-benchmarks (fix-skip-errors-during-benchmark)$ npm run start run stnl

> typescript-runtime-type-benchmarks@1.0.0 start
> ts-node index.ts run stnl

Removing previous results
Executing "stnl"
Loading "stnl"
/home/erik/src/typescript-runtime-type-benchmarks/node_modules/ts-node/dist/index.js:851
            return old(m, filename);
                   ^
Error [ERR_REQUIRE_ESM]: require() of ES Module /home/erik/src/typescript-runtime-type-benchmarks/node_modules/stnl/compilers/validate-json/compose.js from /home/erik/src/typescript-runtime-type-benchmarks/cases/stnl.ts not supported.
Instead change the require of compose.js in /home/erik/src/typescript-runtime-type-benchmarks/cases/stnl.ts to a dynamic import() which is available in all CommonJS modules.
    at Object.require.extensions.<computed> [as .js] (/home/erik/src/typescript-runtime-type-benchmarks/node_modules/ts-node/dist/index.js:851:20)
    at Object.<anonymous> (/home/erik/src/typescript-runtime-type-benchmarks/cases/stnl.ts:6:35)
    at Module.m._compile (/home/erik/src/typescript-runtime-type-benchmarks/node_modules/ts-node/dist/index.js:857:29)
    at Object.require.extensions.<computed> [as .ts] (/home/erik/src/typescript-runtime-type-benchmarks/node_modules/ts-node/dist/index.js:859:16)
    at /home/erik/src/typescript-runtime-type-benchmarks/cases/index.ts:87:72
    at async Object.importCase (/home/erik/src/typescript-runtime-type-benchmarks/cases/index.ts:87:5)
    at async main (/home/erik/src/typescript-runtime-type-benchmarks/index.ts:115:21) {
  code: 'ERR_REQUIRE_ESM'
}
Skipped "stnl" benchmark due to an error

This is the exception that gets ignored:

Error: Command failed: /home/erik/src/typescript-runtime-type-benchmarks/node_modules/.bin/ts-node /home/erik/src/typescript-runtime-type-benchmarks/index.ts run-internal stnl
    at genericNodeError (node:internal/errors:984:15)
    at wrappedFn (node:internal/errors:538:14)
    at checkExecSyncError (node:child_process:890:11)
    at Object.execFileSync (node:child_process:926:15)
    at main (/home/erik/src/typescript-runtime-type-benchmarks/index.ts:64:24)
    at Object.<anonymous> (/home/erik/src/typescript-runtime-type-benchmarks/index.ts:101:1)
    at Module._compile (node:internal/modules/cjs/loader:1358:14)
    at Module.m._compile (/home/erik/src/typescript-runtime-type-benchmarks/node_modules/ts-node/src/index.ts:1618:23)
    at Module._extensions..js (node:internal/modules/cjs/loader:1416:10)
    at Object.require.extensions.<computed> [as .ts] (/home/erik/src/typescript-runtime-type-benchmarks/node_modules/ts-node/src/index.ts:1621:12) {
  status: 1,
  signal: null,
  output: [ null, null, null ],
  pid: 41938,
  stdout: null,
  stderr: null
}