invalid table size Allocation failed - JavaScript heap out of memory

[djcors]

Environment

SaaS (https://sentry.io/)

Version

6.16.1

Steps to Reproduce

we are processing files with 300mb size approximately

• Create a transaction or issue.
• Send logging to sentry

Expected Result

we hope that Sentry works with large objects

Actual Result

<!--StartFragment-->

<--- Last few GCs --->
--
[1:0x4466300] 268773916 ms: Scavenge 273.3 (310.2) -> 273.3 (310.2) MB, 26.9 / 0.0 ms  (average mu = 0.989, current mu = 0.994) allocation failure
[1:0x4466300] 268775116 ms: Scavenge 465.3 (502.2) -> 465.3 (502.2) MB, 52.5 / 0.0 ms  (average mu = 0.989, current mu = 0.994) allocation failure
[1:0x4466300] 268777542 ms: Scavenge 849.3 (886.2) -> 849.3 (886.2) MB, 103.1 / 0.0 ms  (average mu = 0.989, current mu = 0.994) allocation failure
<--- JS stacktrace --->
==== JS stack trace =========================================
0: ExitFrame [pc: 0x13cf019]
Security context: 0x2d4d650008d1 <JSObject>
1: toJSON [0x3ed03c932039] [buffer.js:~1078] [pc=0xcc2219eaa55](this=0x2ae062034151 <Uint8Array map = 0x1884a08e55a9>)
2: walk [0x273ee2e85f19] [/usr/node/app/node_modules/@sentry/utils/dist/object.js:231] [bytecode=0xdeca0b31bc1 offset=70](this=0x04a28ac42a59 <JSGlobal Object>,0x2e22202835d9 <String[#6]: buffer>,0x2ae062034151 <Uint8Array map = 0x1884a08...
FATAL ERROR: invalid table size Allocation failed - JavaScript heap out of memory
1: 0xa093f0 node::Abort() [node]
2: 0xa097fc node::OnFatalError(char const*, char const*) [node]
3: 0xb842ae v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
4: 0xb84629 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
5: 0xd30fe5  [node]
6: 0xf24aea v8::internal::HashTable<v8::internal::NumberDictionary, v8::internal::NumberDictionaryShape>::New(v8::internal::Isolate*, int, v8::internal::AllocationType, v8::internal::MinimumCapacity) [node]
7: 0xf2522b v8::internal::HashTable<v8::internal::NumberDictionary, v8::internal::NumberDictionaryShape>::EnsureCapacity(v8::internal::Isolate*, v8::internal::Handle<v8::internal::NumberDictionary>, int, v8::internal::AllocationType) [node]
8: 0xf2574a v8::internal::Dictionary<v8::internal::NumberDictionary, v8::internal::NumberDictionaryShape>::Add(v8::internal::Isolate*, v8::internal::Handle<v8::internal::NumberDictionary>, unsigned int, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyDetails, int*) [node]
9: 0xe76a6e  [node]
10: 0xedfce2 v8::internal::JSObject::AddDataElement(v8::internal::Handle<v8::internal::JSObject>, unsigned int, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes) [node]
11: 0xf2b9aa v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::Maybe<v8::internal::ShouldThrow>, v8::internal::StoreOrigin) [node]
12: 0xf2c6e7 v8::internal::Object::SetProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::StoreOrigin, v8::Maybe<v8::internal::ShouldThrow>) [node]
13: 0x1060ca4 v8::internal::Runtime::SetObjectProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::StoreOrigin, v8::Maybe<v8::internal::ShouldThrow>) [node]
14: 0xdc18ca v8::internal::Runtime_KeyedStoreIC_Slow(int, unsigned long*, v8::internal::Isolate*) [node]
15: 0x13cf019  [node]

<!--EndFragment-->

[getsentry-release]

Routing to @getsentry/owners-ingest for triage. ⏲️

[getsentry-release]

Routing to @getsentry/team-webplatform for triage. ⏲️

[AbhiPrasad]

Hey, thanks for reaching out!

Is to possible for you to provide a small reproduction of how you are using the SDK? Are you setting these large objects as context or extra on the event - or is this error happening with default SDK usage (no additional context or breadcrumbs set).

Sentry support attachments, https://docs.sentry.io/platforms/javascript/enriching-events/attachments/, allowing you to attach up to 100mb worth of extra data onto an event. Does that help your use case here?

[djcors]

Hello.

We have global handleError which catches all error events from nodejs.

In our case, the files have been sent as a buffer from formData,

I think all formData information including buffer of file is being sent as context, at some point the SDK doesn't handle a buffer of this size correctly

This is our sentry init

import * as ErrorLog from "@sentry/node";
import config from "../../config";

ErrorLog.init({
  dsn: config.params.logger,
  serverName: config.env,
  environment: config.env,
  attachStacktrace: true,
  integrations: [new ErrorLog.Integrations.Http({ tracing: true })],
  release: config.params.loggerRelease,
  tracesSampler: (samplingContext): number => {
    if (samplingContext.transactionContext?.name.includes("/api/ping")) return 0;
    return 1;
  },
});

export default ErrorLog;

// throw capture exceptions

const errorLog = new ExceptionLog(logMessage, { request: req?.body, error: error });
ErrorLog.captureException(errorLog.message, errorLog.toContext());

[AbhiPrasad]

As per our documentation: https://develop.sentry.dev/sdk/event-payloads/

Event payload size limit is 200kb.

So we can only accept events up to a certain size. Beyond that, you'll have to rely on attachments.

We can maybe do a better job at preventing possible OOM errors, maybe by skipping serializing large buffers. Transferring this issue to the JS SDK repository so we can track that.

[djcors]

I'm not sure if this is the problem, with less buffer size (80mb) it doesn't happen this memory leak, I think what happens when it goes over 200mb

[AbhiPrasad]

Internally we have a walk function utility we use to normalize large objects (before they are sent to Sentry). As you pointed out, with large buffers (those over 200mb), this issue occurs, so we should skip them to prevent crashes.

I would recommend against attaching these large buffers onto event context, and instead rely on attachments.

[vladanpaunovic]

I am closing this issue as the follow up is in #4579. Whoever is interested, feel free to subscribe to that issue.