Spans in continueTrace callback are not being recorded

[kparksalku]

Is there an existing issue for this?

• I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• I have reviewed the documentation https://docs.sentry.io/
• I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/aws-serverless

SDK Version

8.18.0

Framework Version

No response

Link to Sentry event

https://alku.sentry.io/performance/trace/d7e316d5f796b8b046d0c0cdbf5ca61c/?pageEnd&pageStart&project=4507617300054016&source=traces&statsPeriod=1h&timestamp=1722461032.632

Reproduction Example/SDK Setup

Initialization code for both Lambda functions:

  Sentry.init({
    debug: true,
    dsn: SENTRY_DSN,
    environment: "sandbox",
    integrations: [
      Sentry.postgresIntegration(),
      nodeProfilingIntegration()
    ],
    tracesSampleRate: 1.0,
    profilesSampleRate: 1.0,
  });

Lambda A:

export const handler = Sentry.wrapHandler(async (event) => {
  // Generate ClientContext headers

  // Invoke Lambda B with generated ClientContext

  // Return a response
});

Lambda B:

export const handler = async (event, context) => {
  const { baggage, sentryTrace } = extractFromLambdaClientContextHeaders(context);

  return await Sentry.continueTrace({ baggage, sentryTrace }, async () => {
    console.log(Sentry.getActiveSpan());
    const queryString = "select count(*) from myTable;";

    return await Sentry.startSpan({
      name: queryString,
      op: "db.sql.execute"
    }, async (span) => {
      console.log(span);
      return await executeQuery();
    });
  });
});

Steps to Reproduce

1. Invoke Lambda A, which invokes Lambda B
2. Check Traces, in Sentry.io

Expected Result

The trace should record the "db.sql.execute" span as a child span in the trace.

Actual Result

There is no record of the "db.sql.execute" span at all.

In the logs, I can see that continueTrace creates a nonRecordingSpan, which is not sent to Sentry. I believe this is the root of the problem. The call to startSpan appears to create a normal (recording) span, but since it is a child of the non-recording span, it is not sent along to Sentry either.

[chargome]

Hey @kparksalku, thanks for reaching out.

So ideally in v8 these spans should be automatically connected, or is there a specific reason why you use continueTrace?

[Lms24]

Just to add to this: Have you tried using our Sentry.wrapHandler wrapper in Lambda function B?

Also, can you confirm that the headers are actually present when Lambda B is invoked?

[kparksalku]

Hey @kparksalku, thanks for reaching out.

So ideally in v8 these spans should be automatically connected, or is there a specific reason why you use continueTrace?

I'm not sure how it's possible that they would be automatically connected, since these are distributed systems. To answer your question, though, the reason I'm using continueTrace is that the documentation says to use it, and moreover, nothing else works.

[kparksalku]

Just to add to this: Have you tried using our Sentry.wrapHandler wrapper in Lambda function B?

Also, can you confirm that the headers are actually present when Lambda B is invoked?

Yes, I tried adding wrapHandler to Lambda B. When I do this, it creates what appears to be a normal (recording) span, but the trace ID is new; it's not the trace ID sent in the ClientContext. This raises another question as to why this never makes it to Sentry; I don't see this trace at all, and I would expect two separate traces show up, but they don't. Inside of this handler, if I keep continueTrace, then the active span inside this callback is a non-recording span, BUT the trace ID matches what's sent in the ClientContext. So it appears this span is getting associated correctly, but it's not being sent for some reason.

I believe the fact that the continueTrace span has the correct trace ID is proof that the headers are making it through to Lambda B, is it not?

[Lms24]

I'm not sure how it's possible that they would be automatically connected, since these are distributed systems.

The idea is that it works automatically in the following situations:

• Lambda is run as CommonJS: OpenTelemetry instrumentation used by our SDK continues the trace
• Lambda is run as ESM: Our wrapHandler function continues the trace

Can you confirm that you run your lambda in ESM (using import syntax)?

Both should pick up our sentry-trace and baggage handlers correctly in the incoming request.

it's not the trace ID sent in the ClientContext

Hmm this is interesting and gave me an idea. Maybe there's a detail I'm not aware of for AWS lambda functions. Our handler extracts the tracing headers from the event object. You extract the headers from the context object. Is there a difference between these two? Perhaps our way is outdated by now and we didn't notice because we can't require the headers to be present 🤔

I believe the fact that the continueTrace span has the correct trace ID is proof that the headers are making it through to Lambda B, is it not?

Technically yes. I just didn't see this in the trace you linked (In fact, I'm not sure if this trace is the one from Lambda A or B or what should be in there vs. what isn't. I can only see that there's just one transaction, so it's not linked).

To debug and fix this we need to reproduce this properly. Can you show us how you invoke Lambda B from Lambda A?
If possible it'd be great if you could provide a minimal but fully reproducible example. Otherwise we have to do this ourselves which likely ends up different from your version.