Skip to content

fix(tests): Limit Remix server integration test stacktrace length. #5711

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 12, 2022

Conversation

onurtemizkan
Copy link
Collaborator

@onurtemizkan onurtemizkan commented Sep 8, 2022

Related: #5650

Updated Remix server-side SDK to use the message of the thrown error instead of the whole object, when the error is not a response.

Limited maximum stacktrace length for Remix server-side integration tests.

Also updated tests to filter by type and separate consecutive multi-type envelopes, and not to trust the capturing order.

@lobsterkatie
Copy link
Member

Updated Remix server-side SDK to use the message of the thrown error instead of the whole object, when the error is not a response.

I know we talked about it briefly earlier today, but I'm having second thoughts about this approach. I have a few questions:

  1. It seems like with this change, the event will end up with no stacktrace at all. Is that correct?

  2. Given that what started all of this was me setting a (perhaps unrealistically high) stacktrace limit in the repo's dev environment, and that it seems the thing that broke wasn't tests failing but the actual testing apparatus breaking down, my guess would be that the problem we're fixing here isn't something an actual SDK user would ever run into. Does that square with what you know?

If so, I think a better idea would be to fix the test environment. What about changing the "test:integration:server" script to "export NODE_OPTIONS='--stack-trace-limit=25' && jest --config=test/integration/jest.config.js test/integration/test/server/"? I tried it out and it seems to fix the problems not already covered by your other fixes here (which I think are definitely good).

@onurtemizkan
Copy link
Collaborator Author

onurtemizkan commented Sep 9, 2022

It seems like with this change, the event will end up with no stacktrace at all. Is that correct?

Yes, that's definitely correct. I saw the frames are populated in the event, and thought it's fine. But they have only included SDK frames, not the actual error. 🤦

Given that what started all of this was me setting a (perhaps unrealistically high) stacktrace limit in the repo's dev environment, and that it seems the thing that broke wasn't tests failing but the actual testing apparatus breaking down, my guess would be that the problem we're fixing here isn't something an actual SDK user would ever run into. Does that square with what you know?

The issue is the JSON parser that Jest uses breaks when such a large string is around. I haven't debugged deeper, so the reason is still unclear to me. The origins of the deep stacktraces are express.lib.router and express.lib.middleware, so they are not necessarily testing-specific. I don't know if there can be a non-testing environment that could be using a setup / parser like that.

But, I have just tested the high stacktrace limit on an actual Remix project, and no issues at all.

Updating the PR now 👍

@onurtemizkan onurtemizkan force-pushed the onur/fix-remix-error-message branch from a10f63d to c0c0747 Compare September 9, 2022 11:54
@onurtemizkan onurtemizkan changed the title fix(remix): Use error message instead of whole error stack. fix(tests): Limit Remix server integration test stack trace size. Sep 9, 2022
@onurtemizkan onurtemizkan changed the title fix(tests): Limit Remix server integration test stack trace size. fix(tests): Limit Remix server integration test stacktrace length. Sep 9, 2022
Copy link
Member

@lobsterkatie lobsterkatie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@lobsterkatie lobsterkatie merged commit 9c01fcc into master Sep 12, 2022
@lobsterkatie lobsterkatie deleted the onur/fix-remix-error-message branch September 12, 2022 05:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants