Some dependencies don't support ES modules

[jimsimon]

Edit 2: Per request, I've simplified this down to a few simple questions and answers.

What did you do?
I attempted to use dom-testing-library in a test suite that's be run via Web Test Runner.

What happened?
The test runner bombed out complaining about CJS modules that are dependencies of dom-testing-library: aria-query, corejs, and lz-string. There may be more but that's as far as I spent time digging into them.

What did you expect to happen?
I expected the test runner to actually run without dependency errors.

Is there a repro?
https://github.com/jimsimon/wtr-dtl/tree/wtr-using-wds (note the branch name)

What have you tried so far?
I've tried using the Web Test Runner's CJS support that leverages Rollup's CJS plugin but that did not help. It appears the dependencies causing the errors are not supported by Rollup's CJS plugin.

Is there a proposed solution?
Upgrade to versions of these dependencies that support native ES modules or migrate to alternatives that do.

Any other notes?
One of the conflicting dependencies (aria-query) was updated in #1058 so that one may no longer be problematic. I haven't had a chance to test it though. I just tried the repro with the latest versions of all dependencies and I'm still having issues.

------------------------------------------------ FIRST EDIT STARTS BELOW -----------------------------------------------------------

Edit: This issue originally started as a request to have dom-testing-library broken down into separate packages to make it easier to consume by non-Jest based test runners. It has since transformed into a discussion about what's preventing dom-testing-libary form working with web-test-runner. In an effort to focus this ticket, I've copied the relevant parts of the conversation below.

First a TLDR: dom-testing-library does not work with web-test-runner. Multiple dependencies used by dom-testing-library do not publish native esm modules and are not compatible with rollup's commonjs plugin (which can be used by web-test-runner). One big offender is aria-query, which may be fixed with #1058 but has not been tested yet. Main repro can be found in https://github.com/jimsimon/wtr-dtl/tree/wtr-using-wds (note the branch name since there are multiple different repros for different attempts to workaround the problem in this repo).

The main problem is that web-test-runner is designed for "buildless" environments using entirely native ESM in the browser. Dom-testing-library has a handful of dependencies that only provide CJS modules that don't play nice in this environment. You can actually use the rollup commonjs plugin for some of these, but some still don't work. Specifically, I recall running into issues with aria-query, corejs, and lz-string. The corejs dependency is some sort of weird prebundled one coming transitively via aria-query.

I'll try to create a sample repo tomorrow and provide a link here, but it's pretty simple to replicate. Create a brand new node project, add web-test-runner and dom-testing-library, write a simple test that uses screen, run the tests via web-test-runner. You'll see web-test-runner just choke and/or throw an error about the CJS deps.

The most success I've had was using snowpack with their web-test-runner plugin in "remote" dependency mode. Unfortunately even that approach errors out.

My suggestion to split the queries out to a separate package was to hopefully reduce the dependency load. A better approach might be to work towards updating to dependencies that also provide native ESM distributions (or remove them altogether).

-- @jimsimon

Does web-test-runner use the module field in package.json?

-- @nickmccurdy

It does as it actually uses https://github.com/rollup/plugins/tree/master/packages/node-resolve under the hood for them. Just for clarity sake, web-test-runner is built on top of web-dev-server, which is what actually handles file transformation.

The main issue seems to be some dependencies that aren't providing ESM distributions. Aria-query in specific is keeping corejs3-runtime as a dependency to support people still on Node 6. There's a ton of discussions about that in A11yance/aria-query#158 and a PR to clean a lot of it up here: A11yance/aria-query#180. So it seems part of the solution here may be to encourage that PR to move forward and then update dom-testing-library accordingly? There are still other dependencies that may need updating as well, but maybe there's some alternatives that provide ESM output available?

-- @jimsimon

I pushed up a couple versions of my demo repo:

Snowpack using remote sources (main branch): https://github.com/jimsimon/wtr-dtl
Snowpack using local sources (snowpack-local branch): https://github.com/jimsimon/wtr-dtl/tree/snowpack-local
WTR using it's built in Web Dev Server + ESBuild (wtr-using-wds branch): https://github.com/jimsimon/wtr-dtl/tree/wtr-using-wds
Hopefully these help. The last link is the one I'd most like to see working and is what spawned this ticket and convo. The Snowpack stuff was an attempt at working around the issue.

Note: The test won't actually pass if you get it to run. That's a different issue/concern regarding being able to pass ShadowRoot to queries. I'm hoping we can address that in a different issue/ticket if we can get this one resolved.

-- @jimsimon

Note that @testing-library/dom itself would not work with native ES modules. But then I see mentions of aria-query which may be fixed with #1058.

So could somebody summarize what the issue is? The issue description is currently missing what is concretely not working (ideally with a repro).

-- @eps1lon

This is the end of the conversation as of this edit. Hopefully this helps!

--------------------------- ORIGINAL ISSUE DESCRIPTION BELOW -------------------------------------------------

Suggested implementation:

Create a new @testing-library/queries package and move all query related code to it.

Describe alternatives you've considered:

The only alternative I can think of would be to reimplement the same queries in an entirely new project. This seems like wasted effort if we can simply extract the ones that testing-library already provides.

Teachability, Documentation, Adoption, Migration Strategy:

There should be no outward facing impact on users as API's would remain backwards compatible. Documentation for the new package would need to be created, and some minor documentation updates might be needed for the main site.

[nickserv]

I'm seeing two related issues here, so I'll split up my response:

Supporting different frameworks

As far as I'm aware, while we primarily focus on Jest support, we still want to support other test frameworks. Testing Library shouldn't necessarily require Jest, it's just a convenient option because of its simple extensible APIs and built in JSDOM support, and it's already popular in the React community.

That being said, I have seen some bugs in some of our packages caused by us assuming a JSDOM environment in Jest, which can cause issues with supporting a real DOM, such as in web-test-runner. I'm assuming that your issue is either related to DOM support or ES module support. Either way, I would appreciate it if you could post an example (preferably one we could run locally) showcasing your specific issues with using DOM Testing Library in web-test-runner.

Using only queries

Because DOM Testing Library doesn't rely on any specific web framework, it works with plain DOM Nodes, so its queries should already be useful in isolation. If you don't want the entire package namespace imported, you have three options:

1. Import screen to use any query on the entire document: import { screen } from "@testing-library/dom"
2. Import specific queries to use them on any element: import { getByText } from "@testing-library/dom"
3. Import queries to use any query on any element: import { queries } from "@testing-library/dom"

From a packaging optimization perspective, I'm not sure how well we support tree shaking, which would let build tools remove other parts of the library that queries don't depend on. If you have a situation where you need to use Testing Library in production but your bundle is too build, I can try to help you look into that if you can post more information, such as the results of running webpack bundle analyzer.

From a package maintenance perspective, separating out our queries into a separate package would require additional infrastructure and manual package dependency bumps (which is a bit trickier to keep track of since we're not using monorepos/workspaces). DOM Testing Library doesn't have too many dependencies, and as long as tree shaking is working properly they can be removed when they're not used anyway. Additionally, I think it's worth noting that our framework specific packages like React Testing Library wrap other DOM Testing Library APIs such as waitFor, so just importing the queries wouldn't be enough.

Example

Let's say you were writing a debugging tool that prints a DOM Node in the console after asking the user for its text content, so you don't care about our other testing utilities such as waitFor, Jest DOM, or React Testing Library. You could import only the queries you want to use and use the DOM Nodes they return.

import { screen } from "@testing-library/dom";

const text = prompt("Type in the text of a Node to get");

if (text) {
  const node = screen.getByText(text);
  console.log(node);
} else {
  console.log("Node not found");
}

[jimsimon]

@nickmccurdy Thanks for the elaborate response. I actually tried the approach that you suggested before creating this ticket, but it still doesn't work. The main problem is that web-test-runner is designed for "buildless" environments using entirely native ESM in the browser. Dom-testing-library has a handful of dependencies that only provide CJS modules that don't play nice in this environment. You can actually use the rollup commonjs plugin for some of these, but some still don't work. Specifically, I recall running into issues with aria-query, corejs, and lz-string. The corejs dependency is some sort of weird prebundled one coming transitively via aria-query.

I'll try to create a sample repo tomorrow and provide a link here, but it's pretty simple to replicate. Create a brand new node project, add web-test-runner and dom-testing-library, write a simple test that uses screen, run the tests via web-test-runner. You'll see web-test-runner just choke and/or throw an error about the CJS deps.

The most success I've had was using snowpack with their web-test-runner plugin in "remote" dependency mode. Unfortunately even that approach errors out.

My suggestion to split the queries out to a separate package was to hopefully reduce the dependency load. A better approach might be to work towards updating to dependencies that also provide native ESM distributions (or remove them altogether).

Edit: fat fingered the close issue button on my phone

[nickserv]

Does web-test-runner use the module field in package.json? We have one, but I'm not sure if the issue is a lack of support in web-test-runner, or that our ES module build still has some CJS module dependencies. If the issue is needing dependencies to be bundled, there's also a browser field we could theoretically start using.

Ideally I think the best solution would be supporting ES module tools as well as we can, but I recognize there can be a lot of complicated inconsistencies between them, and that our dependencies could cause additional issues. If we decided not to go this route and split the packages, we'd be trading ES module maintenance for separate package maintenance, and limiting which parts of Testing Library could be used with similar ES module based tools.

[alexkrolick]

FYI aria-query is a dependency of getByRole anyways, so pulling out the queries package will not fix the module format issue for this use case

[jimsimon]

Does web-test-runner use the module field in package.json?

It does as it actually uses https://github.com/rollup/plugins/tree/master/packages/node-resolve under the hood for them. Just for clarity sake, web-test-runner is built on top of web-dev-server, which is what actually handles file transformation.

...but I'm not sure if the issue is a lack of support in web-test-runner, or that our ES module build still has some CJS module dependencies. If the issue is needing dependencies to be bundled, there's also a browser field we could theoretically start using.

The main issue seems to be some dependencies that aren't providing ESM distributions. Aria-query in specific is keeping corejs3-runtime as a dependency to support people still on Node 6. There's a ton of discussions about that in A11yance/aria-query#158 and a PR to clean a lot of it up here: A11yance/aria-query#180. So it seems part of the solution here may be to encourage that PR to move forward and then update dom-testing-library accordingly? There are still other dependencies that may need updating as well, but maybe there's some alternatives that provide ESM output available?