afterEach(cleanup); isn't cleaning up after each test

[TidyIQ]

• react-testing-library version: 7.0.0
• react version: 16.8.6
• node version: 10.15.3
• npm (or yarn) version: 6.9.0

Relevant code or config:

afterEach(cleanup);

describe("Unit: <OutlinedInput />", (): void => {
  describe.each`
    data                 | showIcon | name                                     | validInput
    ${nameFieldData}     | ${false} | ${"text fields"}                         | ${"John Smith"}
    ${nameFieldData}     | ${true}  | ${"text fields with icon adornment"}     | ${"John Smith"}
    ${passwordFieldData} | ${false} | ${"password fields"}                     | ${"1234567a"}
    ${passwordFieldData} | ${true}  | ${"password fields with icon adornment"} | ${"1234567a"}
    ${emailFieldData}    | ${false} | ${"email fields"}                        | ${"me@domain.com"}
    ${emailFieldData}    | ${true}  | ${"email fields with icon adornment"}    | ${"me@domain.com"}
  `(
    "Initial render",
    ({ data, showIcon, name, validInput }): void => {
      it(`renders as snapshot for ${name}`, (): void => {
        const { asFragment } = render(
          <OutlinedInput fieldData={data} showIcon={showIcon} />
        );
        expect(asFragment()).toMatchSnapshot();
      });

      describe("THEN focus in", (): void => {
        it(`renders as snapshot for ${name}`, (): void => {
          const { asFragment, getByLabelText } = render(
            <OutlinedInput fieldData={data} showIcon={showIcon} />
          );
          const input = getByLabelText(data.label);
          fireEvent.focus(input);
          expect(asFragment()).toMatchSnapshot();
        });
      });

      describe("THEN valid input", (): void => {
        it(`renders as snapshot for ${name}`, (): void => {
          const { asFragment, getByLabelText } = render(
            <OutlinedInput fieldData={data} showIcon={showIcon} />
          );
          const input = getByLabelText(data.label);
          fireEvent.focus(input);
          fireEvent.change(input, { target: { value: validInput } });
          expect(asFragment()).toMatchSnapshot();
        });
      });
    }
  );
});

Notice how I have afterEach(cleanup); declared at the top of the file.

When I run the test, the snapshots created for the first test Unit: <OutlinedInput /> Initial render renders as snapshot for ${name} with icon adornment all include the validInput that is declared in the describe.each() at the top. Only the with adornment tests include the validinput, not the ones without.

This is a bug.

I know this because nowhere in that first snapshot test do I run fireEvent.change(input, { target: { value: validInput } });. That fireEvent is only run in the last snapshot test, and that is the only place that the input value can be changed.

If I change the values of validInput, e.g. to "John Brown", then the snapshot shows this new value, so I know it's getting the value from the describe.each() and not anywhere else in my code.

Also, it only occurs on every second test in the describe.each() (the ones where showIcon === true).

Here are two snapshots that show the input value is incorrectly being added for this test:

exports[`Unit: <OutlinedInput /> Initial render renders as snapshot for text fields 2`] = `
<DocumentFragment>
  <div
    class="MuiFormControl-root MuiFormControl-marginDense MuiFormControl-fullWidth"
  >
    <label
      class="MuiFormLabel-root MuiInputLabel-root MuiInputLabel-formControl MuiInputLabel-animated MuiInputLabel-marginDense MuiInputLabel-outlined makeStyles-inputLabel-1"
      data-shrink="false"
      for="name"
    >
      Name Field Label
    </label>
    <div
      class="MuiInputBase-root MuiOutlinedInput-root MuiInputBase-formControl MuiInputBase-marginDense MuiInputBase-adornedStart MuiOutlinedInput-adornedStart"
      data-testid="outlinedInputInput"
    >
      <fieldset
        aria-hidden="true"
        class="PrivateNotchedOutline-root-61 MuiOutlinedInput-notchedOutline makeStyles-notchedOutline-6"
        style="padding-left: 8px;"
      >
        <legend
          class="PrivateNotchedOutline-legend-62"
          style="width: 0.01px;"
        >
          <span>
            ​
          </span>
        </legend>
      </fieldset>
      <svg
        aria-hidden="true"
        class="MuiSvgIcon-root"
        data-testid="icon"
        focusable="false"
        role="presentation"
        viewBox="0 0 24 24"
      >
        <path
          d="M10 20v-6h4v6h5v-8h3L12 3 2 12h3v8z"
        />
        <path
          d="M0 0h24v24H0z"
          fill="none"
        />
      </svg>
      <hr
        class="MuiDivider-root makeStyles-divider-63"
      />
      <input
        aria-invalid="false"
        class="MuiInputBase-input MuiOutlinedInput-input MuiInputBase-inputMarginDense MuiOutlinedInput-inputMarginDense MuiInputBase-inputAdornedStart MuiOutlinedInput-inputAdornedStart"
        id="name"
        minlength="1"
        pattern="^(?!\\\\s*$).+"
        required=""
        type="text"
        value=""     //  ***** NO ERROR HERE FOR THIS SNAPSHOT *****
      />
    </div>
    <div
      class="MuiCollapse-container MuiCollapse-hidden"
      style="min-height: 0px;"
    >
      <div
        class="MuiCollapse-wrapper"
      >
        <div
          class="MuiCollapse-wrapperInner"
        >
          <p
            class="MuiFormHelperText-root MuiFormHelperText-contained MuiFormHelperText-marginDense"
            data-testid="outlinedInputHelperText"
          />
        </div>
      </div>
    </div>
  </div>
</DocumentFragment>
`;

exports[`Unit: <OutlinedInput /> Initial render renders as snapshot for text fields with icon adornment 2`] = `
<DocumentFragment>
  <div
    class="MuiFormControl-root MuiFormControl-marginDense MuiFormControl-fullWidth"
  >
    <label
      class="MuiFormLabel-root MuiFormLabel-filled MuiInputLabel-root makeStyles-labelValid-722 MuiInputLabel-formControl MuiInputLabel-animated MuiInputLabel-shrink MuiInputLabel-marginDense MuiInputLabel-outlined makeStyles-inputLabel-721"
      data-shrink="true"
      for="name"
    >
      Name Field Label
    </label>
    <div
      class="MuiInputBase-root MuiOutlinedInput-root makeStyles-notchedValid-725 MuiInputBase-formControl MuiInputBase-marginDense MuiInputBase-adornedStart MuiOutlinedInput-adornedStart"
      data-testid="outlinedInputInput"
    >
      <fieldset
        aria-hidden="true"
        class="PrivateNotchedOutline-root-781 MuiOutlinedInput-notchedOutline makeStyles-notchedOutline-726"
        style="padding-left: 8px;"
      >
        <legend
          class="PrivateNotchedOutline-legend-782"
          style="width: 0px;"
        >
          <span>
            ​
          </span>
        </legend>
      </fieldset>
      <svg
        aria-hidden="true"
        class="MuiSvgIcon-root makeStyles-validColor-723"
        data-testid="icon"
        focusable="false"
        role="presentation"
        viewBox="0 0 24 24"
      >
        <path
          d="M10 20v-6h4v6h5v-8h3L12 3 2 12h3v8z"
        />
        <path
          d="M0 0h24v24H0z"
          fill="none"
        />
      </svg>
      <hr
        class="MuiDivider-root makeStyles-divider-783"
      />
      <input
        aria-invalid="false"
        class="MuiInputBase-input MuiOutlinedInput-input MuiInputBase-inputMarginDense MuiOutlinedInput-inputMarginDense MuiInputBase-inputAdornedStart MuiOutlinedInput-inputAdornedStart"
        id="name"
        minlength="1"
        pattern="^(?!\\\\s*$).+"
        required=""
        type="text"
        value="John Smith"    //  ***** ERROR OCCURS HERE. THIS SHOULD NOT BE HERE *****
      />
    </div>
    <div
      class="MuiCollapse-container MuiCollapse-hidden"
      style="min-height: 0px;"
    >
      <div
        class="MuiCollapse-wrapper"
      >
        <div
          class="MuiCollapse-wrapperInner"
        >
          <p
            class="MuiFormHelperText-root MuiFormHelperText-contained MuiFormHelperText-marginDense MuiFormHelperText-filled"
            data-testid="outlinedInputHelperText"
          />
        </div>
      </div>
    </div>
  </div>
</DocumentFragment>
`;

[alexkrolick]

afterEach runs after it or test. describe blocks should not contain assertions. You can use test.each() instead.

[TidyIQ]

I can't use test.each() as it doesn't allow nesting.

Also, the describe block only contains it blocks so afterEach(cleanup) should be running after every test.

If I import the cleanup as import "react-testing-library/cleanup-after-each"; then change the afterEach to afterEach(() => console.log("test"));, I can see that afterEach is working fine. The console shows "test" for every test. The issue is with the cleanup, not with the afterEach().

[TidyIQ]

Using the same code as above that has 3 it blocks and 6 different tests to be run under describe.each(), there is a total of 18 tests performed. If I change the afterEach to:

let i = 0;
afterEach(() => {
  i++;
  return console.log(i);
});

And run the test, the console.log output is:

    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      1
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      2
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      3
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      4
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      5
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      6
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      7
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      8
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      9
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      10
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      11
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      12
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      13
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      14
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      15
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      16
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      17
    console.log components/Inputs/OutlinedInput.unit.test.tsx:67
      18

[alexkrolick]

react-testing-library/cleanup-after-each

Contents:

afterEach(require('./dist').cleanup)

What could be the difference here...?

[TidyIQ]

I can confirm that afterEach(cleanup) has a bug and isn't cleaning up the DOM between tests.

I have another test:

describe("On first click", () => {
  it("renders as snapshot", () => {
    const { asFragment, container } = render(
      <ShowHidePasswordButton {...props} />
    );
    const button = container.querySelector("button");
    fireEvent.click(button);
    expect(asFragment()).toMatchSnapshot();
  });

  it("shows the hide password icon", () => {
    const { container } = render(<ShowHidePasswordButton {...props} />);
    const button = container.querySelector("button");
    fireEvent.click(button);
    const svg = container.querySelector("svg");
    expect(svg.getAttribute("data-testid")).toBe("hidePasswordIcon");
  });
});

When I look at the snapshot created from the first test I can see that the svg element has data-testid value of hidePasswordIcon, however the second test fails with:

    expect(received).toBe(expected) // Object.is equality

    Expected: "hidePasswordIcon"
    Received: "showPasswordIcon"

If I remove the first snapshot test and only run the second test, then it passes fine. OR if I remove the fireEvent.click(button) from the second test then both tests pass fine.

The only possible cause of this is that when the first fireEvent.click(button) is called in the first snapshot test, it isn't cleaned up before the second test starts. This results in the second test starting with a fireEvent.click(button) already in the DOM, so when it runs fireEvent.click(button) within its own test, it's actually behaving as if the button was clicked twice.

The only logical conclusion to come from this, plus my original post, is that afterEach(cleanup) isn't cleaning up the DOM between tests as it's supposed to.

[TidyIQ]

Further proof: All these tests pass when they should be failing due to cleanup. Notice how two of the tests are identical yet they receive back different results for expect(svg.getAttribute("data-testid"))

afterEach(cleanup);

describe("Unit: <ShowHidePasswordButton />", (): void => {
  describe("initial render", (): void => {
    it("renders as snapshot", async (): Promise<void> => {
      const { asFragment } = render(<ShowHidePasswordButton {...props} />);
      expect(asFragment()).toMatchSnapshot();
      const screenshot = await generateImage();
      expect(screenshot).toMatchImageSnapshot();
    });
  });

  describe("On first click", (): void => {
    it("renders as snapshot", async (): Promise<void> => {
      const { asFragment, container } = render(
        <ShowHidePasswordButton {...props} />
      );
      const button: HTMLButtonElement = container.querySelector("button");

      fireEvent.click(button); // ERROR: This should be cleaned up before the next test runs  *******************

      expect(asFragment()).toMatchSnapshot();
      const screenshot = await generateImage();
      expect(screenshot).toMatchImageSnapshot();
    });

    it("shows the hide password icon", (): void => {
      const { container } = render(<ShowHidePasswordButton {...props} />);

      // ERROR: fireEvent.click(button) is not called here, yet the test passes *******************

      const svg = container.querySelector("svg");
      expect(svg.getAttribute("data-testid")).toBe("hidePasswordIcon");
    });
  });

  describe("On second click", (): void => {
    it("renders as snapshot", async (): Promise<void> => {
      const { asFragment, container } = render(
        <ShowHidePasswordButton {...props} />
      );
      const button: HTMLButtonElement = container.querySelector("button");

      fireEvent.click(button);

      expect(asFragment()).toMatchSnapshot();
      const screenshot = await generateImage();
      expect(screenshot).toMatchImageSnapshot();
    });

    it("shows the show password icon", async (): Promise<void> => {
      const { container } = render(<ShowHidePasswordButton {...props} />);
      const svg = container.querySelector("svg");
      expect(svg.getAttribute("data-testid")).toBe("showPasswordIcon");
    });
  });
});

[alexkrolick]

Maybe it has something to do with asFragment?

@TidyIQ can you confirm cleanup-after-each does work but afterEach(cleanup) does not?

[TidyIQ]

Neither of them work.

The asFragment is cleaning up as the snapshots are being saved individually. It's just the fireEvent's that aren't cleaning up.

[alexkrolick]

Can you provide a repository or codesandbox to reproduce this in?

[TidyIQ]

Sure, the repo is here: https://github.com/TidyIQ/tidyiq_website

I've changed the repo from private to public so please let me know once you've cloned it so I can switch it back over to private again.

Once you've installed the packages run npm run unit-test and you'll see that the test in /components/EndAdornments/ShowHidePasswordButton/__tests__/index.unit.test.tsx fails due to this issue.

I've also skipped a test in /components/Inputs/OutlinedInput/__tests__/index.unit.test.tsx that has the same issue with tests failing due to fireEvent not cleaning up. Remove the .skip from the first describe and you'll get those errors too.

Thanks for the help.

[TidyIQ]

Also, I moved afterEach(cleanup) to /jest.config.unit.js as:

  setupFilesAfterEnv: ["react-testing-library/cleanup-after-each"]

as per https://testing-library.com/docs/react-testing-library/setup so I don't need to the afterEach to every test file.

[TidyIQ]

@kentcdodds Any idea what the issue is here?

[TidyIQ]

It seems that the order of how I define each element makes a difference. For example, this won't work:

describe("Unit: <ShowHidePasswordButton />", (): void => {
  describe("Initial render", (): void => {
    it("renders as snapshot", async (): Promise<void> => {
      const { asFragment } = render(<ShowHidePasswordButton {...props} />);
      expect(asFragment()).toMatchSnapshot();
      const screenshot = await generateImage();
      expect(screenshot).toMatchImageSnapshot();
    });
  });

  describe("On first click", (): void => {
    it("renders as snapshot", async (): Promise<void> => {
      const { asFragment, container } = render(
        <ShowHidePasswordButton {...props} />
      );
      const button: HTMLButtonElement = container.querySelector("button");
      fireEvent.click(button);
      expect(asFragment()).toMatchSnapshot();
      const screenshot = await generateImage();
      expect(screenshot).toMatchImageSnapshot();
    });

    it("shows the hide password icon", (): void => {
      const { container } = render(<ShowHidePasswordButton {...props} />);
      const button: HTMLButtonElement = container.querySelector("button");
      fireEvent.click(button);

      const svg = container.querySelector("svg");  // ******** Defined after the fireEvent ***************************

      expect(svg.getAttribute("data-testid")).toBe("hidePasswordIcon");
    });
  });

  describe("On second click", (): void => {
    it("renders as snapshot", async (): Promise<void> => {
      const { asFragment, container } = render(
        <ShowHidePasswordButton {...props} />
      );
      const button: HTMLButtonElement = container.querySelector("button");
      fireEvent.click(button);
      fireEvent.click(button);
      expect(asFragment()).toMatchSnapshot();
      const screenshot = await generateImage();
      expect(screenshot).toMatchImageSnapshot();
    });

    it("shows the show password icon", (): void => {
      const { container } = render(<ShowHidePasswordButton {...props} />);
      const svg = container.querySelector("svg");
      const button: HTMLButtonElement = container.querySelector("button");
      fireEvent.click(button);
      fireEvent.click(button);
      expect(svg.getAttribute("data-testid")).toBe("showPasswordIcon");
    });
  });
});

But this does work:

describe("Unit: <ShowHidePasswordButton />", (): void => {
  describe("Initial render", (): void => {
    it("renders as snapshot", async (): Promise<void> => {
      const { asFragment } = render(<ShowHidePasswordButton {...props} />);
      expect(asFragment()).toMatchSnapshot();
      const screenshot = await generateImage();
      expect(screenshot).toMatchImageSnapshot();
    });
  });

  describe("On first click", (): void => {
    it("renders as snapshot", async (): Promise<void> => {
      const { asFragment, container } = render(
        <ShowHidePasswordButton {...props} />
      );
      const button: HTMLButtonElement = container.querySelector("button");
      fireEvent.click(button);
      expect(asFragment()).toMatchSnapshot();
      const screenshot = await generateImage();
      expect(screenshot).toMatchImageSnapshot();
    });

    it("shows the hide password icon", (): void => {
      const { container } = render(<ShowHidePasswordButton {...props} />);
      const button: HTMLButtonElement = container.querySelector("button");

      const svg = container.querySelector("svg");  // ******** Defined before the fireEvent ***************************

      fireEvent.click(button);
      expect(svg.getAttribute("data-testid")).toBe("hidePasswordIcon");
    });
  });

  describe("On second click", (): void => {
    it("renders as snapshot", async (): Promise<void> => {
      const { asFragment, container } = render(
        <ShowHidePasswordButton {...props} />
      );
      const button: HTMLButtonElement = container.querySelector("button");
      fireEvent.click(button);
      fireEvent.click(button);
      expect(asFragment()).toMatchSnapshot();
      const screenshot = await generateImage();
      expect(screenshot).toMatchImageSnapshot();
    });

    it("shows the show password icon", (): void => {
      const { container } = render(<ShowHidePasswordButton {...props} />);
      const svg = container.querySelector("svg");
      const button: HTMLButtonElement = container.querySelector("button");
      fireEvent.click(button);
      fireEvent.click(button);
      expect(svg.getAttribute("data-testid")).toBe("showPasswordIcon");
    });
  });
});

I'm not sure why this is. I've tried using wait and waitForDomChange in case there is a lag somewhere but they both timeout whenever I try using them.

Also, the test from my original post fails more if I similarly move around the test order. The actual test I have (the one from my OP is an excerpt of it) currently fails 15 times due to this issue, but if I move around the variables in a similar fashion then it fails 21 times.

I really don't understand this. I could really use some help here.

[kentcdodds]

The example code is huge. If you could please try to reproduce this with the minimal amount of code and preferably I'm a codesandbox, then we can determine what the problem is much more easily.

[TidyIQ]

I created a minimal repo here - https://github.com/TidyIQ/rtl_test_failure

git init
git clone https://github.com/TidyIQ/rtl_test_failure.git
npm install
npm run test