Multi sessions screenshotOnFail not working correctly.

[Nozeren]

What are you trying to achieve?

Few months ago I asked if there's a way to save a screenshotAfterFail of a session + the original browser with Webdriver post .
A pr was created but looks like it doesn't work 100% of the time, I can't get all the screenshots like in the PR description.

I made a version of the test created on the PR to test it out.

Scenario('should save screenshot for sessions @session', async ({ I }) => {
  await I.amOnPage('/');
  await I.saveScreenshot('original.png');
  await I.amOnPage('/browsevideos');
  await I.saveScreenshot('main_session.png');
  session('john', async () => {
    await I.amOnPage('/browsevideos');
    await I.see('Fake text'); // Fails here
  });
});

What do you get instead?

I get all the screenshots except the after fail screenshot of the main browser.

but appears as artificat on the output

Output with ---verbose

 › john - Screenshot is saving to ./output/john_should_save_screenshot_for_sessions_@session.failed.png
 › Screenshot of john session has been saved to ./output/john_should_save_screenshot_for_sessions_@session.failed.png
 › <Error: ENOENT: no such file or directory, open './output/should_save_screenshot_for_sessions_@session.failed.png'>

Details

• CodeceptJS version: 3.6.7

[Nozeren]

It helps a lot to have the screenshots for debug, when you have scenarios with more than 2 sessions.
does anyone have the same problem? maybe it's only on my side..

[Nozeren]

@kobenguyent
Ok, I did a investigation to understand what is happening behind the scenes.
Unfortunately I cant run the acceptance session_test.js to make the changes and test it myself...

I will try to explain it here hoping that someone can figure out.

1st Issue:
In screenshotOnFail plugin line 123:
We have a condition that is checking the allureReporter and right after we add the screenshot of the "main browser/session" as allure attachment.

         if (allureReporter) {
            allureReporter.addAttachment(
              'Main session - Last Seen Screenshot',
              fs.readFileSync(path.join(global.output_dir, fileName)),
              dataType,
            )

If the test fails inside a different session ("multiple session") the plugin will try to add the main session screenshot that doesn't exist and the screenshot from the other session will never be attached.
In webdriver.js we add automatically the session name before the screenshot name. (webdriver.js line 1840).

Expected result:

• Attach session screenshot to allure.

Actual Result:

• Plugin fails before multiple session attachment. Currently if we wanna attach a screenshot from another session it requires to have the screenshots from the all the sessions.

2nd Issue:
Here we have a test scenario with 3 sessions (main session + 2 "multiple sessions"), and the test fails inside the second "multiple session" created.

    Scenario("Testing multiple sessions screenshotOnFail", async ({I }) => {
        await I.amOnPage('/');

        await session("first", async() => {
            await I.amOnPage('/');
        });
        await session("second", async() => {
            await I.amOnPage('/');
            await I.see('FAILS HERE');
         });
 });

In this case we expect only the screenshot from the 2nd "multiple session" but the plugin is trying to attach a screenshot that doesn't exist from the 1st session because it didn't fail on that one.
screenshotOnFail line 132 we have a for loop attaching screenshot from all the sessions.

            if (helper.activeSessionName) {
              const sessions = helper.sessionPages || helper.sessionWindows
              for (const sessionName in sessions) {
                const screenshotFileName = `${sessionName}_${fileName}`
                test.artifacts[`${sessionName.replace(/ /g, '_')}_screenshot`] = path.join(
                  global.output_dir,
                  screenshotFileName,
                )
                allureReporter.addAttachment(
                  `${sessionName} - Last Seen Screenshot`,
                  fs.readFileSync(path.join(global.output_dir, screenshotFileName)),
                  dataType,
                )
              }
            }

and the for loop inside webdriver.js line 1836 is saving the screenshot with the 1st session name with the browser of the 2nd session:

if (this.activeSessionName) {
      const browser = this.sessionWindows[this.activeSessionName]

      for (const sessionName in this.sessionWindows) {
        const activeSessionPage = this.sessionWindows[sessionName]
        outputFile = screenshotOutputFolder(`${sessionName}_${fileName}`)

        this.debug(`${sessionName} - Screenshot is saving to ${outputFile}`)

        if (browser) {
          this.debug(`Screenshot of ${sessionName} session has been saved to ${outputFile}`)
          return browser.saveScreenshot(outputFile)
        }
      }
    }

Expected result:

• Attached Screenshot from 2nd session browser with 2nd session name.

Actual result:

• Saves screenshot from 2nd session browser with 1st session name + no attachment on allure report.