Failed expect inside browser.call() method doesn't fail test

user_8808272 0
Viewed 224

I'm using WebdriverIO in sync mode with mocha and devtools-service

//package.json

    "dependencies": {
    "@babel/cli": "^7.11.6",
    "@babel/core": "^7.11.6",
    "@babel/plugin-proposal-class-properties": "^7.10.4",
    "@babel/preset-env": "^7.11.5",
    "@babel/register": "^7.11.5",
    "@wdio/allure-reporter": "^6.6.0",
    "@wdio/cli": "^6.6.0",
    "@wdio/devtools-service": "^6.6.0",
    "@wdio/junit-reporter": "^6.6.0",
    "@wdio/local-runner": "^6.6.0",
    "@wdio/mocha-framework": "^6.6.0",
    "@wdio/spec-reporter": "^6.6.0",
    "@wdio/sync": "^6.6.0",
    "chai": "^4.2.0",
    "chromedriver": "^86.0.0",
    "moment": "^2.29.1",
    "wdio-chromedriver-service": "^6.0.4"
  },

Inside one of the tests I have following code that should open the url, check if non of API responses was 500 and wait for some UI element to show up:

      const puppeteer = browser.getPuppeteer();
      const page = browser.call(() => puppeteer.pages())[0];
      browser.call(() =>
          page.on('response', response => {
             expect(response.status()).not.toEqual(500);
          })
      );
     browser.call(() => page.goto(item.url, { waitUntil: 'networkidle0' }));
     browser.call(() => page.waitForSelector('div.headline'));

The problem is that when I run this test and error 500 occurs, I can see in the log that expect have failed, but the test itself is not failed.

Here is part of the console output that clearly says that expect failed:

[0-0] (node:14952) UnhandledPromiseRejectionWarning: Error: expect(received).not.toEqual(expected) // deep equality

Expected: not 500

    at d:\Robocze\bos-frontend-tests\healthcheck\/healthcheck.gen.js:50:55
    at D:\Robocze\bos-frontend-tests\node_modules\puppeteer-core\lib\cjs\vendor\mitt\src\index.js:51:62
    at Array.map (<anonymous>)
    at Object.emit (D:\Robocze\bos-frontend-tests\node_modules\puppeteer-core\lib\cjs\vendor\mitt\src\index.js:51:43)
    at Page.emit (D:\Robocze\bos-frontend-tests\node_modules\puppeteer-core\lib\cjs\puppeteer\common\EventEmitter.js:72:22)
    at D:\Robocze\bos-frontend-tests\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Page.js:165:101
    at D:\Robocze\bos-frontend-tests\node_modules\puppeteer-core\lib\cjs\vendor\mitt\src\index.js:51:62
    at Array.map (<anonymous>)
    at Object.emit (D:\Robocze\bos-frontend-tests\node_modules\puppeteer-core\lib\cjs\vendor\mitt\src\index.js:51:43)
    at NetworkManager.emit (D:\Robocze\bos-frontend-tests\node_modules\puppeteer-core\lib\cjs\puppeteer\common\EventEmitter.js:72:22)

and the test summary:

 "spec" Reporter:
------------------------------------------------------------------
[chrome 86.0.4240.111 windows #0-0] Spec: d:\Robocze\bos-frontend-tests\healthcheck\healthcheck.gen.js
[chrome 86.0.4240.111 windows #0-0] Running: chrome (v86.0.4240.111) on windows
[chrome 86.0.4240.111 windows #0-0] Session ID: 99e20f03ca9b29678d70c8293aa176d2
[chrome 86.0.4240.111 windows #0-0]
[chrome 86.0.4240.111 windows #0-0] Health check tests generator
[chrome 86.0.4240.111 windows #0-0]    ✓ get all menus
[chrome 86.0.4240.111 windows #0-0]    ✓ perform healthcheck
[chrome 86.0.4240.111 windows #0-0]
[chrome 86.0.4240.111 windows #0-0] 2 passing (13m 38.2s)


Spec Files:      1 passed, 1 total (100% completed) in 00:13:47

So my question why failed expect doesn't fail the test (perform healthcheck test to be precise) and how can I fix it?

webdriver-io puppeteer javascript
1 Answers

As the error says, you should first await for response before asserting it.

I'd suggest using https://webdriver.io/docs/api/browser/mock.html in combination with https://webdriver.io/docs/api/expect-webdriverio.html#toberequestedwith

It's especially for the cases like this and even doesn't require devtools server.

Coming back to your code, it's JS basics to deal with events

const r = await new Promise(resolve => page.on('response', resolve)) expect(r.status()).not.toEqual(500)

P.S. Noticed lot's of interaction with Puppeteer, maybe your intention was using devtools automation protocol instead https://webdriver.io/docs/automationProtocols.html#devtools-protocol?

user_24759870
Related