Skip a spec file on fail and still report it as failed on result

user_6029971 0
Viewed 54

I am aiming to skip a spec files once an it() inside it fails. And I would still like to see the error reported so I know where to focus on.

I have two test scripts that is inside the same folder first.cy.js and second.cy.js:

describe('test body', () => {
    afterEach(() => {
        if (cy.state('test').state == 'failed') {
          Cypress.runner.stop()
        }
    })

    it('should pass first test', function() {
        expect(true).to.eq(true)
    })

    it('should pass second test', function() {
        expect(true).to.eq(true)
    })

    it('should fail third test and skip this script', function() {
        expect(true).to.eq(false)
    })

    it('should skip fourth test', function() {
        expect(true).to.eq(true)
    })
})

My problem is on result reporting, it does not report the failing part: enter image description here

Is there anything I am missing here?

EDIT: I also need to mention that we have retries set on cypress config:

retries: {
    runMode: 2
},

Update: Was able to figure out how I will skip the spec file and still record errors with the help of Alapan das:

describe('skipping with retrying and recording', () => {
    let errorCount = 0

    afterEach(() => {
        if (cy.state('test').state == 'failed') {
            errorCount += 1
            if (errorCount == 3) {
                Cypress.runner.stop()

                cy.state('test').state = 'failed'
            }
        }
    })
})
cypress
2 Answers

The afterEach() is too early to call Cypress.runner.stop(), but it seems to work with test:after:run.

describe('test body', () => {

  Cypress.on('test:after:run', (result) => {
    if (result.currentRetry === result.retries && result.state === 'failed') {
      Cypress.runner.stop()
    }
  })

  it('should pass first test', function() {
    expect(true).to.eq(true)
  })

  it('should pass second test', function() {
    expect(true).to.eq(true)
  })

  it('should fail third test and skip this script', function() {
    expect(true).to.eq(false)
  })

  it('should skip fourth test', function() {
    expect(true).to.eq(true)
  })
})

Running two identical tests in a single run

Before adding the Cypress.on('test:after:run')

enter image description here

After adding the Cypress.on('test:after:run')

enter image description here

user_169977070

I ran your test exactly in my local with cypress version 10.0.3, I am getting the failure count. This is without test re-tries.

describe('test body', () => {
    afterEach(() => {
        if (cy.state('test').state == 'failed') {
            Cypress.runner.stop()
        }
    })

    it('should pass first test', function () {
        expect(true).to.eq(true)
    })

    it('should pass second test', function () {
        expect(true).to.eq(true)
    })

    it('should fail third test and skip this script', function () {
        expect(true).to.eq(false)
    })

    it('should skip fourth test', function () {
        expect(true).to.eq(true)
    })
})

Terminal Screenshot:

Terminal Screenshot

In case of test retries, one way would be to manually set the test state to failed like this:

afterEach(() => {
  if (cy.state('test').state == 'failed') {
    Cypress.runner.stop()
    cy.state('test').state = 'failed'
  }
})
user_45712710
Related