Flaky cypress test with Svelte: Button is sometimes clicked, sometimes not

user_7767808 0
Viewed 516

I am testing my SvelteKit site with Cypress. I sometimes experience flaky tests, similar to what has been described here: https://www.cypress.io/blog/2019/01/22/when-can-the-test-click/. In short, Cypress sometimes finds and clicks a button before the event listeners are attached - as a result, the click goes nowhere. The proposed solution is to simply re-try clicking until the appropriate listeners have been attached. That works in my case as well. However, though I do understand why this can be an issue in the example given in the blog post (it's a large calendar modal), I find it hard to justify that this issue arises when using a simple Svelte button.

Here is a simple example of a button that reveals some content when clicked:

<script>
  let hide = true;
</script>

<button
  on:click={() => {
    console.log('clicked');
    hide = false;
  }}>
  Show
</button>

<span class:hide>Content</span>

<style>
  .hide {
    visibility: hidden;
  }
</style>

The corresponding test sometimes passes, sometimes fails:

it('reveals content on click', () => {
  cy.contains('Show').click();
  cy.contains('Content').should('be.visible');
});

Again, I am aware this can be fixed by re-trying to click the button. And if this is what it takes to make Cypress work with Svelte/SvelteKit, then that's fine with me. But I am wondering: Why would this even be an issue?

Minimal reproduction repo: https://github.com/sophiamersmann/test-svelte-kit-cypress

sveltekit svelte-3 svelte cypress
2 Answers

I think the problem lies with Vite, which uses ES modules to load the page and it's components.

Adding an intercept before the cy.visit() seems to give consistent results. (Note the URL to intercept may vary, you can get it from the last entry in devtools Network).

beforeEach(() => {
  cy.intercept('index.svelte?svelte&type=style&lang.css').as('svelte')
  cy.visit('/');
  cy.wait('@svelte')
});

Using cypress-grep to burn-test

npx cypress run --env burn=100

With intercept

enter image description here

Without intercept

enter image description here

Why is it not hydration?

If you create an equivalent Svelte app with hydratable set to true, it will pass the burn test - IMO because it uses rollup instead of vite to deliver the app to the browser.

enter image description here

user_169977070

SvelteKit will by default do server side rendering (SSR), which means the complete HTML is sent to the browser, including the button. That HTML then needs to be hydrated afterwards to become interactive. This means that some code runs so that Svelte connects to the HTML that already exists. Cypress is likely "too fast" here and clicks the button before that hydration step is completed, therefore nothing happens.

It does not happen with pure Svelte because there's no SSR involved. There's a blank index.html page initially which is completely filled by Svelte's JavaScript inside the browser, so the moment the button is visible, the event listener and everything else is already initialized by Svelte.

Comparison by steps:

SvelteKit with SSR:

  1. Go to page X
  2. Page X is rendered on the server
  3. Page X is sent to the browser, with the complete HTML
  4. Svelte hydrates the HTML (Race condition with Cypress' click test)
  5. Completed

Pure Svelte or SvelteKit without SSR:

  1. Go to page X
  2. Blank page is sent to the browser
  3. Svelte constructs and initializes the HTML inside the browser (No race condition with Cypress' click test)
  4. Completed
user_101961880
Related