I'm trying to validate a network request (that logs a metric) is fired, but the metric isn't tied to a specific user action (a metric fires after the video is left playing for 2 seconds).
From the docs on wait, all of it seems to be based on a model where network requests are fired on some user action or can be sequenced in some way, which in turn generalizes to cases where there's predictability to the order (in the sense that it is free of dependencies on another action) and timing of the network request (that it isn't randomly ordered compared to other network requests that fire to the same endpoint).
So for the below example, this works because this (1) has predictable order (it always follows a click action), and (2) the timing follows the user's click every time (nowhere will there be a second request to the same endpoint that fires in between the click and the network request).
beforeEach(() => {
// omitting setup code
cy.server();
cy.route({ method: 'POST', url: VISUAL_MODE_ENDPOINT }).as('visual-mode-toggle');
});
it('should save the visual mode preference and change the page to dark mode', () => {
cy.get('.visual-mode-toggle').click();
cy.wait('@visual-mode-toggle')
.get(xhr => {/* omitting asserting the request is fired and returns a 200 */});
// UI tests omitted
});
However, for my use case, I'd like to be able to assert that certain metrics are fired. The application I'm building logs many metrics via network calls, but I'd like to test just one specific metric. (Note: all metrics are logged to the same endpooint.) So I tried something like this:
beforeEach(() => {
cy.server();
cy.route({ method: 'POST', url: METRICS_ENDPOINT }).as('metrics');
});
it('should fire a metric when the video is viewed for more than 2 seconds', () => {
cy.wait('@metrics') // this is problematic
.get<Cypress.WaitXHR[]>('@metrics.all')
.then(xhrs => {
const videoPlayInViewportRequests = Array
.from(xhrs)
.filter(isVideoPlayInViewport);
videoPlayInViewportRequests.forEach(xhr => expect(getJSONPayload(xhr)).to.include('viewers'));
expect(videoPlayInViewportRequests.length).to.not.equal(0);
});
});
It turns out that this is flaky (it passes sometimes, but fails more frequently), not because of Cypress per se, but because of how metrics are fired in the application, for two reasons:
Unpredictable order. Apart from this "plays for at least 2 second" metric, there are other metrics that are emitted in between, like time it takes for the player to load the first frame, and other page load metrics. There is no logic guaranteeing the order of these metrics, which makes it impossible to determine the order in which they are fired.
Unpredictable timing. We also don't know when exactly it appears. Sometimes, the metric happens to be fired within the retry limit, but sometimes, it doesn't.
So we get the following scenarios (assume the retry limit is 5 seconds, and all metrics fire via the same endpoint):
- 2 second metric (3 seconds) -> other metrics (7 seconds) (succeeds, since it's the first network request we waited for with
cy.wait) - 2 second metric (6 seconds) -> other metrics (7 seconds) (fails, because it's fired after the retry limit)
- Other metrics (2 seconds) -> 2 second metric (3 seconds) (fails, because we only called
cy.waitonce) - Other metrics (6 seconds) -> 2 second metric (7 seconds) (fails, outside retry limit)
If we wrote cy.wait('@metrics').wait('@metrics'), it could fix scenario 3 but there's no guarantee because there could be more metrics that are fired in between.
So my questions are:
How do we implement wait in this scenario? I was thinking of something like looping until we find what we're looking for, but it seems very un-Cypress like:
let needToWait = true; const startTime = Date.now(); do { cy.wait('@metrics') .get<Cypress.WaitXHR[]>('@metrics.all') .then(xhrs => { const results = Array.from(xhrs).filter(isVideoPlayInViewport); const hasVideoPlayInViewport = results.length !== 0; const timeExceededLimit = (Date.now() - startTime) > 10000; needToWait = !hasVideoPlayInViewport && !timeExceededLimit; }); } while (needToWait);I also thought about a hard wait, but the Cypress guide literally says waiting for an arbitrary amount of time is an anti-pattern.
cy.wait(7000); // this is pretty much the same thing as the above I guess lol, but the above can short circuit the loop once it's found, this doesn't // verify metric is presentDoes testing metrics fall under Cypress' use cases? Are there better strategies to go about validating metrics as part of UI tests?
I have read the documentation on the wait function as well as best practices regarding unnecessary waiting but am drawing a blank.