Correct method to save and replay a sequence of events

we're building a mobile software with React Native where a user can start/build a session of some actions that should be recorded and replayed later. Those actions include playing some videos, seeking a played video, pausing and playing again, showing custom messages for a specific period of time... etc.

The best model we made so far is this:

 {
  "episode": {
    "name": "The New MacBook Pro",
    "events": {
      "1589168577": {
        "source": "stream",
        "action": "play"
        "link": "https://example.com/s1/ep1.mp3"
      },
      "1589168665-1589168223": {
        "source": "video",
        "link": "https://www.youtube.com/watch?v=yoQOmOlWLZo",
        "action": "play"
      },
      "1589169026-1589169827": {
        "source": "video",
        "link": "https://www.youtube.com/watch?v=yoQOmOlWLZo",
        "action": "pause"
      },
      "1589169381-1589169237": {
        "source": "video",
        "link": "https://www.youtube.com/watch?v=yoQOmOlWLZo",
        "action": "play"
      },
      "1589169420-1589169124": {
        "source": "video",
        "link": "https://www.youtube.com/watch?v=yoQOmOlWLZo",
        "action": "seek",
        "seek": "399"
      },
      "1589169540-1589169235": {
        "source": "video",
        "link": "https://www.youtube.com/watch?v=yoQOmOlWLZo",
        "action": "stop"
      },
      "1589169523-1589169235": {
        "source": "message",
        "text": "Keep following up for the discound code",
        "action": "show"
      },
      "1589169542": {
        "source": "stream",
        "action": "stop"
      }
    }
  }
}

On Firebase, we recorde the actions we need ro be replayed (as above), when a user plays the session, all the events must be played in the exact sequence. The events node keys hold the timestamps of starting and ending of the event, separated by a dash. (PS. timestamps above are wrong, and put there just as examples);

The app starts playing the session and compares the current timestamp with the saved ones in events node, if they are equal, the action is played.

Now, the problem is when a user replaying a session, seeked to specific location on the timeline where an action is already being played, we had issues with this, some events are being missed.

I am sure there are better ways to do this but I can't come up with a better way to save the data in a correct way that helps us replay those actions correctly.

A good example of what we're doing is a video being played on Youtube where a user seeks to a spefic time that an AD should appear, and a subscription button appears at another location, subtitles..... etc.

I appreciate your kind advice.