Can I use the Youtube API to search for keywords/timestamps in the caption track?

Question

I want to create a chrome extension that lets you search any youtube videos' caption track for any instance where a desired string occurs. Basically CMD F for any video. type a word/phrase you're looking for and the extension would return a timestamp for every occurrence of that word/phrase

The more I look at the Youtube API, It doesn't look possible unless I'm the owner of the video. If so, any ideas on a way around this? It is possible to open the transcript (with timestamps) manually on the video and watch it play along with the video.

Just thought this sounded like a useful, challenging project to help me learn web development.

user_2179138 · Answer

Welcome Caleb. If you are unable to access the data via the API, then you are left with attempting to access it through your content script. Since none of the web page's JavaScript is exposed to content scripts (for good reason) you can only access/manipulate the DOM, and listen for events.

I will endeavor to give an explanation which hopefully will enlighten you to the general principles involved in attempting this kind of a task, while at the same time giving examples specific to the task you are attempting.

If you are just getting started with writing browser extensions, learn to get familiar with the browser's dev tools, and especially the inspector. In your current challenge for example, if you open the transcript panel, using your inspector you can hover over the panel, and the timestamped entries found there. If you click on one of the entries, it will highlight the corresponding HTML container in the inspector window. Now you are in the ballpark of the code of interest. From there you can begin to inspect the surrounding code, and get the tag names, classNames, and ids (if they exist) of the code of interest.

For example, when I inspect the transcript window, I can see the timestamped entries are all div containers, with the className:

Poking up the tree further, I see that the parent containing all of these entries is a proprietary tag:

The idea is to find the deepest level container which contains all the items you want to search. The deepest level because you want it to contain everything of interest, but little more. Then use JavaScript to access those DOM items. The point is to try to isolate the code of interest as much as possible when you're doing your eg getElementsByClassName() or getElementsByTagName() queries, so you don't risk inadvertently picking up unrelated nodes which may share the same class or tag names.

With that said, I don't know if getElementsByTagName() will find non-standard tags, but if you go one level higher, there is a div which contains solely the container, which has an id:

What I find very useful is to right-click on the outermost container of interest, and select "copy outerHTML". I then run it through an online beautifier, then paste it into my HTML editor for examination. For example, doing so gives me:


  
    
    
      
        00:01
      
      

        
          You may have noticed recently that YouTube
          has changed their layout a bit.
        
        
      
    

    
      
        00:06
      
      

        
          This is as of September 2017.
        
        
      
    

    ...

You can see that each

container contains descendents which contain separately, the text and the timestamp. So once you access the entry container, you can now target and access the distinctive containers containing the information you want, using DOM methods such as getElementsByClassName()

Thus, in your content script, to get a list of all the timestamped entry containers:

// (It seems odd they would use an id of "body" for this, but AFAICT,
// this is unique to the document.)
let transcriptContainer = document.getElementById("body");
let transcriptEntries = tsContainer.getElementsByClassName("cue-group");

Note that I used .cue_group for the class to query. I determined that by examining the use of all the classes within our isolated block to determine that it would be unique to those containers.

One thing to be mindful of, check and make sure the code block you found is not part of a document loaded into an iframe (you can use the inspector for this,) in which case you would have to use methods to access the iframe document. In this case however, the transcript code is part of the root document.

For a basic search:

// There are many ways you could optimize this, this is just a basic example.
function searchTranscript(query) {
  // Normalize the query string.
  query = query.trim().toLowerCase().replace(/ +/g, " ");

  let transcriptContainer = document.getElementById("body");
  let transcriptEntries = transcriptContainer.getElementsByClassName("cue-group");
  let tsMatches = [];

  for (let i = 0; i < transcriptEntries.length; i++) {
    let entry = transcriptEntries[i];
    let textContainer = entry.getElementsByClassName("cue")[0];
    let tsContainer = entry.getElementsByClassName("cue-group-start-offset")[0];

    // Normalize the text entry string.
    let text = textContainer.textContent.trim().toLowerCase().replace(/ +/g, " ");

    // Simple string search, you could get fancier here if you want.
    if (text.indexOf(query) > -1) {
      // If found, record the timestamp.
      tsMatches.push(tsContainer.textContent);
    }
  }

  return tsMatches;
}

Of course this is just my example, and there are many ways you could approach accessing the needed nodes.

Note that this may or may not be dependent on having the transcript window open. It is possible it exists in the DOM when it is closed, only it is hidden. You'll just have to do some poking around. In essence, this is what writing content scripts involve, is doing a lot of poking around. It is also possible that while the content may not be in the DOM on page load, that once the transcript panel has been opened, it will remain there after it has closed again. In such a case you could always programmatically open the panel, then close it (usually undetected by the user,) as I have done in the past for things like this. But that's a whole 'nuther lesson.

Caveat: Relying on methods this way can be fragile, as you are depending on the site not changing the code you are accessing. However, I have written many extensions, and I seldom have had problems in this regard. And sometimes it is just the only way available to get done what you want done.

Edit: The js search code has now been tested and works.

Can I use the Youtube API to search for keywords/timestamps in the caption track?

1 Answers