Nodejs: How to check for particular string repeatedly within raw file data obtained from API request?

I have an application developing using Nodejs. This application is making a request to GitLab API and obtaining the raw file data from it.

I would like to read the particular string which is present after another string and get all similar data from it. I am just a bit confused on this part and unable to proceed further can someone please explain to me how to achieve this?

Following is the sample file data: I would like to read all the numbers if present after the keyword Scenario: i.e in this case I would like to get A001-D002 & K002-M002. These numbers can be anything random and can appear anywhere within the file content. I would like to read them and store them within an array for that particular file.

FileName: File Data
  Background:
    This is some random background

  Scenario: A001-D002 My first scenario
    Given I am sitting on a plane
    When I am offered drinks

  Scenario: K002-M002 My second scenario
    Given when I book the uber taxi
    When I get the notifications

I am not understanding how to iterate over the file content and read every word and match and accordingly obtain the ids.

Following is the code that I have which makes the request to GitLab and obtains the raw file content: ./index.js:

const   express     =   require('express');
const   http        =   require("http");
const   bodyParser  =   require('body-parser');
const   app         =   express();
const   port        =   process.env.PORT || 9000;
const   gitlabDump  =   require("./controller/GitLabDump");

app.use(bodyParser.json()); 
app.use(bodyParser.urlencoded({ extended: true }));

//Make NodeJS to Listen to a particular Port in Localhost
app.listen(port, function(){

        gitlabDump.gitlabDump(type, function(data){
            console.log("Completed Execution for GitLab")
            process.exit();
        })
    
}

My ./controller/GitLabDump.js:

const request = require('request');
const https = require('https');
const axios = require('axios');

exports.gitlabDump = function(callback){
    var gitlabAPI = "https://gitlab.com/api/v4/projects/<project_id>/repository/files/tree/<subfolders>/<fileName>/raw?ref=master&private_token=<privateToken>";

    //Make the request to the each file and read its raw contents
    request(gitlabAPI, function(error, response, body) {
        const featureFileData = JSON.parse(JSON.stringify(body)).toString();

        console.log(featureFileData)
        
        for(const match of featureFileData.matchAll("Scenario:")){
            console.log(match);
        }

        callback("Completed");
    })

    
}

I am able to print the file contents. Can someone please explain me how can I iterate over the raw file contents and get all the required ids?

I suggest you to use a method by analyzing each part of your string by iterating over each lines (i assume that your string is compose like in your exemple). It is easier to understand and coding it than using a regex.

The exemple below represent your request callback function. I split the code in 3 logics :

search the filename
search the line we are interesting with ("Scenario" word)
extract the ID by filter function

You can after that, easily change you ID filter (txt.substr(0, txt.indexOf(' ')) to use a more proper expression to extract your sentence.

The result is sent to a callback function with as first argument the filename, and as second all ids. Like you did in your exemple.

((callback) => {

    const featureFileData = `FileName: File Data
      Background:
        This is some random background

      Scenario: A001-D002 My first scenario
        Given I am sitting on a plane
        When I am offered drinks

      Scenario: K002-M002 My second scenario
        Given when I book the uber taxi
        When I get the notifications`;

    // find "filename"
    const filenames = featureFileData.split('\n')
        .filter(line => line.trim().substr(0,8) === 'FileName')
        .map((raw) => {
            if(!raw) return 'unknown';
            const sentences = raw.trim().split(':');
            if(sentences[1] && sentences[1].length) {
                return sentences[1].trim();
            }
        });

    // filter the "Scenario" lines
    const scenarioLines = featureFileData.split('\n')
        .map((line) => {
            if(line.trim().substr(0,8) === 'Scenario') {
                const sentences = line.trim().split(':');
                if(sentences[1] && sentences[1].length) {
                    return sentences[1].trim();
                }
            }
            return false;
        })
        .filter(r => r !== false);

    // search ids
    const ids = scenarioLines.map(txt => txt.substr(0, txt.indexOf(' ')));

    callback(filenames[0], ids);

})(console.log)

Nodejs: How to check for particular string repeatedly within raw file data obtained from API request?

1 Answers