NodeJS String Replace performance - newlines inside of many sets of xml tags

user_1910490 0
Viewed 146

I have a very large string that I'm running a replace loop on. The length of the string is 15967025 characters (yes its large). Loop taking 4-5 seconds for each iteration on large string.

I'm looping through the string with a while loop and replacing it with a pretty complex regex - the end goal to replace all newline characters inside of matching text:p tags with <text:line-break/>:

const undefinds = /(undefined)/g;
const nulls = /(<[A-z:\s-="\d]*>)(null)(<\/[A-z:\s-="\d]*>)/gi;
const nullsReplace = '$1None$3';

const regexToReplace = /(<text:p ....>)(...\n...)(</text:p>)/g;
const replaceWith = '$1<text:line-break/>$2';

export function cleanOutput(output) {

    while (output.match(regexToReplace)) {
        output = output.replace(regexToReplace, replaceWith);
    }

    return output;
}

The problem I'm seeing is that with every time this loop runs it takes about 4-5 seconds. Is this normal with this size of string? Is there a better way to do this?

** Edit 1: **

I'm adding a more detailed example for the patterns to replace. I don't think I can use the /g flag since I'm doing multiple replacements within some matching tags.

Here's the full example of what I'm doing, the goal is replace newline characters in the <text:p> elements with <text:line-break />. But I want to preserve the wrapping text which is xml.

const newLines = /(<text:p[^/>]*?>[\s\S]*?)(\r\n|\r|\n)+?([\s\S]*?<\/text:p>)/g;
const newLineReplace = '$1<text:line-break/>$3';

export function cleanOutput(output) {
    output = output.replace(undefinds, 'None')
        .replace(nulls, nullsReplace);

    while (output.match(newLines)) {
        output = output.replace(newLines, newLineReplace);
    }

    return output;
}

Edit 2:

I added the missing undefineds/nulls replace. Sorry for not including that. The slowness definitely occurs in the while loop though - it does take about 4-5 seconds between each iteration.

Edit 3:

Here's a sample test case to try:

<text:p>some text\n\nother text\n</text:p>

expected output:

<text:p>some text<text:line-break/><text:line-break/>other text<text:line-break/></text:p>
node.js regex string
1 Answers

From the regex point of view, it is more maintainable to match all substrings between two strings, and later replace all occurrences of some other pattern inside the matches only rather than trying to figure out a universal pattern that will find all occurrences of a pattern between two strings.

So, you need to match all substrings between <text:p> and </text:p> and replace all occurrences of line breaks (let's define a line break as a CR, LF or CRLF char sequences):

let text = `<text:p>some text\n\nother text\n</text:p>`;
const regex = /(<text:p(?:\s[^>]*)?>)([^<]*(?:<(?!\/text:p>)[^<]*)*)(<\/text:p>)/g;
text = text.replace(regex, (_, open, content, close) =>
  `${open}${content.replace(/\r\n?|\n/g, '<text:line-break/>')}${close}`);
console.log(text);

Note (<text:p(?:\s[^>]*)?>)([^<]*(?:<(?!\/text:p>)[^<]*)*)(<\/text:p>) (demo) is an unrolled (=more efficient) (<text:p(?:\s[^>]*)?>)([\w\W]*?)(<\/text:p>) pattern (demo), that simply matches <text:p...> tag and then any text up to the first occurrence of </text:p>.

The \r\n?|\n pattern matches a CRLF or CR pattern with \r\n? and an LF ending with \n.

user_38329700
Related