how to only replace \n which has some char after that

(?<=\S)(?<!\bAbstract)\n(?=\S)

You can try this.See demo.

https://regex101.com/r/crj3aD/1

Python script:

inp = "Balanced Quantum Classical Evolutionary Algorithm(BQCEA)\n\nMuhammad Shahid, Hasan Mujtaba, Muhammad Asim, Omer Beg\n\nAbstract\nWith advancement in Quantum computing, classical algorithms are adapted and integrated\nwith Quantum properties such as qubit representation and entanglement', ' Although these\nproperties perform better however pre-mature convergence is the main issue in Quantum\nEvolutionary Algorithms(QEA) because QEA uses only the best individual to update quan-\ntum population', ' In this paper, we introduced a new way to update the quantum population\nof QEA to avoid premature convergence"

output = re.sub(r'(?<=\S)(?<!\bAbstract)\n(?=\S)', ' ', inp)
print(output)

There are more conditions.

(?<=\S)(?<!\bAbstract)(?:\n|\\n)(?=\S)

Try this for your another condition.

https://regex101.com/r/crj3aD/2

lines = tokenize.sent_tokenize(txt_str)

s = lines.replace('\n', ' ')

print(s)

This should do this for you:

import re
pattern = re.compile(r"^(.*\(BQCEA\))(.*Beg)(Abstract)(With.*)", re.DOTALL)

try:
    with open('sample.txt', 'r') as f:
        line = f.read()
        # remove some unwanted characters
        r = line.replace('\\n', "").replace("'", "").replace("\n", "")
        print(r)
        for match in re.finditer(pattern, r):
            print(match.group(1))
            print('\n')
            print(match.group(2))
            print('\n')
            print(match.group(3))
            print(match.group(4))
except Exception as er:
    print(er)

Output:

Balanced Quantum Classical Evolutionary Algorithm(BQCEA)


Muhammad Shahid, Hasan Mujtaba,Muhammad Asim, Omer Beg


Abstract
With advancement in Quantum computing, classicalalgorithms are adapted and integratedwith Quantum properties such as qubit representationand entanglement,  Although theseproperties perform better however pre-matureconvergence is the main issue in QuantumEvolutionary Algorithms(QEA) because QEA uses onlythe best individual to update quan-tum population,  In this paper, we introduced a newway to update the quantum populationof QEA to avoid premature convergence

Sample:

'Balanced Quantum Classical Evolutionary Algorithm(BQCEA)\n\nMuhammad Shahid, Hasan Mujtaba,
Muhammad Asim, Omer Beg\n\nAbstract\nWith advancement in Quantum computing, classical
algorithms are adapted and integrated\nwith Quantum properties such as qubit representation
and entanglement', ' Although these\nproperties perform better however pre-mature
convergence is the main issue in Quantum\nEvolutionary Algorithms(QEA) because QEA uses only
the best individual to update quan-\ntum population', ' In this paper, we introduced a new
way to update the quantum population\nof QEA to avoid premature convergence'

To answer this problem, each line has to be considered in conjunction with the line that follows it. The rule here is to remove newlines if all of the following apply:

• it is not the last line of the file
• the line contains at least 2 words
• the next line contains at least 1 word

The problem is conveniently solved by using a generator function which will yield pairs of successive lines at a time. This separates cleanly the logic for iterating over the file from the logic for deciding when to remove newlines.

#!/usr/bin/env python

def num_words_line(line):
    return len(line.split())

def iter_lines(input_file):
    """
    yields pairs of adjacent lines
    """
    with open(input_file) as f:
        previous = next(f)
        for line in f:
            yield (previous, line)
            previous = line
        yield (line, None)


def fix_newlines(input_file, output_file):
    with open(output_file, "w") as fout:
        for line, next_line in iter_lines(input_file):
            if (next_line != None and
                num_words_line(line) > 1 and
                num_words_line(next_line) > 0):
                line = line.replace("\n", " ")
            fout.write(line)


if __name__ == '__main__':
    fix_newlines("input.txt", "output.txt")

Gives:

Balanced Quantum Classical Evolutionary Algorithm(BQCEA)

Muhammad Shahid, Hasan Mujtaba, Muhammad Asim, Omer Beg

Abstract
With advancement in Quantum computing, classical algorithms are adapted and integrated with Quantum properties such as qubit representation and entanglement. Although these properties perform better however pre-mature convergence is the main issue in Quantum Evolutionary Algorithms(QEA) because QEA uses only the best individual to update quan- tum population. In this paper, we introduced a new way to update the quantum population of QEA to avoid premature convergence

Note, the number of words in each line is calculated twice. For slightly greater efficiency, this could be fixed, at the expense of slightly more code, by changing iter_lines to call num_words_line on each line as it is read in, and to yield the lengths of each line in the pair as well as the lines themselves. But then there would be less clean separation of the logic between iter_lines and fix_newlines.

A slightly different version could be used for manipulating strings in memory in place of reading and writing files:

#!/usr/bin/env python

def num_words_line(line):
    return len(line.split())


def iter_lines(input_string):
    """
    yields pairs of adjacent lines
    """
    iterator = iter(input_string.strip().split("\n"))
    previous = next(iterator)
    for line in iterator:
        yield (previous, line)
        previous = line
    yield (line, None)


def fix_newlines(input_string, from_file=True):
    output = ''
    for line, next_line in iter_lines(input_string):
        newline = not (next_line != None and
                       num_words_line(line) > 1 and
                       num_words_line(next_line) > 0)
        output += line
        if newline:
            output += "\n"
        else:
            output += " "
    return output

if __name__ == '__main__':

    input_text = ['Balanced Quantum Classical Evolutionary Algorithm(BQCEA)\n\nMuhammad Shahid, Hasan Mujtaba, Muhammad Asim, Omer Beg\n\nAbstract\nWith advancement in Quantum computing, classical algorithms are adapted and integrated\nwith Quantum properties such as qubit representation and entanglement', ' Although these\nproperties perform better however pre-mature convergence is the main issue in Quantum\nEvolutionary Algorithms(QEA) because QEA uses only the best individual to update quan-\ntum population', ' In this paper, we introduced a new way to update the quantum population\nof QEA to avoid premature convergence',]
    str = ' '.join(input_text)

    print(fix_newlines(str))

I will not even try to use a single regex here. The requirement is:

• if the newline if at end of file, keep it
• else if a newline is followed by another newline, keep it
• else if the line is short, keep the newline (short is probably less than 60 or 70)
• else if the newline is preceded by a dash (-) remove both the dash and the newline
• else replace the newline with a space

This should be rather robust, but I cannot imagine to fulfill that with regexes. Assuming input is:

orig = ['Balanced Quantum Classical Evolutionary Algorithm(BQCEA)\n\nMuhammad Shahid, Hasan Mujtaba, Muhammad Asim, Omer Beg\n\nAbstract\nWith advancement in Quantum computing, classical algorithms are adapted and integrated\nwith Quantum properties such as qubit representation and entanglement',
        ' Although these\nproperties perform better however pre-mature convergence is the main issue in Quantum\nEvolutionary Algorithms(QEA) because QEA uses only the best individual to update quan-\ntum population',
        ' In this paper, we introduced a new way to update the quantum population\nof QEA to avoid premature convergence']

I would use:

pre = '.'.join(orig).split('\n')
res = []
old = None
for line in pre:
    if old is None:
        old = line
        continue
    if len(line) == 0:
        res.extend([old, '\n'])
    elif len(old) < 70:
        res.extend([old, '\n'])
    elif old[-1] == '-':
        res.append(old[:-1])
    else:
        res.extend([old, ' '])
    old = line


text = ''.join(res)

It gives for print(text)

Balanced Quantum Classical Evolutionary Algorithm(BQCEA)

Muhammad Shahid, Hasan Mujtaba, Muhammad Asim, Omer Beg

Abstract
With advancement in Quantum computing, classical algorithms are adapted and integrated with Quantum properties such as qubit representation and entanglement. Although these properties perform better however pre-mature convergence is the main issue in Quantum Evolutionary Algorithms(QEA) because QEA uses only the best individual to update quantum population. In this paper, we introduced a new way to update the quantum population