Delete a block of sentences where the start and end of block is clearly defined

Viewed 53

I am using Python 3.6.8

I have a text file like -

###
books 22 feb 2017 21 april 2018
books 22 feb 2017 21
22 feb 2017 21 april
feb 2017 21 april 2018
$$$
###
risk true stories people never thought they d dare share
risk true stories people never
true stories people never thought
stories people never thought they
people never thought they d
never thought they d dare
thought they d dare share
$$$
###
everyone hanging out without me mindy kaling non fiction
everyone hanging out without me
hanging out without me mindy
out without me mindy kaling
without me mindy kaling non
me mindy kaling non fiction
$$$

We generated it with -

for line_no, line in enumerate(books):
    tokens = line.split(" ")
    output = list(ngrams(tokens, 5))
    booksWithNGrams.append("###") #Adding start of block
    booksWithNGrams.append(books[line_no]) # Adding original line
    for x in output: # Adding n-grams
        booksWithNGrams.append(' '.join(x))
    booksWithNGrams.append("$$$") # Adding end of block

As you can see a sentence with its n-grams start with ### and end with $$$. So, the start and end of the block is clearly defined.

Given a sentence, I want to delete a block. For example - If I input 22 feb 2017 21 april, I want to delete -

###
books 22 feb 2017 21 april 2018
books 22 feb 2017 21
22 feb 2017 21 april
feb 2017 21 april 2018
$$$

How can I do that?

1 Answers

As you said the block are limited between # and $. We can treat the text as numerical sequences between those signs. Use finditer to point the blocks limits.

    import re

    starts =[]
    starts = [s.start() for s in re.finditer('###',text)]
    # [0, 105, 349]          

    ends = []          
    ends   = [e.end() for e in re.finditer(re.escape('$$$'),text)] #special char $
    # [104, 348, 558]

    blocks = []
    blocks = list(starts+ends)
    blocks.sort()

    #sequence of blocks
    nBlocks = [blocks[i:i+2] for i in range(0, len(blocks), 2)]
    #[[0, 104], [105, 348], [349, 558]]


    #find where the input text belongs       
    for i in text:       
        find   = '22 feb 2017 21 april'
        where  = text.index(find)
    # 10  

    #removing block elements    
    for n in range(len(nBlocks)):
        if where in range(nBlocks[n][0],nBlocks[n][1]): 
            for x in range(nBlocks[n][0],nBlocks[n][1]+1):
                             #text starts          #text ends
                 cleanText = text[0:nBlocks[n][0]]+text[nBlocks[n][1]+1::]


    print(cleanText)

    ###
    risk true stories people never thought they d dare share
    risk true stories people never
    true stories people never thought
    stories people never thought they
    people never thought they d
    never thought they d dare
    thought they d dare share
    $$$
    ###
    everyone hanging out without me mindy kaling non fiction
    everyone hanging out without me
    hanging out without me mindy
    out without me mindy kaling
    without me mindy kaling non
    me mindy kaling non fiction
    $$$
Related