Select n-grams which contain at least one number

user_7788837 0
Viewed 46

I have a list of n-grams 

('allo', 'stesso', 'modo', 'dell’italia,', 'che')
('stesso', 'modo', 'dell’italia,', 'che', 'sta')
('modo', 'dell’italia,', 'che', 'sta', 'già')
('dell’italia,', 'che', 'sta', 'già', 'pensando')
('che', 'sta', 'già', 'pensando', 'alla')
('sta', 'già', 'pensando', 'alla', 'riapertura')
('soli', '2.900,', 'contando', 'un', 'crollo')
('2.900,', 'contando', 'un', 'crollo', 'del')
('contando', 'un', 'crollo', 'del', '99.9%')
('un', 'crollo', 'del', '99.9%', 'rispetto')
('che', 'prevede', '12,5', 'miliardi', 'di')
('prevede', '12,5', 'miliardi', 'di', 'dollari')
('12,5', 'miliardi', 'di', 'dollari', 'per')
...

created by

from nltk import ngrams

n = 5
list_ngrams=[]

for i in my_list:
    grams = ngrams(i.split(), n)

    for gram in grams:
        print(gram)
        list_ngrams.append(gram)

I would like to select only the n-grams which contain at least one number, e.g. 

('soli', '2.900,', 'contando', 'un', 'crollo')
('2.900,', 'contando', 'un', 'crollo', 'del')
('contando', 'un', 'crollo', 'del', '99.9%')
('un', 'crollo', 'del', '99.9%', 'rispetto')
('che', 'prevede', '12,5', 'miliardi', 'di')
('prevede', '12,5', 'miliardi', 'di', 'dollari')
('12,5', 'miliardi', 'di', 'dollari', 'per')

Could you please help me to select it?

nltk python
2 Answers

You can do this:

l = [('allo', 'stesso', 'modo', 'dell’italia,', 'che'),
     ('stesso', 'modo', 'dell’italia,', 'che', 'sta'),
     ('modo', 'dell’italia,', 'che', 'sta', 'già'),
     ('dell’italia,', 'che', 'sta', 'già', 'pensando'),
     ('che', 'sta', 'già', 'pensando', 'alla'),
     ('sta', 'già', 'pensando', 'alla', 'riapertura'),
     ('soli', '2.900,', 'contando', 'un', 'crollo'),
     ('2.900,', 'contando', 'un', 'crollo', 'del'),
     ('contando', 'un', 'crollo', 'del', '99.9%'),
     ('un', 'crollo', 'del', '99.9%', 'rispetto'),
     ('che', 'prevede', '12,5', 'miliardi', 'di'),
     ('prevede', '12,5', 'miliardi', 'di', 'dollari'),
     ('12,5', 'miliardi', 'di', 'dollari', 'per')]

l2 = [i for i in l if any(any(w.isdigit() for w in s) for s in i)]

print(l2)

Output:

[('soli', '2.900,', 'contando', 'un', 'crollo'), ('2.900,', 'contando', 'un', 'crollo', 'del'), ('contando', 'un', 'crollo', 'del', '99.9%'), ('un', 'crollo', 'del', '99.9%', 'rispetto'), ('che', 'prevede', '12,5', 'miliardi', 'di'), ('prevede', '12,5', 'miliardi', 'di', 'dollari'), ('12,5', 'miliardi', 'di', 'dollari', 'per')]
user_135524700

This will select all tuples that contain any element with some characters in the range 0 - 9. Modify the regex if it needs to be more restrictive:

import re

def filter_ngrams(ngrams):
    return [ngram for ngram in ngrams
            if any(re.match('\d', word) for word in ngram)]

print(filter_ngrams(ngrams))

Output:

[('soli', '2.900,', 'contando', 'un', 'crollo'), ('2.900,', 'contando', 'un', 'crollo', 'del'), ('contando', 'un', 'crollo', 'del', '99.9%'), ('un', 'crollo', 'del', '99.9%', 'rispetto'), ('che', 'prevede', '12,5', 'miliardi', 'di'), ('prevede', '12,5', 'miliardi', 'di', 'dollari'), ('12,5', 'miliardi', 'di', 'dollari', 'per')]
user_92487930
Related