I have developed a code that is responsible for reading the words of a txt file, in my case "elquijote.txt" to then use a dictionary {key: value} to show the words that appear and their occurrences.
For example for a file "test1.txt" with the following words:
hello hello hello good bye bye
The output of my program is:
hello 3
good 1
bye 2
Another of the options that the program has, is that it shows those words that appear a greater number of times than a number introduced by us through an argument.
If in the shell, we put the following command "python readingwords.py text.txt 2" , will show those words contained in the file "test1.txt" that appear more times than the number that we have entered, in this case 2
Output:
hello 3
Now we can introduce a third argument of common words such as determinates conjunctions, which, being so generic, we do not want to be shown or introduced in our dictionary.
My code works correctly, the problem is that using huge files, such as "elquijote.txt" takes a long time to complete the process.
I have been thinking and it is because of the use I make of my auxiliary lists for the elimination of words.
I have thought as a solution not to introduce in my lists those words that appear in txt file that is entered by argument, which contains the words to discard.
Here is my code:
def contar(aux):
counts = {}
for palabra in aux:
palabra = palabra.lower()
if palabra not in counts:
counts[palabra] = 0
counts[palabra] += 1
return counts
def main():
characters = '!?¿-.:;-,><=*»¡'
aux = []
counts = {}
with open(sys.argv[1],'r') as f:
aux = ''.join(c for c in f.read() if c not in characters)
aux = aux.split()
if (len(sys.argv)>3):
with open(sys.argv[3], 'r') as f:
remove = "".join(c for c in f.read())
remove = remove.split()
#Borrar del archivo
for word in aux:
if word in remove:
aux.remove(word)
counts = contar(aux)
for word, count in counts.items():
if count > int(sys.argv[2]):
print word, count
if __name__ == '__main__':
main()
Contar function introduces the words in the dictionary.
And main function introduces in an "aux" list those words that do not contain symbolic characters and then deletes from the same list those "forbidden" words loaded from another .txt file.
I think the correct solution would be to discard the forbidden words where I discard symbols that are not accepted, but after trying several ways I have not managed to do it correctly.
Here you can test my code online: https://repl.it/Nf3S/54 Thanks.