Search for a string in multiple .csv files from a multiple zipped folders

user_17557540 0
Viewed 193

I'm trying to execute a script that will unzip all files in a zipped folder which has multiple txts and .csv files, search only the .csv files for a string, if it contains that string, copy the entire zipped folder to a new folder, if it doesn't, move on to the next zipped folder. I have several scripts that do part of this but can't piece them together. I am a beginner in python so this script looks like it gets complicated.

This script prints the files in the zipped folder, my next step is to search within the .csv files it contains for the string PROGRAM but I don't know how to code it, I'm thinking it goes at the end of this code since it looks like it's running through a loop.

import os
import pandas as pd
import zipfile

curDir = os.getcwd()
zf = zipfile.ZipFile(curDir + '\namedfile.zip')
text_files = zf.infolist()
list_ = []

print ("Uncompressing and reading data... ")

for text_file in text_files:
    print(text_file.filename)

I wrote this script separately, searches for the string PROGRAM in a folder that contains .csv files

import os
from pathlib import Path

#Searches the .csv files within the "AllCSVFiles"
#folder for the string "GBSD"

search_path = "./AllCSVFiles"
file_type = ".csv"
search_str = "PROGRAM"

if not (search_path.endswith("/") or search_path.endswith("\\") ): 
        search_path = search_path + "/"
                                                          
if not os.path.exists(search_path):
        search_path ="."


for fname in os.listdir(path=search_path):
   if fname.endswith(file_type):
        fo = open(search_path + fname)
        line = fo.readline()
        line_no = 1
        while line != '' :
                index = line.find(search_str)
                if ( index != -1) :    
                    print(fname, "[", line_no, ",", index, "] ", sep="")

                line = fo.readline()  
                line_no += 1 
        fo.close()

Is there an easier way to work this code?

script pandas csv search python
1 Answers

I think the first thing is to make sure you know the structure of the solution.

Reading your description, I'd say it's this:

# Create empty list, for marked zip file

# Iterate over zip files
    # Unzip
    # Iterate over files
        # If file ends in .csv
            # If file contains SEARCH_STR
                # Mark this zip file to be copied
                # Stop searching this zip file

# Iterate marked zip files
    # Copy zip file to DEST_DIR

If that is the structure, is this enough to help you see where to put your code?

After that, you can clean up your search for search_str in file quite a bit:

with open(search_path + fname) as csv_file:
    line_no = 0
    for line in csv_file:
        line_no += 1
        if search_str in line:
            search_index = line.index(search_str)
            print(f'{fname}[{line_no},{search_index}]')
            # Mark the zip file this csv_file is in
            # figure out how to stop searching this zip file
  • for line in csv_file: text files opened in Python have a built-in mechanism for iterating over lines
  • if search_str in line: if you don't need to know the line exactly where search_str is, simply test for membership, is search_str in the string line?
user_2468010
Related