Read a csv file with its name changing daily

Viewed 872

I am trying to read a csv file in Pandas. I am automating the script to daily read the data from csv and execute. Daily a new csv is added in the folder where i have the csv files. Newly added csv file have the same name format except the date part in it changes daily and month part change monthly. Year would change yearly. How can i automate to read the csv files daily with its name changing continuously?

Example: If the name for yesterday`s file is:

Name_29Mar2020_data_by_company.csv

Tomorrow`s file will be:

Name_30Mar2020_data_by_company.csv
2 Answers

You can use the datetime module:

import pandas as pd
from datetime import datetime

fname = datetime.today().strftime('Name_%d%b%Y_data_by_company.csv')
df = pd.read_csv(fname)

This is a problem I had, eventually I had given up with file names as they were inconsistent and instead created a function for getting the latest file based off the latest modified time or creation time.

from pathlib import Path

def get_latest_file(src_path,extension,method='st_mtime'):
    """ 
    Takes in a raw path and extension to parse over
    returns a single file with the last modified date

    methods:
    st_mtime: It represents the time of most recent content modification. It is 
    expressed in seconds.
    st_ctime: It represents the time of most recent metadata change on Unix 
    and creation time on Windows. It is expressed in seconds.
    """

    extension = extension if extension[0] != '.' else extension[1:]

    files = (Path(src_path).glob(f'*.{extension}'))

    if method == 'st_mtime':
        file_dictionary = {file : file.stat().st_mtime for file in files}
    elif method == 'st_ctime':
        file_dictionary = {file : file.stat().st_ctime for file in files}
    else:
        raise Exception(f'{method} not valid for this function')

    max_file = max(file_dictionary, key=file_dictionary.get)


    return max_file

latest = get_latest_file('C:/Users/DataNovice',extension='csv',method='st_mtime')

print(latest)

out : WindowsPath('C:/Users/DataNovice/new_file_i_just_created.csv')


df = pd.read_csv(latest)
Related