2021-08-31

looping through a list of dataframes, writing each element of that list to a new .csv file on disk

I have a list of dataframes and am attempting to export each using the pandas.df.to_csv method to a folder on disk. However, only the last item in the list of dataframes is being written to disk as a .csv

Please see code below:

import pandas as pd
import os
import datetime
from pathlib import Path

CSV_Folder = Path('C:\PA_Boundaries\Tests')
Output = r'C:/PA_Boundaries/test_output'

today = datetime.date.today()
date = today.strftime('%Y%m%d')

try:
    dfs = []
    for file in os.listdir(CSV_Folder):
        df = pd.read_csv(CSV_Folder / file)
        dfs.append(df)
    
    new_dfs = []
    for df in dfs:
        new_df = pd.DataFrame()
        new_df['Original Addr string'] = df['StreetConc']
        new_df['Addr #'] = df['AddNum']
        new_df['Prefix'] = df['StPreDir']
        new_df['Street Name'] = df['StName']
        new_df['StreetType'] = df['StType']
        new_df['Suffix'] = df['StDir']
        new_df['Multi-Unit'] = ''
        new_df['City'] = df['City']
        new_df['Zip Code'] = df['PostCode']
        new_df['4'] = df['PostalExt']
        new_df['County'] = df['CountyID']
        new_df['Addr Type'] = ''
        new_df['Precint Part Name'] = ''
        new_df['Lat'] = df['X']
        new_df['Long'] = df['Y']
    
    
        replaced_address_names = []
        for index, row in new_df.iterrows():
            new_row = row['Original Addr string'].replace(',', ' ')
            replaced_address_names.append(new_row)
        
        
        new_df['Original Addr string'] = replaced_address_names
        
        county_id = df.iloc[0, 37]
    
        new_dfs.append(new_df)
        
    for i in range(len(new_dfs)):
        new_dfs[i].to_csv(f'{Output}\ADDR_{county_id}_{date}.csv', index=False)

except FileNotFoundError:
    print(f'{file} not found in {CSV_Folder}')

except PermissionError:
    print('Check syntax of paths')

else:
    print('Process Complete')

new_dfs contains the correct number of dataframes. However, when looping through the new list of dataframes and calling .to_csv on each item in the list, only the last item in the list is written to the disk.



from Recent Questions - Stack Overflow https://ift.tt/2UXHzm4
https://ift.tt/eA8V8J

No comments:

Post a Comment