2022-12-26

Untar specific files using lambda

I'm using a lambda function to untar files. The lambda is supposed to untar files and once it's done it moves the package to an archive folder.

Code below

def untar_file(zip_key,source_bucket,source_path,file):
 zip_obj = s3_resource.Object(bucket_name=source_bucket,key=zip_key)  
 buffer = BytesIO(zip_obj.get()["Body"].read())
 with tarfile.open(fileobj=buffer, mode=('r:gz')) as z:
     for filename in z.getmembers():
         s3_resource.meta.client.upload_fileobj(
         z.extractfile(filename),
         Bucket=source_bucket,
         Key=source_path+f'/{d1}/{filename}.csv'
         )
 copy_objects (zip_key,source_bucket,source_path,file)

I want to only untar specific files in the package. Can I specify which file to not untar? Just to avoid the lambda timeout



No comments:

Post a Comment