I have a s3 folder with a csv file stored on it. I’m trying to download the last modified file. I’m using this script to get the last modified file:
s3_client = boto3.client('s3', aws_access_key_id=s3_extra_data['aws_access_key_id'], aws_secret_access_key=s3_extra_data['aws_secret_access_key']) response = s3_client.list_objects_v2(Bucket='test', Prefix='file_r/') all = response['Contents'] latest = max(all, key=lambda x: x['LastModified']) response = s3_client.list_objects_v2(Bucket='test', Prefix=latest["Key"])[:-52].lower()) all = response['Contents'] latest = max(all, key=lambda x: x['LastModified']) print("LATEST ->" + str(latest["Key"])[:-52].lower()) print("PATH ->" + str(latest["Key"])) s3_client.download_file("test", latest["Key"], str(latest["Key"]))
This code lists my last modified object, the file name is part-00000-40f267f2-38dc-4bab-811c-4c3052fdb1ba-c000.csv
and is inside the file_r
folder.
Although, when I use s3_client.download_file
i get the following error:
'file_r/part-00000-40f267f2-38dc-4bab-811c-4c3052fdb1ba-c000.csv.8cEebaeb'
When i print my path and my file I get the correct values
LATEST -> file_r/part PATH -> file_r/part-00000-40f267f2-38dc-4bab-811c-4c3052fdb1ba-c000.csv
Why the value .8cEebaeb
is appended after the .csv
extension since the PATH is correct.
Any thoughs on that?
Advertisement
Answer
To solve the problem I changed the code to:
s3_client = boto3.client('s3', aws_access_key_id=s3_extra_data['aws_access_key_id'], aws_secret_access_key=s3_extra_data['aws_secret_access_key']) response = s3_client.list_objects_v2(Bucket='test', Prefix='file_r/') all = response['Contents'] latest = max(all, key=lambda x: x['LastModified']) s3_client.download_file("test", latest["Key"], "FILE_NAME"))