Skip to content

Tag: amazon-s3

Query S3 from Python

I am using python to send a query to Athena and get table DDL. I am using start_query_execution and get_query_execution functions in the awswrangler package. The code above creates a dict object that stores query results in an s3 link. The link can be accessed by res[‘ResultConfiguration’][‘OutputLocation’]. It’s a text link: s3://…..txt Can someone help me figure how to access

How to copy .2D file from web to S3 bucket? Failing on decode

I am copying files from a website to a S3 bucket. Everything else is copying fine, even odd extensions that I haven’t heard of before. The extension that I am having problems with is “.2D”. Currently using this code, and it is working for all but the .2D files. Might be a VERSACAD file. Anyone work with this file or

Copy a large amount of files in s3 on the same bucket

I got a “directory” on a s3 bucket with 80 TB ~ and I need do copy everything to another directory in the same bucket source = s3://mybucket/abc/process/ destiny = s3://mybucket/cde/process/ I already tried to use aws s3 sync, but worked only for the big files, still left 50 TB to copy. I’m thinking about to use a boto3 code

S3 appending random string in file name

I have a s3 folder with a csv file stored on it. I’m trying to download the last modified file. I’m using this script to get the last modified file: This code lists my last modified object, the file name is part-00000-40f267f2-38dc-4bab-811c-4c3052fdb1ba-c000.csv and is inside the file_r folder. Although, when I use s3_client.download_file i get the following error: When i