I am copying files from a website to a S3 bucket. Everything else is copying fine, even odd extensions that I haven’t heard of before. The extension that I am having problems with is “.2D”.
Currently using this code, and it is working for all but the .2D files. Might be a VERSACAD file. Anyone work with this file or know how to figure out how to work with this? No, I can’t include an example.
It is failing on the r.data.decode(“utf’8”) line. Using “utf-16” doesn’t work either.
JavaScript
x
3
1
data=r.data.decode("utf-8")
2
key_path="downloaded_docs/{0}/{1}/{2}/{3}".format(year,str(month).zfill(2),str(day).zfill(2),docname)
3
To save to s3 bucket:
JavaScript
1
2
1
s3.Object('s3_bucket_name',key_path).put(Body=data)
2
WHAT WORKED I ended up having to do the following, but it worked. Thanks!!
JavaScript
1
7
1
if extracted_link.endswith('.2D'):
2
r=http.request("GET", url_to_file, preload_content=False)
3
# to get binary data
4
text_b=r.data
5
filepath=<filepath>
6
s3_client.upload_fileobj(io.BytesIO(text_b),Bucket=<bucketname>,Key=filepath)
7
Advertisement
Answer
Upload it in binary
from file path
JavaScript
1
4
1
import boto3
2
client = boto3.client("s3")
3
client.upload_file(2D_filepath, bucket, key)
4
from file object
JavaScript
1
4
1
import boto3, io
2
client = boto3.client("s3")
3
client.upload_fileobj(io.BytesIO(b"binary data"), bucket, key)
4