Skip to content
Advertisement

Uploading folder from local system to a perticular folder in S3

What should I change in my code so that I can upload my entire folder from local system to a particular folder present in my s3 bucket.

import os
import boto3

s3_resource = boto3.resource("s3", region_name="ap-south-1")

def upload_objects():
    try:
        bucket_name = "<S3 bucket-name>" 
        root_path = '<local folder path>' 
        bucket_folder = '<S3 folder name>'

        my_bucket = s3_resource.Bucket(bucket_name)
        # s3 = boto3.resource('s3')

        for path, subdirs, files in os.walk(root_path):
            path = path.replace("\","/")
            directory_name = path.replace(root_path,"")
            for file in files:
                my_bucket.upload_file(os.path.join(path, file), directory_name+'/'+file)
     except Exception as err:
        print(err)

if __name__ == '__main__':
    upload_objects()

Advertisement

Answer

You are not using your bucket_folder at all. This should be the beginning of your S3 prefix as in the S3 there are no folders. Its all about key names and prefixes.

So it should be something as the following:

my_bucket.upload_file(os.path.join(path, file), bucket_folder + '/' + directory_name+'/'+file)
User contributions licensed under: CC BY-SA
8 People found this is helpful
Advertisement