Skip to content
Advertisement

Boto3: grabbing only selected objects from the S3 resource

I can grab and read all the objects in my AWS S3 bucket via

s3 = boto3.resource('s3')
    bucket = s3.Bucket('my-bucket')
    all_objs = bucket.objects.all()
    for obj in all_objs:
        pass
        #filter only the objects I need

and then

obj.key

would give me the path within the bucket.

Is there a way to filter beforehand for only those files respecting a certain starting path (a directory in the bucket) so that I’d avoid looping over all the objects and filtering later?

Advertisement

Answer

Use the filter[1], [2] method of collections like bucket.

s3 = boto3.resource('s3')
bucket = s3.Bucket('my-bucket')
objs = bucket.objects.filter(Prefix='myprefix')
for obj in objs:
    pass
Advertisement