I want to process/download .gz files from S3 bucket. There are more than 10,000 files on S3 so I am using
import boto3 s3 = boto3.resource('s3') bucket = s3.Bucket('my-bucket') objects = bucket.objects.all() for object in objects: print(object.key)
This lists .txt files which I want to avoid. How can I do that?
Advertisement
Answer
The easiest way to filter objects by name or suffix is to do it within Python, such as using .endswith()
to include/exclude objects.
You can Filter
by Prefix, but not by suffix.