I’m new to AWS services. I’ve always used the code below to calculate NDVI for images that were located in a directory.
path = r'images' dirContents = os.listdir(path) for file in dirContents: if os.path.isdir(file): subDir = os.listdir(file) # Assuming only two files in each subdirectory, bands 4 and 8 here if "B04" in subDir[0]: band4 = rasterio.open(subDir[0]) band8 = rasterio.open(subDir[1]) else: band4 = rasterio.open(subDir[1]) band8 = rasterio.open(subDir[0]) red = band4.read(1).astype('float32') nir = band8.read(1).astype('float32') #compute the ndvi ndvi = (NIR.astype(float) - RED.astype(float)) / (NIR+RED) profile = red.meta profile.update(driver='GTiff') profile.update(dtype=rasterio.float32) with rasterio.open(outfile, 'w', **profile) as dst: dst.write(ndvi.astype(rasterio.float32))
Now all the necessary images are in an amazon S3 folder. How do I replace the lines below?
path = r'images' dirContents = os.listdir(path)
Advertisement
Answer
Amazon S3 is not a filesystem. You will need to use different commands to:
- List the contents of a bucket/path
- Download the files to local storage
- Then access the files on local storage
You can use the boto3
AWS SDK for Python to access objects stored in S3.
For example:
import boto3 s3_resource = boto3.resource('s3') # List objects objects = s3_resource.Bucket('your-bucket').objects.filter(Prefix='images/') # Loop through each object for object in objects: s3_resource.Object(object.bucket_name, object.key).download_file('local_filename') # Do something with the file here