How to read image from S3 using OpenCV (cv2.imread)?

I am having trouble reading images from the S3 bucket. I can read images locally like that.

def load_image(path):
    image = cv2.imread(path)
    image = cv2.resize(image, (224, 224))
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    return image

for image_path in os.listdir(path):
    image = load_image(path + "/" + image_path)
    X.append(image)

But I have no idea why S3 said error

TypeError: Can't convert object of type 'list' to 'str' for 'filename'

Answer

You need to first establish a connection to S3, then download the image data, and finally decode the data with OpenCV.

For the first bit (connecting to S3), Boto3 is a good alternative (it’s the Python SDK for AWS).

Full sample code:

from io import BytesIO

import boto3
import cv2
import numpy as np

endpoint_url = {your_s3_endpoint_url}
bucket_name = {your_s3_bucket_name}
key = {your_image_key_within_the_bucket}
aws_access_key_id = {your_aws_access_key_id}  # Better not to directly expose in source code
aws_secret_access_key = {your_aws_secret_access_key}  # Better not to directly expose in source code

bucket = boto3.resource(
                "s3",
                endpoint_url=endpoint_url,
                aws_access_key_id=aws_access_key_id,
                aws_secret_access_key=aws_secret_access_key,
            ).Bucket(bucket_name)
file_stream = BytesIO()
bucket.Object(key).download_fileobj(file_stream)
np_1d_array = np.frombuffer(file_stream.getbuffer(), dtype="uint8")
img = cv2.imdecode(np_1d_array, cv2.IMREAD_COLOR)