is there a way to manipulate image without saving to a file first?

I’m trying to collect img data for my ML/DL project.

I need facial data so, I have to detect the face and crop around it.

I have a bunch of img URLs that I searched online.

Normally I would save them in a file using requests library, but would it be possible to do it in-memory?

response = requests.get(IMG_URL)
img_byte = response.content
# Do image processing without saving to a file

I looked at some image libraries in python such as PIL or OpenCV,

but they all seems to get images loaded from a file in first place.

I think I could speed up the process if I don’t save temporary files. (I/O is a big bottleneck)

I tried playing around with BytesIO functions but I couldn’t figure it out.

Answer

BytesIO is the way to go!. You can store the binary data returned by the request to the In-memory byte buffer(BytesIO) and pass it to Image.open

>>> from PIL import Image
>>> from io import BytesIO
>>> img_obj = Image.open(BytesIO(r.content))

The only requirement of the Image.open API is that the fp argument must be a filename (string), pathlib.Path object or a file object. The file object must implement file.read, file.seek, and file.tell methods, and be opened in binary mode.

BytesIO implements all these methods.

>>> from io import BytesIO
>>>
>>> buffer = BytesIO()
>>> hasattr(buffer, 'read') and hasattr(buffer, 'tell') and hasattr(buffer, 'seek')
True