Not all folders returned by boto3 Bucket.objects.all()

My S3 bucket contains a bunch of files in a multilevel folder structure. I’m trying to identify the top level folders in the hierarchy, but objects.all() returns some but not all folders as distinct ObjectSummary objects. Why?

Sample file structure:

file1.txt
a/file2.txt
a/a1/file3.txt
b/b1/file4.txt

Desired output: [a,b]

What I’m doing:

boto3.resource('s3').Bucket('mybucket').objects.all()

This returns the following ObjectSummary objects:

file1.txt
a/
a/file2.txt
a/a1/file3.txt
b/b1/file4.txt

Notice that a/ is listed as a separate entry, but b/ is not, while the files in b/ are.

I could understand it returning neither, as folders are technically not distinct entities, or both, but why are some folders returned and others not?

I also understand there could be other ways to achieve my objective, but I want to understand why boto3 is behaving this way.

Answer

S3 does have the concept of creating a folder, through a Create Folder button, which creates a dedicated object with just the folder name, separate from the files that have this as a prefix.

a/ in the example above was a folder I created manually, but I hadn’t done this for b/.