In AdapterAmazonS3.getFiles (v3.0.2, dist/AdapterAmazonS3.js:99), a single ListObjectsCommand is issued with MaxKeys: maxFiles (default 10000). AWS S3 caps MaxKeys at 1000 per request regardless of the requested value, and the response's IsTruncated/NextMarker are not checked. For buckets with more than 1000 objects, callers of listFiles / getFiles silently receive only the first page with no indication of truncation.
Reproduction
- Populate an S3 bucket with >1000 objects
- Call
storage.listFiles() on a Storage configured with the S3 adapter
- Result length is 1000, not the actual object count
Suggested fix
Switch to ListObjectsV2Command and loop while IsTruncated is true, passing ContinuationToken: NextContinuationToken until exhausted. The same pattern applies to getFileVersions at dist/AdapterAmazonS3.js:116 (uses ListObjectVersionsCommand with the same missing-pagination issue; IsTruncated + NextKeyMarker / NextVersionIdMarker).
Happy to send a PR if helpful.
In
AdapterAmazonS3.getFiles(v3.0.2,dist/AdapterAmazonS3.js:99), a singleListObjectsCommandis issued withMaxKeys: maxFiles(default 10000). AWS S3 capsMaxKeysat 1000 per request regardless of the requested value, and the response'sIsTruncated/NextMarkerare not checked. For buckets with more than 1000 objects, callers oflistFiles/getFilessilently receive only the first page with no indication of truncation.Reproduction
storage.listFiles()on aStorageconfigured with the S3 adapterSuggested fix
Switch to
ListObjectsV2Commandand loop whileIsTruncatedis true, passingContinuationToken: NextContinuationTokenuntil exhausted. The same pattern applies togetFileVersionsatdist/AdapterAmazonS3.js:116(usesListObjectVersionsCommandwith the same missing-pagination issue;IsTruncated+NextKeyMarker/NextVersionIdMarker).Happy to send a PR if helpful.