Limits
Size, count and request limits on the hosted S3 endpoint.
| Limit | Value | Amazon S3 | What happens past it |
|---|---|---|---|
Single PutObject | 5 GiB | 5 GiB | 400 EntityTooLarge. Use a multipart upload. |
| Multipart part size | 5 MiB to 5 GiB (the last part can be smaller) | Same | 400 EntityTooSmall or 400 EntityTooLarge |
| Parts per upload | 3,000 | 10,000 | 400 InvalidArgument on the part that crosses the limit |
| Largest object | About 14.6 TiB (3,000 × 5 GiB) | 50 TiB | — |
| Object key length | 958 bytes | 1,024 bytes | Keys from 959 to 1,024 bytes fail with 500 InternalError |
Keys per ListObjects page | 1,000 | 1,000 | 400 InvalidArgument for max-keys above 1,000 |
Keys per DeleteObjects request | 1,000 | 1,000 | 400 MalformedXML |
| Clock skew | Under 15 minutes | 15 minutes | 401 |
Content-Length | Required on uploads | Required | 411 MissingContentLength |
Free trial accounts have a storage allowance. An upload that would take you past it is
refused with 400 EntityTooLarge before any data is sent. The
console shows your usage.
Choosing a part size
Because of the 3,000-part limit, your part size sets your largest file:
| Part size | Largest file |
|---|---|
| 8 MiB (AWS CLI and boto3 default) | About 23 GiB |
| 64 MiB | About 187 GiB |
| 512 MiB | About 1.5 TiB |
rclone and DuckDB pick a part size for you when told the limit: max_upload_parts=3000 in
the rclone quickstart, and
s3_uploader_max_parts_per_file = 3000 in the
DuckDB quickstart.
What isn't limited
Objects stay until you delete them. Shelby has no lifecycle rules or automatic expiry.