Troubleshooting
Common S3 errors on Shelby, what causes them, and how to fix them.
Authentication errors
Shelby checks credentials before a request reaches the S3 layer. A rejected request gets
401 Unauthorized with a short plain-text message, not an S3 XML error. boto3 and the AWS
CLI report it with no S3 error code, and the AWS SDK for JavaScript reports an XML parse
error instead. Run the AWS CLI with --debug to see the message:
| Message | Cause | Fix |
|---|---|---|
Invalid S3 credential | The access key ID or API key is wrong, or they come from different keys | Copy both again from console.shelby.xyz. The access key ID starts with SHLB, and the API key is the secret. If they're right and requests still fail, contact us with your access key ID. Never send the API key. |
API key not found | The access key field doesn't hold an access key ID, usually because the API key is in both fields, as older setups had it | Put your access key ID (SHLB…) in the access key field and your API key in the secret field |
Invalid signature | The access key ID or API key is wrong or revoked, or something rewrote a signed header | Copy your access key ID and API key again from console.shelby.xyz. The access key ID starts with SHLB, and the API key is the secret. |
Request timestamp is too far from server time | Your clock is 15 minutes or more off | Turn on automatic time sync |
Anonymous requests are not allowed | The request wasn't signed | Configure credentials in your client. Presigned URLs aren't supported. |
The region doesn't cause authentication errors. Shelby accepts a signature for any region,
but set it to default, the region Shelby reports, so clients don't warn about a mismatch.
Errors by symptom
| Symptom | Cause | Fix |
|---|---|---|
501 NotImplemented: "aws-chunked transfer encoding is not supported" | Your client sends uploads in aws-chunked framing by default (AWS CLI 2.23+, botocore 1.36+) | Set request_checksum_calculation to when_required, as in each quickstart |
501 NotImplemented naming CopyObject | Server-side copy, from aws s3 mv, copy_object, or rclone updating a modification time | Download and upload again. In rclone, add --checksum. |
503 ServiceUnavailable reading a key just after writing or deleting it | Reads wait up to 10 seconds for a recent write to be indexed, and this one took longer | Retry with backoff |
| Object missing from a listing just after upload, or still listed after delete | Listings catch up after a few seconds | Retry, or confirm with HeadObject |
DeleteObjects from the AWS SDK for JavaScript returns a plain-text 404 and deletes nothing | The SDK sends batch deletes in a form Shelby doesn't recognise yet | Delete keys one at a time with DeleteObjectCommand. See Known compatibility issues. |
412 PreconditionFailed on upload | You sent If-None-Match: * and the key already exists | Expected: the create-only check worked |
400 InvalidArgument on a list | max-keys is above 1,000 | Use a page size of 1,000 or less |
400 InvalidArgument partway through a multipart upload | More than 3,000 parts | Raise the part size. Retrying fails the same way. See Limits. |
400 EntityTooSmall | A part other than the last is under 5 MiB | Raise the part size |
400 BadDigest | The Content-MD5 you sent doesn't match the body | Check that nothing changes the body in transit |
404 NoSuchBucket | The bucket name isn't your account's bucket | Copy your bucket name from the console |
403 AccessDenied: "Bucket access denied" | The bucket belongs to a different account | Each access key can read and write only its own account's bucket |
MinIO mc fails every request with invalid header field value for "Authorization" | mc looks up the bucket's region, and Shelby doesn't answer that lookup correctly yet | Set MC_REGION=default before running mc |
400 MetadataTooLarge on upload | More than 2 KB of x-amz-meta-* user metadata, the same limit as S3 | Keep larger metadata in your application |
An object comes back as binary/octet-stream, without the metadata it was uploaded with | It was stored before Shelby kept object metadata | Upload it again to store its metadata |
DuckDB fails with Failed to parse http_proxy | DuckDB reads your HTTPS_PROXY variable but can't parse a proxy URL that includes a username and password | Start DuckDB without the proxy variables set, then set http_proxy (host and port), http_proxy_username and http_proxy_password with SET |
Debugging requests
- AWS CLI: add
--debugto see the signed request and response. - boto3: call
boto3.set_stream_logger("botocore")before creating the client. - rclone: use
--dump headers.-vvdoesn't show HTTP traffic.
Successful responses and S3 XML errors include an x-amz-request-id header. Include it
when you contact support. The plain-text 401 responses don't have one, so include the
message, the time of the request and your bucket name instead.
Getting help
Ask in Discord, or send the team a message. Include the request ID or error details from above.