S3 Gateway

Getting Started

S3-compatible gateway for Shelby blob storage

The Shelby S3 Gateway provides an Amazon S3-compatible API for accessing Shelby blob storage. It allows you to use familiar S3 tools and libraries (AWS SDK, rclone, Cyberduck, DuckDB, etc.) to interact with blobs stored on Shelby without writing custom code.

Key Concepts

  • Buckets map to Shelby account addresses (Aptos addresses)
  • Objects map to Shelby blobs/files
  • Authentication uses standard AWS SigV4 signing

Supported Operations

The gateway currently supports read operations:

OperationDescription
ListBucketsList all configured buckets (accounts)
HeadBucketCheck if a bucket exists and is accessible
ListObjectsV1/V2List objects with pagination support
GetObjectDownload file contents (supports Range headers)
HeadObjectGet object metadata without downloading

Note: Write operations (PutObject, DeleteObject) are not currently supported. Use the Shelby CLI to upload and manage blobs.

Quick Start

Create a config file

Create a shelby.config.yaml file:

shelby.config.yaml
network:
  name: shelbynet
  rpcEndpoint: https://api.shelbynet.shelby.xyz/shelby
  aptosFullnode: https://api.shelbynet.shelby.xyz/v1
  aptosIndexer: https://api.shelbynet.shelby.xyz/v1/graphql
  apiKey: your-api-key

server:
  port: 9000

credentials:
  - accessKeyId: AKIAIOSFODNN7EXAMPLE
    secretAccessKey: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

buckets:
  - "0x0694a79e492d268acf0c6c0b01f42654ac050071a343ebc4226cb6717d63e4ea"

Replace your-api-key with your Shelby API key and the bucket address with your Shelby account address.

Start the gateway

npx @shelby-protocol/s3-gateway --config shelby.config.yaml
Output
Loaded config from: shelby.config.yaml
S3 Gateway is running on http://localhost:9000
API Reference: http://localhost:9000/spec

Verify it's working

Configure rclone and list your buckets:

# Configure rclone (one-time setup)
rclone config create shelby s3 \
  provider=Other \
  access_key_id=AKIAIOSFODNN7EXAMPLE \
  secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \
  endpoint=http://localhost:9000 \
  force_path_style=true

# List buckets
rclone lsd shelby:

# List files in a bucket
rclone ls shelby:0x0694a79e492d268acf0c6c0b01f42654ac050071a343ebc4226cb6717d63e4ea

CLI Options

npx @shelby-protocol/s3-gateway --help
OptionDescription
-c, --config <path>Path to config file
-p, --port <port>Port to listen on (overrides config)
-h, --helpShow help message

Default Credentials

If you don't specify credentials in your config, the gateway uses these defaults:

FieldValue
Access Key IDAKIAIOSFODNN7EXAMPLE
Secret Access KeywJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Regionshelbyland

These are example credentials for development only. For production deployments, configure your own credentials in shelby.config.yaml.

Next Steps