JavaScript (AWS SDK v3)

Upload, list, download and delete Shelby objects with @aws-sdk/client-s3 in Node.js.

Before you start

Node.js 22.18 or later, which runs TypeScript files directly

A project set up for ES modules, with the AWS SDK installed:

npm init -y
npm pkg set type=module
npm install @aws-sdk/client-s3 @aws-sdk/lib-storage

Your access key ID, API key and bucket name from console.shelby.xyz

Export your credentials and bucket name from the console so the commands below can use them:

export SHELBY_ACCESS_KEY_ID="<your access key ID, SHLB…>"
export SHELBY_SECRET_ACCESS_KEY="<your API key>"
export SHELBY_BUCKET="<your bucket name>"

Create a client

shelby-s3.ts

import { S3Client } from "@aws-sdk/client-s3";

export const s3 = new S3Client({
  endpoint: "https://s3.beta.shelby.xyz",
  region: "default",
  credentials: {
    accessKeyId: process.env.SHELBY_ACCESS_KEY_ID!,
    secretAccessKey: process.env.SHELBY_SECRET_ACCESS_KEY!,
  },
  forcePathStyle: true,
  requestChecksumCalculation: "WHEN_REQUIRED",
});

export const Bucket = process.env.SHELBY_BUCKET!;

requestChecksumCalculation is optional. It stops the SDK from computing CRC checksums that Shelby accepts but doesn't verify.

Upload, read back and delete

quickstart.ts

import {
  DeleteObjectCommand,
  GetObjectCommand,
  ListObjectsV2Command,
  PutObjectCommand,
} from "@aws-sdk/client-s3";
import { Bucket, s3 } from "./shelby-s3.ts";

const Key = "examples/hello.txt";

await s3.send(new PutObjectCommand({ Bucket, Key, Body: "Hello, Shelby\n" }));

const object = await s3.send(new GetObjectCommand({ Bucket, Key }));
console.log(await object.Body?.transformToString());

// A new object can take a few seconds to appear in listings.
const listing = await s3.send(
  new ListObjectsV2Command({ Bucket, Prefix: "examples/" }),
);
for (const item of listing.Contents ?? []) console.log(item.Key, item.Size);

await s3.send(new DeleteObjectCommand({ Bucket, Key }));

Run it:

node quickstart.ts

Upload large files

Upload from @aws-sdk/lib-storage splits large bodies into parts. Shelby allows 3,000 parts per upload, so choose a part size that fits your largest file:

import { createReadStream } from "node:fs";
import { Upload } from "@aws-sdk/lib-storage";
import { Bucket, s3 } from "./shelby-s3.ts";

await new Upload({
  client: s3,
  params: {
    Bucket,
    Key: "data/dataset.parquet",
    Body: createReadStream("dataset.parquet"),
  },
  partSize: 64 * 1024 * 1024, // 64 MiB parts
}).done();

What's different with the AWS SDK for JavaScript

Run it on a server. Shelby doesn't support CORS or presigned URLs, so browsers can't call the endpoint directly.
CopyObjectCommand fails with 501. Download and upload again.
DeleteObjectsCommand doesn't work yet. The SDK sends batch deletes in a form Shelby doesn't recognise, so nothing is deleted. Delete keys one at a time with DeleteObjectCommand. See Known compatibility issues.
Authentication failures surface as XML parse errors. Shelby returns a plain-text 401, which the SDK can't parse. See Troubleshooting.

Compatibility matrix · Troubleshooting