# rclone (/tools/s3-gateway/quickstarts/rclone)

## Before you start

* [rclone](https://rclone.org/install/) 1.60 or later
* Your access key ID, API key and bucket name from [console.shelby.xyz](https://console.shelby.xyz)

Export your credentials and bucket name from the
[console](https://console.shelby.xyz) so the commands below can use them:

```bash
export SHELBY_ACCESS_KEY_ID="<your access key ID, SHLB…>"
export SHELBY_SECRET_ACCESS_KEY="<your API key>"
export SHELBY_BUCKET="<your bucket name>"
```

<Steps>
  <Step>
    ### Create a `shelby` remote

    ```bash
    rclone config create shelby s3 \
      provider=Other \
      endpoint=https://s3.beta.shelby.xyz \
      access_key_id="$SHELBY_ACCESS_KEY_ID" \
      secret_access_key="$SHELBY_SECRET_ACCESS_KEY" \
      region=default \
      force_path_style=true \
      max_upload_parts=3000 >/dev/null
    ```

    `max_upload_parts=3000` matches Shelby's part limit. rclone raises the part size for large
    files to fit.

    `rclone config create` prints the new remote's settings, secret included. Redirecting the
    output to `/dev/null` keeps your API key out of your terminal's scrollback, and out of
    an AI agent's transcript.
  </Step>

  <Step>
    ### Copy a directory

    ```bash
    mkdir -p data && echo "Hello, Shelby" > data/hello.txt
    rclone copy ./data "shelby:$SHELBY_BUCKET/data" --checksum --progress
    ```

    Always pass `--checksum` (or `--size-only`) when copying to an existing destination. See
    [Why `--checksum`](#why---checksum) below.
  </Step>

  <Step>
    ### List and verify

    ```bash
    rclone ls "shelby:$SHELBY_BUCKET/data"
    rclone check ./data "shelby:$SHELBY_BUCKET/data" --one-way
    ```

    `rclone check` compares sizes and MD5 hashes, and should report `0 differences found`.
    For a byte-for-byte check of large files, add `--download`.
  </Step>

  <Step>
    ### Download and clean up

    ```bash
    rclone copy "shelby:$SHELBY_BUCKET/data" ./data-copy --checksum
    rclone delete "shelby:$SHELBY_BUCKET/data"
    ```
  </Step>
</Steps>

## Keep a directory in sync

```bash
rclone sync ./data "shelby:$SHELBY_BUCKET/data" --checksum
```

On the second run, unchanged files are skipped and nothing is uploaded.

## Why `--checksum`

By default rclone decides whether a file changed by comparing size and modification time,
which it stores with each object it uploads. When the sizes match but the times don't,
for example after you re-extract an archive, or for objects another tool uploaded, rclone
updates the stored time with a server-side copy. Shelby doesn't support server-side copy,
so that fails with `Failed to set modification time ... 501 NotImplemented`.

`--checksum` compares size and MD5 instead, so unchanged files are skipped. Files rclone
uploads in parts (over 200 MiB by default) have no whole-file MD5, so for those it compares
size only. `--size-only` also works and is faster, but it can't detect an edit that keeps
the same size.

## What's different with rclone

* **Every upload is a full write.** Re-uploading an unchanged file repeats the whole
  upload, which is another reason to always use `--checksum`.
* **Server-side `rclone copy` or `move` between two Shelby paths fails.** Download and
  upload again.
* **`-vv` doesn't show HTTP traffic.** To debug a request, use `--dump headers`.

[Compatibility matrix](/tools/s3-gateway/compatibility) ·
[Troubleshooting](/tools/s3-gateway/troubleshooting)