# Python (boto3) (/tools/s3-gateway/quickstarts/python)

## Before you start

* Python 3.10 or later, with `pip install boto3`
* Your access key ID, API key and bucket name from [console.shelby.xyz](https://console.shelby.xyz)

Export your credentials and bucket name from the
[console](https://console.shelby.xyz) so the commands below can use them:

```bash
export SHELBY_ACCESS_KEY_ID="<your access key ID, SHLB…>"
export SHELBY_SECRET_ACCESS_KEY="<your API key>"
export SHELBY_BUCKET="<your bucket name>"
```

<Steps>
  <Step>
    ### Create a client

    ```python title="shelby_s3.py"
    import os

    import boto3
    from botocore.config import Config

    s3 = boto3.client(
        "s3",
        endpoint_url="https://s3.beta.shelby.xyz",
        aws_access_key_id=os.environ["SHELBY_ACCESS_KEY_ID"],
        aws_secret_access_key=os.environ["SHELBY_SECRET_ACCESS_KEY"],
        region_name="default",
        config=Config(
            s3={"addressing_style": "path"},
            request_checksum_calculation="when_required",
        ),
    )
    bucket = os.environ["SHELBY_BUCKET"]
    ```

    `request_checksum_calculation` stops boto3 from sending uploads in `aws-chunked` framing,
    which it does by default (botocore 1.36 and later) and Shelby doesn't accept yet. Without
    it, uploads fail with `501 NotImplemented`.
  </Step>

  <Step>
    ### Upload, read back and delete

    ```python
    import hashlib

    body = b"Hello, Shelby\n"
    s3.put_object(Bucket=bucket, Key="examples/hello.txt", Body=body)

    data = s3.get_object(Bucket=bucket, Key="examples/hello.txt")["Body"].read()
    assert hashlib.sha256(data).digest() == hashlib.sha256(body).digest()

    # A new object can take a few seconds to appear in listings.
    for obj in s3.list_objects_v2(Bucket=bucket, Prefix="examples/").get("Contents", []):
        print(obj["Key"], obj["Size"])

    s3.delete_object(Bucket=bucket, Key="examples/hello.txt")
    ```
  </Step>
</Steps>

## Upload large files

`upload_file` switches to multipart automatically. Shelby allows 3,000 parts per upload, so
set a part size that fits your largest file:

```python
from boto3.s3.transfer import TransferConfig

s3.upload_file(
    "dataset.parquet",
    bucket,
    "data/dataset.parquet",
    Config=TransferConfig(multipart_chunksize=64 * 1024 * 1024),  # 64 MiB parts
)
```

At the default 8 MiB part size, files over about 23 GiB need a larger part size.

## What's different with boto3

* **`copy_object` and `s3.copy` fail with `501`.** Shelby has no server-side copy. Download
  and upload again.
* **Paginators need `PageSize` of 1,000 or less.** Larger values are rejected.

[Compatibility matrix](/tools/s3-gateway/compatibility) ·
[Troubleshooting](/tools/s3-gateway/troubleshooting)