# Bandwidth Limitations (/apis/bandwidth-limits) import { Accordion, Accordions } from 'fumadocs-ui/components/accordion'; Shelby is designed to serve data with subsecond latency, high throughput, and low costs. Some applications, however, require even lower latencies or exhibit extreme temporal hotspots. This document describes some of your options, such as running your own Shelby RPC or using a CDN. ## Limits The Shelby RPC for Shelbynet that is used by default in the SDK is operated by [Geomi](https://geomi.dev) (from Aptos Labs). To ensure fair usage across all users, the RPC enforces bandwidth limitations. This document describes those limits and what alternatives exist if you need higher bandwidth. ### Per-organization limits See the Geomi docs [here](https://geomi.dev/docs/admin/billing#system-limits) for information about per-organization limits. Look for limits related to the Shelby RPC. The limits apply collectively to all projects and resources within an organization. These are organization-wide caps, not per-project or per-resource. ### Anonymous clients See the Geomi docs [here](https://geomi.dev/docs/faq) for information about anonymous limits. Anonymous clients are those using the Shelby RPC without an API key. These are subject to much lower limits, it is highly recommended you use an API key. Coming soon! Coming soon! ## Use cases ### Video streaming 4k video streaming requires about `4 MiB/s` per stream. At time of writing Geomi allows 512 MiB/s for a single organization. As such, on Shelbynet a single organization can support approximately 64 concurrent viewers (`512 MiB/s / 4 MiB/s = 128`). ## Higher bandwidth needs If your application requires more bandwidth than these limits provide, consider these options: ### 1. Contact the Geomi team The Geomi team is able to grant higher limits in some situations. If there is sufficient demand, we can consider increasing the total overall bandwidth the network is capable of as well. Sign in to [geomi.dev](https://geomi.dev) and use the help button in the top right corner. ### 2. Run your own Shelby RPC Guide coming soon. ### 3. Deploy a CDN Place a CDN in front of the Shelby RPC. This way only the initial read (and therefore the cost for reading from an RPC) for a blob goes to an actual RPC, the rest are served by the CDN. For Shelby mainnet we will be deployed globally, but for Shelbynet / Testnet this can have noteworthy latency advantages as well. **Note**: In this guide we assume the origin is the Geomi-hosted Shelby RPC. While Geomi can abstract read payments away and allow you to query with just an API key, other providers may not work this way. #### 1. Create a Pull Zone In the [Bunny dashboard](https://dash.bunny.net/cdn): 1. Go to **Pull Zones → Add Pull Zone**. 2. Name your zone — e.g. `dport-shelby-cdn`. 3. Use this for the **Origin URL**: ``` https://api.shelbynet.shelby.xyz/shelby ``` 4. Save the Pull Zone. {/* TODO: Down the line it'll be safer to use a URL with the account in the subdomain. That way they won't be able to query other accounts. */} #### 2. Allow the CDN to query the Shelby RPC by attaching a Geomi API key Go to [geomi.dev](https://geomi.dev). From there, [create a new API key](https://geomi.dev/manage/create-api-resource) with `shelbynet` as the network. Now in the [Bunny dashboard](https://dash.bunny.net/cdn) go to Security -> S3 authentication. Paste the API key in to the "AWS key" and "AWS secret" fields. Use "geomi" as the AWS region name. #### 3. Configure other settings These other settings are important to ensure the CDN is operating as intended: * Caching -> General -> Cache expiration time -> See below. * Security -> General -> Enable "Block POST requests" The best cache expiration time will depend on your use of Shelby. Shelby blobs are immutable, but they can be deleted and re-created with the same name. If you delete blobs often, consider a shorter cache expiration time. #### 4. Update your application URLs Before: ``` https://api.shelbynet.shelby.xyz/shelby/v1/blobs/0x0628e72def46bc247908f0b06367bfbf744a50328e429a97632941035ef50a63/whitepaper.pdf ``` After: ``` https://dport-shelby-cdn.b-cdn.net/v1/blobs/0x0628e72def46bc247908f0b06367bfbf744a50328e429a97632941035ef50a63/whitepaper.pdf ``` **Note that `/shelby` is gone from the path.** The first request will fetch from the Shelby RPC. Subsequent requests come from BunnyCDN’s edge nodes. Confirm that this is happening by looking for the `cdn-cache` header in the response. Its value should be `HIT` from the second request onward. #### 5. Optimizations The following changes can improve the security / performance of the CDN further: * General -> Websockets -> Disable * Caching -> General -> Turn on "Optimize for large object delivery" * Consider enabling Origin Shield to reduce the amount of traffic hitting the origin. * Consider some of the Bunny Optimizer features for improved performance. {/* ### 4. Geomi-Managed CDN Coming soon */} # Examples (/examples) Visit the [Shelby Examples](https://github.com/shelby/examples/) repository to view the available examples. # Introduction (/protocol) import { Callout } from 'fumadocs-ui/components/callout'; Shelby Early Access is open. [Apply to build on testnet today](https://developers.shelby.xyz/) and join the [Shelby Discord](https://discord.com/invite/shelbyserves). Shelby is a high-performance decentralized blob storage system designed for demanding read-heavy workloads. Workloads such as video streaming, AI training and inference, and large-scale data analytics require robust storage, significant capacity, and high read bandwidth at reasonable prices, all while maintaining control over their data. To deliver these capabilities to users, Shelby uses: * **Paid reads and user-focused incentive model**. Paying for reads ensure that storage providers deliver good service. * **The Aptos blockchain as a coordination and settlement layer**. Aptos provides a high-performance, reliable foundation for managing system state, economic logic, and enforcing BFT. * **Dedicated private bandwidth**. Performance is limited and inconsistent over the public internet. Shelby uses a dedicated fiber network to ensure consistent high performance. * **A novel auditing system**. Ensures data integrity and rewards honest participation in the network. * **Efficient erasure coding**. Minimizes recovery bandwidth, reducing costs, while ensuring data is safely stored. * **Built by experienced teams**. The Jump Trading Group and Aptos team both are rich with experience building high performance storage and globally distributed systems. ## Key Components {/* Just mean to capture the highest level components here that end users need to know about to meaningfully read the rest of docs */} Users interact primarily with the RPC servers through: 1. Shelby [Typescript SDK](/sdks/typescript) 2. Shelby [CLI](/tools/cli) 3. Shelby web applications, such as the [block explorer](https://explorer.shelby.xyz/testnet), streaming apps, etc. The Shelby system consists of the following major components: 1. **Aptos Smart Contract** - manages the state of the system and manages correctness-critical operations, such as auditing for data correctness. 2. **Storage Provider (SP) servers** - storage servers which store chunks of user data. 3. **Shelby RPC servers** - used by end users to read and write blobs. ## Why Aptos? Shelby uses the Aptos blockchain as its coordination and settlement layer because it offers high transaction throughput, low finality times, and a resource-efficient execution model. This makes it an ideal foundation for managing Shelby's economic logic, including storage commitments, audit outcomes, and payment channels, without compromising scalability. Additionally, the Aptos team brings DNA from Meta's global large-scale platforms, a perfect match for the Shelby project. ## Why Jump Crypto? Shelby's software stack is not built from scratch; it is founded on engineering principles honed through years of experience developing storage and compute systems for Jump Trading Group's high-performance quantitative research infrastructure. This includes expertise in high-performance I/O, efficient concurrency, and low-level code optimizations, allowing Shelby to deliver a system that is both highly scalable and responsive. {/* blurb about user choice on RPC nodes? or no */} # Quick Start (/protocol/quickstart) # Using the CLI Install the CLI following the [CLI quickstart guide](/tools/cli). For a guided sample of using the CLI follow the `README.md` from the [`shelby/shelby-quickstart`](https://github.com/shelby/shelby-quickstart) GitHub repo. # Using the Typescript SDK Follow the [SDK quickstart guide](/sdks/typescript). # Learn More * Explore the [architecture overview](/protocol/architecture/overview) to learn more about how Shelby works. * Explore the CLI Commands: [Uploads](/tools/cli/commands/uploads) and [Downloads](/tools/cli/commands/downloads) * Explore the [Typescript SDK documentation](/sdks/typescript/core) and [file upload guide](/sdks/typescript/node/guides/uploading-file). * Explore the [RPC Server APIs](/apis/rpc) # AI & LLM Integration (/tools/ai-llms) Shelby documentation provides two methods for programmatic access optimized for AI agents and LLMs. ## Full Documentation Endpoint Get all documentation in a single plain text file. ```bash curl https://docs.shelby.xyz/llms-full.txt ``` The response includes all pages formatted as: ```text # Page Title (URL) Page content in markdown... # Next Page Title (URL) Next page content... ``` ## Individual Page Markdown ### Direct URL Access any page's markdown content by appending `.mdx` or `.md` to the page URL: ```bash curl https://docs.shelby.xyz/sdks/typescript.mdx # or curl https://docs.shelby.xyz/sdks/typescript.md ``` Examples: * `/sdks/typescript` → `/sdks/typescript.mdx` or `/sdks/typescript.md` * `/protocol/quickstart` → `/protocol/quickstart.mdx` or `/protocol/quickstart.md` * `/tools/cli` → `/tools/cli.mdx` or `/tools/cli.md` ### Accept Header Alternatively, request any documentation page as markdown by setting the `Accept` header to `text/markdown`: ```bash curl -H "Accept: text/markdown" https://docs.shelby.xyz/docs/quickstart ``` This works for all documentation sections: * `/docs/*` - Core documentation * `/apis/*` - API references * `/protocol/*` - Protocol specifications * `/examples/*` - Code examples * `/sdks/*` - SDK documentation # Shelby Explorer (/tools/explorer) import { Callout } from 'fumadocs-ui/components/callout'; import { Step, Steps } from 'fumadocs-ui/components/steps'; The Shelby Explorer is currently in beta and under active development. Features and functionality may change. The **Shelby Explorer** is a web application that provides a user-friendly interface for exploring the Shelby network. Use it to view blob metadata, monitor network activity, and interact with your account. ### Access the Explorer Visit the Shelby Explorer at: [https://explorer.shelby.xyz](https://explorer.shelby.xyz) # Storage Provider Telemetry (/tools/telemetry) import { Callout } from 'fumadocs-ui/components/callout'; When you run a Shelby storage provider (Cavalier), your node sends telemetry data to help monitor network health and performance. Telemetry is **enabled by default**. **No personal information is collected.** Shelby telemetry only collects software metrics like storage utilization, performance data, and the IP address of your storage provider. ## What is Collected * **Storage metrics** — Blob counts, storage utilization, read/write operations, request latency * **System metrics** — CPU, memory, disk I/O, network connections * **Identification** — Storage provider address and chain ID ## Disabling Telemetry **Disabling telemetry is not recommended.** It makes it harder to identify and debug issues affecting your storage provider. ```toml [telemetry_config] enabled = 0 ``` ## Troubleshooting ### Telemetry not sending 1. Verify `enabled = 1` in config 2. Ensure network access to the telemetry endpoint 3. Check logs for errors # Aptos Tokens (/apis/faucet/aptos) import AptosFaucet from "@/components/faucet/AptosFaucet"; This faucet is used to fund accounts with APT tokens for development purposes. APT tokens are used to pay gas fees when uploading blobs to the Aptos network. # ShelbyUSD Tokens (/apis/faucet/shelbyusd) import ShelbyUSDFaucet from "@/components/faucet/ShelbyUSDFaucet"; This faucet is used to fund accounts with ShelbyUSD tokens for development purposes. ShelbyUSD tokens are used to pay upload fees when uploading blobs to the Shelby network. # Shelby RPC API (/apis/rpc) Choose your network to view the appropriate API documentation: # Networks (/protocol/architecture/networks) import { NetworkEndpoint } from '@/components/NetworkEndpoint'; This page is a reference guide for the network urls and capabilities of the available networks. To actually use the Shelby network, please follow the [Quickstart](/protocol/quickstart). # testnet TBD ## Limits and Capabilities TBD ## `testnet` component URLs and Addresses | Component | URL | | --------------- | ------------------------------------------------------------------ | | Indexer | | | Shelby RPC | | | Aptos Full Node | | The [Shelby Smart Contract](/protocol/architecture/smart-contracts) is deployed by account `0x85fdb9a176ab8ef1d9d9c1b60d60b3924f0800ac1de1cc2085fb0b8bb4988e6a`. See [Aptos Explorer](https://explorer.aptoslabs.com/account/0x85fdb9a176ab8ef1d9d9c1b60d60b3924f0800ac1de1cc2085fb0b8bb4988e6a?network=testnet) to explore this account. ## API Keys Learn about API keys [here](/sdks/typescript/acquire-api-keys). ## Shelby RPC server TBD ## Storage Providers TBD *** # shelbynet Shelby currently operates a single developer prototype network called `shelbynet`. `shelbynet` will be wiped roughly once a week, or faster. To support `shelbynet`, a new set of Aptos validators is running as well, under the network name of `shelbynet`. Use `shelbynet` as the network name when interacting with Aptos Explorer. This network is isolated from the Aptos mainnet, Aptos testnet, and Aptos devnet. ## Limits and Capabilities Storage Capacity: approximately 10 TiB. ## `shelbynet` component URLs and Addresses | Component | URL | | --------------- | ------------------------------------------------------------------------------------------ | | Indexer | [https://api.shelbynet.shelby.xyz/v1/graphql](https://api.shelbynet.shelby.xyz/v1/graphql) | | Shelby RPC | [https://api.shelbynet.shelby.xyz/shelby](https://api.shelbynet.shelby.xyz/shelby) | | Aptos Full Node | [https://api.shelbynet.shelby.xyz/v1](https://api.shelbynet.shelby.xyz/v1) | The [Shelby Smart Contract](/protocol/architecture/smart-contracts) is deployed by account `0x85fdb9a176ab8ef1d9d9c1b60d60b3924f0800ac1de1cc2085fb0b8bb4988e6a`. See [Aptos Explorer](https://explorer.aptoslabs.com/account/0x85fdb9a176ab8ef1d9d9c1b60d60b3924f0800ac1de1cc2085fb0b8bb4988e6a?network=shelbynet) to explore this account. ## API Keys Learn about API keys [here](/sdks/typescript/acquire-api-keys). ## Shelby RPC server A single RPC server is running for Shelby devnet. This server is running in a cloud environment. We use a private network configured in the cloud environment to conect the RPC server to the storage providers. The RPC server is routed to public internet using Geomi, accessible using the URL above. ## Storage Providers For Shelby devnet, we are running 16 storage providers in a cloud environment. These providers are running in a single region, with a private network connecting them. Each Storage Provider has 1 TiB of disk. # Overview (/protocol/architecture/overview) import { Mermaid } from "@/components/mdx/Mermaid"; # Key Components The Shelby system consists of the following major components: 1. **Aptos Smart Contract** - manages the state of the system and manages correctness-critical operations, such as auditing for data correctness. 2. **Storage Provider (SP) servers** - storage servers which store chunks of user data. 3. **Shelby RPC servers** - used by end users to access stored blobs. 4. **Private Network** - Private fiber network used for internal communication. Users connect to Shelby by using the SDK to access a Shelby RPC server, over the public internet. Shelby RPC servers have both public internet connectivity and private network connectivity. The RPC servers will reach Storage Provider servers using this private bandwidth to satisfy user requests. All actors in the network have access to the Aptos L1 blockchain, which includes the Shelby smart contract. Each participant interacts with the smart contract to coordinate their actions. # Accounts and Blob Naming A user's blobs are stored in a user-specific namespace. Shelby uses the hex representation of the Aptos account which is interacting with Shelby as the user namespace. A single user can create as many accounts as they want or need. Blob names are user-defined, and must be unique within the user's namespace. A fully qualified blob name is something like `0x123..../user/defined/path/goes/here.dat`. Blob names can be up to 1024 characters, and must not end in `/`. There are no directories in the system. Accounts only hold blob data. Note that it is possible to create both `/foo` as a blob and `/foo/bar`. The [CLI](../tools/cli) and other tools follow a "canonical" directory layout when uploading or downloading local directory-like structures recursively. That is, if an input directory is laid out as: ``` $ tree . . |-- bar `-- foo |-- baz `-- buzz 2 directories, 3 files ``` The blobs uploaded to Shelby will have these blob names: ``` //bar //foo/baz //foo/buzz ``` Users are still free to later upload `//foo` as a separate object. However, this violates the canonical structure and would prevent standard tools from being able to recursively download the collection as a single directory. # Chunking User data is chunked according to an erasure coding scheme in Shelby. Erasure coding allows the system to recover from data loss without storing a large number of extra copies of user data in the system. Shelby uses [Clay Codes](https://www.usenix.org/system/files/conference/fast18/fast18-vajha.pdf) as its erasure coding scheme. User data (blobs) are first split into fixed size "chunksets". These chunksets are then erasure coded into "chunks". If a user's data size is not a multiple of the chunkset size, the SDK will fill the last chunk with zeros to ensure that chunks are of the appropriate size throughout the system. Reads will never return this zero padding, but it will exist internally within the system. At this time, chunksets are 10 megabytes of user data and each chunk is 1 megabyte. Each chunkset contains 16 total chunks in our erasure coding scheme. The first 10 of these chunks are the original user data. The remaining 6 chunks are "parity" chunks, which contain the data needed to recover chunks lost due to disk or node failure. To read/rebuild a block of data, we need to gather 10 chunks of the chunkset. These can be the 10 data chunks (as these are the original user data chunks), or any 10 chunks combination of data+parity chunks. Shelby can also use the Clay Code bandwidth-optimized repair algorithm to recover chunks lost due to disk or node failure. This algorithm provides a mechanism to read a chunk from a chunkset without fetching 10 full chunks worth of data. Instead it will read a much smaller portion of data from a larger number of servers. This optimized repair algorithm can reduce the network traffic during recovery by as much as 4x compared to standard Reed-Solomon erasure coding while preserving the same optimal storage footprint. # Placement Groups Shelby uses placement groups to efficiently manage where data is stored across the network without requiring massive metadata overhead. Placement groups also allow us to control for data locality, or specify availability zones, in a flexible manner. Instead of tracking the location of every individual chunk, Shelby assigns each blob to a placement group and stores all of that blob's chunks on the same set of storage providers. This dramatically reduces the amount of metadata that must be stored on the Aptos blockchain. When you store a blob, the Shelby system: 1. Randomly assigns the blob to one of many placement groups for load balancing and data availability 2. Stores all chunks of the blob across the 16 storage providers in that placement group Each placement group contains exactly 16 slots for storage providers, matching the erasure coding scheme. All chunks from a blob—both the 10 data chunks and 6 parity chunks—are stored on the same set of 16 storage providers. To read data, the RPC server performing the read looks up which placement group contains the desired blob, then retrieves the chunks from the 16 storage providers assigned to that placement group. For another example of Placement Group usage, see the [Ceph RADOS paper](https://ceph.com/assets/pdfs/weil-rados-pdsw07.pdf). # Read Procedure The following describes what happens in the system when clients request to read data from Shelby: 1. The client selects an available RPC server from the network. 2. The client establishes a payment mechanism and session with the selected RPC server. 3. The client sends HTTP requests to the RPC server specifying the desired blob or byte range, along with payment authorization. 4. (optionally) The RPC server consults a local cache and returns data from local cache if present. 5. The RPC server queries the smart contract to identify which storage providers hold the chunks for the requested blob. 6. The RPC server retrieves the necessary chunks from the storage providers over the DoubleZero private network, using a micropayment channel managed by the smart contract to pay the storage provider for the read. 7. The RPC server validates the chunks against the blob metadata, reassembles the requested data, and returns it to the client. 8. The client can perform additional reads using the same session, with payments deducted incrementally as data is transferred. # Write Procedure The following describes what happens in the system when clients request to write data to Shelby: 1. The client selects an available RPC server from the network. 2. The SDK computes the erasure coded version of the blob locally, processing chunk-by-chunk to minimize memory usage. 3. The SDK calculates cryptographic commitments for each erasure coded chunk. 4. The SDK submits a transaction to the Aptos blockchain containing the blob's metadata and merkle root of chunk commitments. Storage payment is processed on-chain at this point. 5. The SDK transmits the original, non-erasure-coded data to the RPC server to conserve bandwidth. 6. The Shelby RPC server independently erasure codes the received data and recomputes chunk commitments to verify consistency with the on-chain metadata. 7. The RPC server validates that its computed values match the on-chain state. 8. The RPC server distributes the erasure coded chunks to the assigned storage providers based on the blob's placement group. 9. Each storage provider validates its received chunk and returns a signed acknowledgment. 10. The RPC server aggregates the acknowledgments from all storage providers and submits a final transaction to the smart contract. 11. The smart contract transitions the blob to "written" state, confirming it is durably stored and available for reads. # RPCs (/protocol/architecture/rpcs) The Shelby Protocol uses RPC servers as the primary user-facing protocol access layer. A reference implementation, in typescript, of the RPC server exposes a straightforward blob read/write interface, handles Aptos L1 interactions, and interacts with Storage Providers on behalf of end users. The client SDK works natively with the reference implementation endpoints. When reading blobs, it is also possible to interact directly with the HTTP endpoints in situations where it would be complicated to use the SDK. We anticipate value-add RPC services which will build off additional features for specialized usage on top of the reference implementation for example, RPC servers could transform data, process data, cache data, etc. # RPC Server Responsibilities The RPC server exposes friendly HTTP REST APIs to read and write data to and from Storage Provider servers. RPC servers will have a user-facing network interface(s) (typically the public internet) and private network facing network interface(s). ### Core Features * HTTP endpoints: RESTful blob storage APIs with support for standard operations, range requests, and multipart uploads. * Provide user-friendly payment mechanisms and session management. * Storage Provider connection management: Keep these connections healthy and functioning well, gracefully handle loss of storage providers. * Erasure coding and commitment calculations: Done as part of the read and write workflows. * Blockchain integration: Interact with Aptos L1 to inspect blob/chunk state and carry out operations. ### Reading Data The interface exposed for reading is designed to be flexible and composable with other systems. The RPC supports plain `HTTP GET` to fetch entire blobs as well as range requests for advanced usage and concurrent downloads. See the full read path detailed in the [Overview](./overview.mdx) for interactions between components. The RPC server provides graceful degradation when Storage Providers are unavailable, automatically falling back to parity chunks and providing clear error responses when operations cannot be completed. #### Request Hedging Future implementations will include request hedging, which is over-requesting data from Storage Providers and using the first set of valid responses received to reply to the end user. This technique is particularly effective in distributed systems where tail latency can be highly variable. For example, if we need 10 of 16 chunks in our chunkset to answer the user, we can request 14 chunks and use the first 10 full replies to reply at lower client-facing latency. Careful network congestion management and traffic prioritization is required for this technique to be effective. ### Writing Data To write data, the client SDK is the primary interface. The client SDK interacts with the RPC server to perform these writes. See the full write path as detailed in the [Overview](./overview.mdx) for full system interaction. Internal to the SDK and RPC, we support both HTTP PUT and multi-part uploads, which are helpful when large files need to be uploaded over potentially flaky connections (parts can be retried). # Performance Notes The RPC server implementation prioritizes performance through several key architectural decisions that minimize latency, reduce memory usage, and maximize throughput under varying load conditions. ## Streaming Data Pipeline As clients upload blobs, the RPC server begins processing data immediately as it arrives, rather than waiting for complete uploads. The data path uses streams, ensuring that data flows through the system without large buffers and latency bubbles. When data is transformed (i.e. erasure coded), it is done so as a part of this data path in small streaming chunks. This approach provides several benefits: * Reduced time-to-first-byte. * Constant memory usage per connection, allowing a higher connection count and high levels of concurrency. ## Connection Pooling and Reuse Storage Provider connections are maintained in a connection pool. These connections are able to be reused for many concurrent requests through request tracking mechanisms in the protocol. Pooling the connections keeps latency low (no time spent establishing connections) keeps flow control state fresh (no cold start ramp up period as the protocol estimates channel properties). ## Resource Management Other techniques are in use to control resource usage, including: * Bounded queues: Connection pools and processing queues have fixed capacities to prevent memory exhaustion during traffic spikes. * Backpressure handling: When Storage Providers or network connections become congested, the system applies backpressure up the chain rather than buffering unlimited data. * Garbage collection: Sessions, pending uploads, and cached metadata are automatically expired to prevent resource leaks during long-running operations. ## Scalability To scale horizontally, the reference RPC server public interfaces and internals are mostly stateless. Session management requires some database state, currently handled by local persistent databases. All other portions of read and write requests supported by the RPC server are stateless. This allows easy horizontal scaling, more instances can be added and requests sharded/load-balanced across them without much trouble. Because the small amount of persistent session state is carefully managed, these RPC servers can also be started, stopped, and restarted seamlessly. # Monitoring, Observability, and Operational Considerations Every request receives a unique correlation ID that flows through all system components, enabling distributed tracing of complex operations that span multiple Storage Providers and blockchain interactions. This tracing capability is essential for debugging performance issues and understanding system behavior under load. The RPC server exposes operational metrics including request latency, Storage Provider connection health, throughput statistics, and error rates through standard monitoring interfaces. # Smart Contracts (/protocol/architecture/smart-contracts) Shelby's coordination and settlement processes are underpinned by Aptos smart contracts that act as the system's single source of truth. All critical state -- including storage commitments, audit outcomes, micropayment channel metadata, and system participation -- is managed and updated on-chain through these contracts. Maintaining a single on-chain state streamlines interactions among system components (Storage Providers, RPCs, SDKs). ## Blob Metadata The [overview](/protocol/architecture/overview) shows how the system interacts with the smart contract during the read and write of blobs. Each of these flows interacts with the blob metadata stored in the smart contract. ### Write Path Metadata for a blob is initialized when the SDK submits the transaction to register the blob. The user provides the blob and its name to the SDK. The SDK computes then sends a cryptographic blob commitment, along with payment and encoding information, to the smart contract as a signed Aptos transaction. (The construction of the blob commitment allows both simple verification of chunk contents by storage providers and small proofs during audits.) The smart contract executes the transaction: it takes payment of the write according to the current prices of the blob size and length of expiration; it assigns the blob to a placement group, which defines the set of storage providers that will store erasure coded chunks of the blob. The metadata is then updated when storage providers store data. Once a storage provider has stored a chunk, it produces a signed acknowledgement of the stored chunk and sends this to the RPC server, which aggregates and adds the acknowledgements on-chain. (If the RPC is unresponsive, the storage provider can send the acknowledgement directly on-chain.) When enough acknowledgements are registered with the smart contract, it transitions the blob to "written" state, confirming it is durably stored and available for reads. ### Read Path On the read path, the blob metadata is accessed by the SDK and RPC by directly reading the smart contract state or reading derived information via an indexer. The read path does not require updates to the on-chain state, which is necessary for low latency and high throughput. ## Micropayments Shelby deploys a micropayment channel smart contract, which is used by RPC servers to pay storage provider for reads. The micropayment channel only requires an on-chain update for creation and settlement, and all intermediate payments are guaranteed by the sender's signature, which the receiver can settle in bulk on-chain. These optimistic payments allow the read requests to occur fast without on-chain overheads. ## System participation The smart contract manages the set of storage providers, placement groups, and the mapping of storage providers to placement group slots. Storage providers join and leave the system by submitting transactions to the smart contract; when executed the smart contract updates the placement group slots to reflect the new set of providers. ## Audits Data is periodically audited within Shelby, to reward storage providers for storing data, and to punish any storage provider that reneges on their storage promises. When the blob is registered, the write payment is deposited into the smart contract. Only storage providers that have acknowledged writes are paid the write payments, at regular audit intervals. If a storage provider claims to have written a blob but cannot produce a succinct proof of that write to a smart contract audit, it is penalized. More details on the formalation of audits are available in the [whitepaper](/protocol/architecture/white-paper). # Storage Providers (/protocol/architecture/storage-providers) import { Mermaid } from "@/components/mdx/Mermaid"; The Storage Provider nodes store data for the Shelby network. ## Cavalier Cavalier is the Jump Crypto reference implementation of the protocol's Storage Provider. The Cavalier client is a high performance C codebase, written using utilities open sourced as part of the [firedancer](https://github.com/firedancer-io/firedancer) project. ## Tiles Modern CPUs contain large numbers of cores. These cores communicate over local networks, and their caches use sophisticated algorithms to coordinate across the cache hierarchy[^chips]. The relationship between local cpu caches, local-socket memory, and remote cpu caches, must be carefully managed by programmers to have any hope of extracting high performance from these systems. [^chips]: See [AMD's CCX architecture](https://chipsandcheese.com/p/pushing-amds-infinity-fabric-to-its) Multi-threaded applications struggle with this complexity, leaving performance on the table, or worse, experiencing unpredictable performance due to cache conflicts and NUMA penalties. Cavalier leverages a "tile" architecture in which application components run in isolated processes, on dedicated CPU cores, and communicate through shared memory, to avoid these struggles. Building the system around communicating tiles means: * Explicit Communication: Moving data between cores is explicit, avoiding performance surprises. * Resource Locality: A tile controls its core's caches. Explicit core scheduling prevents interference by other processes and threads. * Isolation: Tile state is isolated. Security-sensitive tiles can run in sandboxes; hiding their sensitive state from the rest of the system. The tile model echoes other popular programming approaches seen in languages like erlang and go (independent actors communicating over channels), and frameworks like [Seastar](https://seastar.io/) which embrace the shared-nothing-with-message-passing pattern. ## Workspaces Cavalier uses a shared memory management concept called a "workspace." Workspaces implement the shared memory infrastructure that enables tile communication. A workspace is a section of shared memory that is (usually) backed by huge pages (for TLB efficiency) and is created in a cpu topology aware manner. A workspace can hold sections of application state, shared memory queues, shared memory data buffers, and even dynamic memory allocators (which allocate within a section of the workspace). Workspaces hold information about their layout, allowing debugging tools to inspect state and (where appropriate) allowing tiles to hold persistent state in their workspace across application restarts. # Kinds of Tiles Tile-architecture applications tend to end up with a few different kinds of tiles: * Application Logic: Implement core business logic and track some local state. * State Management: Own application wide databases. Respond to queries over shared memory, and often persist key data in their workspaces. * Hardware Interfacing: Manage network cards, network connections, disks, etc. Implement kernel bypass networking primitives, perform networking stack work, multiplex across many hard drives, etc. * Message routing and multiplexing: Load balance across shared application or state tiles, join queues with robust and consistent ordering, etc. The pattern does not impose an event model for tiles, each can use an appropriate event model for their purpose. For example a tile managing a large number of TCP sessions may want an event-driven epoll-style loop, but something managing hardware might want to spin while polling hardware registers. # Cavalier Tiles Cavalier creates a small number of tiles (potentially many copies of each): 1. Server tile(s) - Manage communication with RPC nodes. 2. Engine tile - Manage access to local drives. 3. Client tile - Connect to Aptos services to gather L1 state and provide SP-local access to relevant state. 4. Rebuild tile - Recover lost chunks using Clay erasure codes. 5. Sign tile - Generate signatures for chunk acknowledgments and Aptos transactions. Cavalier at current size has a minimal footprint, but as the system scales up, we are designing components such that scaling is possible. For example: * As RPC connection requirements grow, we can add more server tiles. * As local metadata grows in size, we can shard access across multiple Aptos client tiles. ## Engine Tile (Drive Interaction) The engine tile manages all physical storage operations using `io_uring` for high-performance asynchronous I/O across multiple drives. We maintain separate read and write queues (of fixed depths) per drive, allowing fine-tuned control over I/O concurrency based on each drive's characteristics. The hard drives have minimal partition tables, and we use direct I/O to bypass page cache, providing predictable performance. The tile listens for I/O requests from other tiles and enqueues requests to the drives (assuming the drive queues are not exceeded). As I/O completions occur, responses are sent back to the server tile through shared memory queues. ## Server Tile The server tile implements a single-threaded event-driven `epoll` style event loop to manage multiple concurrent TCP connections. The tile communicates with both the client and engine tiles through shared memory queues, forwarding metadata validation requests to the client tile and actual I/O operations to the engine tile. The tile tracks each connected client (RPC node), and runs a lightweight protocol using protobufs for communication. Each connection maintains dedicated incoming and outgoing buffers, with ring buffers used for outgoing data to gracefully handle partial writes when the network becomes congested, and some backpressure managment business logic. ## Client Tile The client tile streams events via gRPC from an Aptos Indexer to fetch L1 state, then maintains a database of blobs and chunks in the system. The tile responds to metadata requests from the other tiles, acting like a local database. The client tile also runs audit and rebuild state machines to verify chunk integrity and recover lost data. Additionally, as blobs relevant to this Storage Provider expire, the client tile will inform the engine tile that chunks are ready for deletion. {/*## Operator Notes? */} # Token Economics (/protocol/architecture/token-economics) # The Economics of Shelby *Full Tokenomics of the Shelby protocol, including initial distribution, will be published later.* ## Overview The Shelby network uses both a native token and stablecoins: * **Shelby token** or **Stablecoins** for user payments * **Shelby token** as an internal unit for staking, rewards, and governance *** ## Payment and Conversion Flow 1. **User Payments** * Users pay for storage and read operations in stablecoins or Shelby token. * User read fees are routed directly to the relevant RPC Operators. RPCs use paymentchannels for reading data from SPs. * User storage fees are routed through the protocol on chain. 2. **Conversion to native tokens** * The protocol programmatically converts those storage fees that were paid in stablecoins into Shelby tokens on the open market. * Accumulated Shelby tokens form the source of rewards to Storage Providers (SPs). 3. **Reward Release** * SPs are rewarded in the native token. * Rewards are released gradually and are conditioned on successful audit outcomes. * The release schedule ensures storage integrity and aligns rewards with verified service. 4. **Burns** * A predefined fraction of each conversion is permanently burned. *** ## Staking and Roles ### Storage Providers (SPs) * Must stake the native token to operate and earn rewards. * Storage tasks and reward eligibility are proportional to the individual stake. ### RPC Operators * Stake to access additional benefits (e.g., pricing tiers and read routing). ### Delegators * Can delegate native tokens to SPs or RPC Operators to share in their rewards. *** ## Protocol Fund The **Protocol Fund** is governed by the community. **Sources:** * A predefined fraction of collected usage fees. * Portion of genesis allocation. **Uses (among others):** * Delegation to high-performing operators. * Retroactive public goods funding (RetroPGF). * Ongoing protocol development and maintenance. *** ## Supply and Emissions * Total token supply is capped. (No perpetual inflation.) * Bootstrap rewards for early participants taper over time and are stake-weighted. * Initial allocation with locked releases. * RetroPGF distributions are determined through governance rather than algorithmic issuance. *** Shelby tokenomics structure links storage activity, reward issuance, and supply restrictions within a single closed economic loop. # White Paper (/protocol/architecture/white-paper) import { ExternalLink } from 'lucide-react'; ## Full PDF Versions The Shelby white paper provides an in-depth technical overview of the protocol's architecture, design decisions, and implementation details. ## Abstract Existing decentralized storage protocols fall short of the service required by real-world applications. Their throughput, latency, cost-effectiveness, and availability are insufficient for demanding workloads such as video streaming, large-scale data analytics, or AI training. As a result, Web3 data-intensive applications are predominantly dependent on centralized infrastructure. Shelby is a high-performance decentralized storage protocol designed to meet demanding needs. It achieves fast, reliable access to large volumes of data while preserving decentralization guarantees. The architecture reflects lessons from Web2 systems: it separates control and data planes, uses erasure coding with low replication overhead and minimal repair bandwidth, and operates over a dedicated backbone connecting RPC and storage nodes. Reads are paid, which incentivizes good performance. Shelby also introduces a novel auditing protocol that provides strong cryptoeconomic guarantees without compromising performance, a common limitation of other decentralized solutions. The result is a decentralized system that brings Web2-grade performance to production-scale, read-intensive Web3 applications. # Cavalier Setup (/protocol/node-setup/cavalier) These instructions: * are valid only for Linux x64 * assume familiarity with `udev`, `bash`, `systemd` * assume you have sufficient capabilities, or the ability to escalate to root * assume you already have obtained `shelby-sp-operator-source-bundle-${CAV_COMMIT}.zip` ## Unpack the Bundle ```sh set -euxo pipefail export CAV_COMMIT="LATEST_VERSION_SHA" unzip shelby-sp-operator-source-bundle-${CAV_COMMIT}.zip tar -xzf shelby-sp-operator-source-bundle-${CAV_COMMIT}.tar.gz ``` ## Install Dependencies to Build from Source Refer to `deps.sh` for the up-to-date list of things to install. ### Ubuntu/Debian ```sh apt install unzip git build-essential pkgconf autoconf automake autopoint bison flex gettext libtool cmake python3 python3-pip python3-venv ``` ### RHEL ```sh dnf install unzip git make pkgconf gcc gcc-c++ perl autoconf automake bison flex gettext-devel libtool cmake python3 ``` ## Build the Binaries Build the `cavalier` binary and other necessary executables. ```bash set -euxo pipefail ./shelby/deps.sh fetch check install |& tee cav_deps.log make -C shelby -j bin |& tee cav_build.log export PATH="$PATH:$(readlink -f shelby/build/native/gcc/bin)" ``` The binary is built at `shelby/build/native/gcc/bin/cavalier`. If you move the `cavalier` binary, you MUST ensure that `hugetlbfs.py` and `mountfs.py` (in the same directory) are alongside the `cavalier` binary. It's RECOMMENDED that the `cavalier` binary is built on the machine it runs on, to avoid issues with CPU instruction discrepancies. ## One Time Setup These instructions should only be used when setting up a new node. Some steps are destructive to stored data. They should not be run once you have joined a Shelby network and are storing data. ### Direct Disk Access Decide if you wish to give `cavalier` direct access to disks. This is optional, but may improve performance. Ensure: * the user that will run `cavalier` has access to the block device * the block device you want to use is in `/dev/disk/by-id` (or some other stable identifier) This option WILL cause data on chosen disks to be WIPED! In the config file (described later on), you will set `storage_config.storages` to point to the block devices accordingly. e.g., ```toml title='storage_snippet.toml' [storage_config] storages = [ "/dev/disk/by-id/nvme-nvme.01de-7373746f72616765312d6469736b2d32--00000001" ] ``` #### Via `udev` One way of achieving this is by modifying your `udev` rules. Populate the `{{variables}}` in the following udev snippet with appropriate values from your environment. You can use `udevadm info` to retrieve the `wwid`/`ID_WWN` of your disk (`udevadm info -a -n /dev/nvme3n1 | grep wwid`). ```text title='/usr/lib/udev/rules.d/70-cavalier-disk-ownership-{{wwid}}.rules' SUBSYSTEM=="block", ATTRS{wwid}=="{{wwid}}", OWNER="{{cav_user}}", TAG+="uaccess" ``` Bounce `udevadm` to apply changes. ```sh udevadm control --reload-rules udevadm trigger ``` If you run `cavalier` as root you can skip the `udev` updates. ### Make an Aptos Account Download + install the Aptos CLI: [https://aptos.dev/build/cli/install-cli/install-cli-linux](https://aptos.dev/build/cli/install-cli/install-cli-linux) Run ```console $ aptos init --profile cavtest Configuring for profile cavtest Choose network from [devnet, testnet, mainnet, local, custom | defaults to devnet] testnet Enter your private key as a hex literal (0x...) [Current: None | No input: Generate new key (or keep one if present)] No key given, generating key... --- Aptos CLI is now set up for account 0x00000000000 as profile cavtest! --- The account has not been funded on chain yet. To fund the account and get APT on testnet you must visit https://aptos.dev/network/faucet?address=0x00000000000 ``` This creates a profile for your `cavalier` instance. ### Generate/Retrieve Necessary Keys #### ED25519 Cavalier needs your Aptos private key as a standalone file. ```sh grep private_key .aptos/config.yaml | cut -f3 -d- > cav_privkey ``` `realpath cav_privkey` will be the value of `signature_config.aptos_priv_key_path` in the config file. Retrieve the account address as well. ```sh echo "0x$(grep account .aptos/config.yaml | tr -d ' ' | cut -f2 -d:)" ``` You will need the account address for `client_config.storage_provider` in the config file. #### BLS12-381 You will also need to generate a BLS12-381 keypair with the following command. The pubkey will be compressed and in G2. ```sh python3 -m venv .venv source ./.venv/bin/activate pip install py-ecc python3 shelby/move/scripts/generate_bls_keypairs.py generate --count 1 --filename cav_bls ``` `realpath cav_bls_privkey` will be the value of `signature_config.priv_key_path` in the config file. #### Geomi API Key Follow most of the instructions from [https://aptos.dev/build/guides/build-e2e-dapp#setup-api-key](https://aptos.dev/build/guides/build-e2e-dapp#setup-api-key) to get a Geomi API key. You'll need to make a couple changes: * Set the network to `testnet` * Disable `Client usage` (may be disabled by default) Inform us what organization/email you used during this sign up. ### Config Write the config file. You WILL need to modify some of these values to match your environment. You can `grep PROVIDE_THIS` for values that need updating. Refer to previous sections of this document for values that were generated/computed. ```toml title='testnet.toml' name = "Cavalier, a Shelby Storage Engine by Jump Crypto" # The user that will be used to run Cavalier. The hugetlbfs must be writeable by # this user. username = "YOU_PROVIDE_THIS" tiles = ["SP_SERVER", "SP_ENGINE", "SP_CLIENT", "SP_SIGN", "SP_REBUILD"] # All memory that will be used in Cavalier is pre-allocated in huge pages which # are 2 MiB. This is done to prevent TLB misses which can have a high # performance cost. # # A typical layout of the mounts looks like so: # # /mnt/.cavalier [Mount parent directory specified below] # +-- .huge [Files created in this mount use 2 MiB pages] # +-- scratch1.wksp # +-- scratch2.wksp [hugetlbfs] # The absolute path to a directory in the file system. Cavalier # will mount the hugetlbfs file system for gigantic pages at a # subdirectory named .gigantic under this path, or if the entire # path already exists, will use it as-is. Cavalier will also # mount the hugetlbfs file system for huge pages at a subdirectory # named .huge under this path, or if the entire path already exists, # will use it as-is. If the mount already exists it should be # writable by the Cavalier user. mount_path = "/mnt/.cavalier" # SP server config [server_config] # cavalier listens on this port. this value must match how you register with # the smart contract. port = 39431 max_conns = 40 max_requests_in_flight = 20 # Example of how to restrict the server to listening on a single interface # Note: Only accepts IPv4 at the moment # interface = "64.130.58.54" # metadata checks enable_md_checks = 1 # This only makes sense to enable if the above is enabled. enable_chunk_proofs = 1 # SP client config [client_config] rest_endpoint_url = "WE_PROVIDE_THIS" grpc_endpoint_host = "WE_PROVIDE_THIS" grpc_endpoint_port = "WE PROVIDE THIS" # an integer, so remove the quotes. grpc_use_tls = "WE PROVIDE THIS" # an integer, so remove the quotes. grpc_starting_version = "WE PROVIDE THIS" # an integer, so remove the quotes. # the aptos account address (NOT pubkey and NOT privkey) from the # `Generate/Retrieve Necessary Keys` section above storage_provider = "YOU_PROVIDE_THIS" # get an API key from geomi api_key = "YOU_PROVIDE_THIS" # SP engine config [engine_config] # Path to the file or block device (but NOT a directory) to store engine state. # We recommend a disk or file on a device/filesystem that can: # * maintain high IOPS, # * is separated from your OS or cavalier's storage, and # * has journaling capabilities engine_state = "YOU_PROVIDE_THIS" max_read_per_storage = 2 max_write_per_storage = 2 # SP storage config [storage_config] # *Total* user data storage capacity, in MiB, for this *entire SP*. storage_capacity_MiB = "YOU_PROVIDE_THIS" # an integer, so remove the quotes. # List of storages to be used by this SP. User data will be written here. # # Each storage can be either a block device or a regular file, but NOT a # directory. # # If a storage is a block device, it will be *wiped*, re-partitioned and # formatted with cavalier internal content at time of initialization. # Only raw block device paths (i.e. /dev/sda) should be provided in this # config, not partition paths (e.g. /dev/sda1) or other variants. # # If a storage is a regular file, it will be *wiped*, re-sized # against configured capacity and formatted with cavalier internal # content at time of initialization. # # If no entity exists at provided path, cavalier will attempt to create a # regular file at that location and follow regular file behavior. storages = ["YOU_PROVIDE_THIS"] # SP signature config [signature_config] # Full path to file containing BLS private key (hex string with 0x prefix) for # signing acknowledgments. Refer to Generate/Retrieve Necessary Keys section # above. priv_key_path = "YOU_PROVIDE_THIS" # Full path to file containing Aptos private key (hex string with 0x prefix) for # signing transactions. Refer to Generate/Retrieve Necessary Keys section above. aptos_priv_key_path = "YOU_PROVIDE_THIS" ``` ### Initialize Cavalier Run the `init` subcommand. This does the following: * wipes + initializes disks, if configured to do so * prepares workspaces ```sh sudo shelby/build/native/gcc/bin/cavalier --config testnet.toml init ``` ## Register the SP Before registration, ensure your account is funded with APT for gas fees. You can get testnet APT from the [faucet](https://aptos.dev/network/faucet). ### Using the Registration Script (Recommended) The `sp_register.py` script in the [`sp_operator/`](https://github.com/AptosLabs/shelby/tree/main/sp_operator) directory automates registration in **two phases**: * **Phase 1** — Initialize your storage provider on-chain and set up payment channels. Run this first. * **Phase 2** — Activate your placement group slot(s). Run this only after the Shelby team has assigned your SP to a placement group and asked you to complete registration. ```sh # Create a config file cat > my_sp_config.json << 'EOF' { "sp_index": 0, "sp_ed25519_key": "ed25519-priv-0xYOUR_PRIVATE_KEY", "sp_bls_public_key": "YOUR_BLS_PUBLIC_KEY_HEX", "sp_ip_address": "YOUR_PUBLIC_IP", "sp_port": 39431, "stake": 10000 } EOF # Phase 1: initialize SP and payment channels python3 sp_operator/sp_register.py --network testnet --config-file my_sp_config.json --phase 1 ``` After phase 1 completes, tell the Shelby team that you are ready to have your SP registered on-chain and provide your SP address (your Aptos account address). Once they have assigned your SP to a placement group, they will ask you to run phase 2: ```sh # Phase 2: activate placement group slot(s) python3 sp_operator/sp_register.py --network testnet --config-file my_sp_config.json --phase 2 ``` Phase 2 will find and activate your slot(s) automatically. To activate a specific slot, you can pass `--pg-address` and `--slot-index` (or add `pg_address` and `slot_index` to your config file). The `sp_index` is just a local identifier for organizing your CLI profiles — you can use any number. ### Manual Registration Alternatively, you can run the registration commands manually: ```sh export micropayments_deployer="WE_PROVIDE_THIS" export shelby_contract_deployer="WE_PROVIDE_THIS" # the following will be how other cavaliers talk to you. if you're doublezero'd: export my_ipv4_addr="$(doublezero status --json | jq -r '.[0].response.doublezero_ip')" # otherwise, you may need to talk over the public internet # export my_ipv4_addr="$(curl ifconfig.me)" # initialize payment channels aptos move run --profile cavtest --function-id "${micropayments_deployer}::micropayments::initialize_payment_channels" --assume-yes # initialize storage provider aptos move run --profile cavtest --function-id "${shelby_contract_deployer}::storage_provider::initialize_storage_provider" --args hex:$(cat cav_bls_pubkey) string:${my_ipv4_addr} u64:39431 --max-gas 100000 --assume-yes ``` ## Poke holes in your firewall Allow inbound 39431/TCP (or whatever you've set `sp_port` to) so that other cavaliers can talk to yours (important for recovery, etc.). ## Run Start `cavalier`. You must be funded. ```sh shelby/build/native/gcc/bin/cavalier --config testnet.toml run ``` ### systemd An example service unit for systemd that starts cavalier and keeps it up: ```ini title='~/.config/systemd/user/cavalier.service' [Unit] Description=Cavalier Bootstrap Client After=network.target [Service] Type=exec ExecStart=shelby/build/native/gcc/bin/cavalier --config testnet.toml run TimeoutStopSec=30 Restart=on-failure [Install] WantedBy=default.target ``` # Nodes and Infrastructure (/protocol/node-setup) # Availability Shelby is currently in Early Access in testnet. Access is public and infrastructure setup is managed. Please reach out to the Shelby team via email or Discord if you are interested in running in a testnet node. # RPC Node (/protocol/node-setup/testnet_rpc) ## Testnet Bandwidth management On testnet, to protect the available network bandwidth which is not infinite, `tc` will be used to limit ingress bandwidth at the RPC node to limit bandwidth across the entire network. First determine the double zero interface name to limit, e.g. and set these variables 1. Setup environment variables based on requirements: ```sh # confirm this matches DZ interface, drop the @ DEV="doublezero0" # name of the IFB (keep this unless already using IFBs) IFB="ifb0" # For a 1 Gbps rate limit use RATE="1gbit" BURST="2m" # For a 2 Gbps rate limit use RATE="2gbit" BURST="4m" # For an MTU of 1500 use QUANTUM=1514 # For an MTU of 1476 (DZ GRE Tunnel) QUANTUM=1490 ``` 2. Clean up any prior state ```sh sudo tc qdisc del dev "$DEV" sudo tc qdisc del dev "$IFB" sudo ip link del "$IFB" ``` 3. Load IFB and create the IFB device ```sh sudo modprobe ifb sudo ip link add "$IFB" type ifb sudo ip link set "$IFB" up ``` 4. Attach ingress hook to the real NIC and redirect TCP (IPv4 + IPv6) to IFB using `flower` and the `clsact` qDisc ```sh sudo tc qdisc add dev "$DEV" clsact # IPv4 TCP -> IFB sudo tc filter add dev "$DEV" ingress protocol ip pref 10 flower ip_proto tcp \ action mirred egress redirect dev "$IFB" # IPv6 TCP -> IFB sudo tc filter add dev "$DEV" ingress protocol ipv6 pref 20 flower ip_proto tcp \ action mirred egress redirect dev "$IFB" ``` This takes incoming TCP packets on `$DEV` and feeds them into `$IFB` egress, where we can shape properly. 3. Shape on IFB to `$RATE` with HTB, explicitly setting quantum and burst/cburst, then `fq_codel` ```sh sudo tc qdisc add dev "$IFB" root handle 1: htb default 10 sudo tc class add dev "$IFB" parent 1: classid 1:10 htb \ rate "$RATE" ceil "$RATE" \ quantum "$QUANTUM" \ burst "$BURST" cburst "$BURST" sudo tc qdisc add dev "$IFB" parent 1:10 handle 10: fq_codel ``` ## Verify it’s active ### Device Ingress ```sh tc filter show dev "$DEV" ingress ``` Example output: ``` filter protocol ip pref 10 flower chain 0 filter protocol ip pref 10 flower chain 0 handle 0x1 eth_type ipv4 ip_proto tcp not_in_hw action order 1: mirred (Egress Redirect to device ifb0) stolen index 1 ref 1 bind 1 filter protocol ipv6 pref 20 flower chain 0 filter protocol ipv6 pref 20 flower chain 0 handle 0x1 eth_type ipv6 ip_proto tcp not_in_hw action order 1: mirred (Egress Redirect to device ifb0) stolen index 2 ref 1 bind 1 ``` * Mirrored redirect to `$IFB`. ### IFB Setup ```sh tc -s qdisc show dev "$IFB" ``` Example output: ``` qdisc htb 1: root refcnt 2 r2q 10 default 0x10 direct_packets_stat 48 direct_qlen 32 Sent 23398 bytes 374 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc fq_codel 10: parent 1:10 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 Sent 19476 bytes 311 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 maxpacket 104 drop_overlimit 0 new_flow_count 2 ecn_mark 0 new_flows_len 0 old_flows_len 1 ``` * clsact on `$DEV` * Counters increasing on IFB qdisc ### IFB Class ```sh tc -s class show dev "$IFB" ``` Example output: ``` class htb 1:10 root leaf 10: prio 0 rate 1Gbit ceil 1Gbit burst 2Mb cburst 2Mb Sent 23204 bytes 371 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 lended: 371 borrowed: 0 giants: 0 tokens: 262133 ctokens: 262133 class fq_codel 10:a parent 10: (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 deficit 508 count 0 lastcount 0 ldelay 1us ``` ### Remove / clean up See (1) ## Notes * This shapes aggregate incoming TCP to 1 Gbps, but fq\_codel gives per-flow fairness (much better than ingress policing). * If you want to shape all ingress (TCP+UDP), remove the protocol match and just redirect all protocol ip (or add separate filters). # Getting Started (/sdks/ethereum-kit) # Ethereum Kit SDK The Ethereum Kit SDK enables Ethereum developers to easily integrate with the Shelby Protocol and leverage decentralized blob storage in their applications. ## Installation npm pnpm yarn bun ```bash npm install @shelby-protocol/ethereum-kit ``` ```bash pnpm add @shelby-protocol/ethereum-kit ``` ```bash yarn add @shelby-protocol/ethereum-kit ``` ```bash bun add @shelby-protocol/ethereum-kit ``` ## Entry Points The SDK provides two entry points optimized for different environments: Server-side applications with direct wallet access Browser applications with wallet connections (wagmi) | Entry Point | Environment | Use Case | | ------------------------------------- | ----------- | -------------------------------------- | | `@shelby-protocol/ethereum-kit/node` | Node.js | Backend services, scripts, CLIs | | `@shelby-protocol/ethereum-kit/react` | Browser | React dApps with wagmi wallet adapters | ## Acquire a Shelby API Key API keys authenticate your app and manage rate limits when using Shelby services. Without one, your client runs in "anonymous" mode with much lower limits, which can affect performance. Follow this guide to obtain your Shelby API key ## Quick Start ### Node.js (Server-Side) For backend services with direct wallet access using ethers.js: ```typescript import { Shelby, Network } from "@shelby-protocol/ethereum-kit/node"; import { Wallet } from "ethers"; // Create a Shelby client const shelbyClient = new Shelby({ network: Network.TESTNET, apiKey: "AG-***", }); // Create a storage account from an Ethereum wallet const ethereumWallet = new Wallet("0x...private_key..."); const storageAccount = shelbyClient.createStorageAccount( ethereumWallet, "my-app.com" ); // Upload data await shelbyClient.upload({ blobData: new Uint8Array([1, 2, 3]), signer: storageAccount, blobName: "example.txt", expirationMicros: Date.now() * 1000 + 86400000000, }); ``` ### React (Browser) For browser dApps with wagmi wallet connections: ```tsx "use client"; import { useStorageAccount, Network } from "@shelby-protocol/ethereum-kit/react"; import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { useWalletClient } from "wagmi"; function MyComponent() { const { data: wallet } = useWalletClient(); const shelbyClient = new ShelbyClient({ network: Network.TESTNET, apiKey: "AG-***", }); const { storageAccountAddress, signAndSubmitTransaction } = useStorageAccount({ client: shelbyClient, wallet, }); return (

Storage Account: {storageAccountAddress?.toString()}

); } ``` Works with RainbowKit, Web3Modal, ConnectKit, and other wagmi-based wallet adapters. See the [React guide](/sdks/ethereum-kit/react) for setup details. ## Key Concepts ### Cross-Chain Identity Shelby uses the Aptos blockchain as a coordination and settlement layer. The Ethereum Kit leverages [Aptos Derivable Account Abstraction](https://github.com/aptos-foundation/AIPs/blob/main/aips/aip-113.md) to create a **Shelby Storage Account** from an Ethereum identity. This enables: * Ethereum wallets to control data on Shelby * Cross-chain signatures (SIWE) managed securely on the Aptos network * Application-level isolation through domain scoping ### Ownership Hierarchy The ownership structure is: 1. **Ethereum Wallet** → Controls the Storage Account 2. **Storage Account** → Owns blobs on Shelby Each storage account has a different address on different dApps due to domain scoping. This maintains isolation at the application level. ## Next Steps Server-side integration with ethers.js Browser integration with wagmi # Overview (/sdks/ethereum-kit/overview) The Shelby Protocol Ethereum Kit SDK is built to bridge the gap between Ethereum and the Shelby Protocol, enabling Ethereum developers to use their existing wallets for decentralized blob storage. ## Architecture The SDK provides two entry points for different environments: ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ Your Ethereum Application │ ├─────────────────────────────────────────────────────────────────────────┤ │ @shelby-protocol/ethereum-kit │ │ │ │ ┌─────────────────────────────┐ ┌─────────────────────────────────┐ │ │ │ /node │ │ /react │ │ │ │ ┌───────────────────────┐ │ │ ┌───────────────────────────┐ │ │ │ │ │ Shelby Client │ │ │ │ useStorageAccount │ │ │ │ │ │ (extends SDK) │ │ │ │ (React Hook) │ │ │ │ │ └───────────────────────┘ │ │ └───────────────────────────┘ │ │ │ │ ┌───────────────────────┐ │ │ │ │ │ │ │ ShelbyStorageAccount │ │ │ Uses viem WalletClient for: │ │ │ │ │ (Wallet → Account) │ │ │ - Address derivation │ │ │ │ └───────────────────────┘ │ │ - Transaction signing │ │ │ └─────────────────────────────┘ └─────────────────────────────────┘ │ ├─────────────────────────────────────────────────────────────────────────┤ │ @shelby-protocol/sdk │ │ ┌───────────────────────────┐ ┌───────────────────────────────────┐ │ │ │ Coordination Layer │ │ RPC Operations │ │ │ │ (Aptos Chain) │ │ (Blob Storage) │ │ │ └───────────────────────────┘ └───────────────────────────────────┘ │ └─────────────────────────────────────────────────────────────────────────┘ ``` ## Entry Points ### Node.js (`/node`) For server-side applications with direct access to Ethereum wallets: * **Shelby Client**: Extends the core SDK with Ethereum-specific initialization * **ShelbyStorageAccount**: Converts an ethers.js `Wallet` into a Shelby signer * **Use cases**: Backend services, scripts, CLIs, automated uploads ```typescript import { Shelby, Network } from "@shelby-protocol/ethereum-kit/node"; ``` Complete guide for server-side integration *** ### React (`/react`) For browser applications with wallet connections: * **useStorageAccount Hook**: Derives storage account from connected wallet * **Wallet Integration**: Works with wagmi's `useWalletClient()` and viem types * **Use cases**: React dApps, Next.js applications, browser-based uploads ```typescript import { useStorageAccount } from "@shelby-protocol/ethereum-kit/react"; import { useWalletClient } from "wagmi"; ``` Complete guide for browser integration *** ## Cross-Chain Flow Both entry points use the same underlying cross-chain mechanism: ### Node.js Flow ``` ┌──────────────┐ ┌──────────────────────┐ ┌─────────────────┐ │ ethers.js │────▶│ ShelbyStorageAccount │────▶│ Shelby Client │ │ Wallet │ │ (Signs with SIWE) │ │ (Uploads blob) │ └──────────────┘ └──────────────────────┘ └─────────────────┘ ``` 1. **ethers.js Wallet** → Provided directly (from private key, env, etc.) 2. **ShelbyStorageAccount** → Signs transactions with SIWE envelope 3. **Shelby Client** → Submits to the coordination layer ### React Flow ``` ┌──────────────┐ ┌──────────────────────┐ ┌─────────────────┐ │ Wallet │────▶│ useStorageAccount │────▶│ Shelby Client │ │ (MetaMask) │ │ (Prompts for sign) │ │ (Uploads blob) │ └──────────────┘ └──────────────────────┘ └─────────────────┘ ``` 1. **Wallet** → User connects via wagmi (RainbowKit, Web3Modal, etc.) 2. **useStorageAccount** → Derives address, prompts user to sign 3. **Shelby Client** → Submits to the coordination layer ## SIWE (Sign-In With Ethereum) Both entry points use the SIWE envelope format for transaction signing: 1. **Transaction Data** → Serialized Aptos transaction 2. **SIWE Message** → Human-readable message wrapping the transaction 3. **Ethereum Signature** → ECDSA signature from Ethereum wallet 4. **Authenticator** → Wrapped signature compatible with Aptos This enables Ethereum identities to control accounts on the Aptos-based Shelby network. ## Common Module Both entry points share common functionality: | Export | Description | | --------- | ------------------------------------------- | | `Network` | Network configuration (currently `TESTNET`) | ## Choosing an Entry Point | Requirement | Use Node.js (`/node`) | Use React (`/react`) | | --------------------------- | --------------------- | -------------------- | | Direct wallet access | ✅ | ❌ | | Wallet popup for signing | ❌ | ✅ | | Server-side execution | ✅ | ❌ | | Browser environment | ❌ | ✅ | | Automated/batch operations | ✅ | ❌ | | User-initiated transactions | ✅ | ✅ | # Media Kit (/sdks/media-kit) import { Cards, Card } from 'fumadocs-ui/components/card'; [Shelby Media Kit](https://media-kit.shelby.xyz) A lightweight React video player for adaptive HLS/DASH streaming from Shelby storage. FFmpeg presets and a declarative builder for transcoding video into HLS + CMAF format. # Getting Started (/sdks/react) # React SDK The Shelby Protocol React SDK provides a comprehensive set of React hooks built on top of [`@tanstack/react-query`](https://tanstack.com/query/latest/docs/framework/react/overview) for interacting with the Shelby Protocol. This reference covers all available query and mutation hooks for blob storage operations. ## Installation npm pnpm yarn bun ```bash npm install @shelby-protocol/react @shelby-protocol/sdk @aptos-labs/ts-sdk @tanstack/react-query ``` ```bash pnpm add @shelby-protocol/react @shelby-protocol/sdk @aptos-labs/ts-sdk @tanstack/react-query ``` ```bash yarn add @shelby-protocol/react @shelby-protocol/sdk @aptos-labs/ts-sdk @tanstack/react-query ``` ```bash bun add @shelby-protocol/react @shelby-protocol/sdk @aptos-labs/ts-sdk @tanstack/react-query ``` ## Prerequisites The React SDK requires the following peer dependencies: * `@shelby-protocol/sdk` - Core SDK for Shelby Protocol * `@aptos-labs/ts-sdk` - Aptos TypeScript SDK * `@tanstack/react-query` - React Query for data fetching and state management * `react` and `react-dom` - React framework ### Optional Dependencies * `@aptos-labs/wallet-adapter-react` - For wallet adapter integration (optional) ## Quick Start ### Setting up React Query First, wrap your application with a `QueryClientProvider` and the `ShelbyClientProvider`. This is the recommended route so hooks can read the client from context without passing it into every call. ```tsx import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; import { ShelbyClientProvider } from "@shelby-protocol/react"; import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { Network } from "@aptos-labs/ts-sdk"; const queryClient = new QueryClient(); // Create a Shelby client instance const shelbyClient = new ShelbyClient({ network: Network.TESTNET }); function App() { return ( {/* Your app components */} ); } ``` ### Basic Query Example Query blob metadata for an account: ```tsx import { useAccountBlobs } from "@shelby-protocol/react"; function BlobList({ account }: { account: string }) { const { data: blobs, isLoading, error } = useAccountBlobs({ account, pagination: { limit: 10, offset: 0 }, }); if (isLoading) return
Loading...
; if (error) return
Error: {error.message}
; return (
    {blobs?.map((blob) => (
  • {blob.name}
  • ))}
); } ``` ### Basic Mutation Example Upload blobs to the Shelby network: ```tsx import { useUploadBlobs } from "@shelby-protocol/react"; import { Account } from "@aptos-labs/ts-sdk"; function UploadBlob() { const uploadBlobs = useUploadBlobs({ onSuccess: () => { console.log("Upload complete!"); }, }); const handleUpload = async () => { const signer = Account.generate(); const fileData = new Uint8Array([/* your file data */]); uploadBlobs.mutate({ signer, blobs: [{ blobName: "example.txt", blobData: fileData }], expirationMicros: Date.now() * 1000 + 86400000000, // 1 day }); }; return ( ); } ``` # Overview (/sdks/react/overview) The Shelby Protocol React SDK is built on top of **React Query** (`@tanstack/react-query`) and the **Shelby SDK** (`@shelby-protocol/sdk`) to provide a seamless React experience for interacting with the Shelby Protocol. ## Core Dependencies ### React Query [React Query](https://tanstack.com/query/latest) is a powerful data synchronization library for React that provides: * **Automatic caching**: Query results are cached and shared across components * **Background refetching**: Keep data fresh automatically * **Optimistic updates**: Update UI before server confirmation * **Request deduplication**: Multiple components requesting the same data share a single request * **Error handling**: Built-in error states and retry logic Learn how to use React Query's hooks and features View the source code and contribute to React Query *** ### Shelby SDK The [Shelby SDK](/sdks/typescript) provides the core functionality for interacting with the Shelby Protocol: * **ShelbyClient**: Main client for coordination and RPC operations The React SDK wraps these SDK methods with React Query hooks, providing: * Type-safe React hooks * Automatic query key generation * Integration with React Query's caching and synchronization For most apps, the recommended route is to provide a single client with `ShelbyClientProvider` and let hooks read it from context. You can still pass a client directly when you need multiple clients. Explore the Shelby SDK API and configuration options
# Getting Started (/sdks/solana-kit) # Solana Kit SDK The Solana Kit SDK enables Solana developers to easily integrate with the Shelby Protocol and leverage decentralized blob storage in their applications. ## Installation npm pnpm yarn bun ```bash npm install @shelby-protocol/solana-kit ``` ```bash pnpm add @shelby-protocol/solana-kit ``` ```bash yarn add @shelby-protocol/solana-kit ``` ```bash bun add @shelby-protocol/solana-kit ``` ## Entry Points The SDK provides two entry points optimized for different environments: Server-side applications with direct keypair access Browser applications with wallet connections | Entry Point | Environment | Use Case | | ----------------------------------- | ----------- | -------------------------------- | | `@shelby-protocol/solana-kit/node` | Node.js | Backend services, scripts, CLIs | | `@shelby-protocol/solana-kit/react` | Browser | React dApps with wallet adapters | ## Acquire a Shelby API Key API keys authenticate your app and manage rate limits when using Shelby services. Without one, your client runs in "anonymous" mode with much lower limits, which can affect performance. Follow this guide to obtain your Shelby API key ## Quick Start ### Node.js (Server-Side) For backend services with direct keypair access: ```typescript import { Shelby, Network } from "@shelby-protocol/solana-kit/node"; import { Connection, Keypair } from "@solana/web3.js"; // Create a Solana network connection const connection = new Connection("https://api.devnet.solana.com"); // Create a Shelby client const shelbyClient = new Shelby({ network: Network.TESTNET, connection, apiKey: "AG-***", }); // Create a storage account from a Solana keypair const solanaKeypair = Keypair.generate(); const storageAccount = shelbyClient.createStorageAccount( solanaKeypair, "my-app.com" ); // Upload data await shelbyClient.upload({ blobData: new Uint8Array([1, 2, 3]), signer: storageAccount, blobName: "example.txt", expirationMicros: Date.now() * 1000 + 86400000000, }); ``` ### React (Browser) For browser dApps with wallet connections: ```tsx "use client"; import { useStorageAccount, Network } from "@shelby-protocol/solana-kit/react"; import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { useWalletConnection } from "@solana/react-hooks"; function MyComponent() { const { wallet } = useWalletConnection(); const shelbyClient = new ShelbyClient({ network: Network.TESTNET, apiKey: "AG-***", }); const { storageAccountAddress, signAndSubmitTransaction } = useStorageAccount({ client: shelbyClient, wallet, }); return (

Storage Account: {storageAccountAddress?.toString()}

); } ``` Using `@solana/wallet-adapter-react`? See the [React guide](/sdks/solana-kit/react#using-with-solana-wallet-adapter-react) for adapter compatibility. ## Key Concepts ### Cross-Chain Identity Shelby uses the Aptos blockchain as a coordination and settlement layer. The Solana Kit leverages [Aptos Derivable Account Abstraction](https://github.com/aptos-foundation/AIPs/blob/main/aips/aip-113.md) to create a **Shelby Storage Account** from a Solana identity. This enables: * Solana keypairs to control data on Shelby * Cross-chain signatures managed securely on the Aptos network * Application-level isolation through domain scoping ### Ownership Hierarchy The ownership structure is: 1. **Solana Keypair/Wallet** → Controls the Storage Account 2. **Storage Account** → Owns blobs on Shelby Each storage account has a different address on different dApps due to domain scoping. This maintains isolation at the application level. ## Next Steps Server-side integration with direct keypair access Browser integration with wallet connections Step-by-step guide to uploading files # Overview (/sdks/solana-kit/overview) The Shelby Protocol Solana Kit SDK is built to bridge the gap between Solana and the Shelby Protocol, enabling Solana developers to use their existing keypairs or wallet connections for decentralized blob storage. ## Architecture The SDK provides two entry points for different environments: ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ Your Solana Application │ ├─────────────────────────────────────────────────────────────────────────┤ │ @shelby-protocol/solana-kit │ │ │ │ ┌─────────────────────────────┐ ┌─────────────────────────────────┐ │ │ │ /node │ │ /react │ │ │ │ ┌───────────────────────┐ │ │ ┌───────────────────────────┐ │ │ │ │ │ Shelby Client │ │ │ │ useStorageAccount │ │ │ │ │ │ (extends SDK) │ │ │ │ (React Hook) │ │ │ │ │ └───────────────────────┘ │ │ └───────────────────────────┘ │ │ │ │ ┌───────────────────────┐ │ │ │ │ │ │ │ ShelbyStorageAccount │ │ │ Uses connected wallet for: │ │ │ │ │ (Keypair → Account) │ │ │ - Address derivation │ │ │ │ └───────────────────────┘ │ │ - Transaction signing │ │ │ └─────────────────────────────┘ └─────────────────────────────────┘ │ ├─────────────────────────────────────────────────────────────────────────┤ │ @shelby-protocol/sdk │ │ ┌───────────────────────────┐ ┌───────────────────────────────────┐ │ │ │ Coordination Layer │ │ RPC Operations │ │ │ │ (Aptos Chain) │ │ (Blob Storage) │ │ │ └───────────────────────────┘ └───────────────────────────────────┘ │ └─────────────────────────────────────────────────────────────────────────┘ ``` ## Entry Points ### Node.js (`/node`) For server-side applications with direct access to Solana keypairs: * **Shelby Client**: Extends the core SDK with Solana-specific initialization * **ShelbyStorageAccount**: Converts a Solana `Keypair` into a Shelby signer * **Use cases**: Backend services, scripts, CLIs, automated uploads ```typescript import { Shelby, Network } from "@shelby-protocol/solana-kit/node"; ``` Complete guide for server-side integration *** ### React (`/react`) For browser applications with wallet connections: * **useStorageAccount Hook**: Derives storage account from connected wallet * **Wallet Integration**: Works with `@solana/react-hooks` and Solana wallet extensions * **Use cases**: React dApps, Next.js applications, browser-based uploads ```typescript import { useStorageAccount } from "@shelby-protocol/solana-kit/react"; import { useWalletConnection } from "@solana/react-hooks"; ``` Complete guide for browser integration *** ## Cross-Chain Flow Both entry points use the same underlying cross-chain mechanism: ### Node.js Flow ``` ┌──────────────┐ ┌──────────────────────┐ ┌─────────────────┐ │ Solana │────▶│ ShelbyStorageAccount │────▶│ Shelby Client │ │ Keypair │ │ (Signs with SIWS) │ │ (Uploads blob) │ └──────────────┘ └──────────────────────┘ └─────────────────┘ ``` 1. **Solana Keypair** → Provided directly (from file, env, etc.) 2. **ShelbyStorageAccount** → Signs transactions with SIWS envelope 3. **Shelby Client** → Submits to the coordination layer ### React Flow ``` ┌──────────────┐ ┌──────────────────────┐ ┌─────────────────┐ │ Wallet │────▶│ useStorageAccount │────▶│ Shelby Client │ │ (Phantom) │ │ (Prompts for sign) │ │ (Uploads blob) │ └──────────────┘ └──────────────────────┘ └─────────────────┘ ``` 1. **Wallet** → User connects via `@solana/react-hooks` or wallet adapter 2. **useStorageAccount** → Derives address, prompts user to sign 3. **Shelby Client** → Submits to the coordination layer ## SIWS (Sign-In With Solana) Both entry points use the SIWS envelope format for transaction signing: 1. **Transaction Data** → Serialized Aptos transaction 2. **SIWS Message** → Human-readable message wrapping the transaction 3. **Solana Signature** → Ed25519 signature from Solana keypair/wallet 4. **Authenticator** → Wrapped signature compatible with Aptos This enables Solana identities to control accounts on the Aptos-based Shelby network. ## Common Module Both entry points share common functionality: | Export | Description | | ----------------------------- | --------------------------------------------- | | `Network` | Network configuration (currently `TESTNET`) | | `deriveStorageAccountAddress` | Derives Shelby address from Solana public key | | `createSiwsMessage` | Creates the SIWS message for signing | | `createAuthenticator` | Wraps signature in Aptos authenticator format | ## Choosing an Entry Point | Requirement | Use Node.js (`/node`) | Use React (`/react`) | | --------------------------- | --------------------- | -------------------- | | Direct keypair access | ✅ | ❌ | | Wallet popup for signing | ❌ | ✅ | | Server-side execution | ✅ | ❌ | | Browser environment | ❌ | ✅ | | Automated/batch operations | ✅ | ❌ | | User-initiated transactions | ✅ | ✅ | # Acquire API Keys (/sdks/typescript/acquire-api-keys) import { Step, Steps } from 'fumadocs-ui/components/steps'; # API Keys API keys authenticate your app and manage rate limits when using Shelby services. Without one, your client runs in "anonymous" mode with much lower limits, which can affect performance. ## Overview API keys provide several important benefits: * **Authentication**: Securely identify your application to Shelby services. * **Rate Limiting**: Access higher request limits for better application performance. * **Usage Tracking**: Monitor your API consumption and optimize usage patterns. * **Service Access**: Enable access to premium features and enhanced service tiers. ## Acquiring API Keys To obtain your API keys, you'll need to create an API resource through the Geomi platform: ### Step-by-Step Guide ### Navigate to Geomi Visit [geomi.dev](https://geomi.dev) in your web browser. ### Account Setup Log in to your existing account or create a new account if you haven't already. ### Create API Resource On the overview page, click the "API Resource" card to begin creating a new resource. ### Configure Resource Complete the configuration form with the following settings: * **Network**: Select `Testnet` from the available network options. * **Resource Name**: Provide a descriptive name for your API resource. * **Usage Description**: Briefly describe your intended use case. ### Generate Keys Once submitted, your API keys will be generated and displayed. **Note**: By default the site generates a key for use in a private server context. If you intend to use the key in a frontend context, create a client key. Learn more about API keys at the Geomi [API keys](https://geomi.dev/docs/api-keys) and [billing](https://geomi.dev/docs/admin/billing) pages. ## Implementing API Keys ### Basic Configuration Integrate your API key into the Shelby client configuration as shown below: ```ts import { Network } from "@aptos-labs/ts-sdk"; import { ShelbyNodeClient } from "@shelby/sdk"; const client = new ShelbyNodeClient({ network: Network.TESTNET, apiKey: "aptoslabs_***", // Replace with your actual API key }); ``` Or into an Aptos client: ```ts import { Network, AptosConfig, Aptos } from "@aptos-labs/ts-sdk"; const aptosConfig = new AptosConfig({ network: Network.TESTNET, clientConfig : { API_KEY: "aptoslabs_***", // Replace with your actual API key } }) const aptosClient = new Aptos(aptosConfig) ``` # Acquire shelbyUSD and APT Tokens (/sdks/typescript/fund-your-account) Fund your account with either [ShelbyUSD Tokens](../../apis/faucet/shelbyusd) or [APTOS APT Tokens](../../apis/faucet/aptos) # Getting Started (/sdks/typescript) # TypeScript SDK The Shelby Protocol TypeScript SDK provides both Node.js and browser support for interacting with the Shelby Protocol. This comprehensive reference covers all available types, functions, and classes. ## Installation npm pnpm yarn bun ```bash npm install @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ```bash pnpm add @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ```bash yarn add @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ```bash bun add @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ## Quick Start ### Node.js Environment ```typescript import { ShelbyNodeClient } from "@shelby-protocol/sdk/node"; import { Network } from "@aptos-labs/ts-sdk"; // Create client configuration const config = { network: Network.TESTNET, apiKey: "aptoslabs_***", }; // Initialize the Shelby client const shelbyClient = new ShelbyNodeClient(config); ``` Explore the complete [Node.js client](/sdks/typescript/node) usage ### Browser Environment ```typescript import { ShelbyClient } from '@shelby-protocol/sdk/browser' import { Network } from '@aptos-labs/ts-sdk' // Create client configuration const config = { network: Network.TESTNET apiKey: "aptoslabs_***", } // Initialize the Shelby client const shelbyClient = new ShelbyClient(config) ``` Explore the complete [Browser client](/sdks/typescript/browser) usage ## Examples Explore all of the Shelby examples provided in the examples repo, which demonstrate how to build on Shelby * [Shelby Examples](https://github.com/shelby/examples/tree/main/apps) ## API Reference Explore the complete TypeScript API documentation: * [Core Types & Functions](/sdks/typescript/core) - Shared functionality for both environments # Getting Started (/tools/cli) import { Step, Steps } from "fumadocs-ui/components/steps"; import { Tabs, Tab } from "fumadocs-ui/components/tabs"; import ShelbyUSDFaucet from "@/components/faucet/ShelbyUSDFaucet"; import AptosFaucet from "@/components/faucet/AptosFaucet"; The Shelby CLI offers an intuitive way to interact with Shelby. It lets you upload blobs to and download blobs from Shelby, and also manage multiple accounts or networks (called contexts). ## Installation **Prerequisites:** This guide assumes you have [`Node.js`](https://nodejs.org/) and [`npm`](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) installed. If you don't have them installed, please install them before proceeding. Install the Shelby CLI npm pnpm yarn bun ```bash npm i -g @shelby-protocol/cli ``` ```bash pnpm add -g @shelby-protocol/cli ``` ```bash yarn global add @shelby-protocol/cli ``` ```bash bun add --global @shelby-protocol/cli ``` ## Quick Start ### Initialize Shelby Start off by initializing the CLI with the `shelby init` command. This will create a shelby cli configuration file at `~/.shelby/config.yaml`. ```bash shelby init ``` **Note**: The CLI will ask you to provide an API key. While optional, this step is highly recommended to avoid ratelimits. Learn more about getting an API key [here](/sdks/typescript/acquire-api-keys). If you accept the defaults, your config file (`~/.shelby/config.yaml`) will contain the following: ```bash cat ~/.shelby/config.yaml ``` ```yaml title="~/.shelby/config.yaml" contexts: local: aptos_network: name: local fullnode: http://127.0.0.1:8080/v1 faucet: http://127.0.0.1:8081 indexer: http://127.0.0.1:8090/v1/graphql pepper: https://api.devnet.aptoslabs.com/keyless/pepper/v0 prover: https://api.devnet.aptoslabs.com/keyless/prover/v0 shelby_network: rpc_endpoint: http://localhost:9090/ shelbynet: aptos_network: name: shelbynet fullnode: https://api.shelbynet.shelby.xyz/v1 faucet: https://faucet.shelbynet.shelby.xyz indexer: https://api.shelbynet.shelby.xyz/v1/graphql pepper: https://api.shelbynet.aptoslabs.com/keyless/pepper/v0 prover: https://api.shelbynet.aptoslabs.com/keyless/prover/v0 shelby_network: rpc_endpoint: https://api.shelbynet.shelby.xyz/shelby testnet: aptos_network: name: testnet fullnode: https://api.testnet.aptoslabs.com/v1 indexer: https://api.testnet.aptoslabs.com/v1/graphql pepper: https://api.testnet.aptoslabs.com/keyless/pepper/v0 prover: https://api.testnet.aptoslabs.com/keyless/prover/v0 shelby_network: rpc_endpoint: https://api.testnet.shelby.xyz/shelby accounts: alice: private_key: ed25519-priv-0x8... address: "0xfcba...a51c" default_context: testnet default_account: alice ``` ### List Contexts (Optional) Ensure that the context was created successfully by listing the available contexts (list of networks). The `(default)` network is the one that is currently selected. ```bash shelby context list ``` ```bash title="Output" Aptos Configurations: ┌───────────────────┬──────────┬──────────────────────────────────────────┬──────────────────────────────────────────────┬──────────────────────────────────────┬─────────┐ │ Name │ Network │ Fullnode │ Indexer │ Faucet │ API Key │ ├───────────────────┼──────────┼──────────────────────────────────────────┼──────────────────────────────────────────────┼──────────────────────────────────────┼─────────┤ │ local │ local │ http://127.0.0.1:8080/v1 │ http://127.0.0.1:8090/v1/graphql │ http://127.0.0.1:8081 │ │ ├───────────────────┼──────────┼──────────────────────────────────────────┼──────────────────────────────────────────────┼──────────────────────────────────────┼─────────┤ │ shelbynet │ shelbynet │ https://api.shelbynet.shelby.xyz/v1 │ https://api.shelbynet.shelby.xyz/v1/graphql │ https://faucet.shelbynet.shelby.xyz │ │ ├───────────────────┼──────────┼──────────────────────────────────────────┼──────────────────────────────────────────────┼──────────────────────────────────────┼─────────┤ │ testnet (default) │ testnet │ https://api.testnet.aptoslabs.com/v1 │ https://api.testnet.aptoslabs.com/v1/graphql │ │ │ └───────────────────┴──────────┴──────────────────────────────────────────┴──────────────────────────────────────────────┴──────────────────────────────────────┴─────────┘ Shelby Configurations: ┌───────────────────┬───────────────────────────────────────────┬─────────┬─────────────┬─────────────────┐ │ Name │ RPC │ Indexer │ RPC API Key │ Indexer API Key │ ├───────────────────┼───────────────────────────────────────────┼─────────┼─────────────┼─────────────────┤ │ local │ http://localhost:9090/ │ │ │ │ ├───────────────────┼───────────────────────────────────────────┼─────────┼─────────────┼─────────────────┤ │ shelbynet │ https://api.shelbynet.shelby.xyz/shelby │ │ │ │ ├───────────────────┼───────────────────────────────────────────┼─────────┼─────────────┼─────────────────┤ │ testnet (default) │ https://api.testnet.shelby.xyz/shelby │ │ │ │ └───────────────────┴───────────────────────────────────────────┴─────────┴─────────────┴─────────────────┘ ``` ### List Accounts (optional) To retrieve the list of accounts, you can use the `shelby account list` command. The `(default)` account is the one that is currently selected. ```bash shelby account list ``` ```bash title="Output" ┌──────────────┬────────────────────────────────────────────────┬──────────────────┐ │ Name │ Address │ Private Key │ ├──────────────┼────────────────────────────────────────────────┼──────────────────┤ │ alice │ 0xfcb......................................0fb │ ed25519-priv-0x8 │ │ (default) │ c276e3e598938e00a51c │ adf5... │ └──────────────┴────────────────────────────────────────────────┴──────────────────┘ ``` You will use the value in the `Address` column as the recipient address for funding. ### Fund Account To upload and download files to Shelby, you'll need both 1. **Aptos tokens** (for gas fees) and 2. **ShelbyUSD tokens** (for Shelby operations like upload) #### Aptos Tokens The command below will output the faucet page URL with your active account pre-populated ```bash shelby faucet --network testnet --no-open # remove the --no-open flag to automatically open in browser ``` Make sure the aptos cli is aware of your account and configured. The init command will also print out a command you can run to configure the profile, something like: ```bash aptos init --profile shelby-alice --assume-yes --private-key ed25519-priv-0xa... --network testnet ``` Aptos tokens are used to pay for gas fees. To fund your account with Aptos tokens, you can use `aptos` CLI. ```bash aptos account fund-with-faucet --profile shelby-alice --amount 1000000000000000000 ``` #### ShelbyUSD Tokens The command below will output the faucet page URL with your active account pre-populated ```bash shelby faucet --network testnet --no-open # remove the --no-open flag to automatically open in browser ``` ### Verify Account Balance ```bash shelby account balance ``` ```bash title="Output" 👤 Account Information ──────────────────────────────────────────── 🏷️ Alias: alice 🌐 Context: testnet 🔑 Address:
🔗 Aptos Explorer: https://explorer.aptoslabs.com/account/
?network=testnet 🗂️ Shelby Explorer: https://explorer.shelby.xyz/testnet/account/
──────────────────────────────────────────── 💰 Balance: ┌─────────┬───────────────────────────────────┬─────────────────────┬───────────────────┐ │ Token │ Asset │ Balance │ Raw Units │ ├─────────┼───────────────────────────────────┼─────────────────────┼───────────────────┤ │ APT │ 0x1::aptos_coin::AptosCoin │ 9.998885 APT │ 999,888,500 │ ├─────────┼───────────────────────────────────┼─────────────────────┼───────────────────┤ │ ShelbyU │ 0x1b18363a9f1fe5e6ebf247daba5cc1c │ 9.99993056 │ 999,993,056 │ │ SD │ 18052bb232efdc4c50f556053922d98e1 │ ShelbyUSD │ │ └─────────┴───────────────────────────────────┴─────────────────────┴───────────────────┘ ``` ### Upload a file ```bash # Uploads "filename.txt" to Shelby under a custom path or name (files/filename.txt), expiring tomorrow (auto-confirms payment) # Expiration date/time (required). Examples: "tomorrow", "in 2 days", "next Friday", "2025-12-31", UNIX timestamp shelby upload /Users/User/.../filename.txt files/filename.txt -e tomorrow --assume-yes ``` ```bash title="Output" 🚀 Upload Summary ──────────────────────────────────────────── 📦 File: /Users/User/.../filename.txt 📁 Blob Name: files/filename.txt 🧮 Filelist created (1 entry) ⏱️ Took: 0.00013s ⚙️ Flag: --assume-yes (auto-confirmed) 🕒 Expires: Oct 11, 2025, 4:26:56 PM ✔ Upload complete — took 1.53s 🌐 Aptos Explorer: https://explorer.aptoslabs.com/txn/?network=testnet 🗂️ Shelby Explorer: https://explorer.shelby.xyz/testnet/account/ ──────────────────────────────────────────── ✨ Done! ``` ### Verify Upload You can verify the upload by clicking on the Shelby Explorer link or by running the command below ```bash shelby account blobs ``` ```bash title="Output" 🔍 Retrieving blobs for alice 👤 Address: 🗂️ Shelby Explorer: https://explorer.shelby.xyz/testnet/account/ ✅ Retrieved 2 blobs ──────────────────────────────────────────── 📦 Stored Blobs ┌─────────────────────────────────────────────┬───────────────┬─────────────────────────┐ │ Name │ Size │ Expires │ ├─────────────────────────────────────────────┼───────────────┼─────────────────────────┤ │ │ 494 B │ Oct 11, 2025, 4:03 PM │ └─────────────────────────────────────────────┴───────────────┴─────────────────────────┘ ✨ Done! ``` ### Download the file ```bash shelby download files/filename.txt /Users/User/Desktop/filename.txt ``` *** ## Troubleshooting ### `Insufficient Shelby tokens` Error **Error:** `Insufficient Shelby tokens. Please fund your account with Shelby tokens to continue.` **Solution:** This means you need ShelbyUSD tokens (not just Aptos tokens) to perform uploads. Visit the [Shelby faucet](/apis/faucet/shelbyusd) and fund your account with ShelbyUSD tokens. # CLI Management (/tools/cli/management) import { Step, Steps } from 'fumadocs-ui/components/steps'; ## Check Version To check which version of the Shelby CLI you have installed: ```bash shelby --version ``` ## Find Installation Location To find where the Shelby CLI is installed: ```bash which shelby ``` ## Uninstall To uninstall the Shelby CLI: ### Remove Shelby Config The shelby config is located at `~/.shelby/config.yaml`. ### Uninstall Global Package ```bash npm uninstall -g @shelby-protocol/cli ``` # S3 Compatibility (/tools/s3-gateway/compatibility) The Shelby S3 Gateway implements a subset of the Amazon S3 API. While most S3 tools work out of the box, there are key differences from standard S3 (and S3-compatible services like MinIO or Cloudflare R2) that you should be aware of. ## ETags Use SHA2-256, Not MD5 In standard S3, the ETag for a single-part upload is the MD5 hash of the object data. The Shelby S3 Gateway instead returns the **blob merkle root** — a SHA2-256 based hash derived from Shelby's erasure-coded storage commitments. This means: * ETags are **consistent across all operations** — `PutObject`, `GetObject`, `HeadObject`, and `ListObjects` all return the same ETag for a given object. * ETags are **not** MD5 digests. S3 clients that compute a local MD5 and compare it to the returned ETag for integrity checking will see a mismatch. This does not indicate corruption. * Tools like **rclone** that compare ETags across `ListObjects` and `PutObject` responses to detect changes work correctly, since all routes return the same value. If your S3 client warns about ETag mismatches, you can typically disable MD5 checksum verification. For example, in rclone, the gateway's consistent ETags ensure that `rclone copy` and `rclone sync` correctly detect unchanged files without re-uploading. ## Buckets Are Aptos Addresses In standard S3, bucket names are globally unique strings (e.g., `my-app-data`). In Shelby, a bucket is your **Aptos account address** (e.g., `0x0694a79...`). * You can only write to and delete from the bucket matching your signer's address. * Bucket creation is implicit — there is no `CreateBucket` operation. Buckets exist as long as the corresponding Aptos account exists. * `ListBuckets` returns the set of account addresses configured in your gateway config. ## Expiration Is Required on Upload Standard S3 objects persist indefinitely (unless lifecycle rules are configured at the bucket level). Shelby requires an **explicit expiration** for every blob at upload time. Set the expiration via S3 metadata headers: | Header | Example | | --------------------------------------------- | ------------------ | | `x-amz-meta-expiration-seconds` (recommended) | `86400` (24 hours) | | `x-expiration-seconds` (fallback) | `86400` | Uploads without an expiration header will be rejected with an `InvalidArgument` error. See [Uploads](/tools/s3-gateway/uploads#expiration) for details. ## No Object Overwrite Standard S3 allows overwriting an existing object by uploading to the same key. Shelby does not support in-place overwrites because blobs are registered on-chain with a unique merkle commitment. * Uploading the **same content** to an existing key is idempotent — the gateway detects the matching merkle root and returns `200 OK`. * Uploading **different content** to an existing key returns `409 ObjectAlreadyExists`. To replace an object, delete it first, then upload the new version. ## Supported Operations The gateway implements a focused subset of the S3 API: | Operation | Supported | | ------------------------------- | --------- | | `GetObject` (with Range) | Yes | | `HeadObject` | Yes | | `PutObject` | Yes | | `DeleteObject` | Yes | | `DeleteObjects` (batch) | Yes | | `ListObjectsV1` | Yes | | `ListObjectsV2` | Yes | | `ListBuckets` | Yes | | `HeadBucket` | Yes | | `CreateMultipartUpload` | Yes | | `UploadPart` | Yes | | `CompleteMultipartUpload` | Yes | | `AbortMultipartUpload` | Yes | | `CopyObject` | No | | `CreateBucket` / `DeleteBucket` | No | | `PutBucketPolicy` / ACLs | No | | `Object versioning` | No | | `Object tagging` | No | | `Server-side encryption` (SSE) | No | | `Presigned URLs` | No | ## No Server-Side Copy `CopyObject` is not supported. To copy an object, download it with `GetObject` and re-upload with `PutObject`. ## Unsupported Headers Many standard S3 request headers are accepted but silently ignored. The gateway will not return an error if you include them, but they have no effect: | Header Category | Examples | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------ | | Server-side encryption | `x-amz-server-side-encryption`, `x-amz-server-side-encryption-customer-algorithm`, `x-amz-server-side-encryption-customer-key` | | Storage class | `x-amz-storage-class` (all objects are stored as `STANDARD`) | | Object lock / legal hold | `x-amz-object-lock-mode`, `x-amz-object-lock-legal-hold` | | Tagging | `x-amz-tagging` | | ACL | `x-amz-acl`, `x-amz-grant-read`, `x-amz-grant-write`, `x-amz-grant-full-control` | | Replication | `x-amz-replication-status` | Unrecognized headers are silently ignored and will not cause request failures. This allows most S3 clients to work without disabling features they send by default. ## Authentication The gateway uses standard **AWS SigV4** request signing. The `accessKeyId` and `secretAccessKey` are shared secrets between your S3 client and the gateway — they are **not** AWS credentials. See [Configuration](/tools/s3-gateway/configuration#credentials) for setup. # Configuration (/tools/s3-gateway/configuration) import { Tabs, Tab } from 'fumadocs-ui/components/tabs'; The S3 Gateway supports two configuration formats: * **YAML** — Simple, supports reads and writes (add `aptosPrivateKey` to enable writes) * **TypeScript** — For advanced or dynamic configuration (custom providers, programmatic setup) ## YAML Configuration Create a `shelby.config.yaml` file and run with `--config`: ```bash npx @shelby-protocol/s3-gateway --config shelby.config.yaml ``` ### Full Example ```yaml title="shelby.config.yaml" # Network configuration (required) network: name: shelbynet # "shelbynet" or "local" rpcEndpoint: https://api.shelbynet.shelby.xyz/shelby aptosFullnode: https://api.shelbynet.shelby.xyz/v1 aptosIndexer: https://api.shelbynet.shelby.xyz/v1/graphql apiKey: # Optional # Server settings server: host: localhost # Bind address (default: localhost) port: 9000 # Listen port (default: 9000) region: shelbyland # S3 region for signing (default: shelbyland) verbose: false # Enable request logging (default: false) # S3 credentials for request signing (optional — defaults to dev credentials) credentials: - accessKeyId: AKIAIOSFODNN7EXAMPLE secretAccessKey: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY aptosPrivateKey: # Enables uploads & deletes # Shelby account addresses to expose as S3 buckets (required) # Use your own account address from `shelby account info` # Must be quoted — YAML interprets unquoted 0x… values as numbers buckets: - "" ``` To enable write operations (uploads, deletes), add `aptosPrivateKey` to a credential entry. Without it, the credential only permits read operations. ### Minimal Example ```yaml title="shelby.config.yaml" network: name: shelbynet rpcEndpoint: https://api.shelbynet.shelby.xyz/shelby aptosFullnode: https://api.shelbynet.shelby.xyz/v1 aptosIndexer: https://api.shelbynet.shelby.xyz/v1/graphql apiKey: buckets: - "" ``` With the minimal config, the gateway uses default development credentials and supports read-only access. Add a `credentials` section with `aptosPrivateKey` to enable writes. ## Configuration Reference ### Network Options (Required) | Option | Type | Default | Description | | --------------- | -------------------------- | --------------- | -------------------------- | | `name` | `"shelbynet"` \| `"local"` | `"shelbynet"` | Network name | | `rpcEndpoint` | `string` | Network default | Shelby RPC endpoint URL | | `aptosFullnode` | `string` | - | Aptos fullnode URL | | `aptosIndexer` | `string` | - | Aptos indexer URL | | `apiKey` | `string` | - | API key for authentication | ### Server Options | Option | Type | Default | Description | | --------- | --------- | -------------- | ------------------------------- | | `host` | `string` | `"localhost"` | Bind address for the server | | `port` | `number` | `9000` | Port to listen on | | `region` | `string` | `"shelbyland"` | S3 region for SigV4 signing | | `verbose` | `boolean` | `false` | Enable detailed request logging | The `region` setting must match what your S3 clients use for signing requests. Some tools require a standard AWS region like `"us-east-1"`. ### Credentials Array of S3 credential objects. If not specified, default dev credentials are used. These are **not** AWS credentials. They are shared secrets used for [S3 SigV4 request signing](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html) between your S3 client and the gateway. The same values must be configured in both the gateway config and your S3 client (rclone, boto3, etc.). You can use any values you want — the defaults work for local development. | Field | Type | Description | | ----------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `accessKeyId` | `string` | Public identifier for the credential (used in the `Authorization` header) | | `secretAccessKey` | `string` | Shared secret used for HMAC request signing (never sent over the network) | | `aptosPrivateKey` | `string` (optional) | Your Aptos Ed25519 private key. If you've set up the [Shelby CLI](/tools/cli), copy your `private_key` from `~/.shelby/config.yaml`. Accepts the `ed25519-priv-0x...` format or raw hex (`0x...`). Enables write operations (uploads, deletes) for this credential | ### Buckets (Required) Array of Shelby account addresses to expose as S3 buckets. Must be quoted in YAML (e.g. `"0x1abc..."`) since YAML interprets unquoted `0x…` values as numbers. ## Environment Variables | Variable | Default | Description | | -------- | ------- | ----------------------------------- | | `PORT` | `9000` | Server port (overrides config file) | | `HTTPS` | - | Set to `true` to enable HTTPS | ## CLI Options | Option | Description | | --------------------- | ------------------------------------ | | `-c, --config ` | Path to config file | | `-p, --port ` | Port to listen on (overrides config) | | `-h, --help` | Show help message | ## Advanced: TypeScript Configuration For dynamic configuration or custom credential/bucket providers, use a TypeScript config. A ready-to-use example that reads your existing `~/.shelby/config.yaml` is included in the package at `example.shelby.config.ts`. To use it: ```bash npx @shelby-protocol/s3-gateway --config example.shelby.config.ts ``` Or create your own: ```typescript title="shelby.config.ts" import { Account, Ed25519PrivateKey, Network } from "@aptos-labs/ts-sdk"; import { defineConfig, StaticCredentialProvider, StaticBucketProvider, } from "@shelby-protocol/s3-gateway"; const privateKey = new Ed25519PrivateKey(process.env.APTOS_PRIVATE_KEY!); const aptosSigner = Account.fromPrivateKey({ privateKey }); const signerAddress = aptosSigner.accountAddress.toString(); export default defineConfig({ shelby: { network: Network.SHELBYNET, rpc: { baseUrl: "https://api.shelbynet.shelby.xyz/shelby", }, }, server: { port: 9000, region: "shelbyland", }, credentialProvider: new StaticCredentialProvider({ credentials: { AKIAIOSFODNN7EXAMPLE: { secretKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", aptosSigner, // Enables write operations (uploads, deletes) }, }, }), bucketProvider: new StaticBucketProvider({ buckets: [signerAddress], }), }); ``` The `aptosSigner` field is what enables write operations. Without it, the credential only permits reads. In YAML configs, set `aptosPrivateKey` to achieve the same result. ### Custom Providers Implement custom providers for dynamic credential or bucket lookup: ```typescript title="shelby.config.ts" import { defineConfig } from "@shelby-protocol/s3-gateway"; import type { CredentialProvider } from "@shelby-protocol/s3-gateway"; // Custom credential provider (e.g., from database) class DatabaseCredentialProvider implements CredentialProvider { async getCredential(accessKeyId: string) { const result = await db.query( "SELECT secret_key FROM credentials WHERE access_key_id = ?", [accessKeyId] ); return result ? { accessKeyId, secretAccessKey: result.secret_key } : undefined; } async listAccessKeyIds() { const results = await db.query("SELECT access_key_id FROM credentials"); return results.map((r) => r.access_key_id); } async refresh() { // Refresh cache if needed } } export default defineConfig({ shelby: { /* ... */ }, credentialProvider: new DatabaseCredentialProvider(), bucketProvider: new StaticBucketProvider({ buckets: ["0x..."] }), }); ``` # Deletions (/tools/s3-gateway/deletions) import { Tabs, Tab } from 'fumadocs-ui/components/tabs'; The S3 Gateway supports deleting files from Shelby storage using standard S3 delete operations. This guide covers single object deletion and batch deletion of multiple objects. ## Prerequisites Before using the examples on this page, you need a running S3 Gateway with a valid configuration that includes `aptosPrivateKey` (YAML) or `aptosSigner` (TypeScript). See [Configuration](/tools/s3-gateway/configuration) to set up your `shelby.config.yaml` and [Integrations](/tools/s3-gateway/integrations) to configure your S3 client (i.e., rclone, boto3, AWS CLI). The bucket name must match your signer's Aptos address. You can only delete objects from buckets that correspond to accounts you control. ## Supported Delete Operations | Operation | Description | | --------------- | -------------------------------------- | | `DeleteObject` | Delete a single object | | `DeleteObjects` | Delete multiple objects in one request | ## DeleteObject Delete a single object using the standard S3 DeleteObject operation. Per the S3 specification, DeleteObject returns a 204 success response even if the object doesn't exist. This is idempotent behavior—deleting the same object twice won't cause an error. ```bash # Delete a single file rclone delete shelby:/path/to/file.txt # Delete with verbose output rclone delete shelby:/old-data.json -v ``` ```bash # Delete a single file aws --profile shelby --endpoint-url http://localhost:9000 \ s3 rm s3:///path/to/file.txt # Delete with output aws --profile shelby --endpoint-url http://localhost:9000 \ s3api delete-object \ --bucket \ --key path/to/file.txt ``` ```python import boto3 s3 = boto3.client( 's3', endpoint_url='http://localhost:9000', aws_access_key_id='AKIAIOSFODNN7EXAMPLE', aws_secret_access_key='wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', region_name='shelbyland', ) # Delete a single object s3.delete_object( Bucket='', Key='path/to/file.txt' ) ``` ```typescript import { S3Client, DeleteObjectCommand } from "@aws-sdk/client-s3"; const client = new S3Client({ endpoint: "http://localhost:9000", region: "shelbyland", credentials: { accessKeyId: "AKIAIOSFODNN7EXAMPLE", secretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", }, forcePathStyle: true, }); await client.send(new DeleteObjectCommand({ Bucket: "", Key: "path/to/file.txt", })); ``` ## DeleteObjects (Batch Delete) Delete multiple objects in a single request. This is more efficient than deleting objects one at a time. Batch deletes are atomic—either all objects are deleted or none are. If any object in the batch doesn't exist, the entire operation will fail with a `NoSuchKey` error for all objects. ```bash # Delete multiple objects aws --profile shelby --endpoint-url http://localhost:9000 \ s3api delete-objects \ --bucket \ --delete '{ "Objects": [ {"Key": "file1.txt"}, {"Key": "file2.txt"}, {"Key": "data/file3.json"} ] }' # Delete with quiet mode (suppress successful delete output) aws --profile shelby --endpoint-url http://localhost:9000 \ s3api delete-objects \ --bucket \ --delete '{ "Objects": [{"Key": "file1.txt"}, {"Key": "file2.txt"}], "Quiet": true }' ``` ```python import boto3 s3 = boto3.client( 's3', endpoint_url='http://localhost:9000', aws_access_key_id='AKIAIOSFODNN7EXAMPLE', aws_secret_access_key='wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', region_name='shelbyland', ) # Delete multiple objects response = s3.delete_objects( Bucket='', Delete={ 'Objects': [ {'Key': 'file1.txt'}, {'Key': 'file2.txt'}, {'Key': 'data/file3.json'}, ], 'Quiet': False # Set True to suppress successful delete output } ) # Check results if 'Deleted' in response: for obj in response['Deleted']: print(f"Deleted: {obj['Key']}") if 'Errors' in response: for err in response['Errors']: print(f"Error deleting {err['Key']}: {err['Code']} - {err['Message']}") ``` ```typescript import { S3Client, DeleteObjectsCommand } from "@aws-sdk/client-s3"; const client = new S3Client({ endpoint: "http://localhost:9000", region: "shelbyland", credentials: { accessKeyId: "AKIAIOSFODNN7EXAMPLE", secretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", }, forcePathStyle: true, }); const response = await client.send(new DeleteObjectsCommand({ Bucket: "", Delete: { Objects: [ { Key: "file1.txt" }, { Key: "file2.txt" }, { Key: "data/file3.json" }, ], Quiet: false, }, })); // Check results for (const obj of response.Deleted ?? []) { console.log(`Deleted: ${obj.Key}`); } for (const err of response.Errors ?? []) { console.log(`Error deleting ${err.Key}: ${err.Code} - ${err.Message}`); } ``` ## Delete Directory (Prefix) To delete all objects with a common prefix, first list them, then delete in batch. ```bash # Delete all objects in a "directory" rclone delete shelby:/data/old-files/ # Delete with dry-run first to see what would be deleted rclone delete shelby:/data/old-files/ --dry-run # Purge entire directory (same as delete for S3-like storage) rclone purge shelby:/temp-data/ ``` ```bash # Delete all objects with a prefix aws --profile shelby --endpoint-url http://localhost:9000 \ s3 rm s3:///data/old-files/ --recursive # Dry run to see what would be deleted aws --profile shelby --endpoint-url http://localhost:9000 \ s3 rm s3:///data/old-files/ --recursive --dryrun ``` ```python import boto3 s3 = boto3.client('s3', endpoint_url='http://localhost:9000', ...) bucket = '' prefix = 'data/old-files/' # List all objects with prefix response = s3.list_objects_v2(Bucket=bucket, Prefix=prefix) objects = response.get('Contents', []) if objects: # Delete all found objects delete_response = s3.delete_objects( Bucket=bucket, Delete={ 'Objects': [{'Key': obj['Key']} for obj in objects] } ) print(f"Deleted {len(delete_response.get('Deleted', []))} objects") else: print("No objects found with prefix") ``` ## Error Handling ### Common Errors | Error Code | Description | Solution | | -------------- | ------------------------------------ | ------------------------------------------------------------------- | | `AccessDenied` | No aptosSigner configured | Add `aptosPrivateKey` in YAML or `aptosSigner` in TypeScript config | | `AccessDenied` | Bucket doesn't match signer address | Use your signer's address as bucket | | `NoSuchKey` | Object not found (batch delete only) | Verify object exists before batch delete | | `MalformedXML` | Invalid DeleteObjects request body | Check XML format for batch delete | ### Troubleshooting **"Delete operations require an Aptos signer" error:** Your credential doesn't have an Aptos signer. Add `aptosPrivateKey` to your credential in `shelby.config.yaml`, or add `aptosSigner` in `shelby.config.ts`. See [Configuration](/tools/s3-gateway/configuration#credentials) for details. **"Cannot delete from bucket" error:** The bucket name must match your Aptos signer's address. Check that: 1. Your credential has an `aptosSigner` configured 2. The bucket address matches `aptosSigner.accountAddress.toString()` **Batch delete fails with "NoSuchKey":** Unlike single DeleteObject (which succeeds even if the object doesn't exist), batch DeleteObjects is atomic. If any object in the batch doesn't exist, the entire operation fails. Either: * Verify all objects exist before deleting * Use individual DeleteObject calls if you need idempotent behavior **Deletion succeeded but object still appears in listing:** Shelby deletions are recorded on the blockchain. There may be a brief delay before the indexer reflects the deletion. Wait a few seconds and try listing again. # Downloads (/tools/s3-gateway/downloads) import { Tabs, Tab } from 'fumadocs-ui/components/tabs'; The S3 Gateway supports downloading files from Shelby storage using standard S3 read operations. This guide covers single file downloads, listing objects, and partial downloads with range requests. ## Prerequisites Before using the examples on this page, you need a running S3 Gateway with a valid configuration. See [Configuration](/tools/s3-gateway/configuration) to set up your `shelby.config.yaml` and [Integrations](/tools/s3-gateway/integrations) to configure your S3 client (i.e., rclone, boto3, AWS CLI). ## Supported Read Operations | Operation | Description | | ------------------ | ---------------------------------------- | | `GetObject` | Download file contents | | `HeadObject` | Get object metadata without downloading | | `ListObjectsV1/V2` | List objects in a bucket with pagination | | `ListBuckets` | List all configured buckets | | `HeadBucket` | Check if a bucket exists | ## GetObject Download a file from Shelby storage. ```bash # Download a single file rclone copy shelby:/file.txt ./ # Download to a specific local path rclone copy shelby:/data/report.pdf ./downloads/ # Download with progress rclone copy shelby:/large-file.zip ./ -P ``` ```bash # Download a single file aws --profile shelby --endpoint-url http://localhost:9000 \ s3 cp s3:///file.txt ./ # Download to a specific path aws --profile shelby --endpoint-url http://localhost:9000 \ s3 cp s3:///data/report.pdf ./downloads/ ``` ```python import boto3 s3 = boto3.client( 's3', endpoint_url='http://localhost:9000', aws_access_key_id='AKIAIOSFODNN7EXAMPLE', aws_secret_access_key='wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', region_name='shelbyland', ) # Download to file s3.download_file( '', 'path/to/file.txt', 'local-file.txt' ) # Get object content directly response = s3.get_object( Bucket='', Key='path/to/file.txt' ) content = response['Body'].read() ``` ```typescript import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3"; const client = new S3Client({ endpoint: "http://localhost:9000", region: "shelbyland", credentials: { accessKeyId: "AKIAIOSFODNN7EXAMPLE", secretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", }, forcePathStyle: true, }); const response = await client.send(new GetObjectCommand({ Bucket: "", Key: "path/to/file.txt", })); const content = await response.Body?.transformToString(); ``` ## Range Requests Download partial content using HTTP Range headers. Useful for: * Resuming interrupted downloads * Streaming video/audio * Reading specific parts of large files ```bash # Download first 1KB rclone cat shelby:/large-file.bin --count 1000 > partial.bin # Download from byte 1000 to end rclone cat shelby:/large-file.bin --offset 1000 > rest.bin # Download 500 bytes starting at byte 1000 rclone cat shelby:/large-file.bin --offset 1000 --count 500 > chunk.bin ``` ```python # Download specific byte range response = s3.get_object( Bucket='', Key='large-file.bin', Range='bytes=0-999' # First 1KB ) partial_content = response['Body'].read() print(f"Got {len(partial_content)} bytes") ``` ```typescript const response = await client.send(new GetObjectCommand({ Bucket: "", Key: "large-file.bin", Range: "bytes=0-999", // First 1KB })); const partialContent = await response.Body?.transformToByteArray(); console.log(`Got ${partialContent?.length} bytes`); ``` ## ListObjects List objects in a bucket with optional prefix filtering and pagination. ```bash # List all objects rclone ls shelby: # List with sizes and dates rclone lsl shelby: # List only directories rclone lsd shelby: # List objects with a prefix rclone ls shelby:/data/ ``` ```bash # List all objects aws --profile shelby --endpoint-url http://localhost:9000 \ s3 ls s3:/// # List with prefix aws --profile shelby --endpoint-url http://localhost:9000 \ s3 ls s3:///data/ # Recursive listing aws --profile shelby --endpoint-url http://localhost:9000 \ s3 ls s3:/// --recursive ``` ```python # List objects (v2) response = s3.list_objects_v2( Bucket='', Prefix='data/', # Optional: filter by prefix MaxKeys=100 # Optional: limit results ) for obj in response.get('Contents', []): print(f"{obj['Key']} - {obj['Size']} bytes") # Handle pagination paginator = s3.get_paginator('list_objects_v2') for page in paginator.paginate(Bucket=''): for obj in page.get('Contents', []): print(obj['Key']) ``` ```typescript import { ListObjectsV2Command } from "@aws-sdk/client-s3"; const response = await client.send(new ListObjectsV2Command({ Bucket: "", Prefix: "data/", // Optional MaxKeys: 100, // Optional })); for (const obj of response.Contents ?? []) { console.log(`${obj.Key} - ${obj.Size} bytes`); } ``` ## HeadObject Get object metadata without downloading the content. Returns size, content type, ETag, and expiration. ```python response = s3.head_object( Bucket='', Key='path/to/file.txt' ) print(f"Size: {response['ContentLength']} bytes") print(f"ETag: {response['ETag']}") print(f"Content-Type: {response['ContentType']}") print(f"Expiration: {response.get('Expiration')}") ``` ```typescript import { HeadObjectCommand } from "@aws-sdk/client-s3"; const response = await client.send(new HeadObjectCommand({ Bucket: "", Key: "path/to/file.txt", })); console.log(`Size: ${response.ContentLength} bytes`); console.log(`ETag: ${response.ETag}`); console.log(`Content-Type: ${response.ContentType}`); ``` ## ListBuckets List all configured buckets (Shelby account addresses). ```bash rclone lsd shelby: ``` ```bash aws --profile shelby --endpoint-url http://localhost:9000 s3 ls ``` ```python response = s3.list_buckets() for bucket in response['Buckets']: print(bucket['Name']) ``` ## Sync Directories Synchronize a remote directory to local storage. ```bash # Sync remote to local rclone sync shelby:/data/ ./local-backup/ # Sync with progress rclone sync shelby:/data/ ./local-backup/ -P # Dry run (see what would be copied) rclone sync shelby:/data/ ./local-backup/ --dry-run ``` ## Query with DuckDB [DuckDB](https://duckdb.org/) can query files directly from Shelby storage without downloading them first. ```sql -- Configure S3 settings INSTALL httpfs; LOAD httpfs; SET s3_endpoint = 'localhost:9000'; SET s3_access_key_id = 'AKIAIOSFODNN7EXAMPLE'; SET s3_secret_access_key = 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY'; SET s3_use_ssl = false; SET s3_url_style = 'path'; SET s3_region = 'shelbyland'; -- Query a Parquet file SELECT * FROM read_parquet( 's3:///data/dataset.parquet' ); -- Query CSV SELECT * FROM read_csv('s3:///reports/sales.csv', header = true); -- Query multiple files with glob SELECT * FROM read_parquet('s3:///data/*.parquet'); ``` ## Error Handling ### Common Errors | Error Code | Description | Solution | | -------------- | --------------------- | -------------------------------------- | | `NoSuchKey` | Object does not exist | Verify the key path is correct | | `NoSuchBucket` | Bucket not configured | Add the account address to your config | | `AccessDenied` | Authentication failed | Check credentials and signature | | `InvalidRange` | Invalid range request | Verify range is within file size | ### Troubleshooting **"NoSuchKey" but object exists:** * Check for typos in the key path * Keys are case-sensitive * Ensure you're using the correct bucket (account address) **Slow downloads:** * For large files, consider using range requests to download in parallel * Check network connectivity to the Shelby RPC endpoint **"SignatureDoesNotMatch":** * Ensure your region setting matches the gateway config * Check that your system clock is accurate (within 15 minutes) # Getting Started (/tools/s3-gateway) import { Step, Steps } from 'fumadocs-ui/components/steps'; import { Tabs, Tab } from 'fumadocs-ui/components/tabs'; The Shelby S3 Gateway provides an Amazon S3-compatible API for accessing Shelby blob storage. It allows you to use familiar S3 tools and libraries (AWS SDK, rclone, Cyberduck, DuckDB, etc.) to interact with blobs stored on Shelby without writing custom code. ## Key Concepts | S3 Concept | Shelby Equivalent | Notes | | ----------------- | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Bucket | Shelby account address | Your Aptos address (e.g., `0x0694a79...`). If you've set up the [Shelby CLI](/tools/cli), run `shelby account info` to find it | | Object | Shelby blob/file | Accessed by key (path) within a bucket | | `accessKeyId` | Shared signing identity | **Not** an AWS credential. Identifies which credential signed the request. The gateway rejects requests with an unrecognized `accessKeyId`. Must match between your gateway config and S3 client (i.e., rclone, boto3, AWS CLI) | | `secretAccessKey` | Shared signing secret | **Not** an AWS credential. Your S3 client (i.e., rclone, boto3, AWS CLI) uses this to sign each request; the gateway uses it to verify the signature. Never sent over the network. Must match between your gateway config and S3 client | | `aptosPrivateKey` | Aptos Ed25519 private key | Your Aptos private key. If you've configured the [Shelby CLI](/tools/cli), you can copy your `private_key` from `~/.shelby/config.yaml` | | Region | `shelbyland` | Default S3 region for SigV4 signing. Must match between gateway config and S3 client | | Authentication | AWS SigV4 signing | Standard S3 signature protocol — works with any S3-compatible client | ## Prerequisites * A Shelby account — if you haven't created one yet, install the [Shelby CLI](/tools/cli) and run `shelby account create` * [rclone](https://rclone.org/install/) (optional, for the verification step) ## Quick Start ### Find your account address If you've set up the [Shelby CLI](/tools/cli), run: ```bash shelby account info ``` Note the **address** field (e.g., `0x0694a79...`). This is your Shelby account address and will be used as your S3 bucket name. ### Create a config file Create a `shelby.config.yaml` file in your working directory: ```yaml title="shelby.config.yaml" network: name: shelbynet rpcEndpoint: https://api.shelbynet.shelby.xyz/shelby aptosFullnode: https://api.shelbynet.shelby.xyz/v1 aptosIndexer: https://api.shelbynet.shelby.xyz/v1/graphql apiKey: server: port: 9000 credentials: - accessKeyId: AKIAIOSFODNN7EXAMPLE secretAccessKey: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY aptosPrivateKey: buckets: - "" ``` | Placeholder | Where to find it | | ------------------------------- | --------------------------------------------------------------------------------------------------- | | `` | Your Shelby API key | | `` | If you've set up the [Shelby CLI](/tools/cli), copy your `private_key` from `~/.shelby/config.yaml` | | `` | If you've set up the [Shelby CLI](/tools/cli), the `address` from `shelby account info` | See [Configuration](/tools/s3-gateway/configuration) for the full reference of all options. ### Start the gateway ```bash npx @shelby-protocol/s3-gateway --config shelby.config.yaml ``` ```bash title="Output" Loaded config from: shelby.config.yaml S3 Gateway is running on http://localhost:9000 API Reference: http://localhost:9000/spec ``` ### Connect an S3 client The gateway authenticates requests using S3-standard [SigV4 signing](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html). Your S3 client needs an **access key ID** and **secret access key** to sign requests — these are shared secrets between the client and the gateway, **not** AWS credentials. When no `credentials` are specified in the gateway config, it uses these defaults: | Field | Default value | | ----------------- | ------------------------------------------ | | `accessKeyId` | `AKIAIOSFODNN7EXAMPLE` | | `secretAccessKey` | `wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY` | | `region` | `shelbyland` | Configure your S3 client with these same values. For example, with rclone: ```bash rclone config create shelby s3 \ provider=Other \ access_key_id=AKIAIOSFODNN7EXAMPLE \ secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \ endpoint=http://localhost:9000 \ region=shelbyland \ force_path_style=true ``` See [Integrations](/tools/s3-gateway/integrations) for setup with other tools (boto3, AWS CLI, DuckDB, Cyberduck, etc.). ### Verify it's working ```bash # List buckets rclone lsd shelby: # List files in a bucket (use your account address) rclone ls shelby: ``` ## CLI Options ```bash npx @shelby-protocol/s3-gateway --help ``` | Option | Description | | --------------------- | ------------------------------------ | | `-c, --config ` | Path to config file | | `-p, --port ` | Port to listen on (overrides config) | | `-h, --help` | Show help message | ## Next Steps * [Configuration](/tools/s3-gateway/configuration) — All configuration options including custom credentials * [Downloads](/tools/s3-gateway/downloads) — Download files, list objects, query with DuckDB * [Uploads](/tools/s3-gateway/uploads) — Upload files (requires Aptos signer) * [Integrations](/tools/s3-gateway/integrations) — Tool-specific setup (rclone, boto3, AWS SDK, etc.) # Integrations (/tools/s3-gateway/integrations) import { Tabs, Tab } from 'fumadocs-ui/components/tabs'; The Shelby S3 Gateway works with any S3-compatible tool or library. This guide covers popular integrations with step-by-step examples. ## rclone [rclone](https://rclone.org/) is a command-line tool for managing files on cloud storage. ### Configuration Add a new remote to your rclone config (`~/.config/rclone/rclone.conf`): ```ini title="~/.config/rclone/rclone.conf" [shelby] type = s3 provider = Other access_key_id = AKIAIOSFODNN7EXAMPLE secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY endpoint = http://localhost:9000 force_path_style = true region = shelbyland ``` Or configure interactively: ```bash rclone config ``` ### Usage Examples ```bash # List all buckets rclone lsd shelby: # List files in a bucket (account) rclone ls shelby: # Download a file rclone copy shelby:/path/to/file.txt ./local-folder/ # Sync a directory rclone sync shelby:/data/ ./local-backup/ # Upload a file # --header-upload sets the required blob expiration # --s3-no-check-bucket skips rclone's bucket existence check rclone copyto ./file.txt shelby:/file.txt \ --header-upload "x-amz-meta-expiration-seconds: 86400" \ --s3-no-check-bucket # Upload large file with multipart rclone copyto ./large-file.zip shelby:/large-file.zip \ --header-upload "x-amz-meta-expiration-seconds: 86400" \ --s3-chunk-size 5M \ --s3-no-check-bucket ``` ## DuckDB [DuckDB](https://duckdb.org/) can query files directly from S3-compatible storage, including Parquet, CSV, and JSON files. ### Configuration ```sql -- Install and load the httpfs extension INSTALL httpfs; LOAD httpfs; -- Configure S3 settings SET s3_endpoint = 'localhost:9000'; SET s3_access_key_id = 'AKIAIOSFODNN7EXAMPLE'; SET s3_secret_access_key = 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY'; SET s3_use_ssl = false; SET s3_url_style = 'path'; SET s3_region = 'shelbyland'; ``` ### Query Examples ```sql -- Query a Parquet file SELECT * FROM read_parquet( 's3:///data/dataset.parquet' ); -- Query a CSV file SELECT * FROM read_csv( 's3:///reports/sales.csv', header = true ); -- Query multiple files with glob pattern SELECT * FROM read_parquet('s3:///data/*.parquet'); ``` ## AWS CLI The [AWS CLI](https://aws.amazon.com/cli/) can be used with the gateway by specifying a custom endpoint. ### Configuration Create a profile for Shelby in `~/.aws/credentials`: ```ini title="~/.aws/credentials" [shelby] aws_access_key_id = AKIAIOSFODNN7EXAMPLE aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY ``` And in `~/.aws/config`: ```ini title="~/.aws/config" [profile shelby] region = shelbyland output = json ``` ### Usage Examples ```bash # List buckets aws --profile shelby --endpoint-url http://localhost:9000 s3 ls # List objects in a bucket aws --profile shelby --endpoint-url http://localhost:9000 \ s3 ls s3:/// # Download a file aws --profile shelby --endpoint-url http://localhost:9000 \ s3 cp s3:///file.txt ./file.txt # Download with a specific path aws --profile shelby --endpoint-url http://localhost:9000 \ s3 cp s3:///data/report.pdf ./downloads/ # Upload a file aws --profile shelby --endpoint-url http://localhost:9000 \ s3 cp ./file.txt s3:///file.txt \ --metadata expiration-seconds=86400 ``` ## AWS SDK (JavaScript/TypeScript) Use the AWS SDK with a custom endpoint configuration: ```typescript import { S3Client, ListBucketsCommand, GetObjectCommand } from "@aws-sdk/client-s3"; const client = new S3Client({ endpoint: "http://localhost:9000", region: "shelbyland", credentials: { accessKeyId: "AKIAIOSFODNN7EXAMPLE", secretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", }, forcePathStyle: true, // Required for non-AWS endpoints }); // List buckets const buckets = await client.send(new ListBucketsCommand({})); console.log(buckets.Buckets); // Get an object const object = await client.send(new GetObjectCommand({ Bucket: "", Key: "path/to/file.txt", })); const content = await object.Body?.transformToString(); // Upload an object import { PutObjectCommand } from "@aws-sdk/client-s3"; await client.send(new PutObjectCommand({ Bucket: "", Key: "path/to/file.txt", Body: "file content", Metadata: { "expiration-seconds": "86400" }, })); ``` ## AWS SDK (Python - boto3) ```python import boto3 # Create S3 client s3 = boto3.client( 's3', endpoint_url='http://localhost:9000', aws_access_key_id='AKIAIOSFODNN7EXAMPLE', aws_secret_access_key='wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', region_name='shelbyland', ) # List buckets response = s3.list_buckets() for bucket in response['Buckets']: print(bucket['Name']) # Download a file s3.download_file( '', 'path/to/file.txt', 'local-file.txt' ) # Upload a file s3.put_object( Bucket='', Key='path/to/file.txt', Body=open('local-file.txt', 'rb'), Metadata={'expiration-seconds': '86400'} ) ``` ## Cyberduck [Cyberduck](https://cyberduck.io/) is a GUI file browser for cloud storage. ### Configuration 1. Open Cyberduck and click **Open Connection** 2. Select **Amazon S3** from the dropdown 3. Configure the connection: | Field | Value | | ----------------- | ------------------------------------------ | | Server | `localhost` | | Port | `9000` | | Access Key ID | `AKIAIOSFODNN7EXAMPLE` | | Secret Access Key | `wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY` | 4. Click **More Options** and set: * Uncheck **Use SSL** * Set **Path Style** to **Path** 5. Click **Connect** ## MinIO Client (mc) The [MinIO Client](https://min.io/docs/minio/linux/reference/minio-mc.html) is another CLI option for S3-compatible storage. ### Configuration ```bash # Add the Shelby gateway as an alias mc alias set shelby http://localhost:9000 \ AKIAIOSFODNN7EXAMPLE \ wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY ``` ### Usage Examples ```bash # List buckets mc ls shelby # List objects mc ls shelby/ # Download a file mc cp shelby//file.txt ./local-file.txt # Get file info mc stat shelby//file.txt ``` ## Troubleshooting ### "InvalidAccessKeyId" Error Ensure your access key ID matches exactly what's configured in `shelby.config.ts`. ### "SignatureDoesNotMatch" Error Common causes: * Region mismatch - ensure client region matches `server.region` in config * Clock skew - ensure your system clock is accurate (within 15 minutes) * Wrong secret key ### "NoSuchBucket" Error The bucket (Shelby account address) is not configured in your `bucketProvider`. Add the account address to your `shelby.config.ts`. ### Connection Refused Ensure the gateway is running (`pnpm dev`) and listening on the expected port. ### SSL/TLS Errors For local development, use `http://` instead of `https://` in your endpoint URL. # Uploads (/tools/s3-gateway/uploads) import { Tabs, Tab } from 'fumadocs-ui/components/tabs'; The S3 Gateway supports uploading files to Shelby storage using standard S3 operations. This guide covers single-part uploads (PutObject) and multipart uploads for large files. ## Prerequisites Before using the examples on this page, you need a running S3 Gateway with a valid configuration that includes `aptosPrivateKey` (YAML) or `aptosSigner` (TypeScript). See [Configuration](/tools/s3-gateway/configuration) to set up your `shelby.config.yaml` and [Integrations](/tools/s3-gateway/integrations) to configure your S3 client (i.e., rclone, boto3, AWS CLI). The bucket name must match your signer's Aptos address. You can only upload to buckets that correspond to accounts you control. ## Supported Write Operations | Operation | Description | | ------------------------- | ----------------------------------- | | `PutObject` | Upload a single file | | `CreateMultipartUpload` | Initiate a multipart upload | | `UploadPart` | Upload a part in a multipart upload | | `CompleteMultipartUpload` | Complete a multipart upload | | `AbortMultipartUpload` | Cancel a multipart upload | ## Expiration Unlike AWS S3 (which uses bucket-level lifecycle rules), Shelby requires an explicit expiration for each blob at upload time. Set the expiration using S3 metadata: | Method | Header | | ------------------------- | -------------------------------------- | | S3 Metadata (recommended) | `x-amz-meta-expiration-seconds: 86400` | | Raw Header (fallback) | `x-expiration-seconds: 86400` | The value is the number of seconds until the blob expires (e.g., `86400` = 24 hours). ## PutObject Upload a single file using the standard S3 PutObject operation. ```bash # Upload a file with 24-hour expiration rclone copyto ./local-file.txt \ shelby:/path/to/file.txt \ --header-upload "x-amz-meta-expiration-seconds: 86400" \ --s3-no-check-bucket # Upload with 7-day expiration rclone copyto ./data.json \ shelby:/data.json \ --header-upload "x-amz-meta-expiration-seconds: 604800" \ --s3-no-check-bucket ``` ```bash # Upload a file with 24-hour expiration aws --profile shelby --endpoint-url http://localhost:9000 \ s3 cp ./local-file.txt \ s3:///path/to/file.txt \ --metadata expiration-seconds=86400 ``` ```python import boto3 s3 = boto3.client( 's3', endpoint_url='http://localhost:9000', aws_access_key_id='AKIAIOSFODNN7EXAMPLE', aws_secret_access_key='wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', region_name='shelbyland', ) # Upload with 24-hour expiration s3.put_object( Bucket='', Key='path/to/file.txt', Body=open('local-file.txt', 'rb'), Metadata={'expiration-seconds': '86400'} ) ``` ## Syncing Directories Use `rclone copy` or `rclone sync` to upload entire directories. Because the gateway returns consistent ETags across all operations, rclone can correctly detect unchanged files and skip re-uploading them on subsequent runs. ```bash # Copy a local directory to Shelby (uploads only new/changed files) rclone copy ./local-dir/ shelby:/ \ --header-upload "x-amz-meta-expiration-seconds: 86400" \ --s3-no-check-bucket # Sync (mirror local to remote — deletes remote files not present locally) rclone sync ./local-dir/ shelby:/ \ --header-upload "x-amz-meta-expiration-seconds: 86400" \ --s3-no-check-bucket ``` Re-running `rclone copy` on an unchanged directory completes instantly — rclone compares ETags from `ListObjects` against the ETags returned by previous `PutObject` calls and skips files that match. This works because the Shelby gateway uses consistent SHA2-256 ETags across all routes (see [Compatibility](/tools/s3-gateway/compatibility#etags-use-sha2-256-not-md5)). ## Multipart Uploads For large files, use multipart uploads. This splits the file into parts that are uploaded separately, then combined on the server. rclone and most S3 SDKs handle multipart uploads automatically for large files. You only need to configure the chunk size threshold. ```bash # Upload large file with multipart (5MB chunks) rclone copyto ./large-file.zip \ shelby:/large-file.zip \ --header-upload "x-amz-meta-expiration-seconds: 86400" \ --s3-chunk-size 5M \ --s3-upload-cutoff 5M \ --s3-no-check-bucket \ -v ``` | Option | Description | | ---------------------- | ------------------------------------------------- | | `--s3-chunk-size` | Size of each part (minimum 5MB) | | `--s3-upload-cutoff` | File size threshold to trigger multipart | | `--s3-no-check-bucket` | Skip bucket existence check (required for Shelby) | | `-v` | Verbose output to see upload progress | ```python import boto3 from boto3.s3.transfer import TransferConfig s3 = boto3.client( 's3', endpoint_url='http://localhost:9000', aws_access_key_id='AKIAIOSFODNN7EXAMPLE', aws_secret_access_key='wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', region_name='shelbyland', ) # Configure multipart settings config = TransferConfig( multipart_threshold=5 * 1024 * 1024, # 5MB multipart_chunksize=5 * 1024 * 1024, # 5MB ) # Upload with multipart s3.upload_file( 'large-file.zip', '', 'large-file.zip', Config=config, ExtraArgs={'Metadata': {'expiration-seconds': '86400'}} ) ``` ```typescript import { S3Client } from "@aws-sdk/client-s3"; import { Upload } from "@aws-sdk/lib-storage"; import { createReadStream } from "fs"; const client = new S3Client({ endpoint: "http://localhost:9000", region: "shelbyland", credentials: { accessKeyId: "AKIAIOSFODNN7EXAMPLE", secretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", }, forcePathStyle: true, }); const upload = new Upload({ client, params: { Bucket: "", Key: "large-file.zip", Body: createReadStream("./large-file.zip"), Metadata: { "expiration-seconds": "86400" }, }, partSize: 5 * 1024 * 1024, // 5MB }); upload.on("httpUploadProgress", (progress) => { console.log(`Uploaded ${progress.loaded} of ${progress.total} bytes`); }); await upload.done(); ``` ## Manual Multipart Upload For fine-grained control, you can manage the multipart upload process manually: ```python import boto3 s3 = boto3.client('s3', endpoint_url='http://localhost:9000', ...) bucket = '' key = 'large-file.zip' # 1. Initiate multipart upload response = s3.create_multipart_upload(Bucket=bucket, Key=key) upload_id = response['UploadId'] try: parts = [] part_number = 1 # 2. Upload parts with open('large-file.zip', 'rb') as f: while chunk := f.read(5 * 1024 * 1024): # 5MB chunks response = s3.upload_part( Bucket=bucket, Key=key, PartNumber=part_number, UploadId=upload_id, Body=chunk ) parts.append({ 'PartNumber': part_number, 'ETag': response['ETag'] }) part_number += 1 # 3. Complete upload (include expiration metadata) s3.complete_multipart_upload( Bucket=bucket, Key=key, UploadId=upload_id, MultipartUpload={'Parts': parts}, Metadata={'expiration-seconds': '86400'} # Set expiration here ) except Exception as e: # Abort on failure s3.abort_multipart_upload(Bucket=bucket, Key=key, UploadId=upload_id) raise ``` For multipart uploads, the `expiration-seconds` metadata is set during the `CompleteMultipartUpload` call, not during `CreateMultipartUpload`. ## Conditional Uploads Use the `If-None-Match: *` header to prevent overwriting existing objects: ```python try: s3.put_object( Bucket='', Key='unique-file.txt', Body=b'content', Metadata={'expiration-seconds': '86400'}, IfNoneMatch='*' # Fail if object exists ) except s3.exceptions.ClientError as e: if e.response['Error']['Code'] == 'PreconditionFailed': print("Object already exists!") raise ``` ```typescript import { PutObjectCommand } from "@aws-sdk/client-s3"; try { await client.send(new PutObjectCommand({ Bucket: "", Key: "unique-file.txt", Body: "content", Metadata: { "expiration-seconds": "86400" }, IfNoneMatch: "*", // Fail if object exists })); } catch (error) { if (error.name === "PreconditionFailed") { console.log("Object already exists!"); } throw error; } ``` ## Error Handling ### Common Errors | Error Code | Description | Solution | | --------------------- | --------------------------------------- | ------------------------------------------------------------------- | | `InvalidArgument` | Missing or invalid expiration | Set `expiration-seconds` metadata | | `AccessDenied` | Bucket doesn't match signer address | Use your signer's address as bucket | | `AccessDenied` | No aptosSigner configured | Add `aptosPrivateKey` in YAML or `aptosSigner` in TypeScript config | | `PreconditionFailed` | Object exists (with `If-None-Match: *`) | Object already exists | | `ObjectAlreadyExists` | Different content at same key | Delete existing object first, then upload | | `EntityTooLarge` | Part exceeds 5GB | Use smaller part size | | `EntityTooSmall` | Non-final part \< 5MB | Increase part size | ### Troubleshooting **"Expiration is required" error:** Ensure you're setting the `expiration-seconds` metadata: * rclone: `--header-upload "x-amz-meta-expiration-seconds: 86400"` * boto3: `Metadata={'expiration-seconds': '86400'}` * AWS CLI: `--metadata expiration-seconds=86400` **"Cannot write to bucket" error:** The bucket name must match your Aptos signer's address. Check that: 1. Your credential has an `aptosSigner` configured 2. The bucket address matches `aptosSigner.accountAddress.toString()` **"Write operations require an Aptos signer" error:** Your credential doesn't have an Aptos signer. Add `aptosPrivateKey` to your credential in `shelby.config.yaml`, or add `aptosSigner` in `shelby.config.ts`. See [Configuration](/tools/s3-gateway/configuration#credentials) for details. # Switch to Testnet on Petra (/tools/wallets/petra-setup) import { Step, Steps } from 'fumadocs-ui/components/steps'; import { Callout } from 'fumadocs-ui/components/callout'; import { Tabs, Tab } from 'fumadocs-ui/components/tabs'; import { NetworkEndpoint } from '@/components/NetworkEndpoint'; import { Network } from '@aptos-labs/ts-sdk'; import AptosFaucet from '@/components/faucet/AptosFaucet'; import ShelbyUSDFaucet from '@/components/faucet/ShelbyUSDFaucet'; This guide will walk you through switching to the Testnet network on your Petra wallet, enabling you to interact with Testnet directly from your browser. ## Prerequisites Before you begin, make sure you have: * **Petra Wallet** installed in your browser ([Download here](https://petra.app/)) ## Switching to Testnet Network ### Switch to Aptos Testnet Network 1. Click the **gear icon** (⚙️) in the bottom right to open Settings 2. Navigate to the **Network** section 3. Select **testnet** from the list of networks 4. You should now see your wallet connected to the Aptos Testnet network You're now connected to the Aptos Testnet network! Your account address remains the same across all Aptos networks. ## Funding Your Account After switching to the Testnet network, you'll need both **APT** (for gas fees) and **ShelbyUSD** (for storage operations). Enter your wallet address in the faucet to receive testnet tokens. ### APT Faucet ### ShelbyUSD Faucet # Localhost API (/apis/rpc/localhost) # Localhost API This section contains all API endpoints for **Local Development** (`http://localhost:9090`). Browse the endpoints by category: * **Sessions** - Manage user sessions and micropayment channels * **Storage** - Upload and retrieve video blobs * **Multipart Uploads** - Manage multipart upload sessions for large files # Shelbynet API (/apis/rpc/shelbynet) # Shelbynet API This section contains all API endpoints for **Shelbynet** (`https://api.shelbynet.shelby.xyz/shelby`). Browse the endpoints by category: * **Sessions** - Manage user sessions and micropayment channels * **Storage** - Upload and retrieve video blobs * **Multipart Uploads** - Manage multipart upload sessions for large files # Overview (/sdks/ethereum-kit/node) # Node.js API The Node.js entry point provides server-side functionality for integrating Ethereum applications with the Shelby Protocol. It's designed for backend services, scripts, and CLI applications where you have direct access to Ethereum wallets via ethers.js. ## Installation npm pnpm yarn bun ```bash npm install @shelby-protocol/ethereum-kit ethers ``` ```bash pnpm add @shelby-protocol/ethereum-kit ethers ``` ```bash yarn add @shelby-protocol/ethereum-kit ethers ``` ```bash bun add @shelby-protocol/ethereum-kit ethers ``` ## Usage Import from the `@shelby-protocol/ethereum-kit/node` entry point: ```typescript import { Shelby, Network } from "@shelby-protocol/ethereum-kit/node"; import { Wallet } from "ethers"; // Initialize the Shelby client const shelbyClient = new Shelby({ network: Network.TESTNET, apiKey: "AG-***", }); // Create a storage account from an Ethereum wallet const ethereumWallet = new Wallet("0x...private_key..."); const storageAccount = shelbyClient.createStorageAccount( ethereumWallet, "my-app.com" ); // Upload data await shelbyClient.upload({ blobData: new Uint8Array([1, 2, 3]), signer: storageAccount, blobName: "example.txt", expirationMicros: Date.now() * 1000 + 86400000000, }); ``` ## When to Use Node.js Use the Node.js entry point when you: * Have direct access to Ethereum private keys (e.g., from environment variables or key files) * Building backend services or APIs * Running scripts or CLI tools * Processing data server-side before storing on Shelby For browser applications with wallet connections, use the [React entry point](/sdks/ethereum-kit/react) instead. ## Exports The `@shelby-protocol/ethereum-kit/node` entry point exports: | Export | Description | | ---------------------- | ------------------------------------------------------- | | `Shelby` | Main client class for Ethereum-Shelby integration | | `ShelbyStorageAccount` | Storage account class (also via `createStorageAccount`) | | `Network` | Network configuration constants | | Re-exports from SDK | All exports from `@shelby-protocol/sdk/node` | ## Next Steps Learn about the Shelby client API and configuration Understand storage account creation and usage
# Shelby Client (/sdks/ethereum-kit/node/shelby-client) The `Shelby` class is the main entry point for server-side Ethereum-Shelby integration. It extends the core `ShelbyClient` with Ethereum-specific functionality. ## Import ```typescript import { Shelby, Network } from "@shelby-protocol/ethereum-kit/node"; ``` ## Constructor ```typescript const shelbyClient = new Shelby({ network: Network.TESTNET, apiKey: "AG-***", gasStationApiKey: "GS-***", // Optional }); ``` ### Parameters | Parameter | Type | Required | Description | | ------------------ | --------------- | -------- | ---------------------------------------------- | | `network` | `ShelbyNetwork` | Yes | The Shelby network (e.g., `Network.TESTNET`) | | `apiKey` | `string` | Yes | Your Shelby API key | | `gasStationApiKey` | `string` | No | Gas Station API key for sponsored transactions | ## Methods ### `createStorageAccount` Creates a storage account from an ethers.js Wallet. ```typescript const storageAccount = shelbyClient.createStorageAccount( ethereumWallet, domain, scheme, ); ``` #### Parameters | Parameter | Type | Required | Description | | ---------------- | -------- | -------- | -------------------------------------- | | `ethereumWallet` | `Wallet` | Yes | An ethers.js Wallet instance | | `domain` | `string` | Yes | The dApp domain for address derivation | | `scheme` | `string` | No | URI scheme (default: `"https"`) | #### Returns Returns a `ShelbyStorageAccount` instance that can be used as a signer. ### Inherited Methods The `Shelby` class inherits all methods from `ShelbyClient`: | Method | Description | | ---------- | --------------------------- | | `upload` | Upload a blob to Shelby | | `download` | Download a blob from Shelby | ### Funding your account To upload files, you will need to fund your account with two assets: 1. **APT tokens**: Fund your account with testnet APT through the [Aptos Testnet Faucet](https://aptos.dev/network/faucet). 2. **ShelbyUSD tokens**: Sign up for early access through the [Shelby Discord](https://discord.gg/shelbyprotocol) to receive testnet ShelbyUSD tokens. ## Complete Example ```typescript import { Shelby, Network } from "@shelby-protocol/ethereum-kit/node"; import { Wallet } from "ethers"; async function main() { // Initialize client const shelbyClient = new Shelby({ network: Network.TESTNET, apiKey: process.env.SHELBY_API_KEY!, }); // Create storage account from private key const ethereumWallet = new Wallet(process.env.ETHEREUM_PRIVATE_KEY!); const storageAccount = shelbyClient.createStorageAccount( ethereumWallet, "my-dapp.com", ); console.log("Storage Account:", storageAccount.accountAddress.toString()); // Fund the storage account with APT through https://aptos.dev/network/faucet // Fund the storage account with ShelbyUSD through the Shelby Discord early access program // Upload a file const blobName = `hello-${Date.now()}.txt`; await shelbyClient.upload({ blobData: new TextEncoder().encode("Hello, Shelby!"), signer: storageAccount, blobName, expirationMicros: Date.now() * 1000 + 86400000000, }); console.log("Done! File available at:"); console.log( `https://api.testnet.shelby.xyz/shelby/v1/blobs/${storageAccount.accountAddress}/${blobName}`, ); } main().catch(console.error); ``` ## Gas Station (Sponsored Transactions) To sponsor gas fees for your users, provide a Gas Station API key: ```typescript const shelbyClient = new Shelby({ network: Network.TESTNET, apiKey: "AG-***", gasStationApiKey: "GS-***", }); ``` With a Gas Station key configured, users don't need APT in their accounts for transaction fees. # Storage Account (/sdks/ethereum-kit/node/storage-account) The `ShelbyStorageAccount` class represents a Shelby storage account derived from an Ethereum wallet. It extends `EIP1193DerivedAccount` from `@aptos-labs/derived-wallet-ethereum`. ## Overview A storage account: * Is derived deterministically from an Ethereum address and domain * Can sign transactions on behalf of the Ethereum wallet * Owns blobs stored on Shelby ## Import ```typescript import { ShelbyStorageAccount } from "@shelby-protocol/ethereum-kit/node"; ``` ## Creation The recommended way to create a storage account is via the `Shelby` client: ```typescript import { Shelby, Network } from "@shelby-protocol/ethereum-kit/node"; import { Wallet } from "ethers"; const shelbyClient = new Shelby({ network: Network.TESTNET, apiKey: "AG-***", }); const ethereumWallet = new Wallet("0x...private_key..."); const storageAccount = shelbyClient.createStorageAccount( ethereumWallet, "my-dapp.com", ); ``` Or directly: ```typescript import { ShelbyStorageAccount } from "@shelby-protocol/ethereum-kit/node"; import { Wallet } from "ethers"; const ethereumWallet = new Wallet("0x...private_key..."); const storageAccount = new ShelbyStorageAccount({ ethereumWallet, domain: "my-dapp.com", }); ``` ## Properties | Property | Type | Description | | ------------------------ | ------------------------- | -------------------------------------- | | `accountAddress` | `AccountAddress` | The derived Aptos account address | | `ethereumWallet` | `Wallet` | The underlying ethers.js Wallet | | `domain` | `string` | The domain used for derivation | | `scheme` | `string` | The URI scheme (default: `"https"`) | | `derivedPublicKey` | `EIP1193DerivedPublicKey` | The derived public key instance | | `authenticationFunction` | `string` | The authentication function identifier | ## Address Derivation The storage account address is deterministically derived from: 1. The Ethereum wallet's address 2. The domain string ```typescript // Same wallet + domain = same storage account address const account1 = shelbyClient.createStorageAccount(wallet, "app.com"); const account2 = shelbyClient.createStorageAccount(wallet, "app.com"); account1.accountAddress.toString() === account2.accountAddress.toString(); // true // Different domain = different storage account address const account3 = shelbyClient.createStorageAccount(wallet, "other.com"); account1.accountAddress.toString() !== account3.accountAddress.toString(); // true ``` Domain scoping provides application-level isolation. The same Ethereum wallet will have different storage accounts on different dApps. ## Using as a Signer The storage account implements the signer interface required by Shelby operations: ```typescript // Upload with the storage account as signer await shelbyClient.upload({ blobData: new Uint8Array([1, 2, 3]), signer: storageAccount, blobName: "example.txt", expirationMicros: Date.now() * 1000 + 86400000000, }); ``` ## Methods The `ShelbyStorageAccount` inherits methods from `AbstractedAccount`: | Method | Description | | ----------------------------------------------- | ---------------------------------------- | | `sign(message)` | Signs a message with the Ethereum wallet | | `signTransactionWithAuthenticator(transaction)` | Signs and wraps for Aptos submission | ## Complete Example ```typescript import { Shelby, Network } from "@shelby-protocol/ethereum-kit/node"; import { Wallet } from "ethers"; async function main() { const shelbyClient = new Shelby({ network: Network.TESTNET, apiKey: "AG-***", }); // Create wallet from private key const ethereumWallet = new Wallet(process.env.ETHEREUM_PRIVATE_KEY!); console.log("Ethereum Address:", ethereumWallet.address); // Create storage account const storageAccount = shelbyClient.createStorageAccount( ethereumWallet, "my-dapp.com", ); console.log("Storage Account:", storageAccount.accountAddress.toString()); console.log("Domain:", storageAccount.domain); console.log("Scheme:", storageAccount.scheme); // Fund the storage account with APT through https://aptos.dev/network/faucet // Fund the storage account with ShelbyUSD through the Shelby Discord early access program // Upload await shelbyClient.upload({ blobData: new TextEncoder().encode("Hello from Ethereum!"), signer: storageAccount, blobName: "hello.txt", expirationMicros: Date.now() * 1000 + 86400000000, }); console.log("Upload complete!"); } main().catch(console.error); ``` # Overview (/sdks/ethereum-kit/react) # React API The React entry point provides client-side functionality for integrating Ethereum wallets with the Shelby Protocol. It's designed for browser applications where users connect their Ethereum wallets (MetaMask, Coinbase Wallet, etc.) via wagmi to interact with Shelby storage. ## Installation npm pnpm yarn bun ```bash npm install @shelby-protocol/ethereum-kit ``` ```bash pnpm add @shelby-protocol/ethereum-kit ``` ```bash yarn add @shelby-protocol/ethereum-kit ``` ```bash bun add @shelby-protocol/ethereum-kit ``` You'll also need wagmi and its dependencies for wallet connections: npm pnpm yarn bun ```bash npm install wagmi viem @tanstack/react-query ``` ```bash pnpm add wagmi viem @tanstack/react-query ``` ```bash yarn add wagmi viem @tanstack/react-query ``` ```bash bun add wagmi viem @tanstack/react-query ``` For simplicity, also install the Shelby react package for react hooks helpers: npm pnpm yarn bun ```bash npm install @shelby-protocol/react ``` ```bash pnpm add @shelby-protocol/react ``` ```bash yarn add @shelby-protocol/react ``` ```bash bun add @shelby-protocol/react ``` ## Wallet Setup The Ethereum Kit works with wagmi-based wallet libraries: * **[RainbowKit](https://www.rainbowkit.com/)** - Popular, beautiful wallet connection UI * **[Web3Modal](https://web3modal.com/)** - WalletConnect's official modal * **[ConnectKit](https://docs.family.co/connectkit)** - Family's wallet connector * **[Dynamic](https://www.dynamic.xyz/)** - Multi-chain wallet integration All of these use wagmi under the hood, so you can use `useWalletClient()` directly with the Ethereum Kit. See your wallet library's documentation for setup instructions. Once configured, the `useWalletClient()` hook provides the wallet needed by Ethereum Kit. ## Usage Import from the `@shelby-protocol/ethereum-kit/react` entry point: ```tsx "use client"; import { useStorageAccount, Network } from "@shelby-protocol/ethereum-kit/react"; import { useUploadBlobs } from "@shelby-protocol/react"; import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { useWalletClient } from "wagmi"; function MyComponent() { // Use wagmi wallet hook const { data: wallet } = useWalletClient(); // Initiate Shelby client const shelbyClient = new ShelbyClient({ network: Network.TESTNET, apiKey: "AG-***", }); // Use Shelby ethereum-kit hook const { storageAccountAddress, signAndSubmitTransaction } = useStorageAccount({ client: shelbyClient, wallet, }); // Use Shelby react package upload blob hook const { mutateAsync: uploadBlobs } = useUploadBlobs({ client: shelbyClient, }); const handleUpload = async () => { await uploadBlobs({ signer: { account: storageAccountAddress, signAndSubmitTransaction }, blobs: [ { blobName: "example.txt", blobData: new Uint8Array([1, 2, 3]), }, ], // 30 days from now in microseconds expirationMicros: (1000 * 60 * 60 * 24 * 30 + Date.now()) * 1000, }); }; return (

Storage Account: {storageAccountAddress?.toString()}

); } ``` ## When to Use React Use the React entry point when you: * Building browser-based dApps * Users connect via Ethereum wallet extensions (MetaMask, Coinbase Wallet, etc.) * Need wallet-based signing (user approves each transaction) * Building React/Next.js applications For server-side applications with direct wallet access, use the [Node.js entry point](/sdks/ethereum-kit/node) instead. ## Exports The `@shelby-protocol/ethereum-kit/react` entry point exports: | Export | Description | | ------------------------------- | ----------------------------------------- | | `useStorageAccount` | React hook for storage account operations | | `EthereumWallet` | Type for viem WalletClient | | `UseStorageAccountParams` | Type for hook parameters | | `UseStorageAccountResult` | Type for hook return value | | `SignAndSubmitTransactionInput` | Type for transaction input | | `SignTransactionInput` | Type for sign-only transaction input | | `SubmitTransactionInput` | Type for submit-only transaction input | | `Network` | Network configuration constants | ## Key Differences from Node.js | Feature | Node.js (`/node`) | React (`/react`) | | --------------- | ------------------------------ | -------------------------------------- | | Signing | Direct wallet signing | Wallet popup for user approval | | Storage Account | `ShelbyStorageAccount` class | Derived via hook | | Environment | Server-side | Browser | | Wallet Access | Full wallet (with private key) | Public address only (wallet holds key) | ## Next Steps Complete reference for the storage account hook
# useStorageAccount (/sdks/ethereum-kit/react/use-storage-account) The `useStorageAccount` hook provides a convenient way to interact with Shelby storage from React applications using a connected Ethereum wallet. ## Overview This hook: * Derives a Shelby storage account address from the connected Ethereum wallet * Provides functions to sign and submit transactions to Shelby * Handles the SIWE (Sign-In With Ethereum) authentication flow automatically ## Import ```typescript import { useStorageAccount } from "@shelby-protocol/ethereum-kit/react"; ``` ## Parameters ```typescript interface UseStorageAccountParams { /** The Shelby client instance */ client: ShelbyClient; /** The connected Ethereum wallet from wagmi's useWalletClient() */ wallet: EthereumWallet | null | undefined; } ``` | Parameter | Type | Required | Description | | --------- | ------------------------ | -------- | ----------------------------- | | `client` | `ShelbyClient` | Yes | The Shelby client instance | | `wallet` | `EthereumWallet \| null` | Yes | The connected Ethereum wallet | ### EthereumWallet Type The `EthereumWallet` type is viem's `WalletClient`, which is what wagmi's `useWalletClient()` returns: ```typescript import type { WalletClient, Transport, Chain, Account } from "viem"; type EthereumWallet = WalletClient; ``` This provides full type safety and autocomplete when used with wagmi. ## Return Value ```typescript interface UseStorageAccountResult { /** The derived Shelby storage account address */ storageAccountAddress: AccountAddress | null; /** Sign a transaction for Shelby */ signTransaction: (params: SignTransactionInput) => Promise<{ authenticator: AccountAuthenticatorAbstraction; rawTransaction: AnyRawTransaction; }>; /** Submit a signed transaction to Shelby */ submitTransaction: ( params: SubmitTransactionInput ) => Promise<{ hash: string }>; /** Sign and submit a transaction in one call */ signAndSubmitTransaction: ( params: SignAndSubmitTransactionInput ) => Promise<{ hash: string }>; } ``` | Property | Type | Description | | -------------------------- | ------------------------ | ------------------------------------------- | | `storageAccountAddress` | `AccountAddress \| null` | The derived storage account address | | `signTransaction` | `function` | Signs a transaction (returns authenticator) | | `submitTransaction` | `function` | Submits a pre-signed transaction | | `signAndSubmitTransaction` | `function` | Signs and submits in one operation | ## Basic Usage ```tsx "use client"; import { useStorageAccount, Network } from "@shelby-protocol/ethereum-kit/react"; import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { useWalletClient } from "wagmi"; import { useMemo } from "react"; function StorageComponent() { const { data: wallet } = useWalletClient(); // Create client (memoize to prevent recreation) const shelbyClient = useMemo( () => new ShelbyClient({ network: Network.TESTNET, apiKey: process.env.NEXT_PUBLIC_SHELBY_API_KEY!, }), [] ); const { storageAccountAddress, signAndSubmitTransaction } = useStorageAccount({ client: shelbyClient, wallet, }); if (!wallet) { return

Please connect your wallet

; } return (

Ethereum Address: {wallet.account.address}

Shelby Storage: {storageAccountAddress?.toString()}

); } ``` ## Domain Derivation The storage account address is derived from: 1. The connected wallet's Ethereum address 2. The current domain (`window.location.host`) The same Ethereum wallet will have **different** storage account addresses on different domains. This provides application-level isolation. ```tsx // On app1.com const { storageAccountAddress } = useStorageAccount({...}); // storageAccountAddress = "0x1111..." // On app2.com (same wallet) const { storageAccountAddress } = useStorageAccount({...}); // storageAccountAddress = "0x2222..." ``` ## With File Upload Use with `@shelby-protocol/react` for file uploads: ```tsx "use client"; import { useStorageAccount, Network } from "@shelby-protocol/ethereum-kit/react"; import { useUploadBlobs } from "@shelby-protocol/react"; import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { useWalletClient } from "wagmi"; import { useMemo } from "react"; function FileUploader() { const { data: wallet } = useWalletClient(); const shelbyClient = useMemo( () => new ShelbyClient({ network: Network.TESTNET, apiKey: process.env.NEXT_PUBLIC_SHELBY_API_KEY!, }), [] ); const { storageAccountAddress, signAndSubmitTransaction } = useStorageAccount({ client: shelbyClient, wallet, }); const { mutateAsync: uploadBlobs, isPending } = useUploadBlobs({ client: shelbyClient, }); const handleUpload = async (file: File) => { if (!storageAccountAddress) return; const buffer = await file.arrayBuffer(); await uploadBlobs({ signer: { account: storageAccountAddress, signAndSubmitTransaction }, blobs: [ { blobName: file.name, blobData: new Uint8Array(buffer), }, ], expirationMicros: Date.now() * 1000 + 86400000000 * 30, // 30 days }); }; return (
e.target.files?.[0] && handleUpload(e.target.files[0])} disabled={!wallet || isPending} /> {isPending &&

Uploading...

}
); } ``` ## How It Works 1. **Automatic Address Derivation**: The hook automatically derives the storage account address from: * The connected wallet's Ethereum address * The current page's domain (`window.location.host`) 2. **SIWE Signing**: When signing transactions, the hook uses Sign-In with Ethereum (SIWE) to create signatures that the Aptos blockchain can verify. 3. **Domain Isolation**: Each dApp domain gets a unique storage account for the same wallet, providing application-level isolation. ## Type Safety with wagmi The hook is designed for seamless integration with wagmi's `useWalletClient()`: ```tsx import { useWalletClient } from "wagmi"; function MyComponent() { // wallet is WalletClient | undefined const { data: wallet } = useWalletClient(); // No type casting needed - EthereumWallet matches WalletClient const { storageAccountAddress } = useStorageAccount({ client: shelbyClient, wallet }); } ``` This works with all wagmi-based wallet libraries: * RainbowKit * Web3Modal * ConnectKit * Dynamic # DApp Example (/sdks/react/guides/dapp-example) import { Step, Steps } from 'fumadocs-ui/components/steps'; # Overview In this guide, we will walk you through building a complete file upload page that integrates with the Aptos Wallet Adapter and uses the Shelby React SDK to upload files to the Shelby network. This guide assumes you have a React application set up and understand the basics of React hooks and file handling. ## Getting Started ### Installation First, install the required dependencies: npm pnpm yarn bun ```bash npm install @shelby-protocol/react @shelby-protocol/sdk @aptos-labs/ts-sdk @aptos-labs/wallet-adapter-react @tanstack/react-query ``` ```bash pnpm add @shelby-protocol/react @shelby-protocol/sdk @aptos-labs/ts-sdk @aptos-labs/wallet-adapter-react @tanstack/react-query ``` ```bash yarn add @shelby-protocol/react @shelby-protocol/sdk @aptos-labs/ts-sdk @aptos-labs/wallet-adapter-react @tanstack/react-query ``` ```bash bun add @shelby-protocol/react @shelby-protocol/sdk @aptos-labs/ts-sdk @aptos-labs/wallet-adapter-react @tanstack/react-query ``` ### Setting up Providers Wrap your application with the necessary providers. Create a providers component: ```tsx title="AppProviders.tsx" "use client"; import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; import { AptosWalletAdapterProvider } from "@aptos-labs/wallet-adapter-react"; import { Network } from "@aptos-labs/ts-sdk"; import type { PropsWithChildren } from "react"; const queryClient = new QueryClient(); export function AppProviders({ children }: PropsWithChildren) { return ( {children} ); } ``` Then wrap your app: ```tsx title="App.tsx" import type { PropsWithChildren } from "react"; import { AppProviders } from "./AppProviders"; function App({ children }: PropsWithChildren) { return ( {children} ); } ``` ### Creating the Shelby Client Create a shared Shelby client instance. You can create a utility file: ```tsx title="lib/shelby.ts" import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { Network } from "@aptos-labs/ts-sdk"; export const shelbyClient = new ShelbyClient({ network: Network.TESTNET, }); ``` It is recommended to create a shared instance of the Shelby client and use it throughout your application. This can be done in a utility file or a context provider. ### Building the Upload Component Now let's create a complete upload component with file input: ```tsx title="components/FileUpload.tsx" "use client"; import { useState, useCallback } from "react"; import { useWallet } from "@aptos-labs/wallet-adapter-react"; import { useUploadBlobs } from "@shelby-protocol/react"; import { shelbyClient } from "@/lib/shelby"; export function FileUpload() { const { account, signAndSubmitTransaction, connected } = useWallet(); const [selectedFiles, setSelectedFiles] = useState([]); const uploadBlobs = useUploadBlobs({ client: shelbyClient, onSuccess: () => { alert("Files uploaded successfully!"); setSelectedFiles([]); }, onError: (error) => { alert(`Upload failed: ${error.message}`); }, }); const handleFileSelect = (e: React.ChangeEvent) => { const files = Array.from(e.target.files || []); setSelectedFiles(files); }; const handleUpload = useCallback(async () => { if (!connected || !account || !signAndSubmitTransaction) { alert("Please connect your wallet first"); return; } if (selectedFiles.length === 0) { alert("Please select at least one file"); return; } // Convert files to Uint8Array const blobs = await Promise.all( selectedFiles.map(async (file) => { const arrayBuffer = await file.arrayBuffer(); return { blobName: file.name, blobData: new Uint8Array(arrayBuffer), }; }), ); // Set expiration time to 7 days from now (in microseconds) const expirationMicros = (Date.now() * 1000) + (7 * 24 * 60 * 60 * 1000 * 1000); // Upload the blobs to the Shelby network uploadBlobs.mutate({ signer: { account: account.accountAddress, signAndSubmitTransaction }, blobs, expirationMicros, }); }, [connected, account, signAndSubmitTransaction, selectedFiles, uploadBlobs]); return (

Upload Files to Shelby

{!connected && (

Please connect your wallet to upload files.

)} {connected && (

{selectedFiles.length > 0 && (

Selected Files:

    {selectedFiles.map((file, index) => (
  • {file.name} ({(file.size / 1024).toFixed(2)} KB)
  • ))}
)}

{uploadBlobs.isError && (

Error: {uploadBlobs.error?.message}

)}
)}
); } ```
### Adding Wallet Connection UI Create a simple wallet connection component: ```tsx title="components/WalletConnection.tsx" "use client"; import { useWallet } from "@aptos-labs/wallet-adapter-react"; export function WalletConnection() { const { connect, disconnect, connected, account, wallet } = useWallet(); if (connected && account) { return (

Connected

{account.address.slice(0, 6)}...{account.address.slice(-4)}

); } return (

Connect your wallet

); } ``` For a more polished wallet connection experience, consider using one of the official wallet adapter UI packages. The Aptos Wallet Adapter provides UI components for popular design systems including [shadcn/ui](https://github.com/aptos-labs/aptos-wallet-adapter/blob/main/packages/wallet-adapter-react/READMEV2.md#use-a-ui-package-recommended), [Ant Design](https://github.com/aptos-labs/aptos-wallet-adapter/tree/main/packages/wallet-adapter-ant-design), and [MUI](https://github.com/aptos-labs/aptos-wallet-adapter/tree/main/packages/wallet-adapter-mui-design). These packages provide pre-built wallet selector modals and connect buttons that you can integrate into your application.
### Complete Example Page Here's a complete example page that brings everything together: ```tsx title="pages/UploadPage.tsx" "use client"; import { FileUpload } from "@/components/FileUpload"; import { WalletConnection } from "@/components/WalletConnection"; export default function UploadPage() { return (

Shelby File Upload

); } ```
## Conclusion You now have a complete file upload page that integrates with the Aptos Wallet Adapter! The page allows users to: * Connect their wallet * Select multiple files * Upload files to the Shelby network * See upload status and errors This example demonstrates the core functionality of the Shelby React SDK. For more advanced usage and additional features, check out the [mutation hooks documentation](/sdks/react/mutations/use-upload-blobs) and [query hooks documentation](/sdks/react/queries/use-account-blobs). Learn more about the useUploadBlobs hook Query your uploaded blobs
# useCommitBlobs (/sdks/react/mutations/use-commit-blobs) This mutation uploads multiple blobs data to the RPC endpoint with configurable concurrency control. ## Example ```tsx import { useCommitBlobs } from "@shelby-protocol/react"; function CommitBlobs() { const commitBlobs = useCommitBlobs({ onSuccess: () => { console.log("Blobs committed successfully"); }, }); const handleCommit = () => { commitBlobs.mutate({ account: "0x123...", blobs: [ { blobName: "file1.txt", blobData: new Uint8Array([/* ... */]) }, { blobName: "file2.txt", blobData: new Uint8Array([/* ... */]) }, ], }); }; return ( ); } ``` ## Parameters ### React Query Options ## Mutation Variables ## Return Value # useEncodeBlobs (/sdks/react/mutations/use-encode-blobs) This mutation generates blob commitments (merkle roots and erasure coding chunks) from raw blob data. It supports custom erasure coding providers and progress callbacks for tracking encoding progress. ## Example ```tsx import { useEncodeBlobs } from "@shelby-protocol/react"; function EncodeBlob() { const encodeBlobs = useEncodeBlobs({ onSuccess: (commitments) => { console.log(`Encoded ${commitments.length} blobs`); console.log("Commitments:", commitments); }, }); const handleEncode = () => { const blobData = new Uint8Array([/* your blob data */]); encodeBlobs.mutate({ blobs: [{ blobData }], }); }; return ( ); } ``` ## Parameters ### React Query Options ## Mutation Variables ## Return Value # useRegisterCommitments (/sdks/react/mutations/use-register-commitments) This mutation registers blob commitments (merkle roots) on the Aptos blockchain as part of the blob upload process. It supports both account signers and wallet adapter signers, and handles batch registration of multiple blobs. ## Examples ### Account Signer ```tsx import { Account } from "@aptos-labs/ts-sdk"; import { useRegisterCommitments } from "@shelby-protocol/react"; function RegisterCommitments() { const { mutateAsync: encodeBlobs } = useEncodeBlobs({ onSuccess: (commitments) => { console.log("Encoded", commitments.length, "blobs"); }, }); const registerCommitments = useRegisterCommitments({ onSuccess: ({ hash }) => { console.log("Transaction hash:", hash); }, }); const handleRegister = async () => { const blobCommitments = await encodeBlobs({ blobs: [{ blobName: "file1.txt", blobData: new Uint8Array([...]) }], }); const signer = Account.generate(); const commitments = [ { blobName: "file1.txt", commitment: blobCommitments[0], }, ]; registerCommitments.mutate({ signer, commitments, expirationMicros: Date.now() * 1000 + 86400000000, // 1 day }); }; return ( ); } ``` ### Wallet Adapter ```tsx import { useRegisterCommitments } from "@shelby-protocol/react"; import { useWallet } from "@aptos-labs/wallet-adapter-react"; function RegisterCommitmentsWithWallet() { const { mutateAsync: encodeBlobs } = useEncodeBlobs({ onSuccess: (commitments) => { console.log("Encoded", commitments.length, "blobs"); }, }); const { account, signAndSubmitTransaction } = useWallet(); const registerCommitments = useRegisterCommitments(); const handleRegister = async () => { if (!account || !signAndSubmitTransaction) { alert("Please connect your wallet"); return; } const blobCommitments = await encodeBlobs({ blobs: [{ blobName: "file1.txt", blobData: new Uint8Array([...]) }] }); registerCommitments.mutate({ signer: { account: account.accountAddress, signAndSubmitTransaction }, commitments: [ { blobName: "file1.txt", commitment: blobCommitments[0], }, ], expirationMicros: Date.now() * 1000 + 86400000000, }); }; return ( ); } ``` ## Parameters ### React Query Options ## Mutation Variables ## Return Value # useUploadBlobs (/sdks/react/mutations/use-upload-blobs) This mutation handles the complete blob upload process including encoding blobs with erasure coding, registering commitments on-chain (if not already registered), and uploading blob data to the RPC endpoint. It supports both account signers and wallet adapter signers, and includes logic to skip registration for blobs that already exist. ## Examples ### Account Signer ```tsx import { Account } from "@aptos-labs/ts-sdk"; import { useUploadBlobs } from "@shelby-protocol/react"; function UploadBlob() { const uploadBlobs = useUploadBlobs({ onSuccess: () => { console.log("Upload complete!"); }, }); const handleUpload = async () => { const signer = Account.generate(); const fileData = new Uint8Array([/* your file data */]); uploadBlobs.mutate({ signer, blobs: [{ blobName: "example.txt", blobData: fileData }], expirationMicros: Date.now() * 1000 + 86400000000, // 1 day }); }; return ( ); } ``` ### Wallet Adapter ```tsx import { useUploadBlobs } from "@shelby-protocol/react"; import { useWallet } from "@aptos-labs/wallet-adapter-react"; function UploadBlobWithWallet() { const { account, signAndSubmitTransaction } = useWallet(); const uploadBlobs = useUploadBlobs(); const handleUpload = async () => { if (!account || !signAndSubmitTransaction) { alert("Please connect your wallet"); return; } const fileData = new Uint8Array([/* your file data */]); uploadBlobs.mutate({ signer: { account: account.accountAddress, signAndSubmitTransaction }, blobs: [{ blobName: "example.txt", blobData: fileData }], expirationMicros: Date.now() * 1000 + 86400000000, }); }; return ( ); } ``` ## Parameters ### React Query Options ## Mutation Variables ## Return Value # useAccountBlobs (/sdks/react/queries/use-account-blobs) This query fetches blob metadata for a specific account with support for pagination, filtering, and ordering. ## Example ```tsx import { useAccountBlobs } from "@shelby-protocol/react"; function BlobList({ account }: { account: string }) { const { data: blobs, isLoading, error } = useAccountBlobs({ account, }); if (isLoading) return
Loading...
; if (error) return
Error: {error.message}
; return (
    {blobs?.map((blob) => (
  • {blob.name}
  • ))}
); } ``` ## Parameters ### React Query Options ## Return Value # useBlobMetadata (/sdks/react/queries/use-blob-metadata) This query fetches the metadata for a single blob identified by account and blob name. Returns `null` if the blob is not found. ## Example ```tsx import { useBlobMetadata } from "@shelby-protocol/react"; function BlobDetails({ account, name }: { account: string; name: string }) { const { data: metadata, isLoading, error } = useBlobMetadata({ account, name, }); if (isLoading) return
Loading...
; if (error) return
Error: {error.message}
; if (!metadata) return
Blob not found
; return (

{metadata.name}

Size: {metadata.size} bytes

Created: {new Date(metadata.created_at).toLocaleString()}

Updated: {new Date(metadata.updated_at).toLocaleString()}

); } ``` ## Parameters ### React Query Options ## Return Value # Fund Your Account (/sdks/solana-kit/guides/fund-account) import { Step, Steps } from "fumadocs-ui/components/steps"; import { Tab, Tabs } from "fumadocs-ui/components/tabs"; # Funding Your Storage Account Before uploading files to Shelby, your storage account needs to hold two types of tokens: 1. **ShelbyUSD tokens** - Used to pay for blob storage on the Shelby network 2. **APT tokens** - Used to pay for gas fees when sending transactions ## Funding Methods 1. **APT tokens**: Fund your storage account with testnet APT through the [Aptos Testnet Faucet](https://aptos.dev/network/faucet). 2. **ShelbyUSD tokens**: Sign up for early access through the [Shelby Discord](https://discord.gg/shelbyprotocol) to receive testnet ShelbyUSD tokens. # Retrieve a File (/sdks/solana-kit/guides/retrieve-file) import { Tab, Tabs } from "fumadocs-ui/components/tabs"; # Retrieving Files from Shelby Once you've uploaded files to Shelby, you can retrieve them using several methods. ## Using Shelby Explorer The easiest way to view and download your files is through the [Shelby Explorer](https://explorer.shelby.xyz/testnet): 1. Navigate to the explorer 2. Search for your storage account address 3. Browse and download your blobs ## Using HTTP GET Request You can download files directly using a GET request to the Shelby RPC endpoint. ### URL Format ``` https://api.testnet.shelby.xyz/shelby/v1/blobs/{account_address}/{blob_name} ``` ### cURL Example ```bash curl -X GET "https://api.testnet.shelby.xyz/shelby/v1/blobs/0x1234...abcd/example.txt" ``` ### Node/React Example ```typescript const accountAddress = storageAccount.accountAddress.toString(); const blobName = "example.txt"; const response = await fetch( `https://api.testnet.shelby.xyz/shelby/v1/blobs/${accountAddress}/${blobName}` ); if (response.ok) { const data = await response.arrayBuffer(); console.log("Downloaded:", new Uint8Array(data)); } else { console.error("Download failed:", response.statusText); } ``` ```tsx const { storageAccountAddress } = useStorageAccount({ client: shelbyClient, solanaAddress: publicKey?.toBase58(), signMessageFn: signMessage, }); const handleDownload = async (blobName: string) => { if (!storageAccountAddress) return; const response = await fetch( `https://api.testnet.shelby.xyz/shelby/v1/blobs/${storageAccountAddress}/${blobName}` ); if (response.ok) { const blob = await response.blob(); // Create download link const url = URL.createObjectURL(blob); const a = document.createElement("a"); a.href = url; a.download = blobName; a.click(); URL.revokeObjectURL(url); } }; ``` ## Listing Account Blobs To see all blobs owned by an account, use the Shelby SDK: ```typescript import { Shelby, Network } from "@shelby-protocol/solana-kit/node"; const shelbyClient = new Shelby({ network: Network.TESTNET, connection, apiKey: "AG-***", }); const blobs = await shelbyClient.getAccountBlobs({ account: storageAccount.accountAddress.toString(), }); console.log("Account blobs:"); for (const blob of blobs) { console.log(`- ${blob.name} (owner: ${blob.owner})`); } ``` ```tsx import { useAccountBlobs } from "@shelby-protocol/react"; import { useStorageAccount, Network } from "@shelby-protocol/solana-kit/react"; import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { useEffect, useMemo, useState } from "react"; function BlobList() { const { publicKey, signMessage } = useWallet(); const shelbyClient = useMemo( () => new ShelbyClient({ network: Network.TESTNET, apiKey: process.env.NEXT_PUBLIC_SHELBY_API_KEY!, }), [] ); const { storageAccountAddress } = useStorageAccount({ client: shelbyClient, solanaAddress: publicKey?.toBase58(), signMessageFn: signMessage, }); const { data: blobs } = useAccountBlobs({ client: shelbyClient, account: storageAccountAddress, }); return (
    {blobs.map((blob) => (
  • {blob.blobNameSuffix} -{" "}
  • ))}
); } ```
# Upload a File (/sdks/solana-kit/guides/upload-file) import { Step, Steps } from "fumadocs-ui/components/steps"; import { Tab, Tabs } from "fumadocs-ui/components/tabs"; # Uploading Files to Shelby This guide walks you through uploading files to the Shelby Protocol using the Solana Kit SDK. ## Complete Upload Example ### Set Up the Client ```typescript import { Shelby, Network } from "@shelby-protocol/solana-kit/node"; import { Connection, Keypair } from "@solana/web3.js"; const connection = new Connection("https://api.devnet.solana.com"); const shelbyClient = new Shelby({ network: Network.TESTNET, connection, apiKey: "AG-***", }); ``` ### Create a Storage Account ```typescript const solanaKeypair = Keypair.generate(); const domain = "my-awesome-dapp.com"; const storageAccount = shelbyClient.createStorageAccount(solanaKeypair, domain); ``` ### Fund the Account 1. **APT tokens**: Fund your storage account with testnet APT through the [Aptos Testnet Faucet](https://aptos.dev/network/faucet). 2. **ShelbyUSD tokens**: Sign up for early access through the [Shelby Discord](https://discord.gg/shelbyprotocol) to receive testnet ShelbyUSD tokens. ### Upload the File ```typescript import * as fs from "fs"; // Read file from disk const fileData = fs.readFileSync("./my-file.txt"); const blobData = new Uint8Array(fileData); // Set expiration (1 day from now) const expirationMicros = Date.now() * 1000 + 86400000000; // Upload to Shelby await shelbyClient.upload({ blobData, signer: storageAccount, blobName: "my-file.txt", expirationMicros, }); console.log("File uploaded successfully!"); ``` ### Set Up the Hook ```tsx "use client"; import { useStorageAccount, Network } from "@shelby-protocol/solana-kit/react"; import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { useWalletConnection } from "@solana/react-hooks"; import { useMemo, useState } from "react"; function FileUploader() { const { wallet } = useWalletConnection(); const [status, setStatus] = useState(""); const shelbyClient = useMemo( () => new ShelbyClient({ network: Network.TESTNET, apiKey: process.env.NEXT_PUBLIC_SHELBY_API_KEY!, }), [], ); const { storageAccountAddress, signAndSubmitTransaction } = useStorageAccount( { client: shelbyClient, wallet, }, ); // ... continue in next step } ``` ### Fund the Account (if needed) Fund your storage account with testnet tokens: 1. **APT tokens**: Fund through the [Aptos Testnet Faucet](https://aptos.dev/network/faucet). 2. **ShelbyUSD tokens**: Sign up for early access through the [Shelby Discord](https://discord.gg/shelbyprotocol) to receive testnet ShelbyUSD tokens. ### Handle File Upload ```tsx import { useUploadBlobs } from "@shelby-protocol/react"; const { mutateAsync: uploadBlobs } = useUploadBlobs({ client: shelbyClient, }); const handleFileUpload = async (event: React.ChangeEvent) => { const file = event.target.files?.[0]; if (!file || !storageAccountAddress) return; setStatus("Reading file..."); const arrayBuffer = await file.arrayBuffer(); const blobData = new Uint8Array(arrayBuffer); setStatus("Uploading to Shelby..."); try { const expirationMicros = Date.now() * 1000 + 86400000000; // 1 day await uploadBlobs({ signer: { account: storageAccountAddress, signAndSubmitTransaction }, blobs: [ { blobName: "example.txt", blobData, }, ], expirationMicros, }); setStatus("Upload complete!"); } catch (error) { setStatus(`Error: ${error}`); } }; ``` ### Render the Component ```tsx return (
{!wallet ? (

Please connect your wallet

) : ( <>

Storage Account: {storageAccountAddress?.toString()}

{status}

)}
); ```
## Next Steps After uploading, learn how to retrieve your files: Learn how to download and access your uploaded blobs
# Overview (/sdks/solana-kit/node) # Node.js API The Node.js entry point provides server-side functionality for integrating Solana applications with the Shelby Protocol. It's designed for backend services, scripts, and CLI applications where you have direct access to Solana keypairs. ## Installation npm pnpm yarn bun ```bash npm install @shelby-protocol/solana-kit @solana/web3.js ``` ```bash pnpm add @shelby-protocol/solana-kit @solana/web3.js ``` ```bash yarn add @shelby-protocol/solana-kit @solana/web3.js ``` ```bash bun add @shelby-protocol/solana-kit @solana/web3.js ``` ## Usage Import from the `@shelby-protocol/solana-kit/node` entry point: ```typescript import { Shelby, Network } from "@shelby-protocol/solana-kit/node"; import { Connection, Keypair } from "@solana/web3.js"; // Create a Solana connection const connection = new Connection("https://api.devnet.solana.com"); // Initialize the Shelby client const shelbyClient = new Shelby({ network: Network.TESTNET, connection, apiKey: "AG-***", }); // Create a storage account from a Solana keypair const solanaKeypair = Keypair.generate(); const storageAccount = shelbyClient.createStorageAccount( solanaKeypair, "my-app.com" ); // Upload data await shelbyClient.upload({ blobData: new Uint8Array([1, 2, 3]), signer: storageAccount, blobName: "example.txt", expirationMicros: Date.now() * 1000 + 86400000000, }); ``` ## When to Use Node.js Use the Node.js entry point when you: * Have direct access to Solana keypairs (e.g., from environment variables or key files) * Building backend services or APIs * Running scripts or CLI tools * Processing data server-side before storing on Shelby For browser applications with wallet connections, use the [React entry point](/sdks/solana-kit/react) instead. ## Exports The `@shelby-protocol/solana-kit/node` entry point exports: | Export | Description | | ---------------------- | ------------------------------------------------------- | | `Shelby` | Main client class for Solana-Shelby integration | | `ShelbyStorageAccount` | Storage account class (also via `createStorageAccount`) | | `Network` | Network configuration constants | | Re-exports from SDK | All exports from `@shelby-protocol/sdk/node` | ## Next Steps Learn about the Shelby client API and configuration Understand storage account creation and usage Step-by-step guide to uploading files
# Shelby Client (/sdks/solana-kit/node/shelby-client) The `Shelby` class is the main entry point for interacting with the Shelby Protocol from a Solana application. It extends the core `ShelbyClient` with Solana-specific functionality. ## Constructor ```typescript import { Shelby, Network } from "@shelby-protocol/solana-kit/node"; import { Connection } from "@solana/web3.js"; const shelbyClient = new Shelby({ network: Network.TESTNET, connection: new Connection("https://api.devnet.solana.com"), apiKey: "AG-***", }); ``` ## Parameters | Parameter | Type | Required | Description | | ------------ | --------------- | -------- | ------------------------------------------------------------ | | `network` | `ShelbyNetwork` | Yes | The Shelby network to connect with (e.g., `Network.TESTNET`) | | `connection` | `Connection` | Yes | The Solana network connection | | `apiKey` | `string` | Yes | Your Shelby API key | ## Methods ### createStorageAccount Creates a Shelby storage account derived from a Solana keypair. ```typescript const storageAccount = shelbyClient.createStorageAccount(solanaKeypair, domain); ``` #### Parameters | Parameter | Type | Description | | --------------- | --------- | -------------------------------------------------------- | | `solanaKeypair` | `Keypair` | The Solana keypair that will control the storage account | | `domain` | `string` | The dApp domain for account isolation | #### Returns Returns a `ShelbyStorageAccount` instance that can be used as a signer for transactions. ## Inherited Methods The `Shelby` class extends `ShelbyClient` and inherits all its methods: ### upload Upload blobs to the Shelby network. ```typescript await shelbyClient.upload({ blobData: new Uint8Array([1, 2, 3]), signer: storageAccount, blobName: "example.txt", expirationMicros: Date.now() * 1000 + 86400000000, }); ``` ### Funding your account To upload files, you will need to fund your account with two assets: 1. **APT tokens**: Fund your storage account with testnet APT through the [Aptos Testnet Faucet](https://aptos.dev/network/faucet). 2. **ShelbyUSD tokens**: Sign up for early access through the [Shelby Discord](https://discord.gg/shelbyprotocol) to receive testnet ShelbyUSD tokens. For a complete list of inherited methods, see the [Shelby SDK documentation](/sdks/typescript). # Storage Account (/sdks/solana-kit/node/storage-account) The `ShelbyStorageAccount` class represents a cross-chain storage account that allows Solana keypairs to own and manage data on the Shelby Protocol. ## Overview A Shelby Storage Account is derived from: 1. **Solana Keypair** - Links the original keypair with the storage account 2. **Domain** - Provides application-level isolation This derivation uses [Aptos Derivable Account Abstraction](https://github.com/aptos-foundation/AIPs/blob/main/aips/aip-113.md) to create a unique Aptos address controlled by the Solana keypair. ## Creating a Storage Account ```typescript import { Shelby, Network } from "@shelby-protocol/solana-kit/node"; import { Connection, Keypair } from "@solana/web3.js"; const connection = new Connection("https://api.devnet.solana.com"); const shelbyClient = new Shelby({ network: Network.TESTNET, connection, apiKey: "AG-***", }); const solanaKeypair = Keypair.generate(); const domain = "my-awesome-dapp.com"; const storageAccount = shelbyClient.createStorageAccount(solanaKeypair, domain); ``` ## Parameters | Parameter | Type | Description | | --------------- | --------- | ---------------------------------------------------- | | `solanaKeypair` | `Keypair` | The Solana keypair that controls the storage account | | `domain` | `string` | The dApp domain for account isolation | ## Properties ### accountAddress The address of the storage account. ```typescript console.log(storageAccount.accountAddress.toString()); // Output: "0x1234...abcd" ``` ### solanaKeypair The underlying Solana keypair. ```typescript console.log(storageAccount.solanaKeypair.publicKey.toBase58()); // Output: "7xKX...9abc" ``` ### domain The domain used for account derivation. ```typescript console.log(storageAccount.domain); // Output: "my-awesome-dapp.com" ``` ## Usage as a Signer The storage account can be used as a signer for Shelby operations: ```typescript // Upload a blob await shelbyClient.upload({ blobData: new Uint8Array([1, 2, 3]), signer: storageAccount, // Use as signer blobName: "example.txt", expirationMicros: Date.now() * 1000 + 86400000000, }); ``` ## Domain Isolation Storage accounts are scoped to the dApp domain. The same Solana keypair will have **different** storage account addresses on different domains. ```typescript const keypair = Keypair.generate(); const account1 = shelbyClient.createStorageAccount(keypair, "app1.com"); const account2 = shelbyClient.createStorageAccount(keypair, "app2.com"); // Different addresses! console.log(account1.accountAddress.toString()); // "0x1111..." console.log(account2.accountAddress.toString()); // "0x2222..." ``` This isolation ensures that: * Users have separate storage spaces per application * Applications cannot access each other's data * Domain-specific permissions are enforced # Overview (/sdks/solana-kit/react) # React API The React entry point provides client-side functionality for integrating Solana wallets with the Shelby Protocol. It's designed for browser applications where users connect their Solana wallets (Phantom, Solflare, etc.) to interact with Shelby storage. ## Installation npm pnpm yarn bun ```bash npm install @shelby-protocol/solana-kit ``` ```bash pnpm add @shelby-protocol/solana-kit ``` ```bash yarn add @shelby-protocol/solana-kit ``` ```bash bun add @shelby-protocol/solana-kit ``` For simplicity, also install the Shelby react package for react hooks helpers npm pnpm yarn bun ```bash npm install @shelby-protocol/react ``` ```bash pnpm add @shelby-protocol/react ``` ```bash yarn add @shelby-protocol/react ``` ```bash bun add @shelby-protocol/react ``` ## Solana project setup Set up your Solana project with the recommended stack: npm pnpm yarn bun ```bash npx create-solana-dapp ``` ```bash pnpm dlx create-solana-dapp ``` ```bash yarn dlx create-solana-dapp ``` ```bash bun x create-solana-dapp ``` This scaffolds a project with `@solana/kit` and `@solana/react-hooks` for wallet connections. If you're using `@solana/wallet-adapter-react` instead, see the [adapter compatibility section](#using-with-solana-wallet-adapter-react) below. ## Usage Import from the `@shelby-protocol/solana-kit/react` entry point: ```tsx "use client"; import { useStorageAccount, Network } from "@shelby-protocol/solana-kit/react"; import { useUploadBlobs } from "@shelby-protocol/react"; import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { useWalletConnection } from "@solana/react-hooks"; function MyComponent() { // use Solana wallet hook const { wallet } = useWalletConnection(); // initiate Shelby client const shelbyClient = new ShelbyClient({ network: Network.TESTNET, apiKey: "AG-***", }); // use Shelby solana-kit hook const { storageAccountAddress, signAndSubmitTransaction } = useStorageAccount({ client: shelbyClient, wallet, }); // use Shelby react package upload blob hook const { mutateAsync: uploadBlobs } = useUploadBlobs({ client: shelbyClient, }); const handleUpload = async () => { await uploadBlobs({ signer: { account: storageAccountAddress, signAndSubmitTransaction }, blobs: [ { blobName: "example.txt", blobData: new Uint8Array([1, 2, 3]), }, ], // 30 days from now in microseconds expirationMicros: (1000 * 60 * 60 * 24 * 30 + Date.now()) * 1000, }); }; return (

Storage Account: {storageAccountAddress?.toString()}

); } ``` ## Using with `@solana/wallet-adapter-react` If you're using the older `@solana/wallet-adapter-react` package, adapt the wallet to the expected shape: ```tsx import { useWallet } from "@solana/wallet-adapter-react"; function MyComponent() { const { publicKey, signMessage, signIn } = useWallet(); const { storageAccountAddress, signAndSubmitTransaction } = useStorageAccount({ client: shelbyClient, wallet: publicKey ? { account: { address: publicKey }, signMessage, signIn, } : null, }); // ... } ``` ## When to Use React Use the React entry point when you: * Building browser-based dApps * Users connect via Solana wallet extensions (Phantom, Solflare, etc.) * Need wallet-based signing (user approves each transaction) * Building React/Next.js applications For server-side applications with direct keypair access, use the [Node.js entry point](/sdks/solana-kit/node) instead. ## Exports The `@shelby-protocol/solana-kit/react` entry point exports: | Export | Description | | ------------------------------- | ----------------------------------------- | | `useStorageAccount` | React hook for storage account operations | | `SolanaWallet` | Type for compatible Solana wallet | | `UseStorageAccountParams` | Type for hook parameters | | `UseStorageAccountResult` | Type for hook return value | | `SignAndSubmitTransactionInput` | Type for transaction input | | `SignTransactionInput` | Type for sign-only transaction input | | `SubmitTransactionInput` | Type for submit-only transaction input | | `Network` | Network configuration constants | ## Key Differences from Node.js | Feature | Node.js (`/node`) | React (`/react`) | | --------------- | ------------------------------ | ------------------------------------- | | Signing | Direct keypair signing | Wallet popup for user approval | | Storage Account | `ShelbyStorageAccount` class | Derived via hook | | Environment | Server-side | Browser | | Keypair Access | Full keypair (public + secret) | Public key only (wallet holds secret) | ## Next Steps Complete reference for the storage account hook Step-by-step guide with React examples
# useStorageAccount (/sdks/solana-kit/react/use-storage-account) The `useStorageAccount` hook provides a convenient way to interact with Shelby storage from React applications using a connected Solana wallet. ## Overview This hook: * Derives a Shelby storage account address from the connected Solana wallet * Provides functions to sign and submit transactions to Shelby * Handles the SIWS (Sign-In With Solana) authentication flow automatically ## Import ```typescript import { useStorageAccount } from "@shelby-protocol/solana-kit/react"; ``` ## Parameters ```typescript interface UseStorageAccountParams { /** The Shelby client instance */ client: ShelbyClient; /** The connected Solana wallet */ wallet: SolanaWallet | null; } ``` | Parameter | Type | Required | Description | | --------- | ---------------------- | -------- | --------------------------- | | `client` | `ShelbyClient` | Yes | The Shelby client instance | | `wallet` | `SolanaWallet \| null` | Yes | The connected Solana wallet | ### SolanaWallet Interface The `SolanaWallet` interface is compatible with `@solana/react-hooks`: ```typescript interface SolanaWallet { account: { address: { toString(): string }; }; signMessage?: (message: Uint8Array) => Promise; signIn?: (input: SolanaSignInInput) => Promise; } ``` ## Return Value ```typescript interface UseStorageAccountResult { /** The derived Shelby storage account address */ storageAccountAddress: AccountAddress | null; /** Sign a transaction for Shelby */ signTransaction: (params: SignTransactionInput) => Promise<{ authenticator: AccountAuthenticatorAbstraction; rawTransaction: AnyRawTransaction; }>; /** Submit a signed transaction to Shelby */ submitTransaction: ( params: SubmitTransactionInput ) => Promise<{ hash: string }>; /** Sign and submit a transaction in one call */ signAndSubmitTransaction: ( params: SignAndSubmitTransactionInput ) => Promise<{ hash: string }>; } ``` | Property | Type | Description | | -------------------------- | ------------------------ | ------------------------------------------- | | `storageAccountAddress` | `AccountAddress \| null` | The derived storage account address | | `signTransaction` | `function` | Signs a transaction (returns authenticator) | | `submitTransaction` | `function` | Submits a pre-signed transaction | | `signAndSubmitTransaction` | `function` | Signs and submits in one operation | ## Basic Usage ```tsx "use client"; import { useStorageAccount, Network } from "@shelby-protocol/solana-kit/react"; import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { useWalletConnection } from "@solana/react-hooks"; import { useMemo } from "react"; function StorageComponent() { const { wallet } = useWalletConnection(); // Create client (memoize to prevent recreation) const shelbyClient = useMemo( () => new ShelbyClient({ network: Network.TESTNET, apiKey: process.env.NEXT_PUBLIC_SHELBY_API_KEY!, }), [] ); const { storageAccountAddress, signAndSubmitTransaction } = useStorageAccount({ client: shelbyClient, wallet: wallet, }); if (!wallet) { return

Please connect your wallet

; } return (

Solana Address: {wallet.account.address.toString()}

Shelby Storage: {storageAccountAddress?.toString()}

); } ``` ### Using with `@solana/wallet-adapter-react` If you're using `@solana/wallet-adapter-react`, adapt the wallet to the expected shape: ```tsx import { useWallet } from "@solana/wallet-adapter-react"; function StorageComponent() { const { publicKey, signMessage, signIn } = useWallet(); const { storageAccountAddress } = useStorageAccount({ client: shelbyClient, wallet: publicKey ? { account: { address: publicKey }, signMessage, signIn, } : null, }); // ... } ``` ## Domain Derivation The storage account address is derived from: 1. The connected Solana wallet's public key 2. The current domain (`window.location.host`) The same Solana wallet will have **different** storage account addresses on different domains. This provides application-level isolation. ```tsx // On app1.com const { storageAccountAddress } = useStorageAccount({...}); // storageAccountAddress = "0x1111..." // On app2.com (same wallet) const { storageAccountAddress } = useStorageAccount({...}); // storageAccountAddress = "0x2222..." ``` # Overview (/sdks/typescript/browser) # Browser API Client-side functionality optimized for browser environments with Web APIs. ## Browser Implementation The browser package provides the same core functionality as the Node.js client but optimized for browser environments. It uses the same `ShelbyClient` base class with browser-compatible implementations. ```typescript import { ShelbyClient } from "@shelby-protocol/sdk/browser"; ``` ## Usage ```typescript import { ShelbyClient } from '@shelby-protocol/sdk/browser' import { Network } from '@aptos-labs/ts-sdk' // Create client configuration const config = { network: Network.TESTNET apiKey: process.env.SHELBY_API_KEY, } // Initialize the Shelby client const shelbyClient = new ShelbyClient(config) ``` # Overview (/sdks/typescript/core) # Introduction The core package contains environment agnostic types and functions that can be used to interact with the Shelby network. This shared package will typically contain the types and functions that are shared between the Node.js and browser environments. ## Installation npm pnpm yarn bun ```bash npm install @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ```bash pnpm add @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ```bash yarn add @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ```bash bun add @shelby-protocol/sdk @aptos-labs/ts-sdk ``` This package comes pre-packaged with the Node.js and browser packages. You can access it by using the `@shelby-protocol/sdk` entry point. **Node.js Entry Point** ```typescript import { ShelbyBlob } from '@shelby-protocol/sdk/node' ``` **Browser Entry Point** ```typescript import { ShelbyBlob } from '@shelby-protocol/sdk/browser' ``` # Specifications (/sdks/typescript/core/specifications) import { TypeTable } from 'fumadocs-ui/components/type-table' ## ShelbyBlobClient Blockchain-focused client for managing blob commitments and metadata on-chain. ### Methods #### `registerBlob({account, blobName, blobMerkleRoot, size, expirationMicros, options})` Registers a blob on the blockchain by writing its commitments. | Parameter | Type | Description | | ------------------ | ----------------------------- | --------------------------------------- | | `account` | `Account` | The account to register the blob for | | `blobName` | `BlobName` | The name/path of the blob | | `blobMerkleRoot` | `string` | The merkle root of the blob | | `size` | `number` | The size of the blob in bytes | | `expirationMicros` | `number` | The expiration time in microseconds | | `options` | `WriteBlobCommitmentsOptions` | Optional write blob commitments options | **Returns:** `Promise<{ transaction: PendingTransactionResponse }>` #### `confirmBlobChunks({signer, account, blobName, signedChunksetChunkCommitments, options})` Confirms the blob chunks for a given blob. | Parameter | Type | Description | | -------------------------------- | --------------------------- | ------------------------------------------ | | `signer` | `Account` | The account to confirm the blob chunks for | | `account` | `AccountAddressInput` | The account address | | `blobName` | `string` | The name/path of the blob | | `signedChunksetChunkCommitments` | `SignedChunkCommitment[][]` | The signed chunk commitments for the blob | | `options` | `ConfirmBlobChunksOptions` | Optional confirm blob chunks options | **Returns:** `Promise<{ transaction: PendingTransactionResponse }>` #### `getBlobMetadata({account, name})` Retrieves blob metadata from the blockchain. | Parameter | Type | Description | | --------- | --------------------- | ------------------------- | | `account` | `AccountAddressInput` | The account address | | `name` | `string` | The name/path of the blob | **Returns:** `Promise` #### `getAccountBlobs({account})` Gets all blob metadata for a specific account. | Parameter | Type | Description | | --------- | --------------------- | ------------------- | | `account` | `AccountAddressInput` | The account address | **Returns:** `Promise` *** ## ShelbyRPCClient The client to interact with the Shelby RPC node which is responsible for storing, confirming, and retrieving blobs from the storage layer. ### Methods #### `putBlob({account, blobName, blobData})` Uploads blob data to Shelby storage using multipart upload for reliability. | Parameter | Type | Description | | ---------- | --------------------- | ------------------------------------------- | | `account` | `AccountAddressInput` | The account address to store the blob under | | `blobName` | `string` | The name/path of the blob | | `blobData` | `Uint8Array` | The blob data to upload | #### `getBlob({account, blobName, range?})` Downloads blob data as a readable stream with optional byte range support. | Parameter | Type | Description | | ---------- | --------------------------------- | ------------------------- | | `account` | `AccountAddressInput` | The account address | | `blobName` | `string` | The name/path of the blob | | `range` | `{ start: number; end?: number }` | Optional byte range | **Returns:** `Promise` *** ### ShelbyBlob A blob is a representation of a file (or a part of a file) that is stored on the Shelby network. ### BlobMetadata The metadata of a blob that describes the blob and its properties. ### ClayEncoding The encoding of a blob that describes the encoding of the blob. # Overview (/sdks/typescript/node) # Introduction The Node.js version of the SDK is used to interact with the Shelby network from a Node.js environment. The SDK provides a high level interface for interacting with the different components of the Shelby network: the coordination layer, the RPC layer, and the storage layer. ## Installation npm pnpm yarn bun ```bash npm install @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ```bash pnpm add @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ```bash yarn add @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ```bash bun add @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ## Usage Make sure you [aquired an API Key](/sdks/typescript/acquire-api-keys) Access the Node.js version of the SDK by importing using the `@shelby-protocol/sdk/node` entry point. ```typescript import { ShelbyNodeClient } from "@shelby-protocol/sdk/node"; import { Network } from "@aptos-labs/ts-sdk"; // Create client configuration const config = { network: Network.TESTNET, apiKey: "aptoslabs_***", }; // Initialize the Shelby client const shelbyClient = new ShelbyNodeClient(config); ``` ## Next Steps Reference documentation for the API and specifications Learn how to upload a file to the Shelby network
# Specifications (/sdks/typescript/node/specifications) ## ShelbyNodeClient Node.js-specific implementation that extends the core ShelbyClient with server-side blob operations. The `ShelbyNodeClient` class extends the base `ShelbyClient` and adds Node.js-specific methods for blob operations with multipart upload support and streaming downloads. It combines the functionality of `ShelbyBlobClient` and `ShelbyRPCClient` (both documented in the [core specifications](/sdks/typescript/core/specifications)) to provide a complete Node.js solution. ### Methods #### `upload({ signer, blobData, blobName, expirationMicros, options })` Uploads a blob to the Shelby network, handling both blockchain commitments and storage upload. | Parameter | Type | Description | | ------------------ | ----------------------------- | ----------------------------------------------- | | `signer` | `Account` | The signer of the transaction | | `blobData` | `Buffer` | The data to upload | | `blobName` | `BlobName` | The name of the blob | | `expirationMicros` | `number` | The expiration time of the blob in microseconds | | `options` | `WriteBlobCommitmentsOptions` | The options for the upload | **Returns:** `Promise<{ transaction: CommittedTransactionResponse; blobCommitments: BlobCommitments }>` #### `download({account, blobName, range?})` Downloads blob data as a ShelbyBlob with a readable stream. | Parameter | Type | Description | | ---------- | --------------------------------- | ------------------------- | | `account` | `AccountAddressInput` | The account address | | `blobName` | `string` | The name/path of the blob | | `range` | `{ start: number; end?: number }` | Optional byte range | **Returns:** `Promise` ### Properties The `ShelbyNodeClient` provides access to the underlying clients: #### `coordination: ShelbyBlobClient` The blockchain coordination client for managing blob commitments and metadata. See [ShelbyBlobClient documentation](/sdks/typescript/core/specifications#shelbyblobclient) for details. #### `rpc: ShelbyRPCClient` The RPC client for blob storage operations. See [ShelbyRPCClient documentation](/sdks/typescript/core/specifications#shelbyrcpclient) for details. ### Examples #### Complete Upload and Download Flow ```typescript import { ShelbyNodeClient } from "@shelby-protocol/sdk/node"; import { Account, Network } from "@aptos-labs/ts-sdk"; // Create node client const client = new ShelbyNodeClient({ network: Network.TESTNET, }); // Create or get account const account = Account.generate(); // Prepare blob data const blobData = Buffer.from("Hello, Shelby!"); const blobName = "greeting.txt"; const expirationMicros = Date.now() * 1000 + 3600_000_000; // 1 hour from now // Upload blob (commits to blockchain and uploads to storage) const { transaction, blobCommitments } = await client.upload({ signer: account, blobData, blobName, expirationMicros, }); console.log("Upload completed:", transaction.hash); // Download blob const blob = await client.download({ account: account.accountAddress, blobName, }); console.log("Downloaded blob:", blob.name, blob.contentLength, "bytes"); ``` # Account Management (/tools/cli/commands/account-management) As part of the upload process, the CLI requires that you have a funded account in order to pay for upload and gas fees. The CLI provides a number of commands to help you manage the signer account that is used in the CLI. ## `shelby account` To create a new account, you can use the `shelby account create` command. **Interactive Mode** ```bash shelby account create ``` **Non-interactive Mode** ```bash shelby account create --name --scheme --private-key --address ``` ### Options | Flag | Description | | ----------------------------- | ---------------------------------------------------------------------------------- | | `--name ` | The label to store the credentials under. | | `--scheme ` | Signature scheme for the private key. The CLI currently supports `ed25519`. | | `--private-key ` | Raw private key (`ed25519-priv-0x…` format). Required in non-interactive mode. | | `--address ` | Optional Aptos account address (`0x…`). Useful if you generated the key elsewhere. | To skip the interactive wizard, provide `--name`, `--scheme`, and `--private-key`. `--address` is optional; if omitted, the CLI derives the account address from the private key. ## `shelby account list` To list all accounts, you can use the `shelby account list` command. ```bash shelby account list ``` ```bash title="Example Output" ┌─────────────┬─────────────────────────────────────────────┬────────────────┐ │ Name │ Address │ Private Key │ ├─────────────┼─────────────────────────────────────────────┼────────────────┤ │ alice │ 0xf4b6c29e32ab75d7088886ef5aa2cfebbe4303ba8 │ ed25519-priv-0 │ │ (default) │ 3f3033f76e4e009e0e87fba │ x1ed77... │ ├─────────────┼─────────────────────────────────────────────┼────────────────┤ │ bob │ 0x03206522072ab6bca0d44ea0867d9a3eadb59eb4e │ ed25519-priv-0 │ │ │ 359d9b55f3dc037463caf8c │ x18449... │ └─────────────┴─────────────────────────────────────────────┴────────────────┘ ``` ## `shelby account use` You can choose an account to use by default by using the `shelby account use` command. This account will automatically be used when running any command that requires an account. ```bash shelby account use ``` ```bash title="Example Output" ✅ Now using account 'alice' ``` ## `shelby account delete` Delete an account from the configuration file. ```bash shelby account delete ``` ```bash title="Example Output" ✅ Account 'alice' deleted successfully ``` ## `shelby account blobs` List all blobs associated with an account. ```bash shelby account blobs ``` ```bash title="Example Output" 🔍 Retrieving blobs for alice 👤 Address: 0x0694a79e492d268acf0c6c0b01f42654ac050071a343ebc4226cb6717d63e4ea 🗂️ Shelby Explorer: https://explorer.shelby.xyz/testnet/account/0x0694a79e492d268acf0c6c0b01f42654ac050071a343ebc4226cb6717d63e4ea ✅ Retrieved 2 blobs ──────────────────────────────────────────── 📦 Stored Blobs ┌─────────────────────────────────────────────┬───────────────┬─────────────────────────┐ │ Name │ Size │ Expires │ ├─────────────────────────────────────────────┼───────────────┼─────────────────────────┤ │ .gitignore-v1 │ 494 B │ Oct 11, 2025, 4:03 PM │ ├─────────────────────────────────────────────┼───────────────┼─────────────────────────┤ │ .gitignore-v2 │ 494 B │ Oct 11, 2025, 4:03 PM │ └─────────────────────────────────────────────┴───────────────┴─────────────────────────┘ ✨ Done! ``` `shelby account blobs` requires your active context to include a Shelby indexer endpoint. Configure one with `shelby context create`/`update` before relying on this command. ### Options | Flag | Description | | ---------------------- | ------------------------------------------------------------- | | `-a, --account ` | Optional override for the account whose blobs will be listed. | ## `shelby account balance` Display the Aptos (APT) and ShelbyUSD balances for the active account (or a supplied address). ```bash shelby account balance ``` ```bash title="Example Output" 👤 Account Information ──────────────────────────────────────────── 🏷️ Alias: alice 🌐 Context: testnet 🔑 Address: 0x0694a79e492d268acf0c6c0b01f42654ac050071a343ebc4226cb6717d63e4ea 🔗 Aptos Explorer: https://explorer.aptoslabs.com/account/0x0694a79e492d268acf0c6c0b01f42654ac050071a343ebc4226cb6717d63e4ea?network=testnet 🗂️ Shelby Explorer: https://explorer.shelby.xyz/testnet/account/0x0694a79e492d268acf0c6c0b01f42654ac050071a343ebc4226cb6717d63e4ea ──────────────────────────────────────────── 💰 Balance: ┌─────────┬───────────────────────────────────┬─────────────────────┬───────────────────┐ │ Token │ Asset │ Balance │ Raw Units │ ├─────────┼───────────────────────────────────┼─────────────────────┼───────────────────┤ │ APT │ 0x1::aptos_coin::AptosCoin │ 9.998262 APT │ 999,826,200 │ ├─────────┼───────────────────────────────────┼─────────────────────┼───────────────────┤ │ ShelbyU │ 0x1b18363a9f1fe5e6ebf247daba5cc1c │ 9.99986112 │ 999,986,112 │ │ SD │ 18052bb232efdc4c50f556053922d98e1 │ ShelbyUSD │ │ └─────────┴───────────────────────────────────┴─────────────────────┴───────────────────┘ ``` ### Options | Flag | Description | | ---------------------- | ------------------------------------------------------------------ | | `-a, --account ` | Query a configured account by name instead of the default. | | `-c, --context ` | Use balances from a different context/environment. | | `--address
` | Optional positional Aptos address to query (skips account lookup). | # Commitment (/tools/cli/commands/commitment) ## `shelby commitment` Generate blob and chunkset commitments for a local file without uploading. Useful for debugging, introspection, or pre-computing metadata. The command runs entirely offline but requires the CLI to have access to the Clay erasure-code WASM artifacts (the published package includes them by default). Note that the commitments can also be saved to local files during upload with the `--output-commitments` option. ```bash shelby commitment [options] ``` ### Options None #### Example ``` shelby commitment README.md out.json ``` Output: ``` $ jq . out.json | head { "schema_version": "1.3", "raw_data_size": 1775, "blob_merkle_root": "0xecc399a8cb4a198b29f4b9c3fb3b2d0636a53be3298cd8a5c598153c48a90f07", "chunkset_commitments": [ { "chunkset_root": "0x4cc2bb1793de54665388b00c31580d3fa2df1e121a6b8d8d67ea9be2b911638c", "chunk_commitments": [ "0x01cf2463b8f772d77e93876e9f4ec99d13a3b513b2d073d60f198575ad3fe9d1", "0x30e14955ebf1352266dc2ff8067e68104607e750abb9d3b36582b8af909fcb58", ``` # Context Management (/tools/cli/commands/context-management) The Shelby CLI uses contexts to manage different networks and their endpoints, this is useful for quickly switching between different networks without having to manually update the configuration file each time. By default, the configuration for the contexts are stored in the `~/.shelby/config.yaml` file. ## `shelby context create` To create a new context, you can use the `shelby context create` command. **Interactive Mode** ```bash shelby context create ``` **Non-interactive Mode** ```bash shelby context create --name --shelby-rpc-endpoint --aptos-network ``` ### Options | Flag | Description | | ----------------------------------------- | --------------------------------------------------------------- | | `--name ` | Label for the context. | | `--shelby-rpc-endpoint ` | Shelby RPC endpoint (`https://…`). | | `--shelby-indexer-endpoint ` | Shelby indexer endpoint (`https://…`). | | `--shelby-rpc-api-key ` | API key injected into Shelby RPC requests. | | `--shelby-indexer-api-key ` | API key injected into Shelby indexer requests. | | `--shelby-rpc-receiver-address
` | Shelby RPC receiver address for micropayments. | | `--aptos-network ` | Aptos network name (`local`, `testnet`, `shelbynet`). | | `--aptos-fullnode ` | Override the Aptos fullnode endpoint. | | `--aptos-indexer ` | Override the Aptos indexer endpoint. Required for blob listing. | | `--aptos-faucet ` | Override the Aptos faucet endpoint. | | `--aptos-pepper ` | Override the Aptos pepper endpoint. | | `--aptos-prover ` | Override the Aptos prover endpoint. | | `--aptos-api-key ` | API key injected into Aptos requests. | ## `shelby context list` To list all contexts, you can use the `shelby context list` command. ```bash shelby context list ``` ```bash title="Example Output" Aptos Configurations: ┌───────────────────┬──────────┬────────────────────────────────────────┬─────────────────────────────────────────────────┬──────────────────────────────────────┬─────────┐ │ Name │ Network │ Fullnode │ Indexer │ Faucet │ API Key │ ├───────────────────┼──────────┼────────────────────────────────────────┼─────────────────────────────────────────────────┼──────────────────────────────────────┼─────────┤ │ local │ local │ http://127.0.0.1:8080/v1 │ http://127.0.0.1:8090/v1/graphql │ http://127.0.0.1:8081 │ │ ├───────────────────┼──────────┼────────────────────────────────────────┼─────────────────────────────────────────────────┼──────────────────────────────────────┼─────────┤ │ shelbynet │ shelbynet│ https://api.shelbynet.shelby.xyz/v1 │ https://api.shelbynet.shelby.xyz/v1/graphql │ https://faucet.shelbynet.shelby.xyz │ │ ├───────────────────┼──────────┼────────────────────────────────────────┼─────────────────────────────────────────────────┼──────────────────────────────────────┼─────────┤ │ testnet (default) │ testnet │ https://api.testnet.aptoslabs.com/v1 │ https://api.testnet.aptoslabs.com/v1/graphql │ │ │ └───────────────────┴──────────┴────────────────────────────────────────┴─────────────────────────────────────────────────┴──────────────────────────────────────┴─────────┘ Shelby Configurations: ┌─────────┬────────────────────────┬───────────────────────┬────────┬────────┐ │ Name │ RPC │ Indexer │ RPC │ Index… │ │ │ │ │ API │ API │ │ │ │ │ Key │ Key │ ├─────────┼────────────────────────┼───────────────────────┼────────┼────────┤ │ local │ http://localhost:9090/ │ │ │ │ ├─────────┼────────────────────────┼───────────────────────┼────────┼────────┤ │ shelbyn │ https://api.shelbynet. │ │ │ │ │ et │ shelby.xyz/shelby │ │ │ │ ├─────────┼────────────────────────┼───────────────────────┼────────┼────────┤ │ testnet │ https://api.testnet.sh │ │ │ │ │ (defau │ elby.xyz/shelby │ │ │ │ │ lt) │ │ │ │ │ └─────────┴────────────────────────┴───────────────────────┴────────┴────────┘ ``` ## `shelby context use` You can choose a context to use by default by using the `shelby context use` command. This context will automatically be used when running any command that requires a context. ```bash shelby context use ``` ```bash title="Example Output" ✅ Now using context 'testnet' ``` ## `shelby context update` Update the configuration for a context in place. **Interactive Mode** ```bash shelby context update ``` **Non-interactive Mode** ```bash shelby context update --shelby-rpc-endpoint --aptos-network --aptos-indexer ``` ### Options | Flag | Description | | --------------------------------- | ----------------------------------- | | `--shelby-rpc-endpoint ` | Update the Shelby RPC endpoint. | | `--shelby-indexer-endpoint ` | Update the Shelby indexer endpoint. | | `--shelby-rpc-api-key ` | Update the Shelby RPC API key. | | `--shelby-indexer-api-key ` | Update the Shelby indexer API key. | | `--aptos-network ` | Update the Aptos network name. | | `--aptos-fullnode ` | Update the Aptos fullnode endpoint. | | `--aptos-indexer ` | Update the Aptos indexer endpoint. | | `--aptos-faucet ` | Update the Aptos faucet endpoint. | | `--aptos-pepper ` | Update the Aptos pepper endpoint. | | `--aptos-prover ` | Update the Aptos prover endpoint. | | `--aptos-api-key ` | Update the Aptos API key. | ## `shelby context delete` Delete a context from the configuration file. ```bash shelby context delete ``` ```bash title="Example Output" ✅ Context 'testnet' deleted successfully ``` The CLI refuses to delete the current default context. Run `shelby context use ` first if you need to remove it. # Deletions (/tools/cli/commands/deletions) # Overview The Shelby CLI provides operations for deleting blobs from the Shelby network. Deletion is a soft delete that marks blobs as deleted on the blockchain. It is important that the CLI is properly configured with a network and funded account. If not, please visit the [Getting Started](/tools/cli) quick start guide for more information. ## `shelby delete` Delete a blob or folder of blobs from Shelby. You can only delete blobs that you own. ```bash shelby delete [options] ``` ### Options | Flag | Alias | Type | Required | Default | Description | | -------------- | ----- | ---- | -------- | ------- | -------------------------------------------------- | | `--recursive` | `-r` | flag | | false | Delete all blobs matching the prefix. | | `--assume-yes` | | flag | | false | Skip interactive confirmation. Useful for scripts. | If deleting a single blob, `` must be a valid blob name (does not end in `/`). When deleting recursively, `` must end in `/`. Deletion is permanent. Once a blob is deleted, it cannot be recovered. The CLI will prompt for confirmation before deleting unless `--assume-yes` is provided. ### Basic Example Delete a single blob: ```bash shelby delete files/my-blob.txt ``` ### Recursive Deletion Delete all blobs with a given prefix: ```bash shelby delete folder/ -r shelby delete my-site/ --recursive ``` When using `--recursive`, the destination must end with `/` to indicate it's a prefix, not a blob name. ### Skip Confirmation For scripting or automation, use `--assume-yes` to skip the interactive confirmation prompt: ```bash shelby delete old-files/ -r --assume-yes ``` ### What Happens During Deletion 1. The CLI queries the indexer to find matching blobs 2. For each blob, it verifies: * The blob exists and hasn't expired * You own the blob 3. A delete transaction is submitted to the blockchain 4. The blob is marked as deleted (soft delete) ### Error Cases * **Blob not found**: If the blob doesn't exist or has already been deleted/expired, the CLI will report an error * **Not the owner**: You can only delete blobs that belong to your active account * **Invalid destination**: Single blob deletions cannot end with `/`, and recursive deletions must end with `/` ### Example Output ``` 🗑️ Delete Summary ──────────────────────────────────────────── 📁 Blob Name: files/my-blob.txt 🧮 Delete list created (1 entry) ⏱️ Took: 0.12345s Continue? ❯ Yes No ✔ Delete complete — 1 blob deleted in 2.34s 🌐 Aptos Explorer: https://explorer.aptoslabs.com/txn/?network=testnet 🗂️ Shelby Explorer: https://explorer.shelby.xyz/testnet/account/ ──────────────────────────────────────────── ✨ Done! ``` # Downloads (/tools/cli/commands/downloads) # Overview The Shelby CLI provides simple operations for downloading files from the Shelby network. It is important that the CLI is properly configured with a network and funded account. If not, please visit the [Getting Started](/tools/cli) quick start guide for more information. ## `shelby download` Currently you cannot download blobs that were uploaded by other accounts with the CLI. This will be fixed in the near future. Download a file (or files) from the Shelby network, with progress reporting, for the active account. ```bash shelby download [options] ``` ### Options | Flag | Alias | Type | Required | Description | | ------------- | ----- | ---- | -------- | ------------------------------------------------------------------------------ | | `--recursive` | `-r` | flag | | Treat `src` as a directory prefix. Both the `src` and `dst` must end with `/`. | | `--force` | `-f` | flag | | Overwrite existing files or clear a non-empty directory before downloading. | The downloader validates filesystem state before fetching data. Create the parent directory ahead of time and supply `--force` if you need to overwrite existing content. If downloading a single blob, `` must be a valid blob name (does not end in /). The `` must not end in a directory separator and will be created as a file. When downloading a directory recursively, both `` and ` must end in /. The ` directory will be created if it doesn't exist. Validation Rules: * The parent directory of `` must already exist * Without --force, the `dst` file must not exist, or must be an empty directory for recursive downloads * With --force, any existing `dst` will be completely removed before download ### Basic example ```bash shelby download shelby/blob/name.mp4 ./video.mp4 ``` ### Force Overwrite ```bash shelby download shelby/blob/name.mp4 ./existing-video.mp4 --force ``` ### Directory Download ```bash shelby download -r shelby/blobs/best-videos/hls_video/ ./hls_video/ shelby download --recursive my-site/ ./website/ ``` For recursive downloads, both `src` (the blob prefix) and `dst` (the target directory) must end with `/`. ### Canonical Directory Layout When downloading directories, the command recreates the directory structure locally. If Shelby contains these blobs: ```txt my-files/document.pdf my-files/images/photo1.jpg my-files/images/photo2.jpg ``` Running: ```bash shelby download -r my-files/ ./local-files/ ``` Will create: ```bash $ tree ./local-files/ ./local-files/ ├── document.pdf └── images/ ├── photo1.jpg └── photo2.jpg ``` The download command automatically creates any necessary subdirectories and downloads all files. ### Other Account's Files For now, the CLI only interacts with active account in the CLI's context. Files from other accounts are downloadable using the REST interface from the RPC node, which currently does not require any additional headers or session information for payment. In general: ```bash curl https://api.shelbynet.shelby.xyz/shelby/v1/blobs// ``` For example, if I want to download the blob `foo` stored by account `0x89ca7dfadf5788830b0d5826a56b370ced0d7938c4628f4b57f346ab54f76357` I can use: ```bash curl https://api.shelbynet.shelby.xyz/shelby/v1/blobs/0x89ca7dfadf5788830b0d5826a56b370ced0d7938c4628f4b57f346ab54f76357/foo ``` # Faucet (/tools/cli/commands/faucet) ## `shelby faucet` Opens a browser to faucet website for currently active account. This command only works on `shelbynet`. ```bash shelby faucet ``` # Uploads (/tools/cli/commands/uploads) # Overview The Shelby CLI provides simple operations for uploading files to the Shelby network. It is important that the CLI is properly configured with a network and funded account. If not, please visit the [Getting Started](/tools/cli) quick start guide for more information. ## `shelby upload` Uploads a blob or local directory of blobs to Shelby, using the currently active account as the account to upload to. This will charge SHEL tokens. ```bash shelby upload [options] ``` ### Options | Flag | Alias | Type | Required | Default | Description | | ----------------------------- | ----- | ------ | -------- | ------- | ---------------------------------------------------------------------------- | | `--expiration ` | `-e` | string | yes | - | Expiration timestamp. | | `--recursive` | `-r` | flag | | false | Upload every file under a directory. | | `--assume-yes` | | flag | | false | Skip interactive cost confirmation. Useful for scripts. | | `--output-commitments ` | | string | | - | Persist the generated commitments alongside the upload for later inspection. | If uploading a single blob, `` must always be a valid blob name (does not end in `/`, and no more than 1024 characters). When uploading a directory recursively, `` must end in `/`. The `--expiration` option accepts: * Human language timestamp: * `in 2 days` * `next Friday` * `next month` * `2025-12-31` * UNIX timestamp: `1735689600` (seconds since epoch) * ISO date string: `"2025-01-01T00:00:00Z"` * Human-readable date: `"2025-01-01"` or `"Jan 1, 2025"` The uploaded file list is created before starting the operation. File sizes are saved as the file list is created. If any files change size between the initial listing and the upload, the upload will fail when it reaches the modified files. ### Basic Example ```bash shelby upload local-video.mp4 shelby-output.mp4 -e ``` ### Relative Paths ```bash shelby upload ./local-video.mp4 shelby/blob/name.mp4 -e ``` ### Other Date Formats ```bash shelby upload local-video.mp4 best-videos/video.mp4 --expiration shelby upload local-video.mp4 best-videos/video.mp4 -e "2025-12-31" shelby upload local-video.mp4 best-videos/video.mp4 -e "2025-12-31T00:00:00Z" shelby upload local-video.mp4 best-videos/video.mp4 -e "2025-12-31T00:00:00-0500" ``` On UNIXes, the `date` command can help too: ```bash shelby upload video.mp4 best-videos/video.mp4 -e $(date -d "+1 hour" +%s) ``` ### Directory Upload Example ```bash shelby upload -r ./hls_video/ best-videos/hls_video/ --expiration shelby upload --recursive ./website/ my-site/ -e "2025-12-31" ``` ### Canonical Directory Layout The upload command will use the canonical filesystem layout when uploading directories recursively. That is, if the source is laid out as: ```bash $ tree . . |-- bar `-- foo |-- baz `-- buzz 2 directories, 3 files ``` The blobs uploaded to Shelby will have these blob names: ```text //bar //foo/baz //foo/buzz ``` Note that there is no `//foo` blob in shelby as there is no concept of directory in blob storage. It is possible to later upload `//foo` as a separate blob; this would make it difficult to download the directory tree as a directory tree! # Complete a multipart upload (/apis/rpc/localhost/multipart-uploads/completeMultipartUpload) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Complete a multipart upload session. # Begin a multipart upload (/apis/rpc/localhost/multipart-uploads/startMultipartUpload) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Begin a multipart upload session. # Upload a part (/apis/rpc/localhost/multipart-uploads/uploadPart) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Upload a part for a multipart upload session. # Create a micropayment channel (/apis/rpc/localhost/sessions/createMicropaymentChannel) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Create a micropayment channel for a user. (This is WIP and will be replaced by real blockchain integration.) # Create a session (/apis/rpc/localhost/sessions/createSession) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Create a session for a user with a pre-existing micropayment channel. # Use a session (/apis/rpc/localhost/sessions/useSession) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Use a session, decrementing the number of chunksets left. # Retrieve a blob (/apis/rpc/localhost/storage/getBlob) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Retrieve a blob or a byte range of a blob. # Upload a blob (/apis/rpc/localhost/storage/uploadBlob) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Upload a blob. # Complete a multipart upload (/apis/rpc/shelbynet/multipart-uploads/completeMultipartUpload) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Complete a multipart upload session. # Begin a multipart upload (/apis/rpc/shelbynet/multipart-uploads/startMultipartUpload) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Begin a multipart upload session. # Upload a part (/apis/rpc/shelbynet/multipart-uploads/uploadPart) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Upload a part for a multipart upload session. # Create a micropayment channel (/apis/rpc/shelbynet/sessions/createMicropaymentChannel) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Create a micropayment channel for a user. (This is WIP and will be replaced by real blockchain integration.) # Create a session (/apis/rpc/shelbynet/sessions/createSession) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Create a session for a user with a pre-existing micropayment channel. # Use a session (/apis/rpc/shelbynet/sessions/useSession) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Use a session, decrementing the number of chunksets left. # Retrieve a blob (/apis/rpc/shelbynet/storage/getBlob) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Retrieve a blob or a byte range of a blob. # Upload a blob (/apis/rpc/shelbynet/storage/uploadBlob) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Upload a blob. # Building the Frontend (/sdks/solana-kit/guides/token-gated-solana/frontend) import { Step, Steps } from "fumadocs-ui/components/steps"; import { File, Folder, Files } from "fumadocs-ui/components/files"; import { Accordions, Accordion } from "fumadocs-ui/components/accordion"; # Building the Frontend In this final section, you'll build the React frontend that ties everything together. The UI allows users to: * Connect their Solana wallet * Upload encrypted files with a price * Browse and purchase files * Decrypt and download purchased files ## Project Structure ## Part A: Configuration ### lib/config.ts This file centralizes all configuration including program IDs, RPC endpoints, and ACE settings. ```typescript import type { Address } from "@solana/kit"; /** * Configuration for the token-gated access control dapp. * * Program IDs default to deployed testnet addresses. * Override via environment variables for custom deployments. */ export const config = { // Solana RPC endpoint (used for reading blob metadata) solanaRpcUrl: process.env.NEXT_PUBLIC_SOLANA_RPC_URL || "https://api.testnet.solana.com", // Solana programs programs: { /** Access control program - handles blob registration and purchases */ accessControl: (process.env.NEXT_PUBLIC_ACCESS_CONTROL_PROGRAM_ID || "Ej2KamzNByfcYEkkbx9TT5RCqbKgkmvQ5NCC7rPyyzxq") as Address, /** ACE hook program - verifies access for decryption */ aceHook: (process.env.NEXT_PUBLIC_ACE_HOOK_PROGRAM_ID || "3eQcE44r9fPmNVbfQtZrwZmRWsifjouHRaxmRKgshEND") as Address, }, // ACE threshold decryption committee ace: { /** Worker endpoints for threshold IBE (filter out empty values) */ workerEndpoints: [ process.env.NEXT_PUBLIC_ACE_WORKER_0, process.env.NEXT_PUBLIC_ACE_WORKER_1, ].filter((url): url is string => !!url).length > 0 ? [ process.env.NEXT_PUBLIC_ACE_WORKER_0, process.env.NEXT_PUBLIC_ACE_WORKER_1, ].filter((url): url is string => !!url) : [ "https://ace-worker-0-646682240579.europe-west1.run.app", "https://ace-worker-1-646682240579.europe-west1.run.app", ], /** Threshold for decryption (number of workers needed) */ threshold: Number.parseInt( process.env.NEXT_PUBLIC_ACE_THRESHOLD || "2", 10 ), /** Solana chain name for ACE contract ID */ solanaChainName: (process.env.NEXT_PUBLIC_ACE_CHAIN_NAME || "testnet") as "mainnet-beta" | "testnet" | "devnet" | "localnet", }, // Shelby storage shelby: { /** Seller account address to browse files from (Aptos/Shelby address) */ sellerAccount: process.env.NEXT_PUBLIC_SELLER_ACCOUNT || "", }, // Default pricing pricing: { /** Default price in SOL (string for input binding) */ defaultPriceSol: "0.0005", }, } as const; export const LAMPORTS_PER_SOL = 1_000_000_000n; export const SYSTEM_PROGRAM_ADDRESS = "11111111111111111111111111111111" as Address; /** * Green box encryption scheme for threshold IBE. * This is the protocol-level scheme ID expected by the on-chain program. */ export const GREEN_BOX_SCHEME = 2; ``` ### lib/shelbyClient.ts Create a shared Shelby client instance. ```typescript "use client"; import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { Network } from "@shelby-protocol/solana-kit/react"; export const shelbyClient = new ShelbyClient({ network: Network.TESTNET, apiKey: process.env.NEXT_PUBLIC_TESTNET_API_KEY || "", }); ``` ## Part B: Encryption Helpers ### lib/encryption.ts This file handles both AES-GCM encryption (for files) and ACE threshold encryption (for keys). ```typescript import { ace } from "@aptos-labs/ace-sdk"; import { config } from "./config"; // ============================================================================ // AES-GCM Encryption (for file content - the "redBox") // ============================================================================ const IV_LENGTH = 12; // 96 bits for AES-GCM /** Helper to convert Uint8Array to ArrayBuffer for Web Crypto API */ function toArrayBuffer(arr: Uint8Array): ArrayBuffer { return arr.buffer.slice( arr.byteOffset, arr.byteOffset + arr.byteLength ) as ArrayBuffer; } /** * Generate a random 256-bit key for AES-GCM encryption. */ export function generateRedKey(): Uint8Array { return crypto.getRandomValues(new Uint8Array(32)); } /** * Encrypt file content with AES-GCM using the provided key. * Returns: IV (12 bytes) || ciphertext */ export async function encryptFile( plaintext: Uint8Array, redKey: Uint8Array ): Promise { const iv = crypto.getRandomValues(new Uint8Array(IV_LENGTH)); const cryptoKey = await crypto.subtle.importKey( "raw", toArrayBuffer(redKey), { name: "AES-GCM" }, false, ["encrypt"] ); const ciphertext = await crypto.subtle.encrypt( { name: "AES-GCM", iv }, cryptoKey, toArrayBuffer(plaintext) ); // Prepend IV to ciphertext const result = new Uint8Array(iv.length + ciphertext.byteLength); result.set(iv, 0); result.set(new Uint8Array(ciphertext), iv.length); return result; } /** * Decrypt file content with AES-GCM. * Input format: IV (12 bytes) || ciphertext */ export async function decryptFile( redBox: Uint8Array, redKey: Uint8Array ): Promise { const iv = redBox.slice(0, IV_LENGTH); const ciphertext = redBox.slice(IV_LENGTH); const cryptoKey = await crypto.subtle.importKey( "raw", toArrayBuffer(redKey), { name: "AES-GCM" }, false, ["decrypt"] ); const plaintext = await crypto.subtle.decrypt( { name: "AES-GCM", iv }, cryptoKey, toArrayBuffer(ciphertext) ); return new Uint8Array(plaintext); } // ============================================================================ // ACE Threshold IBE (for key encryption - the "greenBox") // ============================================================================ /** * Create an ACE committee instance. */ export function createAceCommittee(): ace.Committee { return new ace.Committee({ workerEndpoints: [...config.ace.workerEndpoints] as string[], threshold: config.ace.threshold, }); } /** * Create an ACE contract ID for Solana. */ export function createAceContractId(): ace.ContractID { return ace.ContractID.newSolana({ knownChainName: config.ace.solanaChainName, programId: config.programs.aceHook, }); } /** * Encrypt the redKey into a greenBox using threshold IBE. * This greenBox can only be decrypted by users who have purchased access. */ export async function encryptRedKey( redKey: Uint8Array, fullBlobNameBytes: Uint8Array ): Promise { const committee = createAceCommittee(); const contractId = createAceContractId(); // Fetch encryption key from committee const encryptionKeyResult = await ace.EncryptionKey.fetch({ committee, }); const encryptionKey = encryptionKeyResult.unwrapOrThrow( "Failed to fetch encryption key" ); // Encrypt the redKey const encryptResult = ace.encrypt({ encryptionKey, contractId, domain: fullBlobNameBytes, plaintext: redKey, }).unwrapOrThrow("Failed to encrypt redKey"); return encryptResult.ciphertext.toBytes(); } /** * Decrypt the greenBox to recover the redKey using a proof-of-permission transaction. * @param signedTransactionBytes - Serialized signed transaction bytes (from any Solana SDK) */ export async function decryptGreenBox( greenBoxBytes: Uint8Array, fullBlobNameBytes: Uint8Array, signedTransactionBytes: Uint8Array ): Promise { const committee = createAceCommittee(); const contractId = createAceContractId(); // Reconstruct the ciphertext from bytes const greenBox = ace.Ciphertext.fromBytes(greenBoxBytes).unwrapOrThrow( "Failed to parse greenBox ciphertext" ); // Create proof of permission from the signed transaction bytes const pop = ace.ProofOfPermission.createSolana({ txn: signedTransactionBytes as any, }); // Fetch decryption key from committee const decryptionKeyResult = await ace.DecryptionKey.fetch({ committee, contractId, domain: fullBlobNameBytes, proof: pop, }); const decryptionKey = decryptionKeyResult.unwrapOrThrow( "Failed to fetch decryption key" ); // Decrypt the greenBox const plaintext = ace.decrypt({ decryptionKey, ciphertext: greenBox, }).unwrapOrThrow("Failed to decrypt greenBox"); return plaintext; } ``` ## Part C: Anchor Helpers ### lib/anchor.ts This file provides PDA derivation and instruction encoding using Anchor. ```typescript import { AnchorProvider, BN, Program } from "@coral-xyz/anchor"; import { type Address, getAddressEncoder, getProgramDerivedAddress, } from "@solana/kit"; import { Connection, PublicKey } from "@solana/web3.js"; import accessControlIdl from "../../anchor/target/idl/access_control.json"; import aceHookIdl from "../../anchor/target/idl/ace_hook.json"; import type { AccessControl } from "../../anchor/target/types/access_control"; import type { AceHook } from "../../anchor/target/types/ace_hook"; import { config, GREEN_BOX_SCHEME } from "./config"; // ============================================================================ // Anchor Program Helpers // ============================================================================ /** * Create an Anchor provider with a mock wallet for encoding instructions. * The wallet is only used for account resolution, not actual signing. */ function createEncodingProvider(signerPubkey?: PublicKey): AnchorProvider { const connection = new Connection(config.solanaRpcUrl, "confirmed"); // Create a mock wallet that satisfies Anchor's requirements const mockWallet = { publicKey: signerPubkey ?? PublicKey.default, signTransaction: async () => { throw new Error("Not implemented"); }, signAllTransactions: async () => { throw new Error("Not implemented"); }, }; return new AnchorProvider(connection, mockWallet as never, { commitment: "confirmed", }); } /** * Get the AccessControl program instance for encoding/fetching */ function getAccessControlProgram( signerPubkey?: PublicKey ): Program { return new Program( accessControlIdl as AccessControl, createEncodingProvider(signerPubkey) ); } /** * Get the AceHook program instance for encoding/fetching */ function getAceHookProgram(signerPubkey?: PublicKey): Program { return new Program( aceHookIdl as AceHook, createEncodingProvider(signerPubkey) ); } // ============================================================================ // PDA Derivations // ============================================================================ /** * Derive the blob metadata PDA for a given owner and blob name. * Seeds must be raw bytes (no length prefix) to match Anchor's PDA derivation. */ export async function deriveBlobMetadataPda( storageAccountAddressBytes: Uint8Array, blobName: string ): Promise
{ const [pda] = await getProgramDerivedAddress({ programAddress: config.programs.accessControl, seeds: [ new TextEncoder().encode("blob_metadata"), storageAccountAddressBytes, new TextEncoder().encode(blobName), ], }); return pda; } /** * Derive the access receipt PDA for a given owner, blob name, and buyer. * Seeds must be raw bytes (no length prefix) to match Anchor's PDA derivation. */ export async function deriveAccessReceiptPda( storageAccountAddressBytes: Uint8Array, blobName: string, buyerAddress: Address ): Promise
{ const [pda] = await getProgramDerivedAddress({ programAddress: config.programs.accessControl, seeds: [ new TextEncoder().encode("access"), storageAccountAddressBytes, new TextEncoder().encode(blobName), getAddressEncoder().encode(buyerAddress), ], }); return pda; } /** * Check if a user has already purchased access to a blob. * Returns true if the receipt account exists on-chain. */ export async function checkHasPurchased( storageAccountAddressBytes: Uint8Array, blobName: string, buyerAddress: Address ): Promise { const receiptPda = await deriveAccessReceiptPda( storageAccountAddressBytes, blobName, buyerAddress ); const connection = new Connection(config.solanaRpcUrl, "confirmed"); const accountInfo = await connection.getAccountInfo( new PublicKey(receiptPda) ); return accountInfo !== null; } // ============================================================================ // Instruction Data Encoders (using Anchor for type-safe encoding) // ============================================================================ /** * Encode the register_blob instruction data using Anchor. * @param signerAddress - The signer's Solana address (used for account resolution) */ export async function encodeRegisterBlobData( storageAccountAddress: Uint8Array, blobName: string, greenBoxScheme: number, greenBoxBytes: Uint8Array, price: bigint, signerAddress: Address ): Promise { const signerPubkey = new PublicKey(signerAddress); const program = getAccessControlProgram(signerPubkey); // Build the instruction using Anchor's type-safe methods builder const ix = await program.methods .registerBlob( Array.from(storageAccountAddress) as number[], blobName, greenBoxScheme, Buffer.from(greenBoxBytes), new BN(price.toString()) ) .instruction(); return new Uint8Array(ix.data); } /** * Encode the purchase instruction data using Anchor. * @param signerAddress - The buyer's Solana address (the signer) * @param ownerSolanaAddress - The seller's Solana address (receives SOL) */ export async function encodePurchaseData( storageAccountAddress: Uint8Array, blobName: string, signerAddress: Address, ownerSolanaAddress: Address ): Promise { const signerPubkey = new PublicKey(signerAddress); const ownerPubkey = new PublicKey(ownerSolanaAddress); const program = getAccessControlProgram(signerPubkey); // Build the instruction using Anchor's type-safe methods builder // We must provide the `owner` account explicitly since Anchor can't auto-resolve it const ix = await program.methods .purchase(Array.from(storageAccountAddress) as number[], blobName) .accounts({ owner: ownerPubkey, }) .instruction(); return new Uint8Array(ix.data); } /** * Encode the assert_access instruction data using Anchor. * @param signerAddress - The user's Solana address (the signer) * @param blobMetadataPda - The blob metadata PDA * @param receiptPda - The access receipt PDA */ export async function encodeAssertAccessData( fullBlobNameBytes: Uint8Array, signerAddress: Address, blobMetadataPda: Address, receiptPda: Address ): Promise { const signerPubkey = new PublicKey(signerAddress); const program = getAceHookProgram(signerPubkey); // Build the instruction using Anchor's type-safe methods builder // We must provide accounts explicitly since Anchor can't auto-resolve them const ix = await program.methods .assertAccess(Buffer.from(fullBlobNameBytes)) .accounts({ blobMetadata: new PublicKey(blobMetadataPda), receipt: new PublicKey(receiptPda), }) .instruction(); return new Uint8Array(ix.data); } // ============================================================================ // On-chain fetch helpers // ============================================================================ /** * Fetch and decode the blob_metadata account from Solana using Anchor. */ export async function fetchBlobMetadata( storageAccountAddressBytes: Uint8Array, blobName: string ): Promise<{ owner: Address; greenBoxBytes: Uint8Array; price: bigint; seqnum: bigint; }> { const program = getAccessControlProgram(); // Derive PDA using @solana/kit (consistent with rest of codebase) const pda = await deriveBlobMetadataPda(storageAccountAddressBytes, blobName); // Fetch account using Anchor - auto-deserializes the data const metadata = await program.account.blobMetadata.fetch(new PublicKey(pda)); if (metadata.greenBoxScheme !== GREEN_BOX_SCHEME) { throw new Error(`Unsupported green_box_scheme: ${metadata.greenBoxScheme}`); } return { owner: metadata.owner.toBase58() as Address, greenBoxBytes: Buffer.from(metadata.greenBoxBytes), price: BigInt(metadata.price.toString()), seqnum: BigInt(metadata.seqnum.toString()), }; } ``` ### lib/utils.ts Utility functions for blob names and file downloads. ```typescript // ============================================================================ // Full Blob Name Construction // ============================================================================ /** * Construct the full blob name bytes used for ACE encryption. * Format: "0x" (2 bytes) + owner_aptos_addr (32 bytes) + "/" (1 byte) + blob_name */ export function buildFullBlobNameBytes( ownerAptosAddrBytes: Uint8Array, blobName: string ): Uint8Array { const prefix = new TextEncoder().encode("0x"); const separator = new TextEncoder().encode("/"); const nameBytes = new TextEncoder().encode(blobName); const result = new Uint8Array( prefix.length + ownerAptosAddrBytes.length + separator.length + nameBytes.length ); result.set(prefix, 0); result.set(ownerAptosAddrBytes, prefix.length); result.set(separator, prefix.length + ownerAptosAddrBytes.length); result.set( nameBytes, prefix.length + ownerAptosAddrBytes.length + separator.length ); return result; } // ============================================================================ // File Download // ============================================================================ /** * Trigger a file download in the browser. */ export function downloadFile( data: Uint8Array, filename: string, mimeType = "application/octet-stream" ): void { const blob = new Blob([data as BlobPart], { type: mimeType }); const url = URL.createObjectURL(blob); const a = document.createElement("a"); a.href = url; a.download = filename; a.click(); URL.revokeObjectURL(url); } ``` ## Part D: UI Components ### components/providers.tsx Set up the Solana client and React Query providers. ```tsx "use client"; import { autoDiscover, createClient } from "@solana/client"; import { SolanaProvider } from "@solana/react-hooks"; import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; import type { PropsWithChildren } from "react"; import { config } from "../lib/config"; const isSolanaWallet = (wallet: { features: Record }) => { return Object.keys(wallet.features).some((feature) => feature.startsWith("solana:") ); }; const client = createClient({ endpoint: config.solanaRpcUrl, walletConnectors: autoDiscover({ filter: isSolanaWallet }), }); const queryClient = new QueryClient(); export function Providers({ children }: PropsWithChildren) { return ( {children} ); } ``` ### components/file-upload.tsx The seller component for uploading encrypted files. This component uses the [`useStorageAccount`](/sdks/solana-kit/react/use-storage-account) hook to derive a Shelby storage account from the connected Solana wallet. Learn more about [storage accounts](/sdks/solana-kit/node/storage-account). ```tsx "use client"; import { useUploadBlobs } from "@shelby-protocol/react"; import { useStorageAccount } from "@shelby-protocol/solana-kit/react"; import { useSendTransaction, useWalletConnection } from "@solana/react-hooks"; import { PublicKey } from "@solana/web3.js"; import { useQueryClient } from "@tanstack/react-query"; import { useCallback, useRef, useState } from "react"; import type { ReactNode } from "react"; import { config, GREEN_BOX_SCHEME, LAMPORTS_PER_SOL, SYSTEM_PROGRAM_ADDRESS, } from "../lib/config"; import { deriveBlobMetadataPda, encodeRegisterBlobData } from "../lib/anchor"; import { encryptFile, encryptRedKey, generateRedKey } from "../lib/encryption"; import { buildFullBlobNameBytes } from "../lib/utils"; import { shelbyClient } from "../lib/shelbyClient"; type UploadStep = | "idle" | "encrypting" | "uploading" | "registering" | "done" | "error"; export function FileUpload() { const { status, wallet } = useWalletConnection(); const { send, isSending } = useSendTransaction(); const fileInputRef = useRef(null); const queryClient = useQueryClient(); const { storageAccountAddress, signAndSubmitTransaction } = useStorageAccount({ client: shelbyClient, wallet, }); const { mutateAsync: uploadBlobs } = useUploadBlobs({ client: shelbyClient, }); const [selectedFile, setSelectedFile] = useState(null); const [price, setPrice] = useState(config.pricing.defaultPriceSol); const [step, setStep] = useState("idle"); const [statusMessage, setStatusMessage] = useState(null); const handleUpload = useCallback(async () => { if (!selectedFile || !storageAccountAddress || !wallet) return; const walletAddress = wallet.account.address; try { // Step 1: Generate a random AES key and encrypt the file. setStep("encrypting"); setStatusMessage("Generating encryption key and encrypting file..."); const redKey = generateRedKey(); const fileBytes = new Uint8Array(await selectedFile.arrayBuffer()); const redBox = await encryptFile(fileBytes, redKey); // Step 2: Encrypt the AES key with threshold IBE so only buyers can decrypt. const storageAccountAddressBytes = storageAccountAddress.toUint8Array(); const fullBlobNameBytes = buildFullBlobNameBytes( storageAccountAddressBytes, selectedFile.name ); setStatusMessage("Encrypting access key with threshold cryptography..."); const greenBoxBytes = await encryptRedKey(redKey, fullBlobNameBytes); // Step 3: Upload the encrypted file to Shelby storage. setStep("uploading"); setStatusMessage("Uploading encrypted file to Shelby..."); await uploadBlobs({ signer: { account: storageAccountAddress, signAndSubmitTransaction }, blobs: [ { blobName: selectedFile.name, blobData: redBox, }, ], expirationMicros: (1000 * 60 * 60 * 24 * 30 + Date.now()) * 1000, }); // Step 4: Register the encrypted key and price on-chain. setStep("registering"); setStatusMessage("Registering file on Solana..."); const priceLamports = BigInt( Math.floor(Number.parseFloat(price) * Number(LAMPORTS_PER_SOL)) ); const blobMetadataPda = await deriveBlobMetadataPda( storageAccountAddressBytes, selectedFile.name ); const instructionData = await encodeRegisterBlobData( storageAccountAddressBytes, selectedFile.name, GREEN_BOX_SCHEME, greenBoxBytes, priceLamports, walletAddress ); const instruction = { programAddress: config.programs.accessControl, accounts: [ { address: blobMetadataPda, role: 1 }, { address: walletAddress, role: 3 }, { address: SYSTEM_PROGRAM_ADDRESS, role: 0 }, ], data: instructionData, }; const result = await send({ instructions: [instruction] }); setStep("done"); const explorerUrl = `https://explorer.solana.com/tx/${result}?cluster=testnet`; setStatusMessage( <> Successfully uploaded and registered: {selectedFile.name}. View transaction ); setSelectedFile(null); if (fileInputRef.current) { fileInputRef.current.value = ""; } setTimeout(() => { queryClient.invalidateQueries(); }, 1500); } catch (err) { setStep("error"); const message = err instanceof Error ? err.message : "Unknown error"; const cause = err instanceof Error && err.cause instanceof Error ? err.cause.message : undefined; setStatusMessage( cause ? `Error: ${message} — ${cause}` : `Error: ${message}` ); } }, [ selectedFile, storageAccountAddress, wallet, price, send, signAndSubmitTransaction, uploadBlobs, queryClient, ]); const handleFileChange = useCallback( (e: React.ChangeEvent) => { const file = e.target.files?.[0] ?? null; setSelectedFile(file); setStep("idle"); setStatusMessage(null); }, [] ); const handleSelectFile = useCallback(() => { fileInputRef.current?.click(); }, []); const isProcessing = step !== "idle" && step !== "done" && step !== "error"; if (status !== "connected") { return (

Upload Token-Gated File

Connect your wallet to upload encrypted files that can be purchased by others.

Wallet not connected
); } return (

Upload Token-Gated File

Upload an encrypted file to Shelby and register it on Solana. Others can purchase access to decrypt it.

{/* File Input */}
{selectedFile ? (

{selectedFile.name}

{(selectedFile.size / 1024).toFixed(1)} KB

) : (

Click to select a file to sell

)}
{/* Price Input */}
setPrice(e.target.value)} disabled={isProcessing || isSending} className="flex-1 rounded-lg border border-border-low bg-card px-4 py-2.5 text-sm outline-none transition placeholder:text-muted focus:border-foreground/30 disabled:cursor-not-allowed disabled:opacity-60" />
{/* Upload Button */}
{/* Progress Steps */} {step !== "idle" && (
Encrypt file
Upload to Shelby
Register on Solana
)} {/* Status Message */} {statusMessage && (
{statusMessage}
)} {/* Storage Account Info */}

Shelby Storage Account:{" "} {storageAccountAddress?.toString()}

); } function StepIndicator({ active, done }: { active: boolean; done: boolean }) { if (done) { return ( ); } if (active) { return ( ); } return ( ); } ``` The `handleUpload` function encrypts and registers files. Here's how it works: **Step 1: Generate Key & Encrypt File** ``` file → AES key (redKey) → encrypted file (redBox) ``` 1. **Generate `redKey`** — a random 256-bit AES key 2. **Encrypt file** with AES-GCM → produces `redBox` (IV + ciphertext) The `redKey` is the secret that unlocks the file. Anyone with this key can decrypt. **Step 2: Build Encryption Identity** ``` storageAccountAddress + filename → fullBlobNameBytes ``` The full blob name uniquely identifies this file for ACE threshold encryption. Format: `0x{owner_addr}/{filename}` **Step 3: Encrypt the Key** ``` redKey + fullBlobNameBytes → greenBox (threshold-encrypted key) ``` This is the critical step: 1. Fetch IBE encryption key from ACE workers 2. Encrypt `redKey` using threshold IBE with `fullBlobNameBytes` as the identity 3. Result: `greenBox` — can only be decrypted by proving access on-chain **Step 4: Upload to Shelby** ``` redBox → Shelby storage ``` Upload the encrypted file (`redBox`) to decentralized storage. The file is useless without the `redKey`. **Step 5: Register On-Chain** ``` greenBox + price + owner → blob_metadata PDA ``` Call the `register_blob` instruction to store: * `owner` — seller's Solana pubkey (receives payments) * `greenBox` — threshold-encrypted key * `price` — cost in lamports to purchase access **Visual Summary** ``` ┌─────────────────────────────────────────────────────────────────┐ │ 1. GENERATE KEY │ │ crypto.getRandomValues() → redKey (32 bytes) │ ├─────────────────────────────────────────────────────────────────┤ │ 2. ENCRYPT FILE │ │ file + redKey → AES-GCM → redBox │ ├─────────────────────────────────────────────────────────────────┤ │ 3. ENCRYPT KEY (threshold IBE) │ │ redKey + fullBlobName → ACE → greenBox │ ├─────────────────────────────────────────────────────────────────┤ │ 4. UPLOAD TO SHELBY │ │ redBox → decentralized storage │ ├─────────────────────────────────────────────────────────────────┤ │ 5. REGISTER ON-CHAIN │ │ greenBox + price → blob_metadata PDA │ └─────────────────────────────────────────────────────────────────┘ ``` The key insight is **separation of concerns**: the encrypted file goes to Shelby, while the encrypted key and pricing go on-chain. Only users who pay can get the key to decrypt the file. ### components/purchase-card.tsx The buyer component for browsing, purchasing, and downloading files. ```tsx "use client"; import { useAccountBlobs } from "@shelby-protocol/react"; import { type BlobMetadata } from "@shelby-protocol/sdk/browser"; import { appendTransactionMessageInstruction, type Blockhash, compileTransaction, createSolanaRpc, createTransactionMessage, getTransactionEncoder, setTransactionMessageFeePayer, setTransactionMessageLifetimeUsingBlockhash, } from "@solana/kit"; import { useSendTransaction, useWalletConnection } from "@solana/react-hooks"; import { useCallback, useEffect, useState } from "react"; import type { ReactNode } from "react"; import { config, SYSTEM_PROGRAM_ADDRESS } from "../lib/config"; import { shelbyClient } from "../lib/shelbyClient"; import { checkHasPurchased, deriveAccessReceiptPda, deriveBlobMetadataPda, encodeAssertAccessData, encodePurchaseData, fetchBlobMetadata, } from "../lib/anchor"; import { decryptFile, decryptGreenBox } from "../lib/encryption"; import { buildFullBlobNameBytes, downloadFile } from "../lib/utils"; type PurchaseStep = | "idle" | "purchasing" | "decrypting" | "downloading" | "done" | "error"; export function PurchaseCard() { const { status, wallet } = useWalletConnection(); const { send, isSending } = useSendTransaction(); const sellerAccount = config.shelby.sellerAccount; const { data: blobs, isLoading: isBlobsLoading, error: blobsError, } = useAccountBlobs({ client: shelbyClient, account: sellerAccount, enabled: !!sellerAccount, }); const [selectedBlob, setSelectedBlob] = useState(null); const [step, setStep] = useState("idle"); const [statusMessage, setStatusMessage] = useState(null); const [purchasedFiles, setPurchasedFiles] = useState>( new Map() ); const [isCheckingPurchases, setIsCheckingPurchases] = useState(false); useEffect(() => { if (!wallet || !blobs || blobs.length === 0) return; const buyerAddress = wallet.account.address; async function checkAllPurchases() { setIsCheckingPurchases(true); const results = new Map(); await Promise.all( blobs!.map(async (blob) => { try { const hasPurchased = await checkHasPurchased( blob.owner.bcsToBytes(), blob.blobNameSuffix, buyerAddress ); results.set(blob.blobNameSuffix, hasPurchased); } catch (err) { console.error( `Error checking purchase status for ${blob.blobNameSuffix}:`, err ); results.set(blob.blobNameSuffix, false); } }) ); setPurchasedFiles(results); setIsCheckingPurchases(false); } checkAllPurchases(); }, [wallet, blobs]); const handlePurchase = useCallback(async () => { if (!wallet || !selectedBlob) return; const buyerAddress = wallet.account.address; const fileName = selectedBlob.blobNameSuffix; try { // Step 1: Build purchase instruction with PDAs and seller info. setStep("purchasing"); setStatusMessage("Building purchase transaction..."); const storageAccountAddressBytes = selectedBlob.owner.bcsToBytes(); if (storageAccountAddressBytes.length !== 32) { throw new Error("Invalid storage account address: must be 32 bytes"); } const blobMetadataPda = await deriveBlobMetadataPda( storageAccountAddressBytes, fileName ); const receiptPda = await deriveAccessReceiptPda( storageAccountAddressBytes, fileName, buyerAddress ); const { owner: ownerSolanaPubkey } = await fetchBlobMetadata( storageAccountAddressBytes, fileName ); const instructionData = await encodePurchaseData( storageAccountAddressBytes, fileName, buyerAddress, ownerSolanaPubkey ); const instruction = { programAddress: config.programs.accessControl, accounts: [ { address: blobMetadataPda, role: 0 }, { address: receiptPda, role: 1 }, { address: buyerAddress, role: 3 }, { address: ownerSolanaPubkey, role: 1 }, { address: SYSTEM_PROGRAM_ADDRESS, role: 0 }, ], data: instructionData, }; // Step 2: Sign and submit the purchase transaction. setStatusMessage("Awaiting signature..."); const signature = await send({ instructions: [instruction] }); setStep("done"); const explorerUrl = `https://explorer.solana.com/tx/${signature}?cluster=testnet`; setStatusMessage( <> Purchase successful! You can now decrypt the file.{" "} View transaction ); setPurchasedFiles((prev) => new Map(prev).set(fileName, true)); } catch (err) { console.error("Purchase failed:", err); const errorMessage = err instanceof Error ? err.message : "Unknown error"; setStep("error"); setStatusMessage(`Error: ${errorMessage}`); } }, [wallet, selectedBlob, send]); const handleDecrypt = useCallback(async () => { if (!wallet || !selectedBlob) return; const buyerAddress = wallet.account.address; const fileName = selectedBlob.blobNameSuffix; try { setStep("decrypting"); setStatusMessage("Building proof of permission..."); // Step 1: Build the assert_access instruction as proof of permission. const storageAccountAddressBytes = selectedBlob.owner.bcsToBytes(); const fullBlobNameBytes = buildFullBlobNameBytes( storageAccountAddressBytes, fileName ); const blobMetadataPda = await deriveBlobMetadataPda( storageAccountAddressBytes, fileName ); const receiptPda = await deriveAccessReceiptPda( storageAccountAddressBytes, fileName, buyerAddress ); const assertAccessData = await encodeAssertAccessData( fullBlobNameBytes, buyerAddress, blobMetadataPda, receiptPda ); const instruction = { programAddress: config.programs.aceHook, accounts: [ { address: blobMetadataPda, role: 0 }, { address: receiptPda, role: 0 }, { address: buyerAddress, role: 2 }, ], data: assertAccessData, }; // Step 2: Build and sign the transaction (not submitted, only used as proof). const rpc = createSolanaRpc(config.solanaRpcUrl); const { value: latestBlockhash } = await rpc.getLatestBlockhash().send(); const { blockhash, lastValidBlockHeight } = latestBlockhash; let txMessage: any = createTransactionMessage({ version: 0 }); txMessage = setTransactionMessageFeePayer(buyerAddress, txMessage); txMessage = setTransactionMessageLifetimeUsingBlockhash( { blockhash: blockhash as Blockhash, lastValidBlockHeight: BigInt(lastValidBlockHeight), }, txMessage ); txMessage = appendTransactionMessageInstruction(instruction, txMessage); const compiledTx = compileTransaction(txMessage); setStatusMessage("Signing proof of permission..."); if (!wallet.signTransaction) { throw new Error( "Wallet does not support signing transactions. " + "Please use a wallet that supports the signTransaction feature." ); } const signedTx = await wallet.signTransaction(compiledTx as any); const txEncoder = getTransactionEncoder(); const serializedTx = new Uint8Array(txEncoder.encode(signedTx as any)); // Step 3: Send proof to ACE workers and receive the decryption key. const { greenBoxBytes } = await fetchBlobMetadata( storageAccountAddressBytes, fileName ); setStatusMessage("Fetching decryption key from ACE committee..."); const redKey = await decryptGreenBox( greenBoxBytes, fullBlobNameBytes, serializedTx ); // Step 4: Fetch the encrypted file from Shelby storage. setStep("downloading"); setStatusMessage("Fetching encrypted file from Shelby..."); const blob = await shelbyClient.rpc.getBlob({ account: selectedBlob.owner.toString(), blobName: fileName, }); const reader = blob.readable.getReader(); const chunks: Uint8Array[] = []; let totalLength = 0; while (true) { const { done, value } = await reader.read(); if (done) break; chunks.push(value); totalLength += value.length; } const redBox = new Uint8Array(totalLength); let offset = 0; for (const chunk of chunks) { redBox.set(chunk, offset); offset += chunk.length; } // Step 5: Decrypt and download the file. setStatusMessage("Decrypting file..."); const plaintext = await decryptFile(redBox, redKey); downloadFile(plaintext, fileName); setStep("done"); setStatusMessage("File decrypted and downloaded!"); } catch (err) { console.error("Decryption failed:", err); setStep("error"); setStatusMessage( `Error: ${err instanceof Error ? err.message : "Unknown error"}` ); } }, [wallet, selectedBlob]); const isProcessing = step !== "idle" && step !== "done" && step !== "error"; const hasPurchased = selectedBlob ? purchasedFiles.get(selectedBlob.blobNameSuffix) === true : false; const handleAction = useCallback(async () => { if (!selectedBlob) return; if (hasPurchased) { await handleDecrypt(); } else { await handlePurchase(); } }, [selectedBlob, hasPurchased, handleDecrypt, handlePurchase]); if (!sellerAccount) { return (

Browse Files

No seller account configured. Set{" "} NEXT_PUBLIC_SELLER_ACCOUNT {" "} in your .env file.

); } const isConnected = status === "connected"; return (

Browse Files

{isConnected ? "Select a file to purchase access and decrypt." : "Connect your wallet to purchase access to token-gated files."}

{/* File Grid */}
{isBlobsLoading && (
Loading files...
)} {isCheckingPurchases && !isBlobsLoading && (
Checking purchase status...
)} {blobsError && (
Failed to load files: {blobsError.message}
)} {blobs && blobs.length === 0 && (
No files found for this account.
)} {blobs && blobs.length > 0 && (
{blobs.map((blob) => { const isSelected = selectedBlob?.blobNameSuffix === blob.blobNameSuffix; const isPurchased = isConnected && purchasedFiles.get(blob.blobNameSuffix) === true; return ( ); })}
)}
{/* Action Button - only show when a file is selected */} {selectedBlob && ( )} {/* Status Message - only show when a file is selected */} {selectedBlob && statusMessage && (
{statusMessage}
)}
); } ``` The `handleDecrypt` function in `purchase-card.tsx` is the most complex part of the application. Here's how it works step by step: **Step 1: Build Proof of Permission** ``` storageAccountAddressBytes → fullBlobNameBytes → PDAs → instruction ``` First, we construct the data needed to prove access: * **Storage account bytes** — identifies the file owner * **Full blob name** — unique identifier for ACE: `0x{owner_addr}/{filename}` * **PDAs** — `blob_metadata` (stores the encrypted key) and `receipt` (proves purchase) * **`assert_access` instruction** — verifies the buyer has a valid receipt **Step 2: Create & Sign Transaction (Not Submitted!)** ``` instruction → transaction message → compiled tx → signed tx → serialized bytes ``` We build a complete transaction but **never submit it** to Solana: 1. Fetch a recent blockhash (required for valid transaction format) 2. Build transaction message with fee payer and instruction 3. Compile to wire format 4. **Sign** — buyer signs the transaction 5. Serialize to bytes for ACE The signed transaction serves as **cryptographic proof** that the buyer controls the wallet. **Step 3: Fetch Encrypted Key from Chain** ``` blob_metadata PDA → greenBoxBytes (encrypted AES key) ``` The `blob_metadata` account stores the `greenBoxBytes` — the AES key encrypted with threshold IBE during upload. **Step 4: Decrypt Key via ACE Workers** ``` greenBoxBytes + fullBlobNameBytes + signedTx → redKey (AES key) ``` This is where the magic happens: 1. Send to ACE workers: encrypted key, blob identity, and signed transaction 2. Workers **simulate** the `assert_access` transaction to verify it would succeed 3. Simulation passing proves the receipt PDA exists (buyer has purchased) 4. Workers release their key shares → threshold reconstruction recovers `redKey` **Step 5: Fetch & Decrypt File** ``` Shelby → redBox (encrypted file) → decrypt with redKey → plaintext → download ``` Finally: 1. Fetch the encrypted file (`redBox`) from Shelby storage 2. Decrypt using AES-GCM with the recovered `redKey` 3. Trigger browser download **Visual Summary** ``` ┌─────────────────────────────────────────────────────────────────┐ │ 1. BUILD PROOF │ │ owner + filename → PDAs → assert_access instruction │ ├─────────────────────────────────────────────────────────────────┤ │ 2. SIGN (not submit!) │ │ instruction → tx → sign → serialize to bytes │ ├─────────────────────────────────────────────────────────────────┤ │ 3. FETCH ENCRYPTED KEY │ │ blob_metadata PDA → greenBoxBytes │ ├─────────────────────────────────────────────────────────────────┤ │ 4. ACE DECRYPTION │ │ greenBox + signedTx (proof) → workers → redKey │ ├─────────────────────────────────────────────────────────────────┤ │ 5. FETCH & DECRYPT FILE │ │ Shelby → redBox → AES decrypt → download │ └─────────────────────────────────────────────────────────────────┘ ``` The key insight is that the signed transaction proves access **without** being submitted on-chain. ACE workers simulate it to verify the receipt exists, then release the decryption key. ## Part E: Main Page ### app/layout.tsx ```tsx import type { Metadata } from "next"; import { Inter } from "next/font/google"; import "./globals.css"; import { Providers } from "./components/providers"; const inter = Inter({ subsets: ["latin"] }); export const metadata: Metadata = { title: "Token-Gated Files", description: "Buy and sell encrypted files on Solana", }; export default function RootLayout({ children, }: { children: React.ReactNode; }) { return ( {children} ); } ``` ### app/page.tsx ```tsx "use client"; import { FileUpload } from "./components/file-upload"; import { PurchaseCard } from "./components/purchase-card"; export default function Home() { return (

Token-Gated Files

Upload and buy encrypted files

Upload encrypted files to sell, or purchase access to existing files with SOL.

Files for sale

); } ``` ## Run the Application Start the development server: ```bash npm run dev ``` Visit [http://localhost:3000](http://localhost:3000) and test the complete flow: ### Step 1: Upload a File 1. Connect your Solana wallet 2. Note your **Storage Account** address displayed below the upload form 3. Make sure [to fund your Storage Account](/sdks/solana-kit/guides/fund-account) with APT and ShelbyUSD. 4. Upload a file with a price ### Step 2: Configure Seller Account After uploading, add the seller account to your `.env` file so the "Browse Files" section can display files for sale: ```bash # Seller Account (your storage account address from the upload step) NEXT_PUBLIC_SELLER_ACCOUNT=0x-your-storage-account-address ``` Your final `.env` file should look like: ```bash # Shelby Storage NEXT_PUBLIC_TESTNET_API_KEY=AG-your-api-key-here # Solana RPC NEXT_PUBLIC_SOLANA_RPC_URL=your-choice-of-solana-rpc # Program IDs NEXT_PUBLIC_ACCESS_CONTROL_PROGRAM_ID=6XyAbrfHK5sinJAj3nXEVG2ALzKTXQv89JLuYwXictGV NEXT_PUBLIC_ACE_HOOK_PROGRAM_ID=8jDv41SQVKCaVtkbFS1ZaVDDCEtKkAc7QXV3Y1psGts9 # ACE Workers NEXT_PUBLIC_ACE_WORKER_0=https://ace-worker-0-646682240579.europe-west1.run.app NEXT_PUBLIC_ACE_WORKER_1=https://ace-worker-1-646682240579.europe-west1.run.app NEXT_PUBLIC_ACE_THRESHOLD=2 NEXT_PUBLIC_ACE_CHAIN_NAME=testnet # Seller Account NEXT_PUBLIC_SELLER_ACCOUNT=0x-your-storage-account-address ``` ### Step 3: Test the Purchase Flow 1. Restart the development server 2. Your uploaded file should now appear in "Files for sale" 3. (Optional) Connect a different wallet to test purchasing ## Congratulations! 🎉 You've built a complete token-gated file marketplace on Solana! The system provides: * **End-to-end encryption** - Files are encrypted before upload * **Threshold cryptography** - No single party can decrypt without a valid receipt * **SOL payments** - Simple micropayments for file access * **Decentralized storage** - Files stored on Shelby's distributed network ## Next Steps * Implement file listing by multiple sellers * Add support for updating file prices * Add support for deleting a file * Build an admin dashboard for sellers # Introduction (/sdks/solana-kit/guides/token-gated-solana) import { Step, Steps } from "fumadocs-ui/components/steps"; # Token-Gated Files on Solana In this tutorial, you'll build a complete token-gated file marketplace where: * **Sellers** upload encrypted files to Shelby storage and register them on Solana with a price * **Buyers** pay in SOL to purchase access, then decrypt and download the files The encryption uses **threshold cryptography**—the decryption key is only released when a buyer proves they have purchased access via a signed Solana transaction. ## What You'll Build By the end of this tutorial, you'll have a fully functional dApp with: * Two Solana programs (Anchor) for access control * A Next.js frontend with wallet integration * End-to-end encrypted file storage * SOL-based payments for file access ## Architecture The system uses a "double encryption" approach for security: ### Seller Flow ``` ┌─────────────────────────────────────────────────────────────────┐ │ 1. Generate random cipher key (redKey) │ │ 2. Encrypt file with redKey → redBox │ │ 3. Upload redBox to Shelby storage │ │ 4. Encrypt redKey with threshold IBE → greenBox │ │ 5. Register blob on Solana (greenBox + price) │ └─────────────────────────────────────────────────────────────────┘ ``` ### Buyer Flow ``` ┌─────────────────────────────────────────────────────────────────┐ │ 1. Call purchase() on access_control → creates receipt PDA │ │ 2. Sign assert_access() tx as proof-of-permission │ │ 3. Send signed tx to threshold workers → receive key shares │ │ 4. Combine shares → decrypt greenBox → redKey │ │ 5. Fetch redBox from Shelby │ │ 6. Decrypt redBox with redKey → original file │ └─────────────────────────────────────────────────────────────────┘ ``` ## Technology Stack | Tool | Purpose | Documentation | | -------------------- | ----------------------------------------- | ------------------------------------------------------------------------------------- | | `create-solana-dapp` | Project scaffolding with Next.js + Anchor | [Solana Docs](https://solana.com/developers/guides/getstarted/scaffold-nextjs-anchor) | | **Anchor** | Solana smart contract framework | [anchor-lang.com](https://www.anchor-lang.com) | | **ACE SDK** | Decentralized key management | [@aptos-labs/ace-sdk](https://www.npmjs.com/package/@aptos-labs/ace-sdk) | | **Shelby** | Decentralized file storage | [docs.shelby.xyz](https://docs.shelby.xyz) | **Why Anchor?** We use the Anchor framework because threshold IBE workers assume a specific transaction format in practice, and currently only support the Anchor format for verifying proof-of-permission transactions. ## Prerequisites Before starting, ensure you have: * **Node.js 18+** - [Download](https://nodejs.org/) * **Rust toolchain** - [Install via rustup](https://rustup.rs/) * **Solana CLI** - [Installation guide](https://solana.com/docs/intro/installation) * **Anchor CLI** - [Installation guide](https://www.anchor-lang.com/docs/installation) * **A Solana wallet** - [Phantom](https://phantom.app/) or similar Verify your installation: ```bash node --version # v18.0.0 or higher rustc --version # 1.70.0 or higher solana --version # 1.18.0 or higher anchor --version # 0.31.0 or higher ``` ## Dependencies Overview Throughout this tutorial, we'll install these packages: npm pnpm yarn bun ```bash npm install @shelby-protocol/sdk @shelby-protocol/solana-kit @shelby-protocol/react @aptos-labs/ace-sdk ``` ```bash pnpm add @shelby-protocol/sdk @shelby-protocol/solana-kit @shelby-protocol/react @aptos-labs/ace-sdk ``` ```bash yarn add @shelby-protocol/sdk @shelby-protocol/solana-kit @shelby-protocol/react @aptos-labs/ace-sdk ``` ```bash bun add @shelby-protocol/sdk @shelby-protocol/solana-kit @shelby-protocol/react @aptos-labs/ace-sdk ``` | Package | Purpose | | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | | `@shelby-protocol/sdk` | Core SDK for interacting with Shelby decentralized storage | | `@shelby-protocol/solana-kit` | Solana-specific integration that derives [Shelby storage accounts](/sdks/solana-kit/node/storage-account) from Solana wallet addresses | | `@shelby-protocol/react` | React hooks for Shelby operations like `useUploadBlobs` and `useAccountBlobs` | | `@aptos-labs/ace-sdk` | ACE (Access Control Encryption) SDK for encrypting/decrypting keys with distributed key management | ## Environment Configuration Environment variables connect your app to the Shelby storage network and configure the Solana programs. ### Why a Shelby API Key? API keys authenticate your app and manage rate limits when using Shelby services. Without one, your client runs in "anonymous" mode with much lower limits, which can affect performance. ### How to Get a Shelby API Key ### Navigate to Geomi Visit [geomi.dev](https://geomi.dev) in your web browser. ### Account Setup Log in to your existing account or create a new account if you haven't already. ### Create API Resource On the overview page, click the "API Resource" card to begin creating a new resource. ### Configure Resource Complete the configuration form with the following settings: * **Network**: Select `Testnet` from the available network options. * **Resource Name**: Provide a descriptive name for your API resource. * **Usage Description**: Briefly describe your intended use case. ### Generate Keys Once submitted, your API keys will be generated and displayed. Copy this key - you'll configure it in the next step. **Note**: By default the site generates a key for use in a private server context. If you intend to use the key in a frontend context, create a client key. ## Next Steps Ready to start building? Let's set up your development environment # Writing the Programs (/sdks/solana-kit/guides/token-gated-solana/programs) import { Step, Steps } from "fumadocs-ui/components/steps"; import { File, Folder, Files } from "fumadocs-ui/components/files"; # Writing the Programs In this section, you'll create two Anchor programs: 1. **access\_control** - Manages blob registration and purchases 2. **ace\_hook** - Verifies access for threshold decryption The programs are already deployed on Solana testnet: | Program | Address | | ---------------- | ---------------------------------------------- | | `access_control` | `6XyAbrfHK5sinJAj3nXEVG2ALzKTXQv89JLuYwXictGV` | | `ace_hook` | `8jDv41SQVKCaVtkbFS1ZaVDDCEtKkAc7QXV3Y1psGts9` | Feel free to use these deployed programs instead of building and deploying your own. Simply configure the program IDs in your `.env` file and skip to the [frontend section](/sdks/solana-kit/guides/token-gated-solana/frontend). ## Program Architecture Our system needs two programs working together: ``` ┌─────────────────────────────────────────────────────────────────┐ │ access_control │ │ • register_blob() - Store greenBox + price │ │ • purchase() - Transfer SOL, create receipt │ │ • BlobMetadata PDA - Stores encrypted key info │ │ • Receipt PDA - Proves purchase │ └─────────────────────────────────────────────────────────────────┘ │ │ reads ▼ ┌─────────────────────────────────────────────────────────────────┐ │ ace_hook │ │ • assert_access() - Verify receipt for decryption │ │ • Called by buyers as proof-of-permission │ │ • Verified by ACE workers │ └─────────────────────────────────────────────────────────────────┘ ``` ## Part A: The `access_control` Program This program handles the core marketplace functionality: registering encrypted files and processing purchases. ### Create Program Files ### Cargo.toml Create `anchor/programs/access_control/Cargo.toml`: ```toml [package] name = "access_control" version = "0.1.0" description = "Token-gated access control for encrypted files on Shelby" edition = "2021" [lib] crate-type = ["cdylib", "lib"] name = "access_control" [features] default = [] cpi = ["no-entrypoint"] no-entrypoint = [] no-idl = [] no-log-ix-name = [] idl-build = ["anchor-lang/idl-build"] [dependencies] anchor-lang = "0.31.1" sha3 = "0.10" ``` ### lib.rs - Main Program Create `anchor/programs/access_control/src/lib.rs`: ```rust use anchor_lang::prelude::*; declare_id!("YOUR_PROGRAM_ID_HERE"); pub mod instructions; pub use instructions::*; /// Metadata for an encrypted blob registered on-chain. /// Stores the encrypted key (greenBox) and price for access. #[account] pub struct BlobMetadata { /// The Solana owner who registered this blob pub owner: Pubkey, /// Encryption scheme used for the greenBox (2 = threshold IBE) pub green_box_scheme: u8, /// The encrypted cipher key (greenBox) that can be decrypted via ACE pub green_box_bytes: Vec, /// Sequence number for tracking updates pub seqnum: u64, /// Price in lamports to purchase access pub price: u64, } /// Receipt proving a buyer has purchased access to a blob. #[account] pub struct Receipt { /// Sequence number at time of purchase (must match blob's seqnum) pub seqnum: u64, } #[program] pub mod access_control { use super::*; /// Register a new encrypted blob with its greenBox and price. /// Called by the file owner after uploading encrypted content to Shelby. pub fn register_blob( ctx: Context, storage_account_address: [u8; 32], blob_name: String, green_box_scheme: u8, green_box_bytes: Vec, price: u64, ) -> Result<()> { msg!("register_blob: blob_name={}", blob_name); msg!("register_blob: green_box_scheme={}", green_box_scheme); msg!("register_blob: green_box_bytes_len={}", green_box_bytes.len()); msg!("register_blob: price={}", price); instructions::register_blob::handler( ctx, storage_account_address, blob_name, green_box_scheme, green_box_bytes, price, ) } /// Purchase access to a blob by paying the owner. /// Creates a receipt PDA that proves the buyer has paid. pub fn purchase( ctx: Context, storage_account_address: [u8; 32], blob_name: String ) -> Result<()> { instructions::purchase::handler(ctx, storage_account_address, blob_name) } } ``` ### instructions/mod.rs Create `anchor/programs/access_control/src/instructions/mod.rs`: ```rust pub mod register_blob; pub mod purchase; pub use register_blob::*; pub use purchase::*; ``` ### instructions/register\_blob.rs Create `anchor/programs/access_control/src/instructions/register_blob.rs`: ```rust use anchor_lang::prelude::*; use crate::BlobMetadata; #[derive(Accounts)] #[instruction(storage_account_address: [u8; 32], blob_name: String)] pub struct RegisterBlob<'info> { #[account( init, payer = owner, // discriminator + owner + scheme + greenBox + seqnum + price // NOTE: Solana CPI limits account creation to 10KB (10240 bytes) // greenBox is typically ~300 bytes, allocate 1KB to be safe space = 8 // discriminator + 32 // owner + 1 // scheme + 4 + 1024 // green_box_bytes vec len + data (1KB) + 8 // seqnum + 8, // price seeds = [b"blob_metadata", storage_account_address.as_ref(), blob_name.as_bytes()], bump )] pub blob_metadata: Account<'info, BlobMetadata>, #[account(mut)] pub owner: Signer<'info>, pub system_program: Program<'info, System>, } pub fn handler( ctx: Context, _storage_account_address: [u8; 32], _blob_name: String, green_box_scheme: u8, green_box_bytes: Vec, price: u64, ) -> Result<()> { let blob_metadata = &mut ctx.accounts.blob_metadata; blob_metadata.owner = ctx.accounts.owner.key(); blob_metadata.green_box_scheme = green_box_scheme; blob_metadata.green_box_bytes = green_box_bytes; blob_metadata.price = price; blob_metadata.seqnum += 1; Ok(()) } ``` ### instructions/purchase.rs Create `anchor/programs/access_control/src/instructions/purchase.rs`: ```rust use anchor_lang::prelude::*; use crate::{BlobMetadata, Receipt}; #[derive(Accounts)] #[instruction(storage_account_address: [u8; 32], blob_name: String)] pub struct Purchase<'info> { #[account( seeds = [b"blob_metadata", storage_account_address.as_ref(), blob_name.as_bytes()], bump )] pub blob_metadata: Account<'info, BlobMetadata>, #[account( init, payer = buyer, space = 8 + 8, // discriminator + seqnum seeds = [b"access", storage_account_address.as_ref(), blob_name.as_bytes(), buyer.key().as_ref()], bump )] pub receipt: Account<'info, Receipt>, #[account(mut)] pub buyer: Signer<'info>, /// CHECK: We verify this matches blob_metadata.owner #[account(mut)] pub owner: AccountInfo<'info>, pub system_program: Program<'info, System>, } pub fn handler( ctx: Context, _storage_account_address: [u8; 32], _blob_name: String ) -> Result<()> { let blob_metadata = &ctx.accounts.blob_metadata; msg!("purchase: price={}", blob_metadata.price); msg!("purchase: seqnum={}", blob_metadata.seqnum); // Verify the owner account matches the blob's registered owner require!( ctx.accounts.owner.key() == blob_metadata.owner, PurchaseError::InvalidOwner ); // Transfer SOL from buyer to owner anchor_lang::solana_program::program::invoke( &anchor_lang::solana_program::system_instruction::transfer( ctx.accounts.buyer.key, &ctx.accounts.owner.key(), blob_metadata.price, ), &[ ctx.accounts.buyer.to_account_info(), ctx.accounts.owner.to_account_info(), ctx.accounts.system_program.to_account_info(), ], )?; // Record the purchase let receipt = &mut ctx.accounts.receipt; receipt.seqnum = blob_metadata.seqnum; Ok(()) } #[error_code] pub enum PurchaseError { #[msg("Owner account does not match the blob's registered owner")] InvalidOwner, } ``` ## Part B: The `ace_hook` Program This program verifies that a user has purchased access to a blob. It's called by buyers to create a "proof of permission" that ACE workers verify before releasing decryption keys. ### Why a Separate Program? The ACE (Access Control Encryption) system needs to verify that a user has legitimate access before releasing decryption key shares. Workers do this by: 1. Receiving a signed transaction from the buyer 2. Simulating the transaction on-chain 3. Verifying it calls the correct `assert_access` function Having a separate program makes verification clean and auditable—workers can check the exact program ID and instruction being called. ### What are ACE Workers? ACE workers are a distributed network of nodes that collectively manage encryption keys. No single worker holds the complete decryption key—instead, the key is split across multiple workers using threshold cryptography. When a buyer wants to decrypt a file: 1. They sign a transaction calling `assert_access` to prove they have a valid receipt 2. They send this signed transaction to the workers as a "proof of permission" 3. Each worker verifies the transaction on-chain and returns their key share 4. The client combines the shares to reconstruct the decryption key **Deployed Workers (Testnet):** | Worker | Endpoint | | -------- | -------------------------------------------------------- | | Worker 0 | `https://ace-worker-0-646682240579.europe-west1.run.app` | | Worker 1 | `https://ace-worker-1-646682240579.europe-west1.run.app` | The threshold is set to 2, meaning both workers must agree to release their key shares. ### Create Program Files ### Cargo.toml Create `anchor/programs/ace_hook/Cargo.toml`: ```toml [package] name = "ace_hook" version = "0.1.0" description = "ACE hook - allows users to prove access by signing a transaction for decryption key providers" edition = "2021" [lib] crate-type = ["cdylib", "lib"] name = "ace_hook" [features] default = [] cpi = ["no-entrypoint"] no-entrypoint = [] no-idl = [] no-log-ix-name = [] idl-build = ["anchor-lang/idl-build"] [dependencies] anchor-lang = "0.31.1" access_control = { path = "../access_control", features = ["no-entrypoint"] } ``` ### lib.rs - Hook Program Create `anchor/programs/ace_hook/src/lib.rs`: ```rust #![allow(unexpected_cfgs)] use anchor_lang::prelude::*; use access_control::{BlobMetadata, Receipt, ID as ACCESS_CONTROL_PROGRAM_ID}; // This is an ACE-specific access control program. // Apps need to define such a callable so that consumers can prove they are allowed // to access a given data by signing a transaction to call this callable and // handing the signed transaction to decryption key providers. declare_id!("YOUR_ACE_HOOK_PROGRAM_ID_HERE"); #[program] pub mod ace_hook { use super::*; /// Assert that the caller has access to the specified blob. /// This function is called by consumers who sign a transaction proving their access. /// The signed transaction can then be presented to decryption key providers as proof of permission. pub fn assert_access( ctx: Context, full_blob_name_bytes: Vec ) -> Result<()> { // Debug: log what we're checking msg!("blob_metadata.owner = {}", ctx.accounts.blob_metadata.owner); msg!("receipt.owner = {}", ctx.accounts.receipt.owner); msg!("expected = {}", ACCESS_CONTROL_PROGRAM_ID); // Verify blob_metadata account is owned by access_control program if *ctx.accounts.blob_metadata.owner != ACCESS_CONTROL_PROGRAM_ID { msg!("FAIL: blob_metadata owner"); return Err(ErrorCode::InvalidAccountOwner.into()); } // Verify receipt account is owned by access_control program if *ctx.accounts.receipt.owner != ACCESS_CONTROL_PROGRAM_ID { msg!("FAIL: receipt owner"); return Err(ErrorCode::InvalidAccountOwner.into()); } // Parse full_blob_name_bytes: // [0:2] "0x" prefix // [2:34] owner_aptos_addr (32 bytes) // [34] "/" separator // [35:] blob_name if full_blob_name_bytes.len() < 35 || &full_blob_name_bytes[0..2] != b"0x" || full_blob_name_bytes[34] != b'/' { return Err(ErrorCode::InvalidBlobName.into()); } let owner_aptos_addr: [u8; 32] = full_blob_name_bytes[2..34] .try_into() .map_err(|_| ErrorCode::InvalidBlobName)?; let blob_name = &full_blob_name_bytes[35..]; // Derive expected PDA for blob_metadata (using access_control program's ID) let (expected_blob_metadata_pda, _bump) = Pubkey::find_program_address( &[ b"blob_metadata", owner_aptos_addr.as_ref(), blob_name, ], &ACCESS_CONTROL_PROGRAM_ID, ); if ctx.accounts.blob_metadata.key() != expected_blob_metadata_pda { return Err(ErrorCode::InvalidAccountOwner.into()); } // Derive expected PDA for receipt (using access_control program's ID) let (expected_receipt_pda, _bump) = Pubkey::find_program_address( &[ b"access", owner_aptos_addr.as_ref(), blob_name, ctx.accounts.user.key().as_ref(), ], &ACCESS_CONTROL_PROGRAM_ID, ); if ctx.accounts.receipt.key() != expected_receipt_pda { return Err(ErrorCode::InvalidAccountOwner.into()); } // Deserialize accounts owned by access_control program let blob_metadata_data = &ctx.accounts.blob_metadata.try_borrow_data()?; let mut blob_metadata_slice = &blob_metadata_data[8..]; // Skip 8-byte discriminator let blob_metadata = BlobMetadata::deserialize(&mut blob_metadata_slice)?; let receipt_data = &ctx.accounts.receipt.try_borrow_data()?; let mut receipt_slice = &receipt_data[8..]; // Skip 8-byte discriminator let receipt = Receipt::deserialize(&mut receipt_slice)?; require!( blob_metadata.seqnum == receipt.seqnum, ErrorCode::AccessDenied ); Ok(()) } } #[derive(Accounts)] pub struct AssertAccess<'info> { /// CHECK: Account owned by access_control program, we verify ownership and deserialize manually pub blob_metadata: AccountInfo<'info>, /// CHECK: Account owned by access_control program, we verify ownership and deserialize manually pub receipt: AccountInfo<'info>, pub user: Signer<'info>, } #[error_code] pub enum ErrorCode { #[msg("Access denied")] AccessDenied, #[msg("Invalid blob name format")] InvalidBlobName, #[msg("Invalid account owner")] InvalidAccountOwner, } ``` ## Part C: Build and Deploy ### Prerequisites: Fund Your Solana Wallet Before deploying to testnet, your local Solana wallet must be funded with SOL to pay for deployment fees. ```bash # Check your current wallet address solana address # Request an airdrop (testnet only, may take a few tries) solana airdrop 2 --url testnet # Verify your balance solana balance --url testnet ``` Testnet airdrops are rate-limited. If the airdrop fails, wait a few minutes and try again, or use the [Solana Faucet](https://faucet.solana.com/). ### Build the Programs ```bash cd anchor anchor build ``` This generates: * Compiled programs in `target/deploy/` * IDL files in `target/idl/` * TypeScript types in `target/types/` ### Get Program IDs After building, get the generated program IDs: ```bash solana address -k target/deploy/access_control-keypair.json solana address -k target/deploy/ace_hook-keypair.json ``` ### Update Program IDs Update the `declare_id!` macros in your source files with the actual addresses: 1. `anchor/programs/access_control/src/lib.rs`: ```rust declare_id!("YOUR_ACCESS_CONTROL_ADDRESS"); ``` 2. `anchor/programs/ace_hook/src/lib.rs`: ```rust declare_id!("YOUR_ACE_HOOK_ADDRESS"); ``` 3. `anchor/Anchor.toml`: ```toml [programs.testnet] access_control = "YOUR_ACCESS_CONTROL_ADDRESS" ace_hook = "YOUR_ACE_HOOK_ADDRESS" ``` ### Rebuild and Deploy ```bash # Rebuild with updated IDs anchor build # Deploy to testnet anchor deploy --provider.cluster testnet ``` ### Update Environment After deployment, add the following to your `.env` file: ```bash # Program IDs (replace with your deployed addresses) NEXT_PUBLIC_ACCESS_CONTROL_PROGRAM_ID=your-access-control-address NEXT_PUBLIC_ACE_HOOK_PROGRAM_ID=your-ace-hook-address # Solana RPC NEXT_PUBLIC_SOLANA_RPC_URL=your-choice-of-solana-rpc # ACE Workers (testnet) NEXT_PUBLIC_ACE_WORKER_0=https://ace-worker-0-646682240579.europe-west1.run.app NEXT_PUBLIC_ACE_WORKER_1=https://ace-worker-1-646682240579.europe-west1.run.app NEXT_PUBLIC_ACE_THRESHOLD=2 NEXT_PUBLIC_ACE_CHAIN_NAME=testnet ``` Your complete `.env` file should now look like: ```bash # Shelby Storage (from setup step) NEXT_PUBLIC_TESTNET_API_KEY=AG-your-api-key-here # Solana RPC NEXT_PUBLIC_SOLANA_RPC_URL=your-choice-of-solana-rpc # Program IDs NEXT_PUBLIC_ACCESS_CONTROL_PROGRAM_ID=your-access-control-address NEXT_PUBLIC_ACE_HOOK_PROGRAM_ID=your-ace-hook-address # ACE Workers NEXT_PUBLIC_ACE_WORKER_0=https://ace-worker-0-646682240579.europe-west1.run.app NEXT_PUBLIC_ACE_WORKER_1=https://ace-worker-1-646682240579.europe-west1.run.app NEXT_PUBLIC_ACE_THRESHOLD=2 NEXT_PUBLIC_ACE_CHAIN_NAME=testnet ``` If you skipped deployment, use these addresses: | Program | Address | | ---------------- | ---------------------------------------------- | | `access_control` | `6XyAbrfHK5sinJAj3nXEVG2ALzKTXQv89JLuYwXictGV` | | `ace_hook` | `8jDv41SQVKCaVtkbFS1ZaVDDCEtKkAc7QXV3Y1psGts9` | ## Next Steps With the programs deployed, let's build the frontend # Environment Setup (/sdks/solana-kit/guides/token-gated-solana/setup) import { Step, Steps } from "fumadocs-ui/components/steps"; import { File, Folder, Files } from "fumadocs-ui/components/files"; # Environment Setup In this section, you'll scaffold a new Solana dApp using `create-solana-dapp` and configure it for our token-gated file system. ## Scaffold the Project ### Run the Scaffold Command ```bash npx create-solana-dapp@latest token-gated ``` When prompted, select: * **Framework**: Next.js * **SDK**: `@solana/kit` (the new Solana SDK) ### Navigate to the Project ```bash cd token-gated ``` ### Explore the Generated Structure The scaffold creates a project structure like this: ## Clean Up Default Program The scaffold includes a default `vault` program that we don't need. Let's remove it and prepare for our custom programs. ### Delete the Vault Program ```bash rm -rf anchor/programs/vault ``` ### Remove the Vault Frontend Component The scaffold also includes a frontend component for the vault. Remove the vault-related files from the Next.js app: ```bash rm -rf web/components/vault ``` You should also remove the `VaultFeature` import and usage from your main page (`web/app/page.tsx`) if present. ### Update Anchor.toml Replace the contents of `anchor/Anchor.toml`: ```toml [toolchain] anchor_version = "0.31.1" [features] resolution = true skip-lint = false [programs.testnet] access_control = "YOUR_ACCESS_CONTROL_PROGRAM_ID" ace_hook = "YOUR_ACE_HOOK_PROGRAM_ID" [registry] url = "https://api.apr.dev" [provider] cluster = "testnet" wallet = "~/.config/solana/id.json" [scripts] test = "cargo test" ``` We'll update the program IDs after deployment. For now, you can use placeholder values or the pre-deployed addresses from the next section. ## Install Dependencies Install the required packages for Shelby storage, threshold encryption, and Solana integration: ```bash npm install @shelby-protocol/sdk @shelby-protocol/solana-kit @shelby-protocol/react @aptos-labs/ace-sdk @coral-xyz/anchor @tanstack/react-query ``` ## Configure Environment Variables Create a `.env` file in the project root with your Shelby API key: ```bash # .env NEXT_PUBLIC_TESTNET_API_KEY=AG-your-api-key-here ``` We'll add more environment variables (program IDs, RPC URLs, etc.) as we progress through the tutorial. Add `.env` to your `.gitignore` to prevent committing secrets: `bash echo ".env" >> .gitignore ` ## Project Structure After setup, your project structure should look like this: ## Verify Setup Run the development server to ensure everything is configured correctly: ```bash npm run dev ``` Visit [http://localhost:3000](http://localhost:3000). You should see the default scaffold page. ## Next Steps Now that your environment is set up, let's write the Solana programs # Downloading Files (/sdks/typescript/browser/guides/download) ## Overview This guide demonstrates how to retrieve and download files stored on the Shelby network from a browser environment. Files on Shelby are associated with the account address of the uploader and can be accessed through both direct URLs and the SDK API. ## Prerequisites Before downloading files, ensure you have: * The Shelby SDK installed and configured * Access to the account address of the file uploader * A valid API key for the Shelby network ## Download Methods ### Method 1: Direct URL Access Files stored on Shelby can be accessed directly via HTTP using a predictable URL pattern: ``` https://api.testnet.shelby.xyz/shelby/v1/blobs//. ``` **URL Structure:** * ``: The Aptos account address that uploaded the file * ``: The original name of the uploaded file * ``: The file extension (e.g., `.txt`, `.pdf`, `.jpg`) **Example:** ``` https://api.testnet.shelby.xyz/shelby/v1/blobs/0x123.../document.pdf ``` ### Method 2: SDK API Access The Shelby SDK provides programmatic access to retrieve file metadata and download files. #### Initialize the Client First, configure the Shelby client for your browser environment: ```typescript import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { Network } from "@aptos-labs/ts-sdk"; const shelbyClient = new ShelbyClient({ network: Network.TESTNET, apiKey: process.env.SHELBY_API_KEY, }); ``` #### Retrieve Account Files Get a list of all files associated with a specific account: ```typescript const blobs = await shelbyClient.coordination.getAccountBlobs({ account: accountAddress, }); ``` This returns an array containing metadata for all files uploaded by the specified account, including: * File names * Upload timestamps * File sizes * Expiration dates * Blob merkle roots #### Download Specific Files Once you have the file metadata, you can download specific files using the blob information: ```typescript // Example: Download the first file from the account if (blobs.length > 0) { const firstBlob = blobs[0]; // Construct download URL const downloadUrl = `https://api.testnet.shelby.xyz/shelby/v1/blobs/${accountAddress}/${firstBlob.name}`; // Fetch the file const response = await fetch(downloadUrl); const fileData = await response.blob(); // Create download link for user const downloadLink = document.createElement("a"); downloadLink.href = URL.createObjectURL(fileData); downloadLink.download = firstBlob.name; downloadLink.click(); } ``` ## Error Handling When downloading files, handle common error scenarios: ```typescript try { const blobs = await shelbyClient.coordination.getAccountBlobs({ account: accountAddress, }); if (blobs.length === 0) { console.log("No files found for this account"); return; } // Process files... } catch (error) { if (error.message.includes("404")) { console.error("Account or file not found"); } else if (error.message.includes("403")) { console.error("Access denied - check API key"); } else { console.error("Download failed:", error.message); } } ``` # Uploading a File (/sdks/typescript/browser/guides/upload) import { Step, Steps } from "fumadocs-ui/components/steps"; import { NetworkEndpoint } from "@/components/NetworkEndpoint"; import { Network } from "@aptos-labs/ts-sdk"; ## Prerequisites This guide demonstrates how to upload files to the Shelby network from a browser environment. Before proceeding, ensure you have: * A basic understanding of React and TypeScript * An Aptos wallet configured for the Shelby network * ShelbyUSD tokens for file uploads (1 ShelbyUSD per upload) ## Environment Setup To integrate with Aptos wallets, this guide uses the [Aptos Wallet Adapter package](https://aptos.dev/build/sdks/wallet-adapter/dapp). Follow these steps to configure your environment: ### Install the Wallet Adapter Package Install the required wallet adapter dependency: ```bash npm install @aptos-labs/wallet-adapter-react ``` ### Configure the Wallet Provider Initialize the `AptosWalletAdapterProvider` in your application: ```typescript import { AptosWalletAdapterProvider } from "@aptos-labs/wallet-adapter-react"; import { PropsWithChildren } from "react"; import { Network } from "@aptos-labs/ts-sdk"; export const WalletProvider = ({ children }: PropsWithChildren) => { return ( { console.log("Wallet connection error:", error); }} > {children} ); }; ``` ## File Upload Process Uploading a file to the Shelby network involves three sequential steps: 1. **File Encoding**: Split the file into chunks and generate commitment hashes 2. **On-Chain Registration**: Submit a transaction to register the file metadata 3. **RPC Upload**: Upload the actual file data to Shelby storage providers ### Step 1: File Encoding File encoding involves splitting the file into chunks, generating commitment hashes for each chunk, and creating a blob merkle root hash. These hashes are used for verification with storage providers. ```typescript import { type BlobCommitments, createDefaultErasureCodingProvider, generateCommitments, } from "@shelby-protocol/sdk/browser"; export const encodeFile = async (file: File): Promise => { // Convert file to Buffer format const data = Buffer.isBuffer(file) ? file : Buffer.from(await file.arrayBuffer()); // Create the erasure coding provider const provider = await createDefaultErasureCodingProvider(); // Generate commitment hashes for the file const commitments = await generateCommitments(provider, data); return commitments; }; ``` ### Step 2: On-Chain Registration Before uploading, ensure your account is funded: 1. **APT tokens**: Fund your account with testnet APT through the [Aptos Testnet Faucet](https://aptos.dev/network/faucet). 2. **ShelbyUSD tokens**: Sign up for early access through the [Shelby Discord](https://discord.gg/shelbyprotocol) to receive testnet ShelbyUSD tokens. Register the file metadata on the Aptos blockchain by creating and submitting a transaction: ```typescript import { expectedTotalChunksets, ShelbyBlobClient, } from "@shelby-protocol/sdk/browser"; // Create the registration transaction payload const payload = ShelbyBlobClient.createRegisterBlobPayload({ account: account.address, blobName: file.name, blobMerkleRoot: commitments.blob_merkle_root, numChunksets: expectedTotalChunksets(commitments.raw_data_size), expirationMicros: (1000 * 60 * 60 * 24 * 30 + Date.now()) * 1000, // 30 days from now blobSize: commitments.raw_data_size, }); ``` Submit the transaction using the wallet adapter: Ensure your wallet is configured for Aptos Testnet. In addition, to upload a file, you will need your account to have two assets: * **APT tokens**: Used to pay for gas fees when sending transactions * **ShelbyUSD tokens**: Used to pay for uploading the file to the Shelby network 1. **APT tokens**: Fund your account with testnet APT through the [Aptos Testnet Faucet](https://aptos.dev/network/faucet). 2. **ShelbyUSD tokens**: Sign up for early access through the [Shelby Discord](https://discord.gg/shelbyprotocol) to receive testnet ShelbyUSD tokens. Submit the previously created register blob payload with the wallet ```typescript import { Aptos, AptosConfig, Network } from "@aptos-labs/ts-sdk"; import { type InputTransactionData, useWallet, } from "@aptos-labs/wallet-adapter-react"; const { signAndSubmitTransaction } = useWallet(); // Submit the registration transaction const transaction: InputTransactionData = { data: payload, }; const transactionSubmitted = await signAndSubmitTransaction(transaction); // Initialize Aptos client export const aptosClient = new Aptos( new AptosConfig({ network: Network.TESTNET, clientConfig: { API_KEY: process.env.APTOS_API_KEY, }, }), ); // Wait for transaction confirmation await aptosClient.waitForTransaction({ transactionHash: transactionSubmitted.hash, }); ``` ### Step 3: RPC Upload After successful on-chain registration, upload the file data to the Shelby RPC. The RPC validates the file against the registered commitment hashes before accepting the upload. **Important**: The RPC upload must occur after on-chain registration, as the RPC verifies the file's registration status before processing the upload. ```typescript import { ShelbyClient } from "@shelby-protocol/sdk/browser"; import { Network } from "@aptos-labs/ts-sdk"; import { useWallet } from "@aptos-labs/wallet-adapter-react"; const { account } = useWallet(); // Initialize Shelby client const shelbyClient = new ShelbyClient({ network: Network.TESTNET, apiKey: process.env.SHELBY_API_KEY, }); // Upload file data to Shelby RPC await shelbyClient.rpc.putBlob({ account: account.address, blobName: file.name, blobData: new Uint8Array(await file.arrayBuffer()), }); ``` After successful upload, your file is stored on the Shelby network and can be retrieved using the download functionality. # Manually Uploading a File (/sdks/typescript/node/guides/manually-uploading-file) import { Step, Steps } from 'fumadocs-ui/components/steps'; # Overview In this guide, we will walk you through the process of manually uploading a file to the Shelby network from a Node.js environment. As opposed to the [Uploading a File](/sdks/typescript/node/guides/uploading-file) guide, this guide will go through the end-to-end process of generating commitments, writing them to the coordination layer, and then confirming them through the RPC layer. This guide assumes you already have a Node.js environment setup and will be using the Testnet network. ## Getting Started ### Installation and Setup This guide assumes you already have a basic understanding of the [Uploading a File](/sdks/typescript/node/guides/uploading-file) guide. If you do not, please refer to that guide first before proceeding. ### Generating Commitments The first step before uploading a file is to generate the commitments for the file. This can be done by using the `generateCommitments` function from the SDK. The SDK now supports automatic provider management. If you're using a single client, you don't need to create a provider manually. For advanced use cases where you need to share a provider across multiple clients, see the example below. ```typescript title="uploadScript.ts" import { Ed25519Account, Ed25519PrivateKey, Aptos, AptosConfig, Network } from "@aptos-labs/ts-sdk"; import { ShelbyNodeClient, ClayErasureCodingProvider, generateCommitments } from "@shelby-protocol/sdk/node"; import fs from "fs/promises"; // 1. Setup your signer const signer = new Ed25519Account({ privateKey: new Ed25519PrivateKey("ed25519-priv-"), }); // 2. Setup your client (provider will be created automatically) const aptosClient = new Aptos(new AptosConfig({ network: Network.TESTNET })); const shelbyClient = new ShelbyNodeClient({ network: Network.TESTNET, }); // 3. Generate the commitments const blobData = await fs.readFile("file.txt"); // Option A: Create provider explicitly (for direct use with generateCommitments) const provider = await ClayErasureCodingProvider.create(); const blobCommitments = await generateCommitments(provider, blobData); // Option B: Pass provider to client (for sharing across multiple clients) // const provider = await ClayErasureCodingProvider.create(); // const shelbyClient = new ShelbyNodeClient(config, provider); ``` ### Writing Commitments to the Coordination Layer The next step is to write the commitments to the coordination layer. This can be done by using the `registerBlob` function on the `ShelbyBlobClient` class (accessible via `client.coordination`). ```typescript title="uploadScript.ts" import { Ed25519Account, Ed25519PrivateKey, Aptos, AptosConfig, Network } from "@aptos-labs/ts-sdk"; import { ShelbyNodeClient, ClayErasureCodingProvider, generateCommitments } from "@shelby-protocol/sdk/node"; import fs from "fs/promises"; // 1. Setup your signer const signer = new Ed25519Account({ privateKey: new Ed25519PrivateKey("ed25519-priv-"), }); // 2. Setup your client const aptosClient = new Aptos(new AptosConfig({ network: Network.TESTNET })); const shelbyClient = new ShelbyNodeClient({ network: Network.TESTNET, }); // 3. Generate the commitments const blobData = await fs.readFile("file.txt"); // Create provider explicitly for use with generateCommitments const provider = await ClayErasureCodingProvider.create(); const blobCommitments = await generateCommitments(provider, blobData); // 4. Write the commitments to the coordination layer const { transaction: pendingWriteBlobCommitmentsTransaction } = await shelbyClient.coordination.registerBlob({ account: signer, blobName: "path/to/file.txt", blobMerkleRoot: blobCommitments.blob_merkle_root, size: blobData.length, expirationMicros: (1000 * 60 * 60 * 24 * 30 + Date.now()) * 1000, // 30 days }); await aptosClient.waitForTransaction({ transactionHash: pendingWriteBlobCommitmentsTransaction.hash, }); ``` ### Confirming Commitments through the RPC Layer Once the commitments have been written to the coordination layer, we can now confirm them through the RPC layer. This can be done by using the `putBlob` function on the `ShelbyRPCClient` class. ```typescript title="uploadScript.ts" import { Ed25519Account, Ed25519PrivateKey, Aptos, AptosConfig, Network } from "@aptos-labs/ts-sdk"; import { ShelbyNodeClient, ClayErasureCodingProvider, generateCommitments } from "@shelby-protocol/sdk/node"; import fs from "fs/promises"; // 1. Setup your signer const signer = new Ed25519Account({ privateKey: new Ed25519PrivateKey("ed25519-priv-"), }); // 2. Setup your client const aptosClient = new Aptos(new AptosConfig({ network: Network.TESTNET })); const shelbyClient = new ShelbyNodeClient({ network: Network.TESTNET, }); // 3. Generate the commitments const blobData = await fs.readFile("file.txt"); // Create provider explicitly for use with generateCommitments const provider = await ClayErasureCodingProvider.create(); const blobCommitments = await generateCommitments(provider, blobData); // 4. Write the commitments to the coordination layer const { transaction: pendingWriteBlobCommitmentsTransaction } = await shelbyClient.coordination.registerBlob({ account: signer, blobName: "path/to/file.txt", blobMerkleRoot: blobCommitments.blob_merkle_root, size: blobData.length, expirationMicros: (1000 * 60 * 60 * 24 * 30 + Date.now()) * 1000, // 30 days }); await aptosClient.waitForTransaction({ transactionHash: pendingWriteBlobCommitmentsTransaction.hash, }); // 5. Confirm the commitments through the RPC layer await shelbyClient.rpc.putBlob({ account: signer.accountAddress, blobName: "path/to/file.txt", blobData, }); ``` ## Conclusion And that is it! You have now manually uploaded a file to the Shelby network. For more information about the SDK, feel free to refer to the [Specifications](/sdks/typescript/node/specifications) page. Reference documentation for the API and specifications
# Uploading a File (/sdks/typescript/node/guides/uploading-file) import { Step, Steps } from "fumadocs-ui/components/steps"; # Overview In this guide, we will walk you through the process of obtaining ShelbyUSD tokens and uploading a file to the Shelby network. This guide assumes you already have a Node.js environment setup and will be using the Testnet network. ## Getting Started ### Installation To get started, you will need to install the following dependencies in your Node.js environment. npm pnpm yarn bun ```bash npm install @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ```bash pnpm add @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ```bash yarn add @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ```bash bun add @shelby-protocol/sdk @aptos-labs/ts-sdk ``` ### Setting up an account Use the `@aptos-labs/ts-sdk` package to generate a new or existing account ```typescript title="uploadScript.ts" import { Account, Ed25519Account, Ed25519PrivateKey } from "@aptos-labs/ts-sdk"; // Generate a new Ed25519 Account const account = Account.generate(); // Use an existing Ed25519 Account private key const account = new Ed25519Account({ privateKey: new Ed25519PrivateKey("ed25519-priv-..."), }); ``` ### Acquire an API Key To avoid getting rate limited when making calls to the Shelby network, make sure you [acquired an API Key](/sdks/typescript/acquire-api-keys) ### Funding your account To upload a file, you will need to have an account with two assets: * **APT tokens**: Used to pay for gas fees when sending transactions * **ShelbyUSD tokens**: Used to pay for uploading the file to the Shelby network 1. **APT tokens**: Fund your account with testnet APT through the [Aptos Testnet Faucet](https://aptos.dev/network/faucet). 2. **ShelbyUSD tokens**: Sign up for early access through the [Shelby Discord](https://discord.gg/shelbyprotocol) to receive testnet ShelbyUSD tokens. ### Setting up Shelby client Now that you have set up an account with funds, you can start setting up your shelby client to interact with the Shelby network. ```typescript title="uploadScript.ts" import { Network } from "@aptos-labs/ts-sdk"; import { ShelbyNodeClient } from "@shelby-protocol/sdk/node"; const shelbyClient = new ShelbyNodeClient({ network: Network.TESTNET, apiKey: "aptoslabs_***", }); ``` ### Uploading a file Lastly, to upload a file you can use the `upload` function from the `ShelbyNodeClient` class to upload a file to the Shelby network. ```typescript title="uploadScript.ts" import fs from "fs/promises"; // 1. Get the file data const blobData = await fs.readFile("file.txt"); // 2. Upload the file await shelbyClient.upload({ account, blobData, blobName: "path/to/file.txt", expirationMicros: (1000 * 60 * 60 * 24 * 30 + Date.now()) * 1000, // 30 days }); ``` ### Retrieving a file (Optional) To retrieve a file, you can use the `getBlob` function of the `ShelbyRPCClient` class to retrieve a file from the Shelby network. ```typescript title="getBlobScript.ts" import { ShelbyBlob } from "@shelby-protocol/sdk/node"; import fs from "fs"; // 1. Get the file const blob: ShelbyBlob = await shelbyClient.download({ account: account.accountAddress, blobName: "path/to/file.txt", }); // 2. Save the file blob.stream.pipe(fs.createWriteStream("file.txt")); ``` Alternatively, you can directly download the file using a `GET` request to the Shelby RPC endpoint. ```bash curl -X GET "https://api.testnet.shelby.xyz/shelby/v1/blobs/{account_address}/{blob_name}" > file.txt ``` The API documentation is still under development and will be provided at a later date. **Basic Example** | Blob Name | Account Address | | ---------- | -------------------------------------------- | | `file.txt` | `0x1234567890123456789012345678901234567890` | ```bash curl -X GET "https://api.testnet.shelby.xyz/shelby/v1/blobs/0x1234567890123456789012345678901234567890/file.txt" > file.txt ``` **Relative Paths Example** | Blob Name | Account Address | | ------------------ | -------------------------------------------- | | `path/to/file.txt` | `0x1234567890123456789012345678901234567890` | ```bash curl -X GET "https://api.testnet.shelby.xyz/shelby/v1/blobs/0x1234567890123456789012345678901234567890/path/to/file.txt" > file.txt ``` ## Conclusion And that is it! You have now uploaded a file to the Shelby network. For more information about the SDK, feel free to refer to the [Specifications](/sdks/typescript/node/specifications) page. Reference documentation for the API and specifications # Unknown (/apis/rpc/localhost/unknown/swagger/get) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Load the swagger UI. # Unknown (/apis/rpc/shelbynet/unknown/swagger/get) {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */} Load the swagger UI.