AWS CLI and boto3

The AWS CLI is the command-line client for S3, and boto3 is the Python library underneath it. Both work against a Tapedrive bucket once a profile points them at the gateway.

Before you start

You need a bucket and a credential. A bucket is a tape that has delegated writes to the gateway, and a credential is an access key pair the gateway operator issues you. Set up a bucket walks through both.

Configure

Add a profile for the gateway:

~/.aws/config

[profile tapedrive]
region = solana
endpoint_url = https://s3.example.com

~/.aws/credentials

[tapedrive]
aws_access_key_id = TAPE4Q7ZK2M9XH3PD8AW
aws_secret_access_key = mZ9vQ3nL8kR2tW6yB4cX7dF1hJ5pS0aG3uE8iO2q

The gateway ignores the region, so any label works. These pages use solana.

Everyday commands

# list your buckets
aws s3 ls --profile tapedrive

# list objects under a prefix
aws s3 ls s3://my-bucket/photos/ --profile tapedrive

# upload a file
aws s3 cp ./photo.jpg s3://my-bucket/photos/photo.jpg --profile tapedrive

# upload a directory
aws s3 sync ./photos s3://my-bucket/photos/ --profile tapedrive

# download a file
aws s3 cp s3://my-bucket/photos/photo.jpg ./photo.jpg --profile tapedrive

# delete an object
aws s3 rm s3://my-bucket/photos/photo.jpg --profile tapedrive

# a link anyone can open for an hour
aws s3 presign s3://my-bucket/photos/photo.jpg --expires-in 3600 --profile tapedrive

Large files switch to multipart upload on their own.

boto3

The same three values build a client:

import boto3

s3 = boto3.client(
    "s3",
    endpoint_url="https://s3.example.com",
    aws_access_key_id="TAPE4Q7ZK2M9XH3PD8AW",
    aws_secret_access_key="mZ9vQ3nL8kR2tW6yB4cX7dF1hJ5pS0aG3uE8iO2q",
    region_name="solana",
)

# upload and download
s3.upload_file("photo.jpg", "my-bucket", "photos/photo.jpg")
s3.download_file("my-bucket", "photos/photo.jpg", "photo.jpg")

# list a prefix, one page at a time
for page in s3.get_paginator("list_objects_v2").paginate(Bucket="my-bucket", Prefix="photos/"):
    for item in page.get("Contents", []):
        print(item["Key"], item["Size"])

# delete
s3.delete_object(Bucket="my-bucket", Key="photos/photo.jpg")

A 24 MiB upload goes as a three-part multipart upload in about 2 s and downloads in under a second.

Good to know

You cannot create a bucket through S3. Create the tape with tape create and delegate it to the gateway.
ETags are not MD5 hashes. Tools that compare an ETag with a local MD5 report a mismatch that is not corruption (compatibility).
Every object write is a transaction on the chain. A bucket takes one write per block, so many small uploads to one bucket are paced by the block time, not by bandwidth.

Uploads for multipart uploads and overwrites.
Integrations for the errors every S3 client shares.

CLI

S3 Gateway

Integrations

More Tools

Before you start

Configure

Everyday commands

boto3

Good to know

Next

​Before you start

​Configure

​Everyday commands

​boto3

​Good to know

​Next

Before you start

Configure

Everyday commands

boto3

Good to know

Next