Mount An S3/GCS/R2 Bucket - Nullspace

Bucket mounts use FUSE inside the guest. Run these on a FUSE-capable runtime lane, and build or select a template that installs the mount tool you need, such as s3fs for S3/R2 or gcsfuse for Google Cloud Storage.

This guide requires the FUSE-enabled kernel lane. See Choose a per-template kernel for the current private-beta kernel selection status.

S3 or R2

from nullspace import Machine

machine = Machine.create(
    template="my-s3fs-template",
    envs={
        "S3_BUCKET": "my-bucket",
        "AWS_ACCESS_KEY_ID": "AKIA...",
        "AWS_SECRET_ACCESS_KEY": "...",
        "S3_ENDPOINT": "https://ACCOUNT_ID.r2.cloudflarestorage.com",
        "S3_MOUNT_PATH": "/mnt/bucket",
    },
)

try:
    print(machine.commands.run("ls /mnt/bucket", shell=True).stdout)
finally:
    machine.kill()

For a custom template, install the tool explicitly:

from nullspace import Template

builder = (
    Template.from_ubuntu_image("22.04")
    .apt_install(["fuse", "s3fs", "ca-certificates"])
    .run_cmd("mkdir -p /mnt/bucket")
)

Google Cloud Storage

from nullspace import Machine

with open("service-account.json", encoding="utf-8") as handle:
    key_json = handle.read()

machine = Machine.create(
    template="my-gcsfuse-template",
    envs={
        "GCS_BUCKET": "my-bucket",
        "GOOGLE_APPLICATION_CREDENTIALS_JSON": key_json,
        "GCS_MOUNT_PATH": "/mnt/gcs",
    },
)

try:
    print(machine.commands.run("ls /mnt/gcs", shell=True).stdout)
finally:
    machine.kill()

For a custom template, follow the Google Cloud Storage FUSE package setup before running apt-get install gcsfuse.

Manual mount

machine.files.write("/root/.passwd-s3fs", "ACCESS_KEY:SECRET_KEY")
machine.commands.run("chmod 600 /root/.passwd-s3fs", shell=True)
machine.commands.run(
    "mkdir -p /mnt/r2 && "
    "s3fs my-r2-bucket /mnt/r2 "
    "-o passwd_file=/root/.passwd-s3fs "
    "-o url=https://ACCOUNT_ID.r2.cloudflarestorage.com "
    "-o use_path_request_style",
    shell=True,
)

Do not bake long-lived cloud credentials into templates. Pass credentials at machine create time or write short-lived credentials into the machine after it starts. Concept: Create. API reference: createMachine, execCommand, and buildTemplate.