Object stores
On an object store the bucket is the database: kill the process, start one on another machine with the same URI, and the graph is there.
Requirement: PUT-if-absent
NamiDB needs one conditional-write capability from the store:
create-if-absent (If-None-Match: * on S3). It is the compare-and-swap
behind every manifest pointer and WAL commit. Conditional overwrite
(If-Match) is not required for writes, and an eventually consistent
LIST is tolerated.
AWS S3, Google Cloud Storage, Azure Blob Storage, Cloudflare R2, Tigris and
current MinIO releases support it. An “S3-compatible” service that ignores
If-None-Match: * cannot host NamiDB safely: two writers could both
believe they committed. Check a new service before trusting it:
import boto3
s3 = boto3.client("s3", endpoint_url="https://<endpoint>")s3.put_object(Bucket="my-bucket", Key="probe", Body=b"1", IfNoneMatch="*")try: s3.put_object(Bucket="my-bucket", Key="probe", Body=b"2", IfNoneMatch="*") print("UNSAFE: the second create succeeded")except s3.exceptions.ClientError as e: print("ok:", e.response["Error"]["Code"]) # ok: PreconditionFaileds3.delete_object(Bucket="my-bucket", Key="probe")AWS S3
db = namidb.Client("s3://my-bucket/graphs?ns=prod®ion=us-west-2")export AWS_ACCESS_KEY_ID=… AWS_SECRET_ACCESS_KEY=… # or use an instance/task/pod rolenamidb-server --store "s3://my-bucket/graphs?ns=prod®ion=us-west-2" \ --auth-token "$NAMIDB_AUTH_TOKEN"Without a /prefix, objects go under tenants/<ns>/ in the bucket.
Cloudflare R2
db = namidb.Client( "s3://my-bucket?ns=prod" "&endpoint=https://<ACCOUNT_ID>.r2.cloudflarestorage.com®ion=auto")AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY hold the R2 API token’s key id
and secret.
Tigris and other S3-compatible services
Point endpoint= at the service and pass its keys as the AWS variables:
s3://my-bucket?ns=prod&endpoint=https://<service-endpoint>®ion=autoAdd allow_http=true only for plain-HTTP endpoints on a trusted network
(LocalStack, a MinIO in the same Compose project).
Google Cloud Storage
export GOOGLE_APPLICATION_CREDENTIALS=/etc/namidb/gcs-key.jsondb = namidb.Client("gs://my-bucket/graphs?ns=prod")# or name the key file in the URI:db = namidb.Client("gs://my-bucket?ns=prod&service_account=/etc/namidb/gcs-key.json")Azure Blob Storage
export AZURE_STORAGE_ACCOUNT_NAME=myaccountexport AZURE_STORAGE_ACCOUNT_KEY=…db = namidb.Client("az://myaccount/mycontainer?ns=prod")# Azurite emulator:db = namidb.Client("az://devstoreaccount1/mycontainer?ns=dev&use_emulator=true")LocalStack
docker run --rm -p 127.0.0.1:4566:4566 -e SERVICES=s3 localstack/localstackaws --endpoint-url=http://localhost:4566 s3 mb s3://namidb-devexport AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=testdb = namidb.Client( "s3://namidb-dev?ns=local&endpoint=http://localhost:4566&allow_http=true®ion=us-east-1")Bucket settings
- Abort incomplete multipart uploads. Add a lifecycle rule that aborts incomplete multipart uploads after a short window (a day is typical). NamiDB aborts the uploads it can, but a process killed mid-upload leaves parts the bucket keeps billing for.
- Do not expire or transition NamiDB objects with lifecycle rules. The engine deletes what it no longer needs (see Maintenance); an external rule that removes or archives live objects corrupts the namespace.
- Versioning is optional. NamiDB never overwrites its data files; bucket versioning only adds a safety net (and cost) for deletes.
- Use a separate bucket or prefix for backups, with its own credentials if possible. See Backup and restore.
- Keep the server close to the bucket (same region). Each commit is a few round trips to the store, so commit latency follows the store’s latency; group commit amortises it under concurrent writes.
Read cache
On an object store, reads fetch byte ranges of immutable files and keep them in a bounded RAM cache, optionally backed by a persistent local NVMe cache that survives restarts. Both are only caches; see Memory and caches.
Timeouts and retries
The object-store client retries transient errors. Tune it with
NAMIDB_STORE_REQUEST_TIMEOUT, NAMIDB_STORE_RETRY_TIMEOUT and
NAMIDB_STORE_MAX_RETRIES; see
Storage URIs.