Memory, caches and disk
NamiDB reads immutable files from the store and keeps bounded caches in front of them. The defaults suit ordinary graph workloads; size things explicitly for large vector or full-text corpora and for shared machines.
The memory used by a server process is roughly:
caches (≤ NAMIDB_CACHE_MAX_BYTES)+ memtables (≤ --memtable-stall-bytes per active namespace, softly)+ active queries and result rows (≤ the shared query pool, when a ceiling is set)+ transient flush / compaction / index-build work+ Bolt message buffers (≤ NAMIDB_BOLT_MEMORY_BUDGET_BYTES)+ allocator and runtime overheadThe cache ceiling
NAMIDB_CACHE_MAX_BYTES (default 1073741824, 1 GiB) is one
process-wide ceiling for every shared cache: data-file bodies, decoded
row groups, property side-cars, node views, edge readers, bloom filters,
search index bodies, file metadata and the RAM range cache. It is shared by
all namespaces, so a multi-tenant server does not multiply it per tenant.
0 disables the shared caches. Values are exact bytes; a malformed value
stops the server at startup instead of silently falling back.
The ceiling is split between tiers automatically. The older per-tier
variables (NAMIDB_SST_CACHE_BUDGET_MIB, NAMIDB_NODE_CACHE_BUDGET_MIB,
… listed in the configuration reference)
still work as relative requests; when they add up to more than the ceiling,
they are scaled down proportionally.
The startup log shows the result:
INFO namidb_storage::cache_budget: resolved process-wide cache capacities cache_max_bytes=1073741824 cache_assigned_bytes=1073741824 sst_cache_bytes=960716369 search_index_cache_bytes=361681456 cache_path_registry_bytes=11302545 range_page_cache_bytes=22605091 node_cache_bytes=90420364 adjacency_cache_bytes=0Per-query caches are separate and small: each active snapshot keeps at most
NAMIDB_SNAPSHOT_NODE_CACHE_MAX_BYTES,
NAMIDB_SNAPSHOT_ROW_GROUP_CACHE_MAX_BYTES and
NAMIDB_SNAPSHOT_EDGE_READER_CACHE_MAX_BYTES (each min(1 MiB, cache ceiling / 64) by default).
The optional in-RAM adjacency (CSR) cache is off by default because it
holds every edge of a type in memory. Enable it with NAMIDB_ADJACENCY=1
only when you have sized memory for it; it then takes a share of the
ceiling.
The memory governor (server)
--memory-max-bytes / NAMIDB_MEMORY_MAX_BYTES sets a ceiling on the whole
process’s resident memory (RSS on Linux and macOS, working set on Windows):
| Value | Meaning |
|---|---|
0 (default) | Off |
| A byte count | That ceiling |
auto | 90% of the container’s cgroup memory limit; the server refuses to start if there is no finite limit |
When it is set:
- at 90% of the ceiling the server clears reconstructible caches (a 500 ms watchdog does this even when idle);
- at the ceiling, new Cypher work is refused with a retryable 503
memory_pressure(Bolt:Neo.TransientError.General.DatabaseUnavailable), and/v0/healthreportsmemory_pressure: true, until memory falls back; POST /v0/admin/flushand small Bolt control messages stay available so operators and clients can relieve pressure.
Admission is checked when a statement starts. A statement that is already
running is bounded by the per-query budget
below, which the same ceiling turns on. Neither replaces an OS limit: flush,
compaction and other background work can still grow memory for a while. Pair
the ceiling with a container or cgroup limit, for example
NAMIDB_MEMORY_MAX_BYTES=auto with docker run --memory=4g, or
NAMIDB_MEMORY_MAX_BYTES=3758096384 (3.5 GiB) under a 4 GiB limit. It is
server-only; the embedded clients do not install it.
The per-query memory budget (server)
Since 3.2.1, each running statement (HTTP and Bolt, reads and writes) has a
working-memory budget, set by --query-memory-max-bytes /
NAMIDB_QUERY_MEMORY_MAX_BYTES. A query that outgrows it fails with a typed
error and the server keeps serving everything else, instead of growing until
the kernel kills the process and every request in flight with it.
| Value | Meaning |
|---|---|
auto (default) | 50% of --memory-max-bytes; off when that ceiling is 0 |
| A byte count | That budget for each query, with or without a ceiling |
0 | Off |
A server that never set NAMIDB_MEMORY_MAX_BYTES therefore behaves as before,
and one that did gets the budget when it upgrades to 3.2.1.
What is charged. The executor charges the rows each operator
materialises, plus the state that can grow while an operator runs: UNWIND
lists, expansions and variable-length traversals, projections, joins and cross
products, grouping and collect(), DISTINCT, sort keys and range() lists.
It does this on the default and on the factorised (NAMIDB_FACTORIZE=1)
executor. The estimates are conservative: they count the heap the rows really
occupy, allocator overhead included, and err on the high side. On top of that:
- The HTTP response’s copies of the result (the JSON tree and its text) are charged too. For a write they are charged before the commit, so a write whose response would not fit is rolled back and commits nothing.
- A Bolt result stays buffered in the session until the client pulls it, and the statement keeps its budget until the last row is pulled or the result is discarded, reset or closed.
--query-row-capis unchanged and still applies on its own: whichever limit a query reaches first stops it.
The shared pool. With a memory ceiling, all running queries also reserve from one pool of 75% of the ceiling, so many queries that each fit their own budget cannot add up past it. A query running alone always gets its full budget, and a per-query value larger than the pool is clamped to the pool. A growing query also re-samples the process’s resident memory every 16 MiB and stops if the whole process has reached its ceiling, whatever filled it. With an exact budget and no ceiling there is no pool and no re-sampling, only the per-query budget.
The startup log shows the resolved values. In a 384 MiB container with
NAMIDB_MEMORY_MAX_BYTES=auto (a ceiling of 362,387,865 bytes):
INFO namidb_server: per-query working-memory budget enabled query_memory_max_bytes=181193932 query_memory_pool_bytes=271790898What clients see
| Stopped because | HTTP | code | Bolt | gql_status | Retry |
|---|---|---|---|---|---|
| The query needs more than its own budget | 413 | query_memory_limit | Neo.ClientError.Statement.ResourceLimitExceeded | 54000 | No: it fails again unchanged |
| The shared pool is full, or the process reached its ceiling | 503 | memory_pressure | Neo.TransientError.General.DatabaseUnavailable | 53000 | Yes, with backoff |
A 600,000-row UNWIND in that 384 MiB container (the same query with
100,000 rows returns normally):
{"code":"query_memory_limit","error":"read execution failed: query exceeded its working-memory budget: it needs at least 181600304 bytes, the per-query limit (NAMIDB_QUERY_MEMORY_MAX_BYTES) is 181193932 bytes; return fewer or narrower rows (LIMIT, pagination, aggregation)","gql_status":"54000","neo4j_code":"Neo.ClientError.Statement.ResourceLimitExceeded"}Several large queries at once that together fill the pool:
{"code":"memory_pressure","error":"read execution failed: query working memory unavailable: it needs 259293232 bytes but concurrent queries hold the shared query-memory pool of 805306368 bytes; retry after other queries finish","gql_status":"53000","neo4j_code":"Neo.TransientError.General.DatabaseUnavailable"}The server keeps answering after either error, and a Bolt connection stays
usable after the failure. Official Neo4j drivers retry the 503’s
TransientError on their own and do not retry the 413. A statement refused at
admission, because the ceiling was already reached when it arrived, is also a
503 memory_pressure; since 3.2.1 its body carries code, neo4j_code and
gql_status like every other executor error.
Sizing it
- Keep
autounless you have a reason not to. It follows the ceiling: withNAMIDB_MEMORY_MAX_BYTES=autounderdocker run --memory=4g, the ceiling is about 3.6 GiB, one query may use about 1.8 GiB and all running queries together about 2.7 GiB. - Lower it (an exact byte count) when many concurrent queries matter more than one very large one: a smaller budget stops a runaway query sooner and leaves more of the pool to the others.
- Raise it only up to the pool (75% of the ceiling); a larger value is clamped. Giving queries more memory means raising the ceiling and the container limit together.
- Fix the query before raising the budget. A query that hits the 413 does
so on every run. Page it with
ORDER BY … SKIP … LIMITor a key range, return fewer or narrower columns, and aggregate in the query (count(*),sum()) instead of returning orcollect()ing raw rows. - Turn it off with
NAMIDB_QUERY_MEMORY_MAX_BYTES=0. Admission at the ceiling stays; a running query can then grow until the OS limit stops it.
Watch namidb_query_memory_stops_total{reason} and
namidb_query_memory_pool_bytes{kind="peak"} against {kind="capacity"}
(Monitoring).
Frequent pool stops mean the concurrent large queries do not fit together;
process stops mean something else (caches, memtables, background work) is
taking the memory.
Known limits
- Estimates are per operator, not exact. A factorised expansion is metered by the rows it pushes, not by its per-path bookkeeping; the check when the operator returns catches the rest.
- A Bolt client that leaves a large result unpulled keeps its share of the
pool until it pulls, discards, resets or disconnects. That is the memory the
buffered rows really hold, but an idle client can make other queries get
the retryable
memory_pressureuntil then. Consume or close results promptly. - The budget belongs to
namidb-server. The embedded Python and Rust clients and the CLI install none; Rust callers can scope anamidb_query::QueryMemoryBudgetwithwith_query_memory.
Range cache
Index, property and adjacency reads fetch small aligned byte ranges of immutable files and cache them as pages:
| Variable | Default | Meaning |
|---|---|---|
NAMIDB_RAM_PAGE_CACHE_MAX_BYTES | 64 MiB requested, scaled with the other tiers | RAM page cache, carved out of NAMIDB_CACHE_MAX_BYTES (never added to it); 0 disables it |
NAMIDB_NVME_CACHE_PATH | unset | Directory for a persistent local cache tier. Needs the next two variables. |
NAMIDB_NVME_CACHE_MAX_BYTES | unset | Exact capacity of the NVMe tier (never a percentage of the disk) |
NAMIDB_RANGE_CACHE_NAMESPACE | required with NVMe | A stable, non-secret name for this deployment and bucket, mixed into every cache key so a reused cache directory never serves another bucket’s data |
NAMIDB_RANGE_CACHE_PAGE_BYTES | 256 KiB | Read and cache page size |
NAMIDB_RANGE_CACHE_MAX_ENTRY_BYTES | 4 MiB | Largest range admitted |
NAMIDB_NVME_CACHE_BLOCK_BYTES | 16 MiB | NVMe eviction unit (the device must hold at least eight) |
NAMIDB_NVME_CACHE_WRITE_BUFFER_BYTES | ¼ of the RAM tier, at most 16 MiB | Each of the two NVMe write buffers |
The NVMe directory is exclusive to one process and is only a cache; delete it any time and reads become cold. Point it at fast local disk.
Search memory
| Variable | Default | Meaning |
|---|---|---|
NAMIDB_SEARCH_WORKSPACE_MAX_BYTES | 256 MiB | Shared pool for transient search memory (decoded pages, candidate heaps). Queries wait for room; one that can never fit fails with 503 search_workspace_capacity. |
NAMIDB_SEARCH_INDEX_CACHE_MAX_BYTES | proportional share | Exact reservation inside the cache ceiling for decoded vector / full-text index bodies of older (monolithic) formats. |
NAMIDB_SEARCH_MAX_RESULT_BYTES | 64 MiB | Ceiling for unbounded (no k) full-text results; crossing it is a 413 search_result_limit. |
NAMIDB_BM25_MAX_DOCUMENT_BYTES | 1 MiB | Largest document scored by the exact BM25 path (413 search_document_limit beyond it). |
Current vector and full-text formats do not load the corpus into RAM: they
keep bounded routing metadata and read pages through the range cache. A
ten-million-vector namespace therefore does not need a ten-million-vector
cache; smaller settings trade warm latency for more range reads, not
correctness. If a valid index of an older format cannot fit its pool, the
query fails with a retryable 503 search_index_cache_capacity instead of
silently scanning the corpus.
Bolt buffers
NAMIDB_BOLT_MEMORY_BUDGET_BYTES bounds memory used by authenticated Bolt
messages across all connections (default: half the memory ceiling, or about
1 GiB without one). A message that cannot be admitted fails retryably; use
smaller batches. See Bolt limits.
Disk spool
Flush and compaction stream large intermediate data (Parquet outputs, exact-record values, remote compaction inputs, index-build runs) through temporary files instead of RAM:
| Variable | Default | Meaning |
|---|---|---|
NAMIDB_SPOOL_DIR | /var/tmp (Unix bare binary), /var/tmp/namidb-spool (official image), <root>/.namidb-spool (Python with file://) | Spool for flush and compaction |
NAMIDB_INDEX_BUILD_SPOOL_DIR | the compaction spool, then NAMIDB_SPOOL_DIR | Spool for vector / full-text index builds |
NAMIDB_INDEX_BUILD_MEMORY_BYTES | 256 MiB for full rebuilds, 64 MiB for per-flush deltas | In-memory budget of the index builders (one explicit value overrides both) |
NAMIDB_SIDECAR_SORT_MEMORY_BYTES | 8 MiB | Sort buffer per index side-car builder |
Size the spool for a full compaction: about three times the compacted live
node data (roughly 12 to 15 GiB per million 1024-dimension vectors), plus
headroom. Use real disk, never a RAM-backed /tmp or tmpfs. Spool files
are removed automatically on success, failure and exit.
Recipes
A 4 GiB container
docker run --memory=4g \ -e NAMIDB_MEMORY_MAX_BYTES=auto \ -e NAMIDB_CACHE_MAX_BYTES=1073741824 \ -v namidb-spool:/var/tmp/namidb-spool \ …A large vector corpus on an object store
NAMIDB_CACHE_MAX_BYTES=4294967296 # 4 GiB of shared cachesNAMIDB_RAM_PAGE_CACHE_MAX_BYTES=1073741824 # 1 GiB of it for range pagesNAMIDB_NVME_CACHE_PATH=/mnt/nvme/namidb-cacheNAMIDB_NVME_CACHE_MAX_BYTES=107374182400 # 100 GiB persistent cacheNAMIDB_RANGE_CACHE_NAMESPACE=prod-graphs-us-east-1NAMIDB_SEARCH_WORKSPACE_MAX_BYTES=1073741824NAMIDB_SPOOL_DIR=/mnt/nvme/namidb-spoolA small, memory-tight sidecar
NAMIDB_CACHE_MAX_BYTES=134217728 # 128 MiBNAMIDB_SEARCH_WORKSPACE_MAX_BYTES=67108864NAMIDB_MEMORY_MAX_BYTES=auto # with a container memory limit