Skip to content

What a store refuses to hold

Two different things refuse a write, and only one of them is yours to set.

The limit is on the shape of the value. A struct holding a struct holding a Vec of structs is three levels before anything else is counted, and a type that refers to itself - a tree, a menu, an expression - has as many levels as the data does on the day it runs.

Every engine’s codec reads less deeply than it writes. serde_json stops at 128 on the way in and has no limit on the way out; ron stops at 64; rmp_serde has no limit at all, and the stack runs out around three thousand instead (measured on Windows 11 at the default stack size; where it actually gives out depends on the platform and on the stack the thread was started with) - killing the process rather than returning an error, on every later start, because the value is already committed.

So without a check a write past the reader’s ceiling is accepted and cannot be read back: no error anywhere, and on the text engines the whole file is gone, since the document is parsed as one thing. The ceiling is therefore enforced always, whatever else is configured, and the refusal happens at the write.

It is counted during the write, not before it. The value cannot be inspected first - by the time it reaches the store it is a &dyn Serialize, and a five-level struct is indistinguishable from a five-level tree. Nor can it be built and measured, because building it is the dangerous act: on redb that is exactly what overflows the stack. Serde pushes and the store receives, so the levels are counted as they go past, inside the codec’s own pass.

let engine = default_backend();
println!("{}: {} levels", engine.extension(), engine.depth_ceiling());

The numbers are measured rather than looked up - tests/probe_*.rs walked each engine to its boundary:

enginelevels
redb512
JSON127
SQLite127
TOML80
RON64

redb has no limit of its own. rmp_serde recurses until the stack ends, around three thousand levels on Windows 11 at the default stack size, and the process dies rather than returning an error - on every later start, because the value is already committed. That number is a measurement, not a constant: it moves with the platform and with the stack the thread was given. The 512 is imposed for that reason: far above any data anyone means to store, far below where the stack gives out on any of them.

On every text engine a path’s levels become document levels - ui.panels.left is three levels of nesting in the file before the value starts - so the two are counted together. A shallow value at a deep path is as unreadable as a deep value at a shallow one.

SQLite is the exception, since its path is a TEXT key rather than nesting, but paying the path there costs a few levels out of 127 and is not worth a second rule.

When a write is refused the report says where the budget went: how many levels the path took, how many were left for the value, and how many this store reads in all.

Both go through limits, and neither can raise the ceiling above - they only narrow it.

let store = StoreBuilder::new(settings)
.limits(|l| l.key_depth(4))
.build()?;
let deep = StorePath::from_segments(["a", "b", "c", "d", "e"]);
if let Err(refused) = store.set(&deep, &1u32) {
println!("{refused:?}");
}

key_depth refuses a path with more levels than that, before the value is even encoded. Useful for a store whose paths are built at run time, where a bug that nests without bound otherwise shows up as an unreadable file much later.

let store = StoreBuilder::new(settings)
.limits(|l| l.portable_across([default_backend()]))
.build()?;

portable_across names engines the contents must stay readable on beyond the one actually running, and lowers the ceiling to the lowest of them. A store on redb that names RON reads 64 levels rather than 512, so a value too deep for the file it will be exported to is refused now instead of after the export.

Depth is not where it ends. The engines named settle six more properties, each by one rule: the running engine and every engine named must all hold it, or the write is refused.

propertywho loses it
NaN and the infinitiesJSON and SQLite, which have no spelling for them and read back null
enum variantsRON, whose value tree has none - ron#122. A type that writes itself as a string for a reader, an IpAddr among them, is asked in that form
Some(None)every engine but RON: a nested Option collapses to None
a u128 or an i128, whatever it holdsTOML and RON, which have no 128-bit integer type
integers past i64TOML, which has no room for them
integers neither an i64 nor a u64 holdsJSON, which reads a number back as an i64, a u64 or an f64

The running engine counts alongside the named ones for the same reason its ceiling does: a value its own codec cannot read back is lost whatever anyone configured. So a store on JSON refuses a NaN with nothing named at all.

Naming the engines rather than saying all is deliberate. “All” is a moving target - it changes under a store when an engine is added - and the honest requirement is usually narrower: an application shipping JSON on the desktop and SQLite on a phone needs those two and has no opinion about RON.

Depth is what the store enforces. What a format can express is a different question, and no setting changes it: Choosing an engine.