ChainMap vs Merged Dict Lookup: Localhost Lab

Intro — what this post promises

Layered config lookup: collections.ChainMap vs a single merged dict vs manual sequential in across layers. This lab reports ops/s on Linux localhost.

Related links:

​tomllib vs json localhost lab​

​path glob vs fnmatch localhost lab​

​dataclass replace vs manual localhost lab​

​zoneinfo vs utc offset localhost lab​

​heapq merge vs sorted localhost lab​

​islice vs list slice localhost lab​

• stat vs path stat localhost labmmap write vs write localhost lab

Lab honesty (1 Oct 2026 IST): Python 3.13.5. Affiliates: 0. 500 keys; layers cli ⊃ env ⊃ defaults.

Verdict up front: merged dict.get ~21.06 Mops/s; sequential layer in ~18.86; ChainMap.__getitem__ ~3.65; ChainMap.get ~1.71. ChainMap wins when overlays change often; merge wins raw lookup speed.

---

Arms

ArmPatternChainMap(cli, env, defaults).getlayered getChainMap[...]layered getitemmerged dict .getflatten oncesequential in cli→env→defaultshand walkrebuild merge + one sweepcold overlaynew ChainMap + one sweepcheap view

---

Lab topology

500 keys · cli/env/defaults overlays · 200 full sweeps · 7 rounds · p50
metric: ops/s = (sweeps × keys) / p50_s

Script: lab-evidence/118-chainmap-vs-dict-merge/results/run_lab.py.

---

Lead table — warm lookups (p50)

ArmMops/smerged dict.get21.06sequential layer in18.86ChainMap getitem3.65ChainMap.get1.71

---

Overlay rebuild (one sweep)

ArmMops/srebuild merge + sweep18.69new ChainMap + sweep3.96

Creating a ChainMap view is cheap versus copying all keys into a new dict when layers mutate frequently — even though steady-state lookups favor the merged map.

---

Config stack pattern

A common SRE shape is ChainMap(cli_args, os_environ_subset, file_defaults). Keep file defaults immutable and push ephemeral overrides on the left. That avoids defaults.copy(); defaults.update(env) on every boot when only argv changes.

---

vs tomllib/json loads

Labs 112/104 measure parse cost. This lab measures post-parse layered lookup. Parse into dicts, then decide ChainMap vs merge for serving reads.

---

Reading it

• Stable flags / hot path → merge once, look up on one dict.
• Live overlays (feature flags, request-scoped defaults) → ChainMap without rewriting the base.
• ChainMap.get trailed __getitem__ here — prefer cm[k] when keys are known present.
• Semantics matched merge on every key in the fixture.

---

Write semantics

cm[k] = v writes into the first mapping (leftmost). That is usually what you want for overrides, and dangerous if the leftmost map is a shared process-global. Prefer a dedicated per-request dict on the left so defaults stay pristine.

---

Memory shape

ChainMap stores references, not copies. A merged dict duplicates keys/values into one table — faster gets, higher RAM, and a snapshot that drifts if someone mutates a source layer afterward. Pick snapshot-vs-live deliberately.

---

When sequential in appears

Hand-rolled if k in cli elif k in env matched merge speed closely here (~18.9 Mops/s). It still duplicates precedence logic at every call site. ChainMap encodes precedence once; merge encodes it at build time — both beat scattering if ladders across services.

---

Pitfalls

• Mutating a mapping that sits inside a ChainMap (shared views).
• Expecting ChainMap writes to deep-merge nested dicts (they don’t).
• Rebuilding a merged dict on every request when only one layer changed.
• Using ChainMap for speed — it is for structure, not peak Mops/s.

---

Reproduce

python3 lab-evidence/118-chainmap-vs-dict-merge/results/run_lab.py

Evidence: summary.json, summary.txt.

---

Limits

One Linux box. Flat string values. Not nested deep-merge. Not thread-safe mutation patterns.

---

Takeaway

Warm lookups: merged dict ~21.06 Mops/s beat ChainMap getitem ~3.65 Mops/s. Use ChainMap for layered overlays; flatten when lookup rate dominates.