setdefault vs defaultdict: Insert Lab

Intro — what this post promises

Insert-or-default on a dict: is setdefault, an if k not in d, or collections.defaultdict fastest? This lab times list-append groupby and scalar counter patterns on Linux localhost under miss-heavy / mixed / hit-heavy key mixes.

Related links:

• 01shoppercove.comCounter vs dict tally localhost lab
• 02shoppercove.comfunctools partial vs lambda localhost lab
• 03shoppercove.comzip vs index pairing localhost lab
• 04shoppercove.comfrozenset vs set membership localhost lab
• 05shoppercove.comenum vs constants localhost lab
• 06shoppercove.comcontextlib vs try/finally localhost lab
• 07shoppercove.comstatistics vs manual mean localhost lab
• 08shoppercove.combisect vs linear lookup localhost lab

Lab honesty (1 Oct 2026 IST): Python 3.13.5. N=200,000 key ops/arm. List arms seed 5 000 existing keys. Affiliates: 0. Complements Counter tally with the “ensure bucket then mutate” pattern.

Verdict up front (mixed list-append): defaultdict ~5.11M/s ≈ setdefault ~5.08M (~1.01×); hit-heavy defaultdict ~1.08× setdefault. Scalar hit: d.get/defaultdict(int) beat setdefault++= by ~1.36× / ~1.32×. Pick defaultdict for ongoing groupby; use d[k]=d.get(k,0)+1 for counters.

Arms

Hit rates: miss 10%, mixed 50%, hit 90%.

Lab topology

N = 200000 ops; list seed = 5000 keys
scalar miss = all-new keys; scalar hit = 1000 hot keys

Script: lab-evidence/72-setdefault-vs-defaultdict/results/run_lab.py.

Lead table — list-append groupby (p50)

Scalar counters (p50)

Note: setdefault + += does an extra lookup after insert — d[k]=d.get(k,0)+1 avoids that.

Reading it

• List groupby — setdefault ≈ defaultdict on mixed; defaultdict pulls ahead when hits dominate (~1.08×).
• Miss-heavy list — setdefault slightly ahead of defaultdict here (dd÷sd ~0.89×); gaps are small.
• Counters — prefer get/defaultdict(int), not setdefault+mutate for ints.
• try/except — loses on miss-heavy (exception path); fine when hits are near-certain.

setdefault vs defaultdict in APIs

defaultdict changes missing-key behavior for the lifetime of the object — fine inside a function, surprising if you return it to callers who expect KeyError. setdefault keeps a plain dict and only inserts when you ask. For library boundaries, prefer setdefault (or build with defaultdict then dict(d)). For tight internal grouping loops, defaultdict’s __getitem__ path is the ergonomic default and often the speed default on hit-heavy mixes.

Pitfalls

1. setdefault for int counters — double lookup with +=.
2. Leaking defaultdict into APIs that expect plain dict — convert or document.
3. try/except as control flow on cold keys — expensive misses.
4. Microbenching without hit-rate context — rankings flip with mix.

When to pick what

Mixed-hit takeaway

Across miss / mixed / hit, no single helper wins every cell by a wide margin on list buckets — the story is “same ballpark, pick clarity,” except try/except on cold keys and setdefault++= for ints. Profile the hit rate you actually see in production logs before rewriting idioms for a 5% microbench delta.

Reproduce

python3 lab-evidence/72-setdefault-vs-defaultdict/results/run_lab.py

Evidence: /workspace/lab-evidence/72-setdefault-vs-defaultdict/results/.

Closing

defaultdict for buckets; get-add for counters. On this box mixed list-append setdefault and defaultdict tied near ~5.1M/s, hit-heavy favored defaultdict by ~1.08×, and scalar get-add led setdefault by ~1.36×. Match the helper to whether the default is a container or a number.