itertools.chain vs Flatten Lab

Intro — what this post promises

Flattening a list-of-lists: is itertools.chain.from_iterable the right default, or is extend / a comprehension faster? This lab compares flatten strategies on Linux localhost — and times the classic foot-guns sum(lists, []) and out = out + row.

Related links:

• 01shoppercove.comitertools vs python loops localhost lab
• 02shoppercove.comnlargest vs sorted slice localhost lab
• 03shoppercove.comstring concat vs join localhost lab
• 04shoppercove.combytes vs bytearray localhost lab
• 05shoppercove.comCounter vs dict tally localhost lab
• 06shoppercove.comfrozenset vs set membership localhost lab
• 07shoppercove.comattrgetter vs getattr localhost lab
• 08shoppercove.comarray vs list ints localhost lab

Lab honesty (1 Oct 2026 IST): Python 3.13.5. Ops = elements written (or consumed). Affiliates: 0. Complements the general itertools lab with a flatten-only focus.

Verdict up front (balanced 1 000×100): extend ~412M/s, chain.from_iterable ~224M, nested append ~91M; sum(lists, []) ~0.96M (chain ~234× faster). Prefer extend when building a list; chain when you want an iterator.

Arms

Shapes: many_tiny, balanced, few_large, wide.

Lab topology

shapes: 10k×5, 1k×100, 50×5k, 5k×50
metric: p50 ops/s (elements)

Script: lab-evidence/67-itertools-chain-vs-flatten/results/run_lab.py.

Lead table — balanced 1 000×100 (p50)

Anti-pattern scale (chain ÷ plus-loop)

Fewer, larger rows soften the plus/sum tax (fewer recopies) — still far behind extend/chain.

Consume without materializing

For pure reduce, a tight nested loop can edge chain (~1.23× here). Chain still wins on clarity when flattening into another consumer.

Reading it

• list.extend is the throughput king for building one flat list (~4.5× nested append).
• chain.from_iterable is the iterator default — fast enough (~2.5× nested) and lazy until list().
• sum(lists, []) and out + row are quadratic-ish — hundreds of × slower on many rows.
• Avoid chain(*rows) on huge outer length — argument explosion; use from_iterable.

Why plus and sum hurt

Each out = out + row allocates a new list and copies everything seen so far. With thousands of rows that is classic quadratic behavior — the same lesson as string + in a loop. sum(lists, []) is the same idea wearing a functional costume. extend amortizes growth; chain.from_iterable avoids building until you ask for a list.

If you only need to iterate once, skip materialization entirely: for x in chain.from_iterable(rows): ... (or a nested for). The consume arms show nested loops can still win a pure sum — pick clarity for pipelines, micro-optimize only on a profile.

Pitfalls

1. Cute sum(list_of_lists, []) — slow and confusing.
2. out = out + row in a loop — same disease as string +.
3. Materializing with list(chain(...)) when extend suffices — extra pass.
4. Assuming chain always beats nested — consume-only paths may differ.

When to pick what

Reproduce

python3 lab-evidence/67-itertools-chain-vs-flatten/results/run_lab.py

Evidence: /workspace/lab-evidence/67-itertools-chain-vs-flatten/results/.

Closing

Extend to build; chain to iterate; never sum-concat. On this box balanced extend hit ~412M/s, chain ~224M, and sum(lists, []) trailed chain by ~234×. Flatten with the tool that matches whether you need a list or a stream.