Atomic Rename vs Overwrite: Is os.replace Worth the Cost?

Intro — what this post promises

Writing a temp file in the same directory and calling os.replace costs little over overwriting in place: for 4 KiB files it ran at about 13,580 ops/s against 16,566 (about 18% slower), and for 64 KiB files replace was actually about 1.44x faster. Durability is the expensive part, because adding fsync dropped replace to about 3,376 ops/s and fsync of both file and directory to about 2,240, so use os.replace by default for config files and add fsync only when you need data to survive power loss.

This is a hands-on lab with measured numbers:

1. Truncate-overwrite vs temp + os.replace for 64 B / 4 KiB / 64 KiB payloads.
2. Same arms with file fsync after write.
3. Durable-ish pattern: fsync file + replace + fsync directory.
4. Explicit note: no crash/power-fail simulation.

Related links:

• 01shoppercove.comfsync vs fdatasync localhost lab
• 02shoppercove.comSQLite WAL vs DELETE journal localhost lab
• 03shoppercove.composix fadvise sequential vs random localhost lab
• 04shoppercove.comjson vs orjson vs msgpack localhost lab
• 05shoppercove.comWhy your average latency graph is lying (p50 / p95 / p99)

Lab honesty (1 Oct 2026 IST): Shared Linux lab box; Python 3.13.5; files under /tmp on overlay. Affiliates: 0. Throughput ≠ proven crash recovery.

Verdict up front: for 4 KiB, replace costs 18%~~ vs overwrite (13580 vs 16566 ops/s). fsync is the cliff (~~3–5× slower). File+dir fsync rename landed ~2240 ops/s.

Patterns under test

Related links:

• 01shoppercove.comfsync vs fdatasync localhost lab

Lab topology

/tmp workdir on overlay
Arms × payloads: 64 B (n=2000), 4 KiB (n=1000), 64 KiB (n=200)
5 timed batches after warmup; ops/s from p50 batch wall
Also: replace + fsync(file) + fsync(dir_fd) on 4 KiB (n=500)

Script: lab-evidence/42-atomic-rename/results/run_lab.py.

Lead table — 4 KiB (p50)

Atomicity without fsync is cheap here (~18%). Durability is expensive — same story family as the fsync lab.

Related links:

• 01shoppercove.comWhy your average latency graph is lying (p50 / p95 / p99)

Other sizes

64 B: overwrite 16377 ops/s; replace 13698 (~0.84×); overwrite+fsync 4929; replace+fsync 3377.

64 KiB: overwrite 5234; replace 7558 (~1.44× faster replace!). Writing a new temp + rename beat truncating and rewriting the large file on this run — worth re-checking on your FS, but it shows replace is not always the slower path for big blobs.

Crash-safety — what we are not claiming

We did not pull power, kill -9 mid-write, or run dm-flakey. The durable rename pattern is the industry cookbook (fsync data file, rename/replace, fsync directory). This lab times that cookbook; it does not prove your disk honors flush.

Related links:

• 01shoppercove.comSQLite WAL vs DELETE journal localhost lab
• 02shoppercove.comfork COW RSS vs spawn localhost lab

How to read these numbers

• Prefer replace for config/state files readers must never see half-written.
• Budget ~20% throughput on small writes here without fsync — or gain on large rewrites.
• If you need durability across power loss, pay fsync (and likely dir fsync).
• Same-filesystem os.replace is required for atomicity; cross-device becomes copy+unlink.

Pitfalls we hit (or avoided)

1. Calling replace “free” — ~18% on 4 KiB.
2. Calling replace “always slower” — 64 KiB flipped.
3. fsync file only and declaring durable rename done — we timed dir fsync too.
4. Simulating crash with a blog claim — labeled out of scope.
5. Ignoring reader concurrency — atomicity helps; still need locking if two writers.

Practical checklist

• Config updates: temp in same directory + os.replace.
• Need power-fail durability: fsync file (+ dir) and measure.
• Don’t fsync every keystroke unless RPO demands it.
• Keep overwrite for scratch/temp artifacts you can rebuild.
• Pair with the fsync lab numbers when capacity planning.

Related links:

• 01shoppercove.composix fadvise sequential vs random localhost lab
• 02shoppercove.comsha256 vs blake2 xxhash localhost lab
• 03shoppercove.comzstd vs gzip vs lz4 compression localhost lab
• 04shoppercove.comjson vs orjson vs msgpack localhost lab

Methodology footnote

Each op fully writes the payload. Replace arms use unique temp names then os.replace. Directory fsync uses os.open(dir, O_RDONLY) + os.fsync(dir_fd). Ops/s = n / p50_batch_seconds.

Versions / environment

• Python 3.13.5
• Workdir under /tmp on overlay/virtio lab VM

Writable config files are where torn writes hurt humans. Pay the replace tax; add fsync only when the durability contract requires it.

Why same-directory temps matter

os.replace is atomic only on the same filesystem. Writing to /tmp/foo and replacing onto /var/lib/app/config.json can silently become “copy + unlink” semantics across mounts — or fail. This lab kept temps beside the target under /tmp. Production configs usually live on the data volume; create config.json.tmp next to config.json, not on an unrelated tmpfs, unless you intentionally accept non-atomic publish.

Readers during replace

After a successful replace, a reader opening the path sees either the previous complete file or the new complete file — not a mix of both contents from a single truncated write. Readers that already hold an open FD keep the old inode (classic Linux behavior). That is often what you want for long-lived workers; document whether they must reopen to pick up config.

Batch exporters that rewrite large snapshots may find replace faster than overwrite (our 64 KiB case). Profile your size; do not assume the 4 KiB ranking.

Ratio sheet (med 4 KiB ops/s ÷ overwrite)

If you currently overwrite without fsync, switching to replace is the cheap correctness upgrade. If you already fsync every overwrite, moving to replace+fsync (or +dir) is mostly about atomic visibility, not a new performance class — you already paid the barrier.

Compare absolute µs/op to the fsync lab’s per-record sync costs: both say “barriers dominate metadata tricks.”

Ship replace by default for user-visible state files; measure before adding fsync-on-every-write in a tight loop.

Verdict

On this box, 4 KiB atomic replace ran at ~13580 ops/s versus overwrite ~16566 (~0.82×). Adding fsync dropped replace to ~3376 ops/s; fsync file+dir to ~2240. Atomicity is affordable; durability dominates — and we still did not simulate a crash.

Evidence: lab-evidence/42-atomic-rename/results/. Affiliates: 0.

FAQ

Why use os.replace instead of overwriting a file?

Overwriting truncates first, so a crash mid-write can leave a torn file. With a temp file plus os.replace on the same filesystem, readers see either the old complete file or the new one.

Does the temp file need to be in the same directory?

Yes, or at least on the same filesystem. os.replace is atomic only within one filesystem. Across mounts it can turn into copy + unlink or fail, so create config.json.tmp next to config.json.

Do I need to fsync the directory too?

For durability across power loss, the cookbook is fsync the file, replace, then fsync the directory. In this lab that cost 446 µs per op vs 74 µs for replace alone. The lab timed it but did not simulate a crash.