Background Writer and Checkpoint Bottleneck

warningperformance

Optimizing PostgreSQL background writer and checkpoint processes when buffer allocation outpaces background cleaning, causing backends to perform their own writes.

Prompt: “I'm seeing high bgwriter_buffers_backend and low bgwriter_buffers_clean in PostgreSQL metrics. My write performance has degraded and I think backends are writing their own dirty buffers instead of the background writer handling it. How do I tune this?”

Agent Playbook

When an agent encounters this scenario, Schema provides these diagnostic steps automatically.

When backends are writing their own dirty buffers instead of the background writer handling cleanup, start by confirming the symptom in pg_stat_bgwriter ratios, then check if the bgwriter is hitting its per-round limits. Next examine checkpoint frequency and I/O spread patterns, since excessive checkpoints can overwhelm the buffer system. Finally look for emergency fsync calls and potential autovacuum I/O contention.

1Confirm backends are doing the heavy lifting

Query pg_stat_bgwriter and compare `buffers_backend` to `buffers_clean` plus `buffer-checkpoint`. If `buffer-backend` is significantly higher (say, 2-3x or more), you've confirmed that backends are writing their own dirty buffers instead of the background writer cleaning them. This is the smoking gun described in the `background-writer-overload` insight and means your bgwriter configuration is undersized for your workload.

Background writer cannot keep up with I/O demand buffers_backendbuffers_cleanbuffers_checkpoint

2Check if bgwriter is hitting its per-round limit

Look at `postgresql-bgwriter-maxwritten` to see how often the background writer stops early because it hit bgwriter_lru_maxpages in a single round. High values here mean the bgwriter wants to clean more buffers but is artificially constrained. The fix is to increase bgwriter_lru_maxpages (try doubling from default 100 to 200-500) and bgwriter_lru_multiplier (from 2.0 to 3.0-5.0) so it can clean more aggressively.

Background writer cannot keep up with I/O demand postgresql.bgwriter.maxwritten

3Examine checkpoint frequency and type

Check the ratio of scheduled checkpoints (`checkpoint-timed`) to requested checkpoints (checkpoint_req in pg_stat_bgwriter). If requested checkpoints dominate, it means you're hitting max_wal_size too frequently and forcing early checkpoints. The `excessive-checkpoint-frequency` insight recommends increasing max_wal_size (e.g., from 1GB to 4GB for OLTP workloads) to reduce checkpoint pressure and give the bgwriter more breathing room between checkpoint flushes.

Checkpoint frequency exceeds sustainable rate Checkpoint and WAL configuration mismatches cause I/O spikes checkpoint_timed

4Assess checkpoint I/O spread and timing

Look at `stat-checkpointer-write-time` and check your checkpoint_completion_target setting (should be 0.9 to spread writes over 90% of the checkpoint interval). If checkpoints are completing too quickly, you get I/O spikes that overwhelm both the bgwriter and backends. The `checkpoint-io-spikes` insight warns that low checkpoint_completion_target values cause exactly this problem—checkpoint traffic floods the buffer pool faster than bgwriter can keep up.

Checkpoint spikes cause I/O contention and query latency Checkpoint and WAL configuration mismatches cause I/O spikes checkpointer_write_timebuffers_checkpoint

5Look for emergency backend fsync calls

Check `buffer-backend-fsync` to see if backends are not just writing buffers but also forced to perform their own fsync operations. Non-zero values here indicate severe I/O bottlenecks where backends couldn't find clean buffers and had to synchronously flush dirty pages to disk themselves. This is the worst case described in `background-writer-overload` and means your tuning adjustments from steps 2-4 are critical.

Background writer cannot keep up with I/O demand buffers_backend_fsyncfsyncs

6Consider autovacuum I/O contention

If you've tuned bgwriter and checkpoint settings but still see high `buffer-backend`, check if autovacuum processes are saturating I/O during your workload's peak write times. The `autovacuum-io-saturation-high-write-volume` insight describes how high-write workloads can cause I/O spikes to 100% when autovacuum runs. Monitor I/O utilization; if autovacuum is the culprit, consider isolating high-write databases or adjusting autovacuum_vacuum_cost_delay to throttle autovacuum I/O.

High-volume write workload causes I/O saturation when autovacuum runs buffers_backend