~/blog/2026-10-25/ $ cat post.md

annotating a deliberate data race in percpu_counter

syzbot flagged a lockless read in percpu_counter_limited_add(). the read was on purpose; the annotation was missing. here is what KCSAN actually checks, and why READ_ONCE() alone does not silence it.

the patch is small: one line, plus a comment. it changes no logic, and it is upstream now. the interesting part is not the line. it is the three wrong answers I had to throw away before it.

this is the story of a KCSAN report that looked like a missing lock, turned out to be a deliberate lockless read, and still needed fixing - because "we did it on purpose" is not something a data-race detector can infer from the source.

what syzbot reported

on 2026 june 19, syzbot filed a report titled:

BUG: KCSAN: data-race in __percpu_counter_limited_add / percpu_counter_add_batch

the object is an s64: sbinfo->used_blocks, a per-CPU counter on a tmpfs superblock. two plain accesses hit the same 8 bytes, with no common lock between them:

write to 0xffff88812a80e9d0 of 8 bytes by task 11484 on cpu 0:
 percpu_counter_add_batch+0x101/0x120      lib/percpu_counter.c:107
 percpu_counter_sub                        include/linux/percpu_counter.h:277
 shmem_inode_unacct_blocks                  mm/shmem.c:263
 shmem_recalc_inode+0x143/0x1f0            mm/shmem.c:466
 shmem_undo_range+0xb20/0xb60              mm/shmem.c:1272
 shmem_evict_inode / evict / iput / filename_unlinkat / __x64_sys_unlink

read to 0xffff88812a80e9d0 of 8 bytes by task 16836 on cpu 1:
 __percpu_counter_limited_add+0xc4/0x3f0
 percpu_counter_limited_add               include/linux/percpu_counter.h:77
 shmem_inode_acct_blocks+0xf4/0x230        mm/shmem.c:232
 shmem_get_folio_gfp+0x5d3/0xd90           mm/shmem.c:2502
 shmem_write_begin+0xfc/0x1f0              mm/shmem.c:3239
 generic_perform_write / shmem_file_write_iter / __kernel_write_iter

the write is on tmpfs's free path: truncate or unlink, through shmem_recalc_inode() and shmem_inode_unacct_blocks(), into percpu_counter_sub(). the read is on the allocation path: writing into a file, through shmem_get_folio() and shmem_inode_acct_blocks(), into percpu_counter_limited_add(). same superblock, same counter, one side under fbc->lock and the other side not.

the two lines in question, in lib/percpu_counter.c:

fbc->count += count + amount;      /* under fbc->lock */
...
if (abs(count + amount) <= batch &&
    ((amount > 0 && fbc->count + unknown <= limit) ||   /* no lock */
     (amount < 0 && fbc->count - unknown >= limit))) {

syzbot had no reproducer, just the splat. that is usually a hint that the path is a little unusual.

what KCSAN actually detects

KCSAN is a dynamic data-race detector built on compiler instrumentation. it is not a static analyzer, and it is not KASAN. in fact KCSAN depends on !KASAN; you pick one sanitizer for a given build. KASAN would never see this report anyway, because nothing here is out of bounds - it is an 8-byte load racing an 8-byte store on the same address.

the mechanism matters for everything that follows. for each plain access it instruments, KCSAN runs a short state machine:

  1. check. is there a watchpoint on this address? if there is, and at least one side is a write, report. this is the common case, and it is where this report came from.
  2. setup. otherwise, with a low probability, arm a watchpoint on the address and remember the current value. most accesses are never sampled at all - that is the "watchpoint-based sampling" part, and it is why a real race can take a while to surface.
  3. delay. if a watchpoint was armed, stall for a short random time, to widen the window in which the other CPU can touch the address.
  4. re-check. read the value again; if it changed under us, report a race of unknown origin.

how often step 2 fires and how long step 3 stalls are tunable at runtime, which is the first thing to reach for when a report is rare:

/sys/module/kcsan/parameters/skip_watch   # accesses to skip before arming another watchpoint
/sys/module/kcsan/parameters/udelay_task  # microseconds to stall after arming one
/sys/kernel/debug/kcsan                   # "on" / "off": the whole detector, if built in but disabled

it also explains the two report shapes. a normal report prints both stacks, because both sides were instrumented. the other shape - "race at unknown origin" - is printed when the watchpoint saw the value change but the conflicting access was not instrumented at all, so there is no second stack to show.

marked atomic accesses - READ_ONCE(), WRITE_ONCE(), atomic_*, and friends - are instrumented too, but only to check for a watchpoint. they never set one.

that asymmetry is the whole trap. annotating one side of a race with READ_ONCE() does not silence KCSAN: the plain access still sets a watchpoint, and the marked access still walks into it and reports. and it is not wrong to report it - the two accesses really are unordered.

why the race initially looks suspicious

two threads touching an s64 without a common lock is the textbook definition of a data race, and it is easy to stop reading there and go hunting for the missing lock. worse, the value is used to enforce a limit:

if (... fbc->count + unknown <= limit)

a tmpfs mount can carry a size= limit, which becomes sbinfo->max_blocks. if a stale read lets a page allocation through, could a tmpfs exceed the size the user asked for? that sounds like a real bug, not a false positive.

so the question is not "which lock did someone forget". it is "is this read allowed to be approximate, and by how much". that requires reading the counter's design, not just the backtrace.

how percpu_counter works

the structure is a compromise between accuracy and cache-line contention:

struct percpu_counter {
    raw_spinlock_t lock;
    s64 count;              /* the global part */
    s32 __percpu *counters; /* one s32 per CPU */
};
  • percpu_counter_add_batch() adds to this CPU's local s32. when the local value would reach batch, it takes fbc->lock, folds the local amount into the global count, and zeroes the local slot.
  • percpu_counter_sum() takes the lock and adds the global count plus every CPU's local slot.
  • plain reads are deliberately cheap and approximate: percpu_counter_read() just loads the global count, with no lock at all.

so the global count lags the true total by whatever is parked in the per-CPU slots. each slot is bounded by batch, so the total in flight is bounded by unknown = batch * num_online_cpus(). that is the slack term.

percpu_counter_limited_add(limit, amount) is the interesting one: it adds amount only if the resulting total is still within limit. it is a three-stage fallback, each stage more accurate and more expensive than the last:

  1. fast path, no lock. if this CPU's slot stays within batch and the global count plus the slack is clearly inside the limit, add to the local slot and return. this is the path the report hits.
  2. locked estimate. otherwise take fbc->lock and re-test the limit against the global count, again with the unknown slack. if the answer is unambiguous either way, stop here - no per-CPU summation needed.
  3. exact sum. only if the slack straddles the limit does it walk the CPUs (for_each_cpu_or(cpu, cpu_online_mask, cpu_dying_mask)), add every slot, and decide on the true total.
/* stage 1: skip taking the lock when safe */
if (abs(count + amount) <= batch &&
    ((amount > 0 && gcount + unknown <= limit) ||
     (amount < 0 && gcount - unknown >= limit))) {
    this_cpu_add(*fbc->counters, amount);
    return true;
}

/* stage 2: take the lock; skip the full sum when the slack does not straddle the limit */
raw_spin_lock(&fbc->lock);
count = fbc->count + amount;
...

/* stage 3: still ambiguous - sum every CPU's slot and decide on the exact total */

that shape is what makes the lockless read defensible as an approximation rather than a guess: the fast path may only skip the lock when the answer is unambiguous even after the worst-case slack is added in, so the error it can introduce is bounded by unknown and nothing more.

why the read is intentionally lockless

used_blocks is touched on every tmpfs page allocation and free. taking fbc->lock on each one would serialize allocation against free, which is exactly what per-CPU counters exist to avoid. so the fast path reads fbc->count without the lock, on purpose.

is that read "safe because it is conservative"? no. my first version of the comment said something like that, and it was wrong. the honest statement is narrower:

  • a concurrent flush moves value between the global count and a per-CPU slot;
  • but other CPUs' locked slow paths also add to the global count while the fast path adds to a slot;
  • so a stale low read is not a lower bound on anything. it can let the fast path proceed where the locked slow path would have refused.

what limits the damage is the per-CPU slack. the abs(count + amount) <= batch guard keeps this CPU's slot within batch across the update, and every other slot is bounded by batch too, so the total cannot run ahead by more than unknown. the read is a cheap approximation used to skip the lock when there is clearly room, not a hard guarantee.

that is the part a quick read of the diff gets wrong, and it is worth writing down in the code so the next person does not "fix" the read by taking the lock:

/*
 * Lockless on purpose: gcount may be stale, so this is only an
 * approximation, bounded by the per-cpu slack ("unknown").
 */
gcount = data_race(READ_ONCE(fbc->count));

the local is named gcount, not total, because it is the global part - total already means the summed value including the per-CPU slots everywhere else in this file. naming matters when the whole point is not to mislead the next reader.

why READ_ONCE() isn't enough for KCSAN

this is the trap that cost me the most time. READ_ONCE(fbc->count) looks like the fix: it tells the compiler this is a single, deliberate load. but to KCSAN it is a marked atomic access, and marked accesses only ever check watchpoints, they never create them. so:

  • the plain compound store fbc->count += ... sets a watchpoint and stalls inside its critical section;
  • the READ_ONCE() read on the other CPU looks for a watchpoint, finds it, and reports.

if anything, the READ_ONCE() made the code look more careful without silencing anything. the tool is not asking "did you tell the compiler"; it is asking "do you acknowledge this race as intended".

there is a second trap: READ_ONCE() on an s64 is not a 64-bit atomic load on 32-bit. it compiles to a single volatile access of that type, and on a 32-bit target that can still be two loads. it stops the compiler from refetching or caching the value, which is what percpu_counter_read_positive() already relied on, but it does not give you atomicity - that is what atomic64_t is for.

why data_race(READ_ONCE()) is the correct annotation

data_race() is the only thing that actually removes an access from KCSAN's view: it disables instrumentation around the expression, so the access neither sets nor consumes a watchpoint. and include/linux/compiler.h documents exactly this shape:

  • accesses to a lock-protected variable stay plain;
  • deliberate lockless accesses to that same variable use data_race();
  • if the access must also be atomic, use data_race(READ_ONCE(x)).

keeping the writers plain matters. if you sprinkle WRITE_ONCE() on the locked writes, you make them marked, and a future genuinely-buggy lockless write - the kind you actually want KCSAN to catch - gets harder to see. annotate the one intentional side, leave the rest visible.

the final hunk:

count = __this_cpu_read(*fbc->counters);
/*
 * Lockless on purpose: gcount may be stale, so this is only an
 * approximation, bounded by the per-cpu slack ("unknown").
 */
gcount = data_race(READ_ONCE(fbc->count));

/* Skip taking the lock when safe */
if (abs(count + amount) <= batch &&
    ((amount > 0 && gcount + unknown <= limit) ||
     (amount < 0 && gcount - unknown >= limit))) {

no logic change. the syzbot report was a claim about the C memory model, and the claim was true; the race simply was not a bug. what was missing was saying so in a way the tool understands.

testing and verification

two separate things need evidence: that the code does not change, and that KCSAN goes quiet for this pair.

object code. a defconfig build, gcc 16.2.1, before and after, comparing the disassembly of one function with objdump -d - same compiler, same config, so the patch is the only variable. __percpu_counter_limited_add() keeps the same size (0x23b), still contains the same four fbc->count loads, and has identical branch targets. the .text differs in 45 of 2171 bytes, all of it register allocation and compare-operand order. as a control, a comment-only rebuild is byte-identical - so "no functional change intended" is not just a phrase.

KCSAN A/B. build one kernel with CONFIG_KCSAN=y and CONFIG_KASAN=n (the two sanitizers are mutually exclusive - you pick one per build), but leave KCSAN off at boot and switch it on from init by writing on to /sys/kernel/debug/kcsan. the first attempt used KCSAN_STRICT + KCSAN_WEAK_MEMORY + early enable and never finished booting in 420 s under KVM; deferring it keeps boot sane, and the stall knob (udelay_task) is what widens the window enough for a 120 s run to mean something.

the workload is a 120 s stress on one size=64M tmpfs in an 8 vCPU QEMU/KVM guest: threads doing write() and unlink()/ftruncate() concurrently, which is exactly shmem_inode_acct_blocks against shmem_inode_unacct_blocks. the base kernel produces the report almost immediately:

BUG: KCSAN: data-race in __percpu_counter_limited_add / percpu_counter_add_batch

write to ... by percpu_counter_add_batch+0xcc/0x150
 shmem_inode_unacct_blocks / shmem_recalc_inode / shmem_undo_range
 shmem_setattr / do_ftruncate / __x64_sys_ftruncate

read to ... by __percpu_counter_limited_add+0x8a/0x2e0
 shmem_inode_acct_blocks / shmem_get_folio_gfp / shmem_write_begin
 generic_perform_write / shmem_file_write_iter / vfs_write

base: 38 reports of that exact title, the first one 0.3 s after the workers started. with the annotation: 0 percpu_counter reports in the same 120 s. unrelated races (d_make_discardable/dentry_kill, osq_lock) still appear, which is a useful control - the detector was definitely awake, it just had nothing to say about this pair.

KCSAN is sampling, so "0 reports" is strong evidence, not proof. but 38 hits in a third of a second on one side, and nothing in two minutes on the other, is about as clean as a reproduction gets.

tag choices in the commit message

the patch went to akpm and linux-mm, cc Hugh Dickins - who wrote the original _limited_add() - and Baolin Wang. two tag choices are worth calling out:

  • Reported-by: carries the syzbot address, and checkpatch wants Closes: immediately after it with the bug URL, so the report is linked rather than just thanked.
  • there is deliberately no Fixes: tag. this annotates an intentional race; it is not a regression and it does not fix a crash. a Fixes: tag invites stable/AUTOSEL backports and frames the change as a bug fix, which is the wrong frame. checkpatch does warn "perhaps it also needs a 'Fixes:' tag" - that warning is expected and was left in on purpose. if a maintainer wants the tag, they will ask.

takeaways

  • a KCSAN splat is a statement about the C memory model, not automatically a bug. some races are intentional.
  • intentional means "annotate", not "ignore": the next reader deserves to know why, and the tool needs to be told.
  • READ_ONCE()/WRITE_ONCE() do not silence KCSAN; data_race() does. use data_race(READ_ONCE(x)) when you need both.
  • annotate narrowly. keep the locked accesses plain so real lockless bugs stay visible.
  • be precise about lockless approximations. "conservative" was wrong; "bounded by the per-CPU slack" is right.
← Back to blog