1*49ab51b0SPaul E. McKenneyMARKING SHARED-MEMORY ACCESSES 2*49ab51b0SPaul E. McKenney============================== 3*49ab51b0SPaul E. McKenney 4*49ab51b0SPaul E. McKenneyThis document provides guidelines for marking intentionally concurrent 5*49ab51b0SPaul E. McKenneynormal accesses to shared memory, that is "normal" as in accesses that do 6*49ab51b0SPaul E. McKenneynot use read-modify-write atomic operations. It also describes how to 7*49ab51b0SPaul E. McKenneydocument these accesses, both with comments and with special assertions 8*49ab51b0SPaul E. McKenneyprocessed by the Kernel Concurrency Sanitizer (KCSAN). This discussion 9*49ab51b0SPaul E. McKenneybuilds on an earlier LWN article [1]. 10*49ab51b0SPaul E. McKenney 11*49ab51b0SPaul E. McKenney 12*49ab51b0SPaul E. McKenneyACCESS-MARKING OPTIONS 13*49ab51b0SPaul E. McKenney====================== 14*49ab51b0SPaul E. McKenney 15*49ab51b0SPaul E. McKenneyThe Linux kernel provides the following access-marking options: 16*49ab51b0SPaul E. McKenney 17*49ab51b0SPaul E. McKenney1. Plain C-language accesses (unmarked), for example, "a = b;" 18*49ab51b0SPaul E. McKenney 19*49ab51b0SPaul E. McKenney2. Data-race marking, for example, "data_race(a = b);" 20*49ab51b0SPaul E. McKenney 21*49ab51b0SPaul E. McKenney3. READ_ONCE(), for example, "a = READ_ONCE(b);" 22*49ab51b0SPaul E. McKenney The various forms of atomic_read() also fit in here. 23*49ab51b0SPaul E. McKenney 24*49ab51b0SPaul E. McKenney4. WRITE_ONCE(), for example, "WRITE_ONCE(a, b);" 25*49ab51b0SPaul E. McKenney The various forms of atomic_set() also fit in here. 26*49ab51b0SPaul E. McKenney 27*49ab51b0SPaul E. McKenney 28*49ab51b0SPaul E. McKenneyThese may be used in combination, as shown in this admittedly improbable 29*49ab51b0SPaul E. McKenneyexample: 30*49ab51b0SPaul E. McKenney 31*49ab51b0SPaul E. McKenney WRITE_ONCE(a, b + data_race(c + d) + READ_ONCE(e)); 32*49ab51b0SPaul E. McKenney 33*49ab51b0SPaul E. McKenneyNeither plain C-language accesses nor data_race() (#1 and #2 above) place 34*49ab51b0SPaul E. McKenneyany sort of constraint on the compiler's choice of optimizations [2]. 35*49ab51b0SPaul E. McKenneyIn contrast, READ_ONCE() and WRITE_ONCE() (#3 and #4 above) restrict the 36*49ab51b0SPaul E. McKenneycompiler's use of code-motion and common-subexpression optimizations. 37*49ab51b0SPaul E. McKenneyTherefore, if a given access is involved in an intentional data race, 38*49ab51b0SPaul E. McKenneyusing READ_ONCE() for loads and WRITE_ONCE() for stores is usually 39*49ab51b0SPaul E. McKenneypreferable to data_race(), which in turn is usually preferable to plain 40*49ab51b0SPaul E. McKenneyC-language accesses. 41*49ab51b0SPaul E. McKenney 42*49ab51b0SPaul E. McKenneyKCSAN will complain about many types of data races involving plain 43*49ab51b0SPaul E. McKenneyC-language accesses, but marking all accesses involved in a given data 44*49ab51b0SPaul E. McKenneyrace with one of data_race(), READ_ONCE(), or WRITE_ONCE(), will prevent 45*49ab51b0SPaul E. McKenneyKCSAN from complaining. Of course, lack of KCSAN complaints does not 46*49ab51b0SPaul E. McKenneyimply correct code. Therefore, please take a thoughtful approach 47*49ab51b0SPaul E. McKenneywhen responding to KCSAN complaints. Churning the code base with 48*49ab51b0SPaul E. McKenneyill-considered additions of data_race(), READ_ONCE(), and WRITE_ONCE() 49*49ab51b0SPaul E. McKenneyis unhelpful. 50*49ab51b0SPaul E. McKenney 51*49ab51b0SPaul E. McKenneyIn fact, the following sections describe situations where use of 52*49ab51b0SPaul E. McKenneydata_race() and even plain C-language accesses is preferable to 53*49ab51b0SPaul E. McKenneyREAD_ONCE() and WRITE_ONCE(). 54*49ab51b0SPaul E. McKenney 55*49ab51b0SPaul E. McKenney 56*49ab51b0SPaul E. McKenneyUse of the data_race() Macro 57*49ab51b0SPaul E. McKenney---------------------------- 58*49ab51b0SPaul E. McKenney 59*49ab51b0SPaul E. McKenneyHere are some situations where data_race() should be used instead of 60*49ab51b0SPaul E. McKenneyREAD_ONCE() and WRITE_ONCE(): 61*49ab51b0SPaul E. McKenney 62*49ab51b0SPaul E. McKenney1. Data-racy loads from shared variables whose values are used only 63*49ab51b0SPaul E. McKenney for diagnostic purposes. 64*49ab51b0SPaul E. McKenney 65*49ab51b0SPaul E. McKenney2. Data-racy reads whose values are checked against marked reload. 66*49ab51b0SPaul E. McKenney 67*49ab51b0SPaul E. McKenney3. Reads whose values feed into error-tolerant heuristics. 68*49ab51b0SPaul E. McKenney 69*49ab51b0SPaul E. McKenney4. Writes setting values that feed into error-tolerant heuristics. 70*49ab51b0SPaul E. McKenney 71*49ab51b0SPaul E. McKenney 72*49ab51b0SPaul E. McKenneyData-Racy Reads for Approximate Diagnostics 73*49ab51b0SPaul E. McKenney 74*49ab51b0SPaul E. McKenneyApproximate diagnostics include lockdep reports, monitoring/statistics 75*49ab51b0SPaul E. McKenney(including /proc and /sys output), WARN*()/BUG*() checks whose return 76*49ab51b0SPaul E. McKenneyvalues are ignored, and other situations where reads from shared variables 77*49ab51b0SPaul E. McKenneyare not an integral part of the core concurrency design. 78*49ab51b0SPaul E. McKenney 79*49ab51b0SPaul E. McKenneyIn fact, use of data_race() instead READ_ONCE() for these diagnostic 80*49ab51b0SPaul E. McKenneyreads can enable better checking of the remaining accesses implementing 81*49ab51b0SPaul E. McKenneythe core concurrency design. For example, suppose that the core design 82*49ab51b0SPaul E. McKenneyprevents any non-diagnostic reads from shared variable x from running 83*49ab51b0SPaul E. McKenneyconcurrently with updates to x. Then using plain C-language writes 84*49ab51b0SPaul E. McKenneyto x allows KCSAN to detect reads from x from within regions of code 85*49ab51b0SPaul E. McKenneythat fail to exclude the updates. In this case, it is important to use 86*49ab51b0SPaul E. McKenneydata_race() for the diagnostic reads because otherwise KCSAN would give 87*49ab51b0SPaul E. McKenneyfalse-positive warnings about these diagnostic reads. 88*49ab51b0SPaul E. McKenney 89*49ab51b0SPaul E. McKenneyIn theory, plain C-language loads can also be used for this use case. 90*49ab51b0SPaul E. McKenneyHowever, in practice this will have the disadvantage of causing KCSAN 91*49ab51b0SPaul E. McKenneyto generate false positives because KCSAN will have no way of knowing 92*49ab51b0SPaul E. McKenneythat the resulting data race was intentional. 93*49ab51b0SPaul E. McKenney 94*49ab51b0SPaul E. McKenney 95*49ab51b0SPaul E. McKenneyData-Racy Reads That Are Checked Against Marked Reload 96*49ab51b0SPaul E. McKenney 97*49ab51b0SPaul E. McKenneyThe values from some reads are not implicitly trusted. They are instead 98*49ab51b0SPaul E. McKenneyfed into some operation that checks the full value against a later marked 99*49ab51b0SPaul E. McKenneyload from memory, which means that the occasional arbitrarily bogus value 100*49ab51b0SPaul E. McKenneyis not a problem. For example, if a bogus value is fed into cmpxchg(), 101*49ab51b0SPaul E. McKenneyall that happens is that this cmpxchg() fails, which normally results 102*49ab51b0SPaul E. McKenneyin a retry. Unless the race condition that resulted in the bogus value 103*49ab51b0SPaul E. McKenneyrecurs, this retry will with high probability succeed, so no harm done. 104*49ab51b0SPaul E. McKenney 105*49ab51b0SPaul E. McKenneyHowever, please keep in mind that a data_race() load feeding into 106*49ab51b0SPaul E. McKenneya cmpxchg_relaxed() might still be subject to load fusing on some 107*49ab51b0SPaul E. McKenneyarchitectures. Therefore, it is best to capture the return value from 108*49ab51b0SPaul E. McKenneythe failing cmpxchg() for the next iteration of the loop, an approach 109*49ab51b0SPaul E. McKenneythat provides the compiler much less scope for mischievous optimizations. 110*49ab51b0SPaul E. McKenneyCapturing the return value from cmpxchg() also saves a memory reference 111*49ab51b0SPaul E. McKenneyin many cases. 112*49ab51b0SPaul E. McKenney 113*49ab51b0SPaul E. McKenneyIn theory, plain C-language loads can also be used for this use case. 114*49ab51b0SPaul E. McKenneyHowever, in practice this will have the disadvantage of causing KCSAN 115*49ab51b0SPaul E. McKenneyto generate false positives because KCSAN will have no way of knowing 116*49ab51b0SPaul E. McKenneythat the resulting data race was intentional. 117*49ab51b0SPaul E. McKenney 118*49ab51b0SPaul E. McKenney 119*49ab51b0SPaul E. McKenneyReads Feeding Into Error-Tolerant Heuristics 120*49ab51b0SPaul E. McKenney 121*49ab51b0SPaul E. McKenneyValues from some reads feed into heuristics that can tolerate occasional 122*49ab51b0SPaul E. McKenneyerrors. Such reads can use data_race(), thus allowing KCSAN to focus on 123*49ab51b0SPaul E. McKenneythe other accesses to the relevant shared variables. But please note 124*49ab51b0SPaul E. McKenneythat data_race() loads are subject to load fusing, which can result in 125*49ab51b0SPaul E. McKenneyconsistent errors, which in turn are quite capable of breaking heuristics. 126*49ab51b0SPaul E. McKenneyTherefore use of data_race() should be limited to cases where some other 127*49ab51b0SPaul E. McKenneycode (such as a barrier() call) will force the occasional reload. 128*49ab51b0SPaul E. McKenney 129*49ab51b0SPaul E. McKenneyIn theory, plain C-language loads can also be used for this use case. 130*49ab51b0SPaul E. McKenneyHowever, in practice this will have the disadvantage of causing KCSAN 131*49ab51b0SPaul E. McKenneyto generate false positives because KCSAN will have no way of knowing 132*49ab51b0SPaul E. McKenneythat the resulting data race was intentional. 133*49ab51b0SPaul E. McKenney 134*49ab51b0SPaul E. McKenney 135*49ab51b0SPaul E. McKenneyWrites Setting Values Feeding Into Error-Tolerant Heuristics 136*49ab51b0SPaul E. McKenney 137*49ab51b0SPaul E. McKenneyThe values read into error-tolerant heuristics come from somewhere, 138*49ab51b0SPaul E. McKenneyfor example, from sysfs. This means that some code in sysfs writes 139*49ab51b0SPaul E. McKenneyto this same variable, and these writes can also use data_race(). 140*49ab51b0SPaul E. McKenneyAfter all, if the heuristic can tolerate the occasional bogus value 141*49ab51b0SPaul E. McKenneydue to compiler-mangled reads, it can also tolerate the occasional 142*49ab51b0SPaul E. McKenneycompiler-mangled write, at least assuming that the proper value is in 143*49ab51b0SPaul E. McKenneyplace once the write completes. 144*49ab51b0SPaul E. McKenney 145*49ab51b0SPaul E. McKenneyPlain C-language stores can also be used for this use case. However, 146*49ab51b0SPaul E. McKenneyin kernels built with CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC=n, this 147*49ab51b0SPaul E. McKenneywill have the disadvantage of causing KCSAN to generate false positives 148*49ab51b0SPaul E. McKenneybecause KCSAN will have no way of knowing that the resulting data race 149*49ab51b0SPaul E. McKenneywas intentional. 150*49ab51b0SPaul E. McKenney 151*49ab51b0SPaul E. McKenney 152*49ab51b0SPaul E. McKenneyUse of Plain C-Language Accesses 153*49ab51b0SPaul E. McKenney-------------------------------- 154*49ab51b0SPaul E. McKenney 155*49ab51b0SPaul E. McKenneyHere are some example situations where plain C-language accesses should 156*49ab51b0SPaul E. McKenneyused instead of READ_ONCE(), WRITE_ONCE(), and data_race(): 157*49ab51b0SPaul E. McKenney 158*49ab51b0SPaul E. McKenney1. Accesses protected by mutual exclusion, including strict locking 159*49ab51b0SPaul E. McKenney and sequence locking. 160*49ab51b0SPaul E. McKenney 161*49ab51b0SPaul E. McKenney2. Initialization-time and cleanup-time accesses. This covers a 162*49ab51b0SPaul E. McKenney wide variety of situations, including the uniprocessor phase of 163*49ab51b0SPaul E. McKenney system boot, variables to be used by not-yet-spawned kthreads, 164*49ab51b0SPaul E. McKenney structures not yet published to reference-counted or RCU-protected 165*49ab51b0SPaul E. McKenney data structures, and the cleanup side of any of these situations. 166*49ab51b0SPaul E. McKenney 167*49ab51b0SPaul E. McKenney3. Per-CPU variables that are not accessed from other CPUs. 168*49ab51b0SPaul E. McKenney 169*49ab51b0SPaul E. McKenney4. Private per-task variables, including on-stack variables, some 170*49ab51b0SPaul E. McKenney fields in the task_struct structure, and task-private heap data. 171*49ab51b0SPaul E. McKenney 172*49ab51b0SPaul E. McKenney5. Any other loads for which there is not supposed to be a concurrent 173*49ab51b0SPaul E. McKenney store to that same variable. 174*49ab51b0SPaul E. McKenney 175*49ab51b0SPaul E. McKenney6. Any other stores for which there should be neither concurrent 176*49ab51b0SPaul E. McKenney loads nor concurrent stores to that same variable. 177*49ab51b0SPaul E. McKenney 178*49ab51b0SPaul E. McKenney But note that KCSAN makes two explicit exceptions to this rule 179*49ab51b0SPaul E. McKenney by default, refraining from flagging plain C-language stores: 180*49ab51b0SPaul E. McKenney 181*49ab51b0SPaul E. McKenney a. No matter what. You can override this default by building 182*49ab51b0SPaul E. McKenney with CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC=n. 183*49ab51b0SPaul E. McKenney 184*49ab51b0SPaul E. McKenney b. When the store writes the value already contained in 185*49ab51b0SPaul E. McKenney that variable. You can override this default by building 186*49ab51b0SPaul E. McKenney with CONFIG_KCSAN_REPORT_VALUE_CHANGE_ONLY=n. 187*49ab51b0SPaul E. McKenney 188*49ab51b0SPaul E. McKenney c. When one of the stores is in an interrupt handler and 189*49ab51b0SPaul E. McKenney the other in the interrupted code. You can override this 190*49ab51b0SPaul E. McKenney default by building with CONFIG_KCSAN_INTERRUPT_WATCHER=y. 191*49ab51b0SPaul E. McKenney 192*49ab51b0SPaul E. McKenneyNote that it is important to use plain C-language accesses in these cases, 193*49ab51b0SPaul E. McKenneybecause doing otherwise prevents KCSAN from detecting violations of your 194*49ab51b0SPaul E. McKenneycode's synchronization rules. 195*49ab51b0SPaul E. McKenney 196*49ab51b0SPaul E. McKenney 197*49ab51b0SPaul E. McKenneyACCESS-DOCUMENTATION OPTIONS 198*49ab51b0SPaul E. McKenney============================ 199*49ab51b0SPaul E. McKenney 200*49ab51b0SPaul E. McKenneyIt is important to comment marked accesses so that people reading your 201*49ab51b0SPaul E. McKenneycode, yourself included, are reminded of the synchronization design. 202*49ab51b0SPaul E. McKenneyHowever, it is even more important to comment plain C-language accesses 203*49ab51b0SPaul E. McKenneythat are intentionally involved in data races. Such comments are 204*49ab51b0SPaul E. McKenneyneeded to remind people reading your code, again, yourself included, 205*49ab51b0SPaul E. McKenneyof how the compiler has been prevented from optimizing those accesses 206*49ab51b0SPaul E. McKenneyinto concurrency bugs. 207*49ab51b0SPaul E. McKenney 208*49ab51b0SPaul E. McKenneyIt is also possible to tell KCSAN about your synchronization design. 209*49ab51b0SPaul E. McKenneyFor example, ASSERT_EXCLUSIVE_ACCESS(foo) tells KCSAN that any 210*49ab51b0SPaul E. McKenneyconcurrent access to variable foo by any other CPU is an error, even 211*49ab51b0SPaul E. McKenneyif that concurrent access is marked with READ_ONCE(). In addition, 212*49ab51b0SPaul E. McKenneyASSERT_EXCLUSIVE_WRITER(foo) tells KCSAN that although it is OK for there 213*49ab51b0SPaul E. McKenneyto be concurrent reads from foo from other CPUs, it is an error for some 214*49ab51b0SPaul E. McKenneyother CPU to be concurrently writing to foo, even if that concurrent 215*49ab51b0SPaul E. McKenneywrite is marked with data_race() or WRITE_ONCE(). 216*49ab51b0SPaul E. McKenney 217*49ab51b0SPaul E. McKenneyNote that although KCSAN will call out data races involving either 218*49ab51b0SPaul E. McKenneyASSERT_EXCLUSIVE_ACCESS() or ASSERT_EXCLUSIVE_WRITER() on the one hand 219*49ab51b0SPaul E. McKenneyand data_race() writes on the other, KCSAN will not report the location 220*49ab51b0SPaul E. McKenneyof these data_race() writes. 221*49ab51b0SPaul E. McKenney 222*49ab51b0SPaul E. McKenney 223*49ab51b0SPaul E. McKenneyEXAMPLES 224*49ab51b0SPaul E. McKenney======== 225*49ab51b0SPaul E. McKenney 226*49ab51b0SPaul E. McKenneyAs noted earlier, the goal is to prevent the compiler from destroying 227*49ab51b0SPaul E. McKenneyyour concurrent algorithm, to help the human reader, and to inform 228*49ab51b0SPaul E. McKenneyKCSAN of aspects of your concurrency design. This section looks at a 229*49ab51b0SPaul E. McKenneyfew examples showing how this can be done. 230*49ab51b0SPaul E. McKenney 231*49ab51b0SPaul E. McKenney 232*49ab51b0SPaul E. McKenneyLock Protection With Lockless Diagnostic Access 233*49ab51b0SPaul E. McKenney----------------------------------------------- 234*49ab51b0SPaul E. McKenney 235*49ab51b0SPaul E. McKenneyFor example, suppose a shared variable "foo" is read only while a 236*49ab51b0SPaul E. McKenneyreader-writer spinlock is read-held, written only while that same 237*49ab51b0SPaul E. McKenneyspinlock is write-held, except that it is also read locklessly for 238*49ab51b0SPaul E. McKenneydiagnostic purposes. The code might look as follows: 239*49ab51b0SPaul E. McKenney 240*49ab51b0SPaul E. McKenney int foo; 241*49ab51b0SPaul E. McKenney DEFINE_RWLOCK(foo_rwlock); 242*49ab51b0SPaul E. McKenney 243*49ab51b0SPaul E. McKenney void update_foo(int newval) 244*49ab51b0SPaul E. McKenney { 245*49ab51b0SPaul E. McKenney write_lock(&foo_rwlock); 246*49ab51b0SPaul E. McKenney foo = newval; 247*49ab51b0SPaul E. McKenney do_something(newval); 248*49ab51b0SPaul E. McKenney write_unlock(&foo_rwlock); 249*49ab51b0SPaul E. McKenney } 250*49ab51b0SPaul E. McKenney 251*49ab51b0SPaul E. McKenney int read_foo(void) 252*49ab51b0SPaul E. McKenney { 253*49ab51b0SPaul E. McKenney int ret; 254*49ab51b0SPaul E. McKenney 255*49ab51b0SPaul E. McKenney read_lock(&foo_rwlock); 256*49ab51b0SPaul E. McKenney do_something_else(); 257*49ab51b0SPaul E. McKenney ret = foo; 258*49ab51b0SPaul E. McKenney read_unlock(&foo_rwlock); 259*49ab51b0SPaul E. McKenney return ret; 260*49ab51b0SPaul E. McKenney } 261*49ab51b0SPaul E. McKenney 262*49ab51b0SPaul E. McKenney int read_foo_diagnostic(void) 263*49ab51b0SPaul E. McKenney { 264*49ab51b0SPaul E. McKenney return data_race(foo); 265*49ab51b0SPaul E. McKenney } 266*49ab51b0SPaul E. McKenney 267*49ab51b0SPaul E. McKenneyThe reader-writer lock prevents the compiler from introducing concurrency 268*49ab51b0SPaul E. McKenneybugs into any part of the main algorithm using foo, which means that 269*49ab51b0SPaul E. McKenneythe accesses to foo within both update_foo() and read_foo() can (and 270*49ab51b0SPaul E. McKenneyshould) be plain C-language accesses. One benefit of making them be 271*49ab51b0SPaul E. McKenneyplain C-language accesses is that KCSAN can detect any erroneous lockless 272*49ab51b0SPaul E. McKenneyreads from or updates to foo. The data_race() in read_foo_diagnostic() 273*49ab51b0SPaul E. McKenneytells KCSAN that data races are expected, and should be silently 274*49ab51b0SPaul E. McKenneyignored. This data_race() also tells the human reading the code that 275*49ab51b0SPaul E. McKenneyread_foo_diagnostic() might sometimes return a bogus value. 276*49ab51b0SPaul E. McKenney 277*49ab51b0SPaul E. McKenneyHowever, please note that your kernel must be built with 278*49ab51b0SPaul E. McKenneyCONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC=n in order for KCSAN to 279*49ab51b0SPaul E. McKenneydetect a buggy lockless write. If you need KCSAN to detect such a 280*49ab51b0SPaul E. McKenneywrite even if that write did not change the value of foo, you also 281*49ab51b0SPaul E. McKenneyneed CONFIG_KCSAN_REPORT_VALUE_CHANGE_ONLY=n. If you need KCSAN to 282*49ab51b0SPaul E. McKenneydetect such a write happening in an interrupt handler running on the 283*49ab51b0SPaul E. McKenneysame CPU doing the legitimate lock-protected write, you also need 284*49ab51b0SPaul E. McKenneyCONFIG_KCSAN_INTERRUPT_WATCHER=y. With some or all of these Kconfig 285*49ab51b0SPaul E. McKenneyoptions set properly, KCSAN can be quite helpful, although it is not 286*49ab51b0SPaul E. McKenneynecessarily a full replacement for hardware watchpoints. On the other 287*49ab51b0SPaul E. McKenneyhand, neither are hardware watchpoints a full replacement for KCSAN 288*49ab51b0SPaul E. McKenneybecause it is not always easy to tell hardware watchpoint to conditionally 289*49ab51b0SPaul E. McKenneytrap on accesses. 290*49ab51b0SPaul E. McKenney 291*49ab51b0SPaul E. McKenney 292*49ab51b0SPaul E. McKenneyLock-Protected Writes With Lockless Reads 293*49ab51b0SPaul E. McKenney----------------------------------------- 294*49ab51b0SPaul E. McKenney 295*49ab51b0SPaul E. McKenneyFor another example, suppose a shared variable "foo" is updated only 296*49ab51b0SPaul E. McKenneywhile holding a spinlock, but is read locklessly. The code might look 297*49ab51b0SPaul E. McKenneyas follows: 298*49ab51b0SPaul E. McKenney 299*49ab51b0SPaul E. McKenney int foo; 300*49ab51b0SPaul E. McKenney DEFINE_SPINLOCK(foo_lock); 301*49ab51b0SPaul E. McKenney 302*49ab51b0SPaul E. McKenney void update_foo(int newval) 303*49ab51b0SPaul E. McKenney { 304*49ab51b0SPaul E. McKenney spin_lock(&foo_lock); 305*49ab51b0SPaul E. McKenney WRITE_ONCE(foo, newval); 306*49ab51b0SPaul E. McKenney ASSERT_EXCLUSIVE_WRITER(foo); 307*49ab51b0SPaul E. McKenney do_something(newval); 308*49ab51b0SPaul E. McKenney spin_unlock(&foo_wlock); 309*49ab51b0SPaul E. McKenney } 310*49ab51b0SPaul E. McKenney 311*49ab51b0SPaul E. McKenney int read_foo(void) 312*49ab51b0SPaul E. McKenney { 313*49ab51b0SPaul E. McKenney do_something_else(); 314*49ab51b0SPaul E. McKenney return READ_ONCE(foo); 315*49ab51b0SPaul E. McKenney } 316*49ab51b0SPaul E. McKenney 317*49ab51b0SPaul E. McKenneyBecause foo is read locklessly, all accesses are marked. The purpose 318*49ab51b0SPaul E. McKenneyof the ASSERT_EXCLUSIVE_WRITER() is to allow KCSAN to check for a buggy 319*49ab51b0SPaul E. McKenneyconcurrent lockless write. 320*49ab51b0SPaul E. McKenney 321*49ab51b0SPaul E. McKenney 322*49ab51b0SPaul E. McKenneyLockless Reads and Writes 323*49ab51b0SPaul E. McKenney------------------------- 324*49ab51b0SPaul E. McKenney 325*49ab51b0SPaul E. McKenneyFor another example, suppose a shared variable "foo" is both read and 326*49ab51b0SPaul E. McKenneyupdated locklessly. The code might look as follows: 327*49ab51b0SPaul E. McKenney 328*49ab51b0SPaul E. McKenney int foo; 329*49ab51b0SPaul E. McKenney 330*49ab51b0SPaul E. McKenney int update_foo(int newval) 331*49ab51b0SPaul E. McKenney { 332*49ab51b0SPaul E. McKenney int ret; 333*49ab51b0SPaul E. McKenney 334*49ab51b0SPaul E. McKenney ret = xchg(&foo, newval); 335*49ab51b0SPaul E. McKenney do_something(newval); 336*49ab51b0SPaul E. McKenney return ret; 337*49ab51b0SPaul E. McKenney } 338*49ab51b0SPaul E. McKenney 339*49ab51b0SPaul E. McKenney int read_foo(void) 340*49ab51b0SPaul E. McKenney { 341*49ab51b0SPaul E. McKenney do_something_else(); 342*49ab51b0SPaul E. McKenney return READ_ONCE(foo); 343*49ab51b0SPaul E. McKenney } 344*49ab51b0SPaul E. McKenney 345*49ab51b0SPaul E. McKenneyBecause foo is accessed locklessly, all accesses are marked. It does 346*49ab51b0SPaul E. McKenneynot make sense to use ASSERT_EXCLUSIVE_WRITER() in this case because 347*49ab51b0SPaul E. McKenneythere really can be concurrent lockless writers. KCSAN would 348*49ab51b0SPaul E. McKenneyflag any concurrent plain C-language reads from foo, and given 349*49ab51b0SPaul E. McKenneyCONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC=n, also any concurrent plain 350*49ab51b0SPaul E. McKenneyC-language writes to foo. 351*49ab51b0SPaul E. McKenney 352*49ab51b0SPaul E. McKenney 353*49ab51b0SPaul E. McKenneyLockless Reads and Writes, But With Single-Threaded Initialization 354*49ab51b0SPaul E. McKenney------------------------------------------------------------------ 355*49ab51b0SPaul E. McKenney 356*49ab51b0SPaul E. McKenneyFor yet another example, suppose that foo is initialized in a 357*49ab51b0SPaul E. McKenneysingle-threaded manner, but that a number of kthreads are then created 358*49ab51b0SPaul E. McKenneythat locklessly and concurrently access foo. Some snippets of this code 359*49ab51b0SPaul E. McKenneymight look as follows: 360*49ab51b0SPaul E. McKenney 361*49ab51b0SPaul E. McKenney int foo; 362*49ab51b0SPaul E. McKenney 363*49ab51b0SPaul E. McKenney void initialize_foo(int initval, int nkthreads) 364*49ab51b0SPaul E. McKenney { 365*49ab51b0SPaul E. McKenney int i; 366*49ab51b0SPaul E. McKenney 367*49ab51b0SPaul E. McKenney foo = initval; 368*49ab51b0SPaul E. McKenney ASSERT_EXCLUSIVE_ACCESS(foo); 369*49ab51b0SPaul E. McKenney for (i = 0; i < nkthreads; i++) 370*49ab51b0SPaul E. McKenney kthread_run(access_foo_concurrently, ...); 371*49ab51b0SPaul E. McKenney } 372*49ab51b0SPaul E. McKenney 373*49ab51b0SPaul E. McKenney /* Called from access_foo_concurrently(). */ 374*49ab51b0SPaul E. McKenney int update_foo(int newval) 375*49ab51b0SPaul E. McKenney { 376*49ab51b0SPaul E. McKenney int ret; 377*49ab51b0SPaul E. McKenney 378*49ab51b0SPaul E. McKenney ret = xchg(&foo, newval); 379*49ab51b0SPaul E. McKenney do_something(newval); 380*49ab51b0SPaul E. McKenney return ret; 381*49ab51b0SPaul E. McKenney } 382*49ab51b0SPaul E. McKenney 383*49ab51b0SPaul E. McKenney /* Also called from access_foo_concurrently(). */ 384*49ab51b0SPaul E. McKenney int read_foo(void) 385*49ab51b0SPaul E. McKenney { 386*49ab51b0SPaul E. McKenney do_something_else(); 387*49ab51b0SPaul E. McKenney return READ_ONCE(foo); 388*49ab51b0SPaul E. McKenney } 389*49ab51b0SPaul E. McKenney 390*49ab51b0SPaul E. McKenneyThe initialize_foo() uses a plain C-language write to foo because there 391*49ab51b0SPaul E. McKenneyare not supposed to be concurrent accesses during initialization. The 392*49ab51b0SPaul E. McKenneyASSERT_EXCLUSIVE_ACCESS() allows KCSAN to flag buggy concurrent unmarked 393*49ab51b0SPaul E. McKenneyreads, and the ASSERT_EXCLUSIVE_ACCESS() call further allows KCSAN to 394*49ab51b0SPaul E. McKenneyflag buggy concurrent writes, even if: (1) Those writes are marked or 395*49ab51b0SPaul E. McKenney(2) The kernel was built with CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC=y. 396*49ab51b0SPaul E. McKenney 397*49ab51b0SPaul E. McKenney 398*49ab51b0SPaul E. McKenneyChecking Stress-Test Race Coverage 399*49ab51b0SPaul E. McKenney---------------------------------- 400*49ab51b0SPaul E. McKenney 401*49ab51b0SPaul E. McKenneyWhen designing stress tests it is important to ensure that race conditions 402*49ab51b0SPaul E. McKenneyof interest really do occur. For example, consider the following code 403*49ab51b0SPaul E. McKenneyfragment: 404*49ab51b0SPaul E. McKenney 405*49ab51b0SPaul E. McKenney int foo; 406*49ab51b0SPaul E. McKenney 407*49ab51b0SPaul E. McKenney int update_foo(int newval) 408*49ab51b0SPaul E. McKenney { 409*49ab51b0SPaul E. McKenney return xchg(&foo, newval); 410*49ab51b0SPaul E. McKenney } 411*49ab51b0SPaul E. McKenney 412*49ab51b0SPaul E. McKenney int xor_shift_foo(int shift, int mask) 413*49ab51b0SPaul E. McKenney { 414*49ab51b0SPaul E. McKenney int old, new, newold; 415*49ab51b0SPaul E. McKenney 416*49ab51b0SPaul E. McKenney newold = data_race(foo); /* Checked by cmpxchg(). */ 417*49ab51b0SPaul E. McKenney do { 418*49ab51b0SPaul E. McKenney old = newold; 419*49ab51b0SPaul E. McKenney new = (old << shift) ^ mask; 420*49ab51b0SPaul E. McKenney newold = cmpxchg(&foo, old, new); 421*49ab51b0SPaul E. McKenney } while (newold != old); 422*49ab51b0SPaul E. McKenney return old; 423*49ab51b0SPaul E. McKenney } 424*49ab51b0SPaul E. McKenney 425*49ab51b0SPaul E. McKenney int read_foo(void) 426*49ab51b0SPaul E. McKenney { 427*49ab51b0SPaul E. McKenney return READ_ONCE(foo); 428*49ab51b0SPaul E. McKenney } 429*49ab51b0SPaul E. McKenney 430*49ab51b0SPaul E. McKenneyIf it is possible for update_foo(), xor_shift_foo(), and read_foo() to be 431*49ab51b0SPaul E. McKenneyinvoked concurrently, the stress test should force this concurrency to 432*49ab51b0SPaul E. McKenneyactually happen. KCSAN can evaluate the stress test when the above code 433*49ab51b0SPaul E. McKenneyis modified to read as follows: 434*49ab51b0SPaul E. McKenney 435*49ab51b0SPaul E. McKenney int foo; 436*49ab51b0SPaul E. McKenney 437*49ab51b0SPaul E. McKenney int update_foo(int newval) 438*49ab51b0SPaul E. McKenney { 439*49ab51b0SPaul E. McKenney ASSERT_EXCLUSIVE_ACCESS(foo); 440*49ab51b0SPaul E. McKenney return xchg(&foo, newval); 441*49ab51b0SPaul E. McKenney } 442*49ab51b0SPaul E. McKenney 443*49ab51b0SPaul E. McKenney int xor_shift_foo(int shift, int mask) 444*49ab51b0SPaul E. McKenney { 445*49ab51b0SPaul E. McKenney int old, new, newold; 446*49ab51b0SPaul E. McKenney 447*49ab51b0SPaul E. McKenney newold = data_race(foo); /* Checked by cmpxchg(). */ 448*49ab51b0SPaul E. McKenney do { 449*49ab51b0SPaul E. McKenney old = newold; 450*49ab51b0SPaul E. McKenney new = (old << shift) ^ mask; 451*49ab51b0SPaul E. McKenney ASSERT_EXCLUSIVE_ACCESS(foo); 452*49ab51b0SPaul E. McKenney newold = cmpxchg(&foo, old, new); 453*49ab51b0SPaul E. McKenney } while (newold != old); 454*49ab51b0SPaul E. McKenney return old; 455*49ab51b0SPaul E. McKenney } 456*49ab51b0SPaul E. McKenney 457*49ab51b0SPaul E. McKenney 458*49ab51b0SPaul E. McKenney int read_foo(void) 459*49ab51b0SPaul E. McKenney { 460*49ab51b0SPaul E. McKenney ASSERT_EXCLUSIVE_ACCESS(foo); 461*49ab51b0SPaul E. McKenney return READ_ONCE(foo); 462*49ab51b0SPaul E. McKenney } 463*49ab51b0SPaul E. McKenney 464*49ab51b0SPaul E. McKenneyIf a given stress-test run does not result in KCSAN complaints from 465*49ab51b0SPaul E. McKenneyeach possible pair of ASSERT_EXCLUSIVE_ACCESS() invocations, the 466*49ab51b0SPaul E. McKenneystress test needs improvement. If the stress test was to be evaluated 467*49ab51b0SPaul E. McKenneyon a regular basis, it would be wise to place the above instances of 468*49ab51b0SPaul E. McKenneyASSERT_EXCLUSIVE_ACCESS() under #ifdef so that they did not result in 469*49ab51b0SPaul E. McKenneyfalse positives when not evaluating the stress test. 470*49ab51b0SPaul E. McKenney 471*49ab51b0SPaul E. McKenney 472*49ab51b0SPaul E. McKenneyREFERENCES 473*49ab51b0SPaul E. McKenney========== 474*49ab51b0SPaul E. McKenney 475*49ab51b0SPaul E. McKenney[1] "Concurrency bugs should fear the big bad data-race detector (part 2)" 476*49ab51b0SPaul E. McKenney https://lwn.net/Articles/816854/ 477*49ab51b0SPaul E. McKenney 478*49ab51b0SPaul E. McKenney[2] "Who's afraid of a big bad optimizing compiler?" 479*49ab51b0SPaul E. McKenney https://lwn.net/Articles/793253/ 480