1b00aedf9SAmol Grover.. _rcu_dereference_doc: 2b00aedf9SAmol Grover 3b00aedf9SAmol GroverPROPER CARE AND FEEDING OF RETURN VALUES FROM rcu_dereference() 4b00aedf9SAmol Grover=============================================================== 5b00aedf9SAmol Grover 6b00aedf9SAmol GroverMost of the time, you can use values from rcu_dereference() or one of 7b00aedf9SAmol Groverthe similar primitives without worries. Dereferencing (prefix "*"), 8b00aedf9SAmol Groverfield selection ("->"), assignment ("="), address-of ("&"), addition and 9b00aedf9SAmol Groversubtraction of constants, and casts all work quite naturally and safely. 10b00aedf9SAmol Grover 11b00aedf9SAmol GroverIt is nevertheless possible to get into trouble with other operations. 12b00aedf9SAmol GroverFollow these rules to keep your RCU code working properly: 13b00aedf9SAmol Grover 14b00aedf9SAmol Grover- You must use one of the rcu_dereference() family of primitives 15b00aedf9SAmol Grover to load an RCU-protected pointer, otherwise CONFIG_PROVE_RCU 16b00aedf9SAmol Grover will complain. Worse yet, your code can see random memory-corruption 17b00aedf9SAmol Grover bugs due to games that compilers and DEC Alpha can play. 18b00aedf9SAmol Grover Without one of the rcu_dereference() primitives, compilers 19b00aedf9SAmol Grover can reload the value, and won't your code have fun with two 20b00aedf9SAmol Grover different values for a single pointer! Without rcu_dereference(), 21b00aedf9SAmol Grover DEC Alpha can load a pointer, dereference that pointer, and 22*b33994efSPaul E. McKenney return data preceding initialization that preceded the store 23*b33994efSPaul E. McKenney of the pointer. (As noted later, in recent kernels READ_ONCE() 24*b33994efSPaul E. McKenney also prevents DEC Alpha from playing these tricks.) 25b00aedf9SAmol Grover 26b00aedf9SAmol Grover In addition, the volatile cast in rcu_dereference() prevents the 27b00aedf9SAmol Grover compiler from deducing the resulting pointer value. Please see 28b00aedf9SAmol Grover the section entitled "EXAMPLE WHERE THE COMPILER KNOWS TOO MUCH" 29b00aedf9SAmol Grover for an example where the compiler can in fact deduce the exact 30b00aedf9SAmol Grover value of the pointer, and thus cause misordering. 31b00aedf9SAmol Grover 3286b5a738SPaul E. McKenney- In the special case where data is added but is never removed 3386b5a738SPaul E. McKenney while readers are accessing the structure, READ_ONCE() may be used 3486b5a738SPaul E. McKenney instead of rcu_dereference(). In this case, use of READ_ONCE() 3586b5a738SPaul E. McKenney takes on the role of the lockless_dereference() primitive that 3686b5a738SPaul E. McKenney was removed in v4.15. 3786b5a738SPaul E. McKenney 38*b33994efSPaul E. McKenney- You are only permitted to use rcu_dereference() on pointer values. 39b00aedf9SAmol Grover The compiler simply knows too much about integral values to 40b00aedf9SAmol Grover trust it to carry dependencies through integer operations. 41b00aedf9SAmol Grover There are a very few exceptions, namely that you can temporarily 42b00aedf9SAmol Grover cast the pointer to uintptr_t in order to: 43b00aedf9SAmol Grover 44b00aedf9SAmol Grover - Set bits and clear bits down in the must-be-zero low-order 45b00aedf9SAmol Grover bits of that pointer. This clearly means that the pointer 46b00aedf9SAmol Grover must have alignment constraints, for example, this does 47e3879ecdSAkira Yokosawa *not* work in general for char* pointers. 48b00aedf9SAmol Grover 49b00aedf9SAmol Grover - XOR bits to translate pointers, as is done in some 50b00aedf9SAmol Grover classic buddy-allocator algorithms. 51b00aedf9SAmol Grover 52b00aedf9SAmol Grover It is important to cast the value back to pointer before 53b00aedf9SAmol Grover doing much of anything else with it. 54b00aedf9SAmol Grover 55b00aedf9SAmol Grover- Avoid cancellation when using the "+" and "-" infix arithmetic 56b00aedf9SAmol Grover operators. For example, for a given variable "x", avoid 57b00aedf9SAmol Grover "(x-(uintptr_t)x)" for char* pointers. The compiler is within its 58b00aedf9SAmol Grover rights to substitute zero for this sort of expression, so that 59b00aedf9SAmol Grover subsequent accesses no longer depend on the rcu_dereference(), 60b00aedf9SAmol Grover again possibly resulting in bugs due to misordering. 61b00aedf9SAmol Grover 62b00aedf9SAmol Grover Of course, if "p" is a pointer from rcu_dereference(), and "a" 63b00aedf9SAmol Grover and "b" are integers that happen to be equal, the expression 64b00aedf9SAmol Grover "p+a-b" is safe because its value still necessarily depends on 65b00aedf9SAmol Grover the rcu_dereference(), thus maintaining proper ordering. 66b00aedf9SAmol Grover 67b00aedf9SAmol Grover- If you are using RCU to protect JITed functions, so that the 68b00aedf9SAmol Grover "()" function-invocation operator is applied to a value obtained 69b00aedf9SAmol Grover (directly or indirectly) from rcu_dereference(), you may need to 70b00aedf9SAmol Grover interact directly with the hardware to flush instruction caches. 71b00aedf9SAmol Grover This issue arises on some systems when a newly JITed function is 72b00aedf9SAmol Grover using the same memory that was used by an earlier JITed function. 73b00aedf9SAmol Grover 74b00aedf9SAmol Grover- Do not use the results from relational operators ("==", "!=", 75b00aedf9SAmol Grover ">", ">=", "<", or "<=") when dereferencing. For example, 76b00aedf9SAmol Grover the following (quite strange) code is buggy:: 77b00aedf9SAmol Grover 78b00aedf9SAmol Grover int *p; 79b00aedf9SAmol Grover int *q; 80b00aedf9SAmol Grover 81b00aedf9SAmol Grover ... 82b00aedf9SAmol Grover 83b00aedf9SAmol Grover p = rcu_dereference(gp) 84b00aedf9SAmol Grover q = &global_q; 85b00aedf9SAmol Grover q += p > &oom_p; 86b00aedf9SAmol Grover r1 = *q; /* BUGGY!!! */ 87b00aedf9SAmol Grover 88b00aedf9SAmol Grover As before, the reason this is buggy is that relational operators 89b00aedf9SAmol Grover are often compiled using branches. And as before, although 90b00aedf9SAmol Grover weak-memory machines such as ARM or PowerPC do order stores 91b00aedf9SAmol Grover after such branches, but can speculate loads, which can again 92b00aedf9SAmol Grover result in misordering bugs. 93b00aedf9SAmol Grover 94b00aedf9SAmol Grover- Be very careful about comparing pointers obtained from 95b00aedf9SAmol Grover rcu_dereference() against non-NULL values. As Linus Torvalds 96b00aedf9SAmol Grover explained, if the two pointers are equal, the compiler could 97b00aedf9SAmol Grover substitute the pointer you are comparing against for the pointer 98b00aedf9SAmol Grover obtained from rcu_dereference(). For example:: 99b00aedf9SAmol Grover 100b00aedf9SAmol Grover p = rcu_dereference(gp); 101b00aedf9SAmol Grover if (p == &default_struct) 102b00aedf9SAmol Grover do_default(p->a); 103b00aedf9SAmol Grover 104b00aedf9SAmol Grover Because the compiler now knows that the value of "p" is exactly 105b00aedf9SAmol Grover the address of the variable "default_struct", it is free to 106b00aedf9SAmol Grover transform this code into the following:: 107b00aedf9SAmol Grover 108b00aedf9SAmol Grover p = rcu_dereference(gp); 109b00aedf9SAmol Grover if (p == &default_struct) 110b00aedf9SAmol Grover do_default(default_struct.a); 111b00aedf9SAmol Grover 112b00aedf9SAmol Grover On ARM and Power hardware, the load from "default_struct.a" 113b00aedf9SAmol Grover can now be speculated, such that it might happen before the 114b00aedf9SAmol Grover rcu_dereference(). This could result in bugs due to misordering. 115b00aedf9SAmol Grover 116b00aedf9SAmol Grover However, comparisons are OK in the following cases: 117b00aedf9SAmol Grover 118b00aedf9SAmol Grover - The comparison was against the NULL pointer. If the 119b00aedf9SAmol Grover compiler knows that the pointer is NULL, you had better 120b00aedf9SAmol Grover not be dereferencing it anyway. If the comparison is 121b00aedf9SAmol Grover non-equal, the compiler is none the wiser. Therefore, 122b00aedf9SAmol Grover it is safe to compare pointers from rcu_dereference() 123b00aedf9SAmol Grover against NULL pointers. 124b00aedf9SAmol Grover 125b00aedf9SAmol Grover - The pointer is never dereferenced after being compared. 126b00aedf9SAmol Grover Since there are no subsequent dereferences, the compiler 127b00aedf9SAmol Grover cannot use anything it learned from the comparison 128b00aedf9SAmol Grover to reorder the non-existent subsequent dereferences. 129b00aedf9SAmol Grover This sort of comparison occurs frequently when scanning 130b00aedf9SAmol Grover RCU-protected circular linked lists. 131b00aedf9SAmol Grover 132022d1b35SPaul E. McKenney Note that if the pointer comparison is done outside 133022d1b35SPaul E. McKenney of an RCU read-side critical section, and the pointer 134022d1b35SPaul E. McKenney is never dereferenced, rcu_access_pointer() should be 135022d1b35SPaul E. McKenney used in place of rcu_dereference(). In most cases, 136022d1b35SPaul E. McKenney it is best to avoid accidental dereferences by testing 137022d1b35SPaul E. McKenney the rcu_access_pointer() return value directly, without 138022d1b35SPaul E. McKenney assigning it to a variable. 139022d1b35SPaul E. McKenney 140022d1b35SPaul E. McKenney Within an RCU read-side critical section, there is little 141022d1b35SPaul E. McKenney reason to use rcu_access_pointer(). 142b00aedf9SAmol Grover 143b00aedf9SAmol Grover - The comparison is against a pointer that references memory 144b00aedf9SAmol Grover that was initialized "a long time ago." The reason 145b00aedf9SAmol Grover this is safe is that even if misordering occurs, the 146b00aedf9SAmol Grover misordering will not affect the accesses that follow 147b00aedf9SAmol Grover the comparison. So exactly how long ago is "a long 148b00aedf9SAmol Grover time ago"? Here are some possibilities: 149b00aedf9SAmol Grover 150b00aedf9SAmol Grover - Compile time. 151b00aedf9SAmol Grover 152b00aedf9SAmol Grover - Boot time. 153b00aedf9SAmol Grover 154b00aedf9SAmol Grover - Module-init time for module code. 155b00aedf9SAmol Grover 156b00aedf9SAmol Grover - Prior to kthread creation for kthread code. 157b00aedf9SAmol Grover 158b00aedf9SAmol Grover - During some prior acquisition of the lock that 159b00aedf9SAmol Grover we now hold. 160b00aedf9SAmol Grover 161b00aedf9SAmol Grover - Before mod_timer() time for a timer handler. 162b00aedf9SAmol Grover 163b00aedf9SAmol Grover There are many other possibilities involving the Linux 164b00aedf9SAmol Grover kernel's wide array of primitives that cause code to 165b00aedf9SAmol Grover be invoked at a later time. 166b00aedf9SAmol Grover 167b00aedf9SAmol Grover - The pointer being compared against also came from 168b00aedf9SAmol Grover rcu_dereference(). In this case, both pointers depend 169b00aedf9SAmol Grover on one rcu_dereference() or another, so you get proper 170b00aedf9SAmol Grover ordering either way. 171b00aedf9SAmol Grover 172b00aedf9SAmol Grover That said, this situation can make certain RCU usage 173b00aedf9SAmol Grover bugs more likely to happen. Which can be a good thing, 174b00aedf9SAmol Grover at least if they happen during testing. An example 175b00aedf9SAmol Grover of such an RCU usage bug is shown in the section titled 176b00aedf9SAmol Grover "EXAMPLE OF AMPLIFIED RCU-USAGE BUG". 177b00aedf9SAmol Grover 178b00aedf9SAmol Grover - All of the accesses following the comparison are stores, 179b00aedf9SAmol Grover so that a control dependency preserves the needed ordering. 180b00aedf9SAmol Grover That said, it is easy to get control dependencies wrong. 181b00aedf9SAmol Grover Please see the "CONTROL DEPENDENCIES" section of 182b00aedf9SAmol Grover Documentation/memory-barriers.txt for more details. 183b00aedf9SAmol Grover 184e3879ecdSAkira Yokosawa - The pointers are not equal *and* the compiler does 185b00aedf9SAmol Grover not have enough information to deduce the value of the 186b00aedf9SAmol Grover pointer. Note that the volatile cast in rcu_dereference() 187b00aedf9SAmol Grover will normally prevent the compiler from knowing too much. 188b00aedf9SAmol Grover 189b00aedf9SAmol Grover However, please note that if the compiler knows that the 190b00aedf9SAmol Grover pointer takes on only one of two values, a not-equal 191b00aedf9SAmol Grover comparison will provide exactly the information that the 192b00aedf9SAmol Grover compiler needs to deduce the value of the pointer. 193b00aedf9SAmol Grover 194b00aedf9SAmol Grover- Disable any value-speculation optimizations that your compiler 195b00aedf9SAmol Grover might provide, especially if you are making use of feedback-based 196b00aedf9SAmol Grover optimizations that take data collected from prior runs. Such 197b00aedf9SAmol Grover value-speculation optimizations reorder operations by design. 198b00aedf9SAmol Grover 199b00aedf9SAmol Grover There is one exception to this rule: Value-speculation 200b00aedf9SAmol Grover optimizations that leverage the branch-prediction hardware are 201b00aedf9SAmol Grover safe on strongly ordered systems (such as x86), but not on weakly 202b00aedf9SAmol Grover ordered systems (such as ARM or Power). Choose your compiler 203b00aedf9SAmol Grover command-line options wisely! 204b00aedf9SAmol Grover 205b00aedf9SAmol Grover 206b00aedf9SAmol GroverEXAMPLE OF AMPLIFIED RCU-USAGE BUG 207b00aedf9SAmol Grover---------------------------------- 208b00aedf9SAmol Grover 209b00aedf9SAmol GroverBecause updaters can run concurrently with RCU readers, RCU readers can 210b00aedf9SAmol Groversee stale and/or inconsistent values. If RCU readers need fresh or 211b00aedf9SAmol Groverconsistent values, which they sometimes do, they need to take proper 212b00aedf9SAmol Groverprecautions. To see this, consider the following code fragment:: 213b00aedf9SAmol Grover 214b00aedf9SAmol Grover struct foo { 215b00aedf9SAmol Grover int a; 216b00aedf9SAmol Grover int b; 217b00aedf9SAmol Grover int c; 218b00aedf9SAmol Grover }; 219b00aedf9SAmol Grover struct foo *gp1; 220b00aedf9SAmol Grover struct foo *gp2; 221b00aedf9SAmol Grover 222b00aedf9SAmol Grover void updater(void) 223b00aedf9SAmol Grover { 224b00aedf9SAmol Grover struct foo *p; 225b00aedf9SAmol Grover 226b00aedf9SAmol Grover p = kmalloc(...); 227b00aedf9SAmol Grover if (p == NULL) 228b00aedf9SAmol Grover deal_with_it(); 229b00aedf9SAmol Grover p->a = 42; /* Each field in its own cache line. */ 230b00aedf9SAmol Grover p->b = 43; 231b00aedf9SAmol Grover p->c = 44; 232b00aedf9SAmol Grover rcu_assign_pointer(gp1, p); 233b00aedf9SAmol Grover p->b = 143; 234b00aedf9SAmol Grover p->c = 144; 235b00aedf9SAmol Grover rcu_assign_pointer(gp2, p); 236b00aedf9SAmol Grover } 237b00aedf9SAmol Grover 238b00aedf9SAmol Grover void reader(void) 239b00aedf9SAmol Grover { 240b00aedf9SAmol Grover struct foo *p; 241b00aedf9SAmol Grover struct foo *q; 242b00aedf9SAmol Grover int r1, r2; 243b00aedf9SAmol Grover 244*b33994efSPaul E. McKenney rcu_read_lock(); 245b00aedf9SAmol Grover p = rcu_dereference(gp2); 246b00aedf9SAmol Grover if (p == NULL) 247b00aedf9SAmol Grover return; 248b00aedf9SAmol Grover r1 = p->b; /* Guaranteed to get 143. */ 249b00aedf9SAmol Grover q = rcu_dereference(gp1); /* Guaranteed non-NULL. */ 250b00aedf9SAmol Grover if (p == q) { 251b00aedf9SAmol Grover /* The compiler decides that q->c is same as p->c. */ 252b00aedf9SAmol Grover r2 = p->c; /* Could get 44 on weakly order system. */ 253*b33994efSPaul E. McKenney } else { 254*b33994efSPaul E. McKenney r2 = p->c - r1; /* Unconditional access to p->c. */ 255b00aedf9SAmol Grover } 256*b33994efSPaul E. McKenney rcu_read_unlock(); 257b00aedf9SAmol Grover do_something_with(r1, r2); 258b00aedf9SAmol Grover } 259b00aedf9SAmol Grover 260b00aedf9SAmol GroverYou might be surprised that the outcome (r1 == 143 && r2 == 44) is possible, 261b00aedf9SAmol Groverbut you should not be. After all, the updater might have been invoked 262b00aedf9SAmol Grovera second time between the time reader() loaded into "r1" and the time 263b00aedf9SAmol Groverthat it loaded into "r2". The fact that this same result can occur due 264b00aedf9SAmol Groverto some reordering from the compiler and CPUs is beside the point. 265b00aedf9SAmol Grover 266b00aedf9SAmol GroverBut suppose that the reader needs a consistent view? 267b00aedf9SAmol Grover 268b00aedf9SAmol GroverThen one approach is to use locking, for example, as follows:: 269b00aedf9SAmol Grover 270b00aedf9SAmol Grover struct foo { 271b00aedf9SAmol Grover int a; 272b00aedf9SAmol Grover int b; 273b00aedf9SAmol Grover int c; 274b00aedf9SAmol Grover spinlock_t lock; 275b00aedf9SAmol Grover }; 276b00aedf9SAmol Grover struct foo *gp1; 277b00aedf9SAmol Grover struct foo *gp2; 278b00aedf9SAmol Grover 279b00aedf9SAmol Grover void updater(void) 280b00aedf9SAmol Grover { 281b00aedf9SAmol Grover struct foo *p; 282b00aedf9SAmol Grover 283b00aedf9SAmol Grover p = kmalloc(...); 284b00aedf9SAmol Grover if (p == NULL) 285b00aedf9SAmol Grover deal_with_it(); 286b00aedf9SAmol Grover spin_lock(&p->lock); 287b00aedf9SAmol Grover p->a = 42; /* Each field in its own cache line. */ 288b00aedf9SAmol Grover p->b = 43; 289b00aedf9SAmol Grover p->c = 44; 290b00aedf9SAmol Grover spin_unlock(&p->lock); 291b00aedf9SAmol Grover rcu_assign_pointer(gp1, p); 292b00aedf9SAmol Grover spin_lock(&p->lock); 293b00aedf9SAmol Grover p->b = 143; 294b00aedf9SAmol Grover p->c = 144; 295b00aedf9SAmol Grover spin_unlock(&p->lock); 296b00aedf9SAmol Grover rcu_assign_pointer(gp2, p); 297b00aedf9SAmol Grover } 298b00aedf9SAmol Grover 299b00aedf9SAmol Grover void reader(void) 300b00aedf9SAmol Grover { 301b00aedf9SAmol Grover struct foo *p; 302b00aedf9SAmol Grover struct foo *q; 303b00aedf9SAmol Grover int r1, r2; 304b00aedf9SAmol Grover 305*b33994efSPaul E. McKenney rcu_read_lock(); 306b00aedf9SAmol Grover p = rcu_dereference(gp2); 307b00aedf9SAmol Grover if (p == NULL) 308b00aedf9SAmol Grover return; 309b00aedf9SAmol Grover spin_lock(&p->lock); 310b00aedf9SAmol Grover r1 = p->b; /* Guaranteed to get 143. */ 311b00aedf9SAmol Grover q = rcu_dereference(gp1); /* Guaranteed non-NULL. */ 312b00aedf9SAmol Grover if (p == q) { 313b00aedf9SAmol Grover /* The compiler decides that q->c is same as p->c. */ 314b00aedf9SAmol Grover r2 = p->c; /* Locking guarantees r2 == 144. */ 315*b33994efSPaul E. McKenney } else { 316*b33994efSPaul E. McKenney spin_lock(&q->lock); 317*b33994efSPaul E. McKenney r2 = q->c - r1; 318*b33994efSPaul E. McKenney spin_unlock(&q->lock); 319b00aedf9SAmol Grover } 320*b33994efSPaul E. McKenney rcu_read_unlock(); 321b00aedf9SAmol Grover spin_unlock(&p->lock); 322b00aedf9SAmol Grover do_something_with(r1, r2); 323b00aedf9SAmol Grover } 324b00aedf9SAmol Grover 325b00aedf9SAmol GroverAs always, use the right tool for the job! 326b00aedf9SAmol Grover 327b00aedf9SAmol Grover 328b00aedf9SAmol GroverEXAMPLE WHERE THE COMPILER KNOWS TOO MUCH 329b00aedf9SAmol Grover----------------------------------------- 330b00aedf9SAmol Grover 331b00aedf9SAmol GroverIf a pointer obtained from rcu_dereference() compares not-equal to some 332b00aedf9SAmol Groverother pointer, the compiler normally has no clue what the value of the 333b00aedf9SAmol Groverfirst pointer might be. This lack of knowledge prevents the compiler 334b00aedf9SAmol Groverfrom carrying out optimizations that otherwise might destroy the ordering 335b00aedf9SAmol Groverguarantees that RCU depends on. And the volatile cast in rcu_dereference() 336b00aedf9SAmol Grovershould prevent the compiler from guessing the value. 337b00aedf9SAmol Grover 338b00aedf9SAmol GroverBut without rcu_dereference(), the compiler knows more than you might 339b00aedf9SAmol Groverexpect. Consider the following code fragment:: 340b00aedf9SAmol Grover 341b00aedf9SAmol Grover struct foo { 342b00aedf9SAmol Grover int a; 343b00aedf9SAmol Grover int b; 344b00aedf9SAmol Grover }; 345b00aedf9SAmol Grover static struct foo variable1; 346b00aedf9SAmol Grover static struct foo variable2; 347b00aedf9SAmol Grover static struct foo *gp = &variable1; 348b00aedf9SAmol Grover 349b00aedf9SAmol Grover void updater(void) 350b00aedf9SAmol Grover { 351b00aedf9SAmol Grover initialize_foo(&variable2); 352b00aedf9SAmol Grover rcu_assign_pointer(gp, &variable2); 353b00aedf9SAmol Grover /* 354b00aedf9SAmol Grover * The above is the only store to gp in this translation unit, 355b00aedf9SAmol Grover * and the address of gp is not exported in any way. 356b00aedf9SAmol Grover */ 357b00aedf9SAmol Grover } 358b00aedf9SAmol Grover 359b00aedf9SAmol Grover int reader(void) 360b00aedf9SAmol Grover { 361b00aedf9SAmol Grover struct foo *p; 362b00aedf9SAmol Grover 363b00aedf9SAmol Grover p = gp; 364b00aedf9SAmol Grover barrier(); 365b00aedf9SAmol Grover if (p == &variable1) 366b00aedf9SAmol Grover return p->a; /* Must be variable1.a. */ 367b00aedf9SAmol Grover else 368b00aedf9SAmol Grover return p->b; /* Must be variable2.b. */ 369b00aedf9SAmol Grover } 370b00aedf9SAmol Grover 371b00aedf9SAmol GroverBecause the compiler can see all stores to "gp", it knows that the only 372b00aedf9SAmol Groverpossible values of "gp" are "variable1" on the one hand and "variable2" 373b00aedf9SAmol Groveron the other. The comparison in reader() therefore tells the compiler 374b00aedf9SAmol Groverthe exact value of "p" even in the not-equals case. This allows the 375b00aedf9SAmol Grovercompiler to make the return values independent of the load from "gp", 376b00aedf9SAmol Groverin turn destroying the ordering between this load and the loads of the 377b00aedf9SAmol Groverreturn values. This can result in "p->b" returning pre-initialization 378*b33994efSPaul E. McKenneygarbage values on weakly ordered systems. 379b00aedf9SAmol Grover 380e3879ecdSAkira YokosawaIn short, rcu_dereference() is *not* optional when you are going to 381b00aedf9SAmol Groverdereference the resulting pointer. 382b00aedf9SAmol Grover 383b00aedf9SAmol Grover 384b00aedf9SAmol GroverWHICH MEMBER OF THE rcu_dereference() FAMILY SHOULD YOU USE? 385b00aedf9SAmol Grover------------------------------------------------------------ 386b00aedf9SAmol Grover 387b00aedf9SAmol GroverFirst, please avoid using rcu_dereference_raw() and also please avoid 388b00aedf9SAmol Groverusing rcu_dereference_check() and rcu_dereference_protected() with a 389b00aedf9SAmol Groversecond argument with a constant value of 1 (or true, for that matter). 390b00aedf9SAmol GroverWith that caution out of the way, here is some guidance for which 391b00aedf9SAmol Grovermember of the rcu_dereference() to use in various situations: 392b00aedf9SAmol Grover 393b00aedf9SAmol Grover1. If the access needs to be within an RCU read-side critical 394b00aedf9SAmol Grover section, use rcu_dereference(). With the new consolidated 395b00aedf9SAmol Grover RCU flavors, an RCU read-side critical section is entered 396b00aedf9SAmol Grover using rcu_read_lock(), anything that disables bottom halves, 397b00aedf9SAmol Grover anything that disables interrupts, or anything that disables 398b00aedf9SAmol Grover preemption. 399b00aedf9SAmol Grover 400b00aedf9SAmol Grover2. If the access might be within an RCU read-side critical section 401b00aedf9SAmol Grover on the one hand, or protected by (say) my_lock on the other, 402b00aedf9SAmol Grover use rcu_dereference_check(), for example:: 403b00aedf9SAmol Grover 404b00aedf9SAmol Grover p1 = rcu_dereference_check(p->rcu_protected_pointer, 405b00aedf9SAmol Grover lockdep_is_held(&my_lock)); 406b00aedf9SAmol Grover 407b00aedf9SAmol Grover 408b00aedf9SAmol Grover3. If the access might be within an RCU read-side critical section 409b00aedf9SAmol Grover on the one hand, or protected by either my_lock or your_lock on 410b00aedf9SAmol Grover the other, again use rcu_dereference_check(), for example:: 411b00aedf9SAmol Grover 412b00aedf9SAmol Grover p1 = rcu_dereference_check(p->rcu_protected_pointer, 413b00aedf9SAmol Grover lockdep_is_held(&my_lock) || 414b00aedf9SAmol Grover lockdep_is_held(&your_lock)); 415b00aedf9SAmol Grover 416b00aedf9SAmol Grover4. If the access is on the update side, so that it is always protected 417b00aedf9SAmol Grover by my_lock, use rcu_dereference_protected():: 418b00aedf9SAmol Grover 419b00aedf9SAmol Grover p1 = rcu_dereference_protected(p->rcu_protected_pointer, 420b00aedf9SAmol Grover lockdep_is_held(&my_lock)); 421b00aedf9SAmol Grover 422b00aedf9SAmol Grover This can be extended to handle multiple locks as in #3 above, 423b00aedf9SAmol Grover and both can be extended to check other conditions as well. 424b00aedf9SAmol Grover 425b00aedf9SAmol Grover5. If the protection is supplied by the caller, and is thus unknown 426b00aedf9SAmol Grover to this code, that is the rare case when rcu_dereference_raw() 427b00aedf9SAmol Grover is appropriate. In addition, rcu_dereference_raw() might be 428b00aedf9SAmol Grover appropriate when the lockdep expression would be excessively 429b00aedf9SAmol Grover complex, except that a better approach in that case might be to 430b00aedf9SAmol Grover take a long hard look at your synchronization design. Still, 431b00aedf9SAmol Grover there are data-locking cases where any one of a very large number 432b00aedf9SAmol Grover of locks or reference counters suffices to protect the pointer, 433b00aedf9SAmol Grover so rcu_dereference_raw() does have its place. 434b00aedf9SAmol Grover 435b00aedf9SAmol Grover However, its place is probably quite a bit smaller than one 436b00aedf9SAmol Grover might expect given the number of uses in the current kernel. 437b00aedf9SAmol Grover Ditto for its synonym, rcu_dereference_check( ... , 1), and 438b00aedf9SAmol Grover its close relative, rcu_dereference_protected(... , 1). 439b00aedf9SAmol Grover 440b00aedf9SAmol Grover 441b00aedf9SAmol GroverSPARSE CHECKING OF RCU-PROTECTED POINTERS 442b00aedf9SAmol Grover----------------------------------------- 443b00aedf9SAmol Grover 444*b33994efSPaul E. McKenneyThe sparse static-analysis tool checks for non-RCU access to RCU-protected 445b00aedf9SAmol Groverpointers, which can result in "interesting" bugs due to compiler 446b00aedf9SAmol Groveroptimizations involving invented loads and perhaps also load tearing. 447b00aedf9SAmol GroverFor example, suppose someone mistakenly does something like this:: 448b00aedf9SAmol Grover 449b00aedf9SAmol Grover p = q->rcu_protected_pointer; 450b00aedf9SAmol Grover do_something_with(p->a); 451b00aedf9SAmol Grover do_something_else_with(p->b); 452b00aedf9SAmol Grover 453b00aedf9SAmol GroverIf register pressure is high, the compiler might optimize "p" out 454b00aedf9SAmol Groverof existence, transforming the code to something like this:: 455b00aedf9SAmol Grover 456b00aedf9SAmol Grover do_something_with(q->rcu_protected_pointer->a); 457b00aedf9SAmol Grover do_something_else_with(q->rcu_protected_pointer->b); 458b00aedf9SAmol Grover 459b00aedf9SAmol GroverThis could fatally disappoint your code if q->rcu_protected_pointer 460b00aedf9SAmol Groverchanged in the meantime. Nor is this a theoretical problem: Exactly 461b00aedf9SAmol Groverthis sort of bug cost Paul E. McKenney (and several of his innocent 462b00aedf9SAmol Grovercolleagues) a three-day weekend back in the early 1990s. 463b00aedf9SAmol Grover 464b00aedf9SAmol GroverLoad tearing could of course result in dereferencing a mashup of a pair 465b00aedf9SAmol Groverof pointers, which also might fatally disappoint your code. 466b00aedf9SAmol Grover 467b00aedf9SAmol GroverThese problems could have been avoided simply by making the code instead 468b00aedf9SAmol Groverread as follows:: 469b00aedf9SAmol Grover 470b00aedf9SAmol Grover p = rcu_dereference(q->rcu_protected_pointer); 471b00aedf9SAmol Grover do_something_with(p->a); 472b00aedf9SAmol Grover do_something_else_with(p->b); 473b00aedf9SAmol Grover 474b00aedf9SAmol GroverUnfortunately, these sorts of bugs can be extremely hard to spot during 475b00aedf9SAmol Groverreview. This is where the sparse tool comes into play, along with the 476b00aedf9SAmol Grover"__rcu" marker. If you mark a pointer declaration, whether in a structure 477b00aedf9SAmol Groveror as a formal parameter, with "__rcu", which tells sparse to complain if 478b00aedf9SAmol Groverthis pointer is accessed directly. It will also cause sparse to complain 479b00aedf9SAmol Groverif a pointer not marked with "__rcu" is accessed using rcu_dereference() 480b00aedf9SAmol Groverand friends. For example, ->rcu_protected_pointer might be declared as 481b00aedf9SAmol Groverfollows:: 482b00aedf9SAmol Grover 483b00aedf9SAmol Grover struct foo __rcu *rcu_protected_pointer; 484b00aedf9SAmol Grover 485b00aedf9SAmol GroverUse of "__rcu" is opt-in. If you choose not to use it, then you should 486b00aedf9SAmol Groverignore the sparse warnings. 487