Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull locking/atomics update from Thomas Gleixner:
"The locking, atomics and memory model brains delive

Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull locking/atomics update from Thomas Gleixner:
"The locking, atomics and memory model brains delivered:

- A larger update to the atomics code which reworks the ordering
barriers, consolidates the atomic primitives, provides the new
atomic64_fetch_add_unless() primitive and cleans up the include
hell.

- Simplify cmpxchg() instrumentation and add instrumentation for
xchg() and cmpxchg_double().

- Updates to the memory model and documentation"

* 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (48 commits)
locking/atomics: Rework ordering barriers
locking/atomics: Instrument cmpxchg_double*()
locking/atomics: Instrument xchg()
locking/atomics: Simplify cmpxchg() instrumentation
locking/atomics/x86: Reduce arch_cmpxchg64*() instrumentation
tools/memory-model: Rename litmus tests to comply to norm7
tools/memory-model/Documentation: Fix typo, smb->smp
sched/Documentation: Update wake_up() & co. memory-barrier guarantees
locking/spinlock, sched/core: Clarify requirements for smp_mb__after_spinlock()
sched/core: Use smp_mb() in wake_woken_function()
tools/memory-model: Add informal LKMM documentation to MAINTAINERS
locking/atomics/Documentation: Describe atomic_set() as a write operation
tools/memory-model: Make scripts executable
tools/memory-model: Remove ACCESS_ONCE() from model
tools/memory-model: Remove ACCESS_ONCE() from recipes
locking/memory-barriers.txt/kokr: Update Korean translation to fix broken DMA vs. MMIO ordering example
MAINTAINERS: Add Daniel Lustig as an LKMM reviewer
tools/memory-model: Fix ISA2+pooncelock+pooncelock+pombonce name
tools/memory-model: Add litmus test for full multicopy atomicity
locking/refcount: Always allow checked forms
...

show more ...

tools/memory-model: Rename litmus tests to comply to norm7

norm7 produces the 'normalized' name of a litmus test, when the test
can be generated from a single cycle that passes through each process

tools/memory-model: Rename litmus tests to comply to norm7

norm7 produces the 'normalized' name of a litmus test, when the test
can be generated from a single cycle that passes through each process
exactly once. The commit renames such tests in order to comply to the
naming scheme implemented by this tool.

Signed-off-by: Andrea Parri <andrea.parri@amarulasolutions.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Cc: Akira Yokosawa <akiyks@gmail.com>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Jade Alglave <j.alglave@ucl.ac.uk>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luc Maranget <luc.maranget@inria.fr>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Will Deacon <will.deacon@arm.com>
Cc: linux-arch@vger.kernel.org
Cc: parri.andrea@gmail.com
Link: http://lkml.kernel.org/r/20180716180605.16115-14-paulmck@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>

show more ...

tools/memory-model: Remove ACCESS_ONCE() from recipes

Since commit:

b899a850431e2dd0 ("compiler.h: Remove ACCESS_ONCE()")

... there has been no definition of ACCESS_ONCE() in the kernel tree,
an

tools/memory-model: Remove ACCESS_ONCE() from recipes

Since commit:

b899a850431e2dd0 ("compiler.h: Remove ACCESS_ONCE()")

... there has been no definition of ACCESS_ONCE() in the kernel tree,
and it has been necessary to use READ_ONCE() or WRITE_ONCE() instead.

Let's update the exmaples in recipes.txt likewise for consistency, using
READ_ONCE() for reads.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Andrea Parri <andrea.parri@amarulasolutions.com>
Cc: Akira Yokosawa <akiyks@gmail.com>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Jade Alglave <j.alglave@ucl.ac.uk>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luc Maranget <luc.maranget@inria.fr>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Will Deacon <will.deacon@arm.com>
Cc: linux-arch@vger.kernel.org
Cc: parri.andrea@gmail.com
Link: http://lkml.kernel.org/r/20180716180605.16115-5-paulmck@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>

show more ...

Merge branch 'next' into for-linus

Prepare input updates for 4.18 merge window.

Merge branches 'clk-warn', 'clk-core', 'clk-spear' and 'clk-qcom-msm8998' into clk-next

* clk-warn:
clk: Print the clock name and warning cause

* clk-core:
clk: Remove clk_init_cb typedef

* cl

Merge branches 'clk-warn', 'clk-core', 'clk-spear' and 'clk-qcom-msm8998' into clk-next

* clk-warn:
clk: Print the clock name and warning cause

* clk-core:
clk: Remove clk_init_cb typedef

* clk-spear:
clk: spear: fix WDT clock definition on SPEAr600

* clk-qcom-msm8998:
clk: qcom: Add MSM8998 Global Clock Control (GCC) driver

show more ...

Merge tag 'v4.17-rc6' into next

Sync up with mainline to bring in Atmel controller changes for Caroline.

Merge branch 'drm-intel-next-queued' into gvt-next

Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>

Merge remote-tracking branch 'drm/drm-next' into drm-misc-next

drm-misc-next is still based on v4.16-rc7, and was getting a bit stale.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.inte

Merge remote-tracking branch 'drm/drm-next' into drm-misc-next

drm-misc-next is still based on v4.16-rc7, and was getting a bit stale.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>

show more ...

Merge drm/drm-next into drm-intel-next-queued

Need d224985a5e31 ("sched/wait, drivers/drm: Convert wait_on_atomic_t()
usage to the new wait_var_event() API") in dinq to be able to fix
https://bugs.f

Merge drm/drm-next into drm-intel-next-queued

Need d224985a5e31 ("sched/wait, drivers/drm: Convert wait_on_atomic_t()
usage to the new wait_var_event() API") in dinq to be able to fix
https://bugs.freedesktop.org/show_bug.cgi?id=106085.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>

show more ...

Merge tag 'v4.17-rc3' into apparmor-next

Linux v4.17-rc3

Merge in v4.17 for LSM updates

Signed-off-by: John Johansen <john.johansen@canonical.com>

Merge tag 'v4.17-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux into fbdev-for-next

Linux 4.17-rc2

Merge tag 'v4.17-rc2' into next-general

Sync to Linux 4.17-rc2 for developers.

Back merge 'drm-intel-fixes' into gvt-fixes

Need for 4.17-rc1

Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Merge branch 'WIP.x86/asm' into x86/urgent, because the topic is ready

Signed-off-by: Ingo Molnar <mingo@kernel.org>

Merge branch 'linus' into x86/pti to pick up upstream changes

Signed-off-by: Ingo Molnar <mingo@kernel.org>

Merge branch 'linus' into sched/urgent, to pick up fixes and updates

Signed-off-by: Ingo Molnar <mingo@kernel.org>

Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull locking updates from Ingo Molnar:
"The main changes in the locking subsystem in this cycle were:

Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull locking updates from Ingo Molnar:
"The main changes in the locking subsystem in this cycle were:

- Add the Linux Kernel Memory Consistency Model (LKMM) subsystem,
which is an an array of tools in tools/memory-model/ that formally
describe the Linux memory coherency model (a.k.a.
Documentation/memory-barriers.txt), and also produce 'litmus tests'
in form of kernel code which can be directly executed and tested.

Here's a high level background article about an earlier version of
this work on LWN.net:

https://lwn.net/Articles/718628/

The design principles:

"There is reason to believe that Documentation/memory-barriers.txt
could use some help, and a major purpose of this patch is to
provide that help in the form of a design-time tool that can
produce all valid executions of a small fragment of concurrent
Linux-kernel code, which is called a "litmus test". This tool's
functionality is roughly similar to a full state-space search.
Please note that this is a design-time tool, not useful for
regression testing. However, we hope that the underlying
Linux-kernel memory model will be incorporated into other tools
capable of analyzing large bodies of code for regression-testing
purposes."

[...]

"A second tool is klitmus7, which converts litmus tests to
loadable kernel modules for direct testing. As with herd7, the
klitmus7 code is freely available from

http://diy.inria.fr/sources/index.html

(and via "git" at https://github.com/herd/herdtools7)"

[...]

Credits go to:

"This patch was the result of a most excellent collaboration
founded by Jade Alglave and also including Alan Stern, Andrea
Parri, and Luc Maranget."

... and to the gents listed in the MAINTAINERS entry:

LINUX KERNEL MEMORY CONSISTENCY MODEL (LKMM)
M: Alan Stern <stern@rowland.harvard.edu>
M: Andrea Parri <parri.andrea@gmail.com>
M: Will Deacon <will.deacon@arm.com>
M: Peter Zijlstra <peterz@infradead.org>
M: Boqun Feng <boqun.feng@gmail.com>
M: Nicholas Piggin <npiggin@gmail.com>
M: David Howells <dhowells@redhat.com>
M: Jade Alglave <j.alglave@ucl.ac.uk>
M: Luc Maranget <luc.maranget@inria.fr>
M: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>

The LKMM project already found several bugs in Linux locking
primitives and improved the understanding and the documentation of
the Linux memory model all around.

- Add KASAN instrumentation to atomic APIs (Dmitry Vyukov)

- Add RWSEM API debugging and reorganize the lock debugging Kconfig
(Waiman Long)

- ... misc cleanups and other smaller changes"

* 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (31 commits)
locking/Kconfig: Restructure the lock debugging menu
locking/Kconfig: Add LOCK_DEBUGGING_SUPPORT to make it more readable
locking/rwsem: Add DEBUG_RWSEMS to look for lock/unlock mismatches
lockdep: Make the lock debug output more useful
locking/rtmutex: Handle non enqueued waiters gracefully in remove_waiter()
locking/atomic, asm-generic, x86: Add comments for atomic instrumentation
locking/atomic, asm-generic: Add KASAN instrumentation to atomic operations
locking/atomic/x86: Switch atomic.h to use atomic-instrumented.h
locking/atomic, asm-generic: Add asm-generic/atomic-instrumented.h
locking/xchg/alpha: Remove superfluous memory barriers from the _local() variants
tools/memory-model: Finish the removal of rb-dep, smp_read_barrier_depends(), and lockless_dereference()
tools/memory-model: Add documentation of new litmus test
tools/memory-model: Remove mention of docker/gentoo image
locking/memory-barriers: De-emphasize smp_read_barrier_depends() some more
locking/lockdep: Show unadorned pointers
mutex: Drop linkage.h from mutex.h
tools/memory-model: Remove rb-dep, smp_read_barrier_depends, and lockless_dereference
tools/memory-model: Convert underscores to hyphens
tools/memory-model: Add a S lock-based external-view litmus test
tools/memory-model: Add required herd7 version to README file
...

show more ...

Merge branch 'lkmm-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into locking/urgent

Pull the "Linux kernel memory model" tooling implementation from Paul E. McKenney

Merge branch 'lkmm-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into locking/urgent

Pull the "Linux kernel memory model" tooling implementation from Paul E. McKenney:

'This pull request contains a single commit that adds a memory model to
the tools directory. This memory model can (roughly speaking) be thought
of as an automated version of memory-barriers.txt. It is written in the
"cat" language, which is executable by the externally provided "herd7"
simulator, which exhaustively explores the state space of small litmus
tests.'

Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra <peterz@infradead.org>

show more ...

Automate memory-barriers.txt; provide Linux-kernel memory model

There is some reason to believe that Documentation/memory-barriers.txt
could use some help, and a major purpose of this patch is to pr

Automate memory-barriers.txt; provide Linux-kernel memory model

There is some reason to believe that Documentation/memory-barriers.txt
could use some help, and a major purpose of this patch is to provide
that help in the form of a design-time tool that can produce all valid
executions of a small fragment of concurrent Linux-kernel code, which is
called a "litmus test". This tool's functionality is roughly similar to
a full state-space search. Please note that this is a design-time tool,
not useful for regression testing. However, we hope that the underlying
Linux-kernel memory model will be incorporated into other tools capable
of analyzing large bodies of code for regression-testing purposes.

The main tool is herd7, together with the linux-kernel.bell,
linux-kernel.cat, linux-kernel.cfg, linux-kernel.def, and lock.cat files
added by this patch. The herd7 executable takes the other files as input,
and all of these files collectively define the Linux-kernel memory memory
model. A brief description of each of these other files is provided
in the README file. Although this tool does have its limitations,
which are documented in the README file, it does improve on the version
reported on in the LWN series (https://lwn.net/Articles/718628/ and
https://lwn.net/Articles/720550/) by supporting locking and arithmetic,
including a much wider variety of read-modify-write atomic operations.
Please note that herd7 is not part of this submission, but is freely
available from http://diy.inria.fr/sources/index.html (and via "git"
at https://github.com/herd/herdtools7).

A second tool is klitmus7, which converts litmus tests to loadable
kernel modules for direct testing. As with herd7, the klitmus7
code is freely available from http://diy.inria.fr/sources/index.html
(and via "git" at https://github.com/herd/herdtools7).

Of course, litmus tests are not always the best way to fully understand a
memory model, so this patch also includes Documentation/explanation.txt,
which describes the memory model in detail. In addition,
Documentation/recipes.txt provides example known-good and known-bad use
cases for those who prefer working by example.

This patch also includes a few sample litmus tests, and a great many
more litmus tests are available at https://github.com/paulmckrcu/litmus.

This patch was the result of a most excellent collaboration founded
by Jade Alglave and also including Alan Stern, Andrea Parri, and Luc
Maranget. For more details on the history of this collaboration, please
refer to the Linux-kernel memory model presentations at 2016 LinuxCon EU,
2016 Kernel Summit, 2016 Linux Plumbers Conference, 2017 linux.conf.au,
or 2017 Linux Plumbers Conference microconference. However, one aspect
of the history does bear repeating due to weak copyright tracking earlier
in this project, which extends back to early 2015. This weakness came
to light in late 2017 after an LKMM presentation by Paul in which an
audience member noted the similarity of some LKMM code to code in early
published papers. This prompted a copyright review.

From Alan Stern:

To say that the model was mine is not entirely accurate.
Pieces of it (especially the Scpv and Atomic axioms) were taken
directly from Jade's models. And of course the Happens-before
and Propagation relations and axioms were heavily based on
Jade and Luc's work, even though they weren't identical to the
earlier versions. Only the RCU portion was completely original.

. . .

One can make a much better case that I wrote the bulk of lock.cat.
However, it was inspired by Luc's earlier version (and still
shares some elements in common), and of course it benefited from
feedback and testing from all members of our group.

The model prior to Alan's was Luc Maranget's. From Luc:

I totally agree on Alan Stern's account of the linux kernel model
genesis. I thank him for his acknowledgments of my participation
to previous model drafts. I'd like to complete Alan Stern's
statement: any bell cat code I have written has its roots in
discussions with Jade Alglave and Paul McKenney. Moreover I
have borrowed cat and bell code written by Jade Alglave freely.

This copyright review therefore resulted in late adds to the copyright
statements of several files.

Discussion of v1 has raised several issues, which we do not believe should
block acceptance given that this level of change will be ongoing, just
as it has been with memory-barriers.txt:

o Under what conditions should ordering provided by pure locking
be seen by CPUs not holding the relevant lock(s)? In particular,
should the message-passing pattern be forbidden?

o Should examples involving C11 release sequences be forbidden?
Note that this C11 is still a moving target for this issue:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0735r0.html

o Some details of the handling of internal dependencies for atomic
read-modify-write atomic operations are still subject to debate.

o Changes recently accepted into mainline greatly reduce the need
to handle DEC Alpha as a special case. These changes add an
smp_read_barrier_depends() to READ_ONCE(), thus causing Alpha
to respect ordering of dependent reads. If these changes stick,
the memory model can be simplified accordingly.

o Will changes be required to accommodate RISC-V?

Differences from v1:
(http://lkml.kernel.org/r/20171113184031.GA26302@linux.vnet.ibm.com)

o Add SPDX notations to .bell and .cat files, replacing
textual license statements.

o Add reference to upcoming ASPLOS paper to .bell and .cat files.

o Updated identifier names in .bell and .cat files to match those
used in the ASPLOS paper.

o Updates to READMEs and other documentation based on review
feedback.

o Added a memory-ordering cheatsheet.

o Update sigs to new Co-Developed-by and add acks and
reviewed-bys.

o Simplify rules detecting nested RCU read-side critical sections.

o Update copyright statements as noted above.

Co-Developed-by: Alan Stern <stern@rowland.harvard.edu>
Co-Developed-by: Andrea Parri <parri.andrea@gmail.com>
Co-Developed-by: Jade Alglave <j.alglave@ucl.ac.uk>
Co-Developed-by: Luc Maranget <luc.maranget@inria.fr>
Co-Developed-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Signed-off-by: Jade Alglave <j.alglave@ucl.ac.uk>
Signed-off-by: Luc Maranget <luc.maranget@inria.fr>
Signed-off-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Reviewed-by: Boqun Feng <boqun.feng@gmail.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Acked-by: Nicholas Piggin <npiggin@gmail.com>
Acked-by: David Howells <dhowells@redhat.com>
Acked-by: "Reshetova, Elena" <elena.reshetova@intel.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Akira Yokosawa <akiyks@gmail.com>
Cc: <linux-arch@vger.kernel.org>

show more ...