ARM: assembler: introduce adr_l, ldr_l and str_l macros

commit 0b1674638a5c69cbace63278625c199100955490 upstream.

Like arm64, ARM supports position independent code sequences that

ARM: assembler: introduce adr_l, ldr_l and str_l macros

commit 0b1674638a5c69cbace63278625c199100955490 upstream.

Like arm64, ARM supports position independent code sequences that
produce symbol references with a greater reach than the ordinary
adr/ldr instructions. Since on ARM, the adrl pseudo-instruction is
only supported in ARM mode (and not at all when using Clang), having
a adr_l macro like we do on arm64 is useful, and increases symmetry
as well.

Currently, we use open coded instruction sequences involving literals
and arithmetic operations. Instead, we can use movw/movt pairs on v7
CPUs, circumventing the D-cache entirely.

E.g., on v7+ CPUs, we can emit a PC-relative reference as follows:

movw <reg>, #:lower16:<sym> - (1f + 8)
movt <reg>, #:upper16:<sym> - (1f + 8)
1: add <reg>, <reg>, pc

For older CPUs, we can emit the literal into a subsection, allowing it
to be emitted out of line while retaining the ability to perform
arithmetic on label offsets.

E.g., on pre-v7 CPUs, we can emit a PC-relative reference as follows:

ldr <reg>, 2f
1: add <reg>, <reg>, pc
.subsection 1
2: .long <sym> - (1b + 8)
.previous

This is allowed by the assembler because, unlike ordinary sections,
subsections are combined into a single section in the object file, and
so the label references are not true cross-section references that are
visible as relocations. (Subsections have been available in binutils
since 2004 at least, so they should not cause any issues with older
toolchains.)

So use the above to implement the macros mov_l, adr_l, ldr_l and str_l,
all of which will use movw/movt pairs on v7 and later CPUs, and use
PC-relative literals otherwise.

Reviewed-by: Nicolas Pitre <nico@fluxnic.net>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm

Pull ARM updates from Russell King:

- remove a now unnecessary usage of the KERNEL_DS for
sys_oabi_epoll_ct

Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm

Pull ARM updates from Russell King:

- remove a now unnecessary usage of the KERNEL_DS for
sys_oabi_epoll_ctl()

- update my email address in a number of drivers

- decompressor EFI updates from Ard Biesheuvel

- module unwind section handling updates

- sparsemem Kconfig cleanups

- make act_mm macro respect THREAD_SIZE

* tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm:
ARM: 8980/1: Allow either FLATMEM or SPARSEMEM on the multiplatform build
ARM: 8979/1: Remove redundant ARCH_SPARSEMEM_DEFAULT setting
ARM: 8978/1: mm: make act_mm() respect THREAD_SIZE
ARM: decompressor: run decompressor in place if loaded via UEFI
ARM: decompressor: move GOT into .data for EFI enabled builds
ARM: decompressor: defer loading of the contents of the LC0 structure
ARM: decompressor: split off _edata and stack base into separate object
ARM: decompressor: move headroom variable out of LC0
ARM: 8976/1: module: allow arch overrides for .init section names
ARM: 8975/1: module: fix handling of unwind init sections
ARM: 8974/1: use SPARSMEM_STATIC when SPARSEMEM is enabled
ARM: 8971/1: replace the sole use of a symbol with its definition
ARM: 8969/1: decompressor: simplify libfdt builds
Update rmk's email address in various drivers
ARM: compat: remove KERNEL_DS usage in sys_oabi_epoll_ctl()

show more ...

ARM: uaccess: consolidate uaccess asm to asm/uaccess-asm.h

Consolidate the user access assembly code to asm/uaccess-asm.h. This
moves the csdb, check_uaccess, uaccess_mask_range_ptr, ua

ARM: uaccess: consolidate uaccess asm to asm/uaccess-asm.h

Consolidate the user access assembly code to asm/uaccess-asm.h. This
moves the csdb, check_uaccess, uaccess_mask_range_ptr, uaccess_enable,
uaccess_disable, uaccess_save, uaccess_restore macros, and creates two
new ones for exception entry and exit - uaccess_entry and uaccess_exit.

This makes the uaccess_save and uaccess_restore macros private to
asm/uaccess-asm.h.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

show more ...

ARM: 8971/1: replace the sole use of a symbol with its definition

ALT_UP_B macro sets symbol up_b_offset via .equ to an expression
involving another symbol. The macro gets expanded twice

ARM: 8971/1: replace the sole use of a symbol with its definition

ALT_UP_B macro sets symbol up_b_offset via .equ to an expression
involving another symbol. The macro gets expanded twice when
arch/arm/kernel/sleep.S is assembled, creating a scenario where
up_b_offset is set to another expression involving symbols while its
current value is based on symbols. LLVM integrated assembler does not
allow such cases, and based on the documentation of binutils, "Values
that are based on expressions involving other symbols are allowed, but
some targets may restrict this to only being done once per assembly", so
it may be better to avoid such cases as it is not clearly stated which
targets should support or disallow them. The fix in this case is simple,
as up_b_offset has only one use, so we can replace the use with the
definition and get rid of up_b_offset.

Link:https://github.com/ClangBuiltLinux/linux/issues/920

Reviewed-by: Stefan Agner <stefan@agner.ch>

Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Jian Cai <caij2003@gmail.com>
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

show more ...

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500

Based on 2 normalized pattern(s):

this program is free software you can redistribute it and or modify
it u

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500

Based on 2 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation #

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-only

has been chosen to replace the boilerplate/reference in 4122 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Enrico Weigelt <info@metux.net>
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

ARM: 8843/1: use unified assembler in headers

Use unified assembler syntax (UAL) in headers. Divided syntax is
considered deprecated. This will also allow to build the kernel
using L

ARM: 8843/1: use unified assembler in headers

Use unified assembler syntax (UAL) in headers. Divided syntax is
considered deprecated. This will also allow to build the kernel
using LLVM's integrated assembler.

Signed-off-by: Stefan Agner <stefan@agner.ch>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

show more ...

ARM: 8812/1: Optimise copy_{from/to}_user for !CPU_USE_DOMAINS

ARMv6+ processors do not use CONFIG_CPU_USE_DOMAINS and use privileged
ldr/str instructions in copy_{from/to}_user. They a

ARM: 8812/1: Optimise copy_{from/to}_user for !CPU_USE_DOMAINS

ARMv6+ processors do not use CONFIG_CPU_USE_DOMAINS and use privileged
ldr/str instructions in copy_{from/to}_user. They are currently
unnecessarily using single ldr/str instructions and can use ldm/stm
instructions instead like memcpy does (but with appropriate fixup
tables).

This speeds up a "dd if=foo of=bar bs=32k" on a tmpfs filesystem by
about 4% on my Cortex-A9.

before:134217728 bytes (128.0MB) copied, 0.543848 seconds, 235.4MB/s
before:134217728 bytes (128.0MB) copied, 0.538610 seconds, 237.6MB/s
before:134217728 bytes (128.0MB) copied, 0.544356 seconds, 235.1MB/s
before:134217728 bytes (128.0MB) copied, 0.544364 seconds, 235.1MB/s
before:134217728 bytes (128.0MB) copied, 0.537130 seconds, 238.3MB/s
before:134217728 bytes (128.0MB) copied, 0.533443 seconds, 240.0MB/s
before:134217728 bytes (128.0MB) copied, 0.545691 seconds, 234.6MB/s
before:134217728 bytes (128.0MB) copied, 0.534695 seconds, 239.4MB/s
before:134217728 bytes (128.0MB) copied, 0.540561 seconds, 236.8MB/s
before:134217728 bytes (128.0MB) copied, 0.541025 seconds, 236.6MB/s

after:134217728 bytes (128.0MB) copied, 0.520445 seconds, 245.9MB/s
after:134217728 bytes (128.0MB) copied, 0.527846 seconds, 242.5MB/s
after:134217728 bytes (128.0MB) copied, 0.519510 seconds, 246.4MB/s
after:134217728 bytes (128.0MB) copied, 0.527231 seconds, 242.8MB/s
after:134217728 bytes (128.0MB) copied, 0.525030 seconds, 243.8MB/s
after:134217728 bytes (128.0MB) copied, 0.524236 seconds, 244.2MB/s
after:134217728 bytes (128.0MB) copied, 0.523659 seconds, 244.4MB/s
after:134217728 bytes (128.0MB) copied, 0.525018 seconds, 243.8MB/s
after:134217728 bytes (128.0MB) copied, 0.519249 seconds, 246.5MB/s
after:134217728 bytes (128.0MB) copied, 0.518527 seconds, 246.9MB/s

Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

show more ...

Merge branches 'fixes', 'misc' and 'spectre' into for-next

ARM: 8796/1: spectre-v1,v1.1: provide helpers for address sanitization

Introduce C and asm helpers to sanitize user address, taking the
address range they target into account.

U

ARM: 8796/1: spectre-v1,v1.1: provide helpers for address sanitization

Introduce C and asm helpers to sanitize user address, taking the
address range they target into account.

Use asm helper for existing sanitization in __copy_from_user().

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

show more ...

Merge branches 'fixes', 'misc' and 'spectre' into for-linus

Conflicts:
arch/arm/include/asm/uaccess.h

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

ARM: spectre-v1: mitigate user accesses

Spectre variant 1 attacks are about this sequence of pseudo-code:

index = load(user-manipulated pointer);
access(base + i

ARM: spectre-v1: mitigate user accesses

Spectre variant 1 attacks are about this sequence of pseudo-code:

index = load(user-manipulated pointer);
access(base + index * stride);

In order for the cache side-channel to work, the access() must me made
to memory which userspace can detect whether cache lines have been
loaded. On 32-bit ARM, this must be either user accessible memory, or
a kernel mapping of that same user accessible memory.

The problem occurs when the load() speculatively loads privileged data,
and the subsequent access() is made to user accessible memory.

Any load() which makes use of a user-maniplated pointer is a potential
problem if the data it has loaded is used in a subsequent access. This
also applies for the access() if the data loaded by that access is used
by a subsequent access.

Harden the get_user() accessors against Spectre attacks by forcing out
of bounds addresses to a NULL pointer. This prevents get_user() being
used as the load() step above. As a side effect, put_user() will also
be affected even though it isn't implicated.

Also harden copy_from_user() by redoing the bounds check within the
arm_copy_from_user() code, and NULLing the pointer if out of bounds.

Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

show more ...

Merge branches 'fixes', 'misc' and 'spectre' into for-linus

ARM: spectre-v1: add speculation barrier (csdb) macros

Add assembly and C macros for the new CSDB instruction.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
Acked-by:

ARM: spectre-v1: add speculation barrier (csdb) macros

Add assembly and C macros for the new CSDB instruction.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Boot-tested-by: Tony Lindgren <tony@atomide.com>
Reviewed-by: Tony Lindgren <tony@atomide.com>

show more ...

ARM: 8772/1: kprobes: Prohibit kprobes on get_user functions

Since do_undefinstr() uses get_user to get the undefined
instruction, it can be called before kprobes processes
recursive

ARM: 8772/1: kprobes: Prohibit kprobes on get_user functions

Since do_undefinstr() uses get_user to get the undefined
instruction, it can be called before kprobes processes
recursive check. This can cause an infinit recursive
exception.
Prohibit probing on get_user functions.

Fixes: 24ba613c9d6c ("ARM kprobes: core code")
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: stable@vger.kernel.org
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

show more ...

ARM: BUG if jumping to usermode address in kernel mode

Detect if we are returning to usermode via the normal kernel exit paths
but the saved PSR value indicates that we are in kernel mod

ARM: BUG if jumping to usermode address in kernel mode

Detect if we are returning to usermode via the normal kernel exit paths
but the saved PSR value indicates that we are in kernel mode. This
could occur due to corrupted stack state, which has been observed with
"ftracetest".

This ensures that we catch the problem case before we get to user code.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

show more ...

ARM: Prepare for randomized task_struct

With the new task struct randomization, we can run into a build
failure for certain random seeds, which will place fields beyond
the allow imm

ARM: Prepare for randomized task_struct

With the new task struct randomization, we can run into a build
failure for certain random seeds, which will place fields beyond
the allow immediate size in the assembly:

arch/arm/kernel/entry-armv.S: Assembler messages:
arch/arm/kernel/entry-armv.S:803: Error: bad immediate value for offset (4096)

Only two constants in asm-offset.h are affected, and I'm changing
both of them here to work correctly in all configurations.

One more macro has the problem, but is currently unused, so this
removes it instead of adding complexity.

Suggested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
[kees: Adjust commit log slightly]
Signed-off-by: Kees Cook <keescook@chromium.org>

show more ...

ARM: 8605/1: V7M: fix notrace variant of save_and_disable_irqs

Commit 8e43a905 "ARM: 7325/1: fix v7 boot with lockdep enabled"
introduced notrace variant of save_and_disable_irqs to bala

ARM: 8605/1: V7M: fix notrace variant of save_and_disable_irqs

Commit 8e43a905 "ARM: 7325/1: fix v7 boot with lockdep enabled"
introduced notrace variant of save_and_disable_irqs to balance notrace
variant of restore_irqs; however V7M case has been missed. It was not
noticed because cache-v7.S the only place where notrace variant is used.
So fix it, since we are going to extend V7 cache routines to handle V7M
case too.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Tested-by: Andras Szemzo <sza@esh.hu>
Tested-by: Joachim Eastwood <manabian@gmail.com>
Tested-by: Alexandre TORGUE <alexandre.torgue@st.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

show more ...

ARM: introduce svc_pt_regs structure

Since the privileged mode pt_regs are an extended version of the saved
userland pt_regs, introduce a new svc_pt_regs structure to describe this
l

ARM: introduce svc_pt_regs structure

Since the privileged mode pt_regs are an extended version of the saved
userland pt_regs, introduce a new svc_pt_regs structure to describe this
layout.

Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

show more ...

ARM: rename S_FRAME_SIZE to PT_REGS_SIZE

S_FRAME_SIZE is no longer the size of the kernel stack frame, so this
name is misleading. It is the size of the kernel pt_regs structure.
Na

ARM: rename S_FRAME_SIZE to PT_REGS_SIZE

S_FRAME_SIZE is no longer the size of the kernel stack frame, so this
name is misleading. It is the size of the kernel pt_regs structure.
Name it so.

Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

show more ...

Merge branch 'fixes' of git://ftp.arm.linux.org.uk/~rmk/linux-arm

Pull ARM fixes from Russell King:
"A number of fixes for the merge window, fixing a number of cases
missed when t

Merge branch 'fixes' of git://ftp.arm.linux.org.uk/~rmk/linux-arm

Pull ARM fixes from Russell King:
"A number of fixes for the merge window, fixing a number of cases
missed when testing the uaccess code, particularly cases which only
show up with certain compiler versions"

* 'fixes' of git://ftp.arm.linux.org.uk/~rmk/linux-arm:
ARM: 8431/1: fix alignement of __bug_table section entries
arm/xen: Enable user access to the kernel before issuing a privcmd call
ARM: domains: add memory dependencies to get_domain/set_domain
ARM: domains: thread_info.h no longer needs asm/domains.h
ARM: uaccess: fix undefined instruction on ARMv7M/noMMU
ARM: uaccess: remove unneeded uaccess_save_and_disable macro
ARM: swpan: fix nwfpe for uaccess changes
ARM: 8429/1: disable GCC SRA optimization

show more ...

ARM: uaccess: remove unneeded uaccess_save_and_disable macro

This macro is never referenced, remove it.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

Merge branches 'cleanup', 'fixes', 'misc', 'omap-barrier' and 'uaccess' into for-linus

ARM: software-based priviledged-no-access support

Provide a software-based implementation of the priviledged no access
support found in ARMv8.1.

Userspace pages are mapped using

ARM: software-based priviledged-no-access support

Provide a software-based implementation of the priviledged no access
support found in ARMv8.1.

Userspace pages are mapped using a different domain number from the
kernel and IO mappings. If we switch the user domain to "no access"
when we enter the kernel, we can prevent the kernel from touching
userspace.

However, the kernel needs to be able to access userspace via the
various user accessor functions. With the wrapping in the previous
patch, we can temporarily enable access when the kernel needs user
access, and re-disable it afterwards.

This allows us to trap non-intended accesses to userspace, eg, caused
by an inadvertent dereference of the LIST_POISON* values, which, with
appropriate user mappings setup, can be made to succeed. This in turn
can allow use-after-free bugs to be further exploited than would
otherwise be possible.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

show more ...

ARM: entry: provide uaccess assembly macro hooks

Provide hooks into the kernel entry and exit paths to permit control
of userspace visibility to the kernel. The intended use is:

ARM: entry: provide uaccess assembly macro hooks

Provide hooks into the kernel entry and exit paths to permit control
of userspace visibility to the kernel. The intended use is:

- on entry to kernel from user, uaccess_disable will be called to
disable userspace visibility
- on exit from kernel to user, uaccess_enable will be called to
enable userspace visibility
- on entry from a kernel exception, uaccess_save_and_disable will be
called to save the current userspace visibility setting, and disable
access
- on exit from a kernel exception, uaccess_restore will be called to
restore the userspace visibility as it was before the exception
occurred.

These hooks allows us to keep userspace visibility disabled for the
vast majority of the kernel, except for localised regions where we
want to explicitly access userspace.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

show more ...

ARM: entry: efficiency cleanups

Make the "fast" syscall return path fast again. The addition of IRQ
tracing and context tracking has made this path grossly inefficient.
We can do mu

ARM: entry: efficiency cleanups

Make the "fast" syscall return path fast again. The addition of IRQ
tracing and context tracking has made this path grossly inefficient.
We can do much better if these options are enabled if we save the
syscall return code on the stack - we then don't need to save a bunch
of registers around every single callout to C code.

Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

show more ...