Revision tags: v6.6.25, v6.6.24, v6.6.23 |
|
#
79491ddf |
| 13-Feb-2024 |
Mark Brown <broonie@kernel.org> |
arm64/sme: Restore SMCR_EL1.EZT0 on exit from suspend
[ Upstream commit d7b77a0d565b048cb0808fa8a4fb031352b22a01 ]
The fields in SMCR_EL1 reset to an architecturally UNKNOWN value. Since we do not
arm64/sme: Restore SMCR_EL1.EZT0 on exit from suspend
[ Upstream commit d7b77a0d565b048cb0808fa8a4fb031352b22a01 ]
The fields in SMCR_EL1 reset to an architecturally UNKNOWN value. Since we do not otherwise manage the traps configured in this register at runtime we need to reconfigure them after a suspend in case nothing else was kind enough to preserve them for us. Do so for SMCR_EL1.EZT0.
Fixes: d4913eee152d ("arm64/sme: Add basic enumeration for SME2") Reported-by: Jackson Cooper-Driver <Jackson.Cooper-Driver@arm.com> Signed-off-by: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20240213-arm64-sme-resume-v3-2-17e05e493471@kernel.org Signed-off-by: Will Deacon <will@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
#
7c892383 |
| 13-Feb-2024 |
Mark Brown <broonie@kernel.org> |
arm64/sme: Restore SME registers on exit from suspend
[ Upstream commit 9533864816fb4a6207c63b7a98396351ce1a9fae ]
The fields in SMCR_EL1 and SMPRI_EL1 reset to an architecturally UNKNOWN value. Si
arm64/sme: Restore SME registers on exit from suspend
[ Upstream commit 9533864816fb4a6207c63b7a98396351ce1a9fae ]
The fields in SMCR_EL1 and SMPRI_EL1 reset to an architecturally UNKNOWN value. Since we do not otherwise manage the traps configured in this register at runtime we need to reconfigure them after a suspend in case nothing else was kind enough to preserve them for us.
The vector length will be restored as part of restoring the SME state for the next SME using task.
Fixes: a1f4ccd25cc2 ("arm64/sme: Provide Kconfig for SME") Reported-by: Jackson Cooper-Driver <Jackson.Cooper-Driver@arm.com> Signed-off-by: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20240213-arm64-sme-resume-v3-1-17e05e493471@kernel.org Signed-off-by: Will Deacon <will@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
Revision tags: v6.6.16, v6.6.15 |
|
#
60480c6b |
| 30-Jan-2024 |
Mark Brown <broonie@kernel.org> |
arm64/signal: Don't assume that TIF_SVE means we saved SVE state
commit 61da7c8e2a602f66be578cbbcebe8638c10e0f48 upstream.
When we are in a syscall we will only save the FPSIMD subset even though t
arm64/signal: Don't assume that TIF_SVE means we saved SVE state
commit 61da7c8e2a602f66be578cbbcebe8638c10e0f48 upstream.
When we are in a syscall we will only save the FPSIMD subset even though the task still has access to the full register set, and on context switch we will only remove TIF_SVE when loading the register state. This means that the signal handling code should not assume that TIF_SVE means that the register state is stored in SVE format, it should instead check the format that was recorded during save.
Fixes: 8c845e273104 ("arm64/sve: Leave SVE enabled on syscall if we don't context switch") Signed-off-by: Mark Brown <broonie@kernel.org> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20240130-arm64-sve-signal-regs-v2-1-9fc6f9502782@kernel.org Signed-off-by: Will Deacon <will@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
show more ...
|
Revision tags: v6.6.14, v6.6.13 |
|
#
569156e4 |
| 15-Jan-2024 |
Mark Brown <broonie@kernel.org> |
arm64/sme: Always exit sme_alloc() early with existing storage
commit dc7eb8755797ed41a0d1b5c0c39df3c8f401b3d9 upstream.
When sme_alloc() is called with existing storage and we are not flushing we
arm64/sme: Always exit sme_alloc() early with existing storage
commit dc7eb8755797ed41a0d1b5c0c39df3c8f401b3d9 upstream.
When sme_alloc() is called with existing storage and we are not flushing we will always allocate new storage, both leaking the existing storage and corrupting the state. Fix this by separating the checks for flushing and for existing storage as we do for SVE.
Callers that reallocate (eg, due to changing the vector length) should call sme_free() themselves.
Fixes: 5d0a8d2fba50 ("arm64/ptrace: Ensure that SME is set up for target when writing SSVE state") Signed-off-by: Mark Brown <broonie@kernel.org> Cc: <stable@vger.kernel.org> Link: https://lore.kernel.org/r/20240115-arm64-sme-flush-v1-1-7472bd3459b7@kernel.org Signed-off-by: Will Deacon <will@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
show more ...
|
Revision tags: v6.6.12, v6.6.11, v6.6.10, v6.6.9, v6.6.8, v6.6.7, v6.6.6, v6.6.5, v6.6.4, v6.6.3, v6.6.2, v6.5.11, v6.6.1, v6.5.10, v6.6, v6.5.9, v6.5.8, v6.5.7, v6.5.6, v6.5.5, v6.5.4, v6.5.3, v6.5.2, v6.1.51, v6.5.1, v6.1.50, v6.5, v6.1.49, v6.1.48, v6.1.46, v6.1.45 |
|
#
5d0a8d2f |
| 10-Aug-2023 |
Mark Brown <broonie@kernel.org> |
arm64/ptrace: Ensure that SME is set up for target when writing SSVE state
When we use NT_ARM_SSVE to either enable streaming mode or change the vector length for a process we do not currently do an
arm64/ptrace: Ensure that SME is set up for target when writing SSVE state
When we use NT_ARM_SSVE to either enable streaming mode or change the vector length for a process we do not currently do anything to ensure that there is storage allocated for the SME specific register state. If the task had not previously used SME or we changed the vector length then the task will not have had TIF_SME set or backing storage for ZA/ZT allocated, resulting in inconsistent register sizes when saving state and spurious traps which flush the newly set register state.
We should set TIF_SME to disable traps and ensure that storage is allocated for ZA and ZT if it is not already allocated. This requires modifying sme_alloc() to make the flush of any existing register state optional so we don't disturb existing state for ZA and ZT.
Fixes: e12310a0d30f ("arm64/sme: Implement ptrace support for streaming mode SVE registers") Reported-by: David Spickett <David.Spickett@arm.com> Signed-off-by: Mark Brown <broonie@kernel.org> Cc: <stable@vger.kernel.org> # 5.19.x Link: https://lore.kernel.org/r/20230810-arm64-fix-ptrace-race-v1-1-a5361fad2bd6@kernel.org Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
show more ...
|
Revision tags: v6.1.44, v6.1.43 |
|
#
01948b09 |
| 31-Jul-2023 |
Mark Brown <broonie@kernel.org> |
arm64/fpsimd: Only provide the length to cpufeature for xCR registers
For both SVE and SME we abuse the generic register field comparison support in the cpufeature code as part of our detection of u
arm64/fpsimd: Only provide the length to cpufeature for xCR registers
For both SVE and SME we abuse the generic register field comparison support in the cpufeature code as part of our detection of unsupported variations in the vector lengths available to PEs, reporting the maximum vector lengths via ZCR_EL1.LEN and SMCR_EL1.LEN. Since these are configuration registers rather than identification registers the assumptions the cpufeature code makes about how unknown bitfields behave are invalid, leading to warnings when SME features like FA64 are enabled and we hotplug a CPU:
CPU features: SANITY CHECK: Unexpected variation in SYS_SMCR_EL1. Boot CPU: 0x0000000000000f, CPU3: 0x0000008000000f CPU features: Unsupported CPU feature variation detected.
SVE has no controls other than the vector length so is not yet impacted but the same issue will apply there if any are defined.
Since the only field we are interested in having the cpufeature code handle is the length field and we use a custom read function to obtain the value we can avoid these warnings by filtering out all other bits when we return the register value, if we're doing that we don't need to bother reading the register at all and can simply use the RDVL/RDSVL value we were filling in instead.
Fixes: 2e0f2478ea37 ("arm64/sve: Probe SVE capabilities and usable vector lengths") FixeS: b42990d3bf77 ("arm64/sme: Identify supported SME vector lengths at boot") Signed-off-by: Mark Brown <broonie@kernel.org> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Link: https://lore.kernel.org/r/20230731-arm64-sme-fa64-hotplug-v2-1-7714c00dd902@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
show more ...
|
#
69af56ae |
| 03-Aug-2023 |
Mark Brown <broonie@kernel.org> |
arm64/fpsimd: Sync and zero pad FPSIMD state for streaming SVE
We have a function sve_sync_from_fpsimd_zeropad() which is used by the ptrace code to update the SVE state when the user writes to the
arm64/fpsimd: Sync and zero pad FPSIMD state for streaming SVE
We have a function sve_sync_from_fpsimd_zeropad() which is used by the ptrace code to update the SVE state when the user writes to the the FPSIMD register set. Currently this checks that the task has SVE enabled but this will miss updates for tasks which have streaming SVE enabled if SVE has not been enabled for the thread, also do the conversion if the task has streaming SVE enabled.
Fixes: e12310a0d30f ("arm64/sme: Implement ptrace support for streaming mode SVE registers") Signed-off-by: Mark Brown <broonie@kernel.org> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20230803-arm64-fix-ptrace-ssve-no-sve-v1-3-49df214bfb3e@kernel.org Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
show more ...
|
#
507ea5dd |
| 03-Aug-2023 |
Mark Brown <broonie@kernel.org> |
arm64/fpsimd: Sync FPSIMD state with SVE for SME only systems
Currently we guard FPSIMD/SVE state conversions with a check for the system supporting SVE but SME only systems may need to sync streami
arm64/fpsimd: Sync FPSIMD state with SVE for SME only systems
Currently we guard FPSIMD/SVE state conversions with a check for the system supporting SVE but SME only systems may need to sync streaming mode SVE state so add a check for SME support too. These functions are only used by the ptrace code.
Fixes: e12310a0d30f ("arm64/sme: Implement ptrace support for streaming mode SVE registers") Signed-off-by: Mark Brown <broonie@kernel.org> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20230803-arm64-fix-ptrace-ssve-no-sve-v1-2-49df214bfb3e@kernel.org Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
show more ...
|
#
c9bb40b7 |
| 02-Aug-2023 |
Mark Brown <broonie@kernel.org> |
arm64/fpsimd: Clear SME state in the target task when setting the VL
When setting SME vector lengths we clear TIF_SME to reenable SME traps, doing a reallocation of the backing storage on next use.
arm64/fpsimd: Clear SME state in the target task when setting the VL
When setting SME vector lengths we clear TIF_SME to reenable SME traps, doing a reallocation of the backing storage on next use. We do this using clear_thread_flag() which operates on the current thread, meaning that when setting the vector length via ptrace we may both not force traps for the target task and force a spurious flush of any SME state that the tracing task may have.
Clear the flag in the target task.
Fixes: e12310a0d30f ("arm64/sme: Implement ptrace support for streaming mode SVE registers") Reported-by: David Spickett <David.Spickett@arm.com> Signed-off-by: Mark Brown <broonie@kernel.org> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20230803-arm64-fix-ptrace-tif-sme-v1-1-88312fd6fbfd@kernel.org Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
show more ...
|
Revision tags: v6.1.42 |
|
#
05d881b8 |
| 26-Jul-2023 |
Mark Brown <broonie@kernel.org> |
arm64/sme: Set new vector length before reallocating
As part of fixing the allocation of the buffer for SVE state when changing SME vector length we introduced an immediate reallocation of the SVE s
arm64/sme: Set new vector length before reallocating
As part of fixing the allocation of the buffer for SVE state when changing SME vector length we introduced an immediate reallocation of the SVE state, this is also done when changing the SVE vector length for consistency. Unfortunately this reallocation is done prior to writing the new vector length to the task struct, meaning the allocation is done with the old vector length and can lead to memory corruption due to an undersized buffer being used.
Move the update of the vector length before the allocation to ensure that the new vector length is taken into account.
For some reason this isn't triggering any problems when running tests on the arm64 fixes branch (even after repeated tries) but is triggering issues very often after merge into mainline.
Fixes: d4d5be94a878 ("arm64/fpsimd: Ensure SME storage is allocated after SVE VL changes") Signed-off-by: Mark Brown <broonie@kernel.org> Cc: <stable@vger.kernel.org> Link: https://lore.kernel.org/r/20230726-arm64-fix-sme-fix-v1-1-7752ec58af27@kernel.org Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
show more ...
|
Revision tags: v6.1.41 |
|
#
3421ddbe |
| 24-Jul-2023 |
Mark Brown <broonie@kernel.org> |
arm64/fpsimd: Don't flush SME register hardware state along with thread
We recently changed the fpsimd thread flush to flush the physical SME state as well as the thread state for the current thread
arm64/fpsimd: Don't flush SME register hardware state along with thread
We recently changed the fpsimd thread flush to flush the physical SME state as well as the thread state for the current thread. Unfortunately this leads to intermittent corruption in interaction with the lazy FPSIMD register switching. When under heavy load such as can be triggered by the startup phase of fp-stress it is possible that the current thread may not be scheduled prior to returning to userspace, and indeed we may end up returning to the last thread that was scheduled on the PE without ever exiting the kernel to any other task. If that happens then we will not reload the register state from memory, leading to loss of any SME register state.
Since this was purely an attempt to defensively close off potential problems revert the change.
Fixes: af3215fd0230 ("arm64/fpsimd: Exit streaming mode when flushing tasks") Signed-off-by: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20230724-arm64-dont-flush-smstate-v1-1-9a8b637ace6c@kernel.org Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
show more ...
|
Revision tags: v6.1.40 |
|
#
d4d5be94 |
| 20-Jul-2023 |
Mark Brown <broonie@kernel.org> |
arm64/fpsimd: Ensure SME storage is allocated after SVE VL changes
When we reconfigure the SVE vector length we discard the backing storage for the SVE vectors and then reallocate on next SVE use, l
arm64/fpsimd: Ensure SME storage is allocated after SVE VL changes
When we reconfigure the SVE vector length we discard the backing storage for the SVE vectors and then reallocate on next SVE use, leaving the SME specific state alone. This means that we do not enable SME traps if they were already disabled. That means that userspace code can enter streaming mode without trapping, putting the task in a state where if we try to save the state of the task we will fault.
Since the ABI does not specify that changing the SVE vector length disturbs SME state, and since SVE code may not be aware of SME code in the process, we shouldn't simply discard any ZA state. Instead immediately reallocate the storage for SVE, and disable SME if we change the SVE vector length while there is no SME state active.
Disabling SME traps on SVE vector length changes would make the overall code more complex since we would have a state where we have valid SME state stored but might get a SME trap.
Fixes: 9e4ab6c89109 ("arm64/sme: Implement vector length configuration prctl()s") Reported-by: David Spickett <David.Spickett@arm.com> Signed-off-by: Mark Brown <broonie@kernel.org> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20230720-arm64-fix-sve-sme-vl-change-v2-1-8eea06b82d57@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
show more ...
|
Revision tags: v6.1.39, v6.1.38, v6.1.37, v6.1.36, v6.4, v6.1.35, v6.1.34 |
|
#
af3215fd |
| 09-Jun-2023 |
Mark Brown <broonie@kernel.org> |
arm64/fpsimd: Exit streaming mode when flushing tasks
Ensure there is no path where we might attempt to save SME state after we flush a task by updating the SVCR register state as well as updating o
arm64/fpsimd: Exit streaming mode when flushing tasks
Ensure there is no path where we might attempt to save SME state after we flush a task by updating the SVCR register state as well as updating our in memory state. I haven't seen a specific case where this is happening or seen a path where it might happen but for the cost of a single low overhead instruction it seems sensible to close the potential gap.
Signed-off-by: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20230607-arm64-flush-svcr-v2-1-827306001841@kernel.org Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
show more ...
|
Revision tags: v6.1.33, v6.1.32, v6.1.31, v6.1.30, v6.1.29, v6.1.28, v6.1.27, v6.1.26, v6.3, v6.1.25, v6.1.24, v6.1.23, v6.1.22, v6.1.21 |
|
#
97b5576b |
| 17-Mar-2023 |
Dongxu Sun <sundongxu3@huawei.com> |
arm64/sme: Fix some comments of ARM SME
When TIF_SME is clear, fpsimd_restore_current_state will disable SME trap during ret_to_user, then SME access trap is impossible in userspace, not SVE.
Besid
arm64/sme: Fix some comments of ARM SME
When TIF_SME is clear, fpsimd_restore_current_state will disable SME trap during ret_to_user, then SME access trap is impossible in userspace, not SVE.
Besides, fix typo: alocated->allocated.
Signed-off-by: Dongxu Sun <sundongxu3@huawei.com> Reviewed-by: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20230317124915.1263-5-sundongxu3@huawei.com Signed-off-by: Will Deacon <will@kernel.org>
show more ...
|
Revision tags: v6.1.20, v6.1.19, v6.1.18, v6.1.17, v6.1.16, v6.1.15, v6.1.14, v6.1.13, v6.2, v6.1.12 |
|
#
0269680e |
| 09-Feb-2023 |
Mark Brown <broonie@kernel.org> |
arm64/fpsimd: Remove warning for SME without SVE
Support for SME without SVE is architecturally valid and has now been tested well enough so let's remove the warning message that is displayed at boo
arm64/fpsimd: Remove warning for SME without SVE
Support for SME without SVE is architecturally valid and has now been tested well enough so let's remove the warning message that is displayed at boot.
Signed-off-by: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20230209-arm64-sme-no-sve-v1-1-74eb3df2f878@kernel.org Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
show more ...
|
Revision tags: v6.1.11, v6.1.10, v6.1.9, v6.1.8, v6.1.7 |
|
#
95fcec71 |
| 16-Jan-2023 |
Mark Brown <broonie@kernel.org> |
arm64/sme: Implement context switching for ZT0
When the system supports SME2 the ZT0 register must be context switched as part of the floating point state. This register is stored immediately after
arm64/sme: Implement context switching for ZT0
When the system supports SME2 the ZT0 register must be context switched as part of the floating point state. This register is stored immediately after ZA in memory and is only accessible when PSTATE.ZA is set so we handle it in the same functions we use to save and restore ZA.
Signed-off-by: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-10-f2fa0aef982f@kernel.org Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
show more ...
|
#
d4913eee |
| 16-Jan-2023 |
Mark Brown <broonie@kernel.org> |
arm64/sme: Add basic enumeration for SME2
Add basic feature detection for SME2, detecting that the feature is present and disabling traps for ZT0.
Signed-off-by: Mark Brown <broonie@kernel.org> Lin
arm64/sme: Add basic enumeration for SME2
Add basic feature detection for SME2, detecting that the feature is present and disabling traps for ZT0.
Signed-off-by: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-8-f2fa0aef982f@kernel.org Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
show more ...
|
#
ce514000 |
| 16-Jan-2023 |
Mark Brown <broonie@kernel.org> |
arm64/sme: Rename za_state to sme_state
In preparation for adding support for storage for ZT0 to the thread_struct rename za_state to sme_state. Since ZT0 is accessible when PSTATE.ZA is set just li
arm64/sme: Rename za_state to sme_state
In preparation for adding support for storage for ZT0 to the thread_struct rename za_state to sme_state. Since ZT0 is accessible when PSTATE.ZA is set just like ZA itself we will extend the allocation done for ZA to cover it, avoiding the need to further expand task_struct for non-SME tasks.
No functional changes.
Signed-off-by: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-1-f2fa0aef982f@kernel.org Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
show more ...
|
Revision tags: v6.1.6, v6.1.5, v6.0.19, v6.0.18, v6.1.4, v6.1.3, v6.0.17, v6.1.2, v6.0.16 |
|
#
fcd3d2c0 |
| 27-Dec-2022 |
Mark Brown <broonie@kernel.org> |
arm64/sme: Don't use streaming mode to probe the maximum SME VL
During development the architecture added the RDSVL instruction which means we do not need to enter streaming mode to enumerate the SM
arm64/sme: Don't use streaming mode to probe the maximum SME VL
During development the architecture added the RDSVL instruction which means we do not need to enter streaming mode to enumerate the SME VLs, use it when we probe the maximum supported VL. Other users were already updated.
No functional change.
Signed-off-by: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20221223-arm64-sme-probe-max-v1-1-cbde68f67ad0@kernel.org Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
show more ...
|
#
0cab5b49 |
| 27-Dec-2022 |
Mark Brown <broonie@kernel.org> |
arm64/sme: Fix context switch for SME only systems
When refactoring fpsimd_load() to support keeping SVE enabled over syscalls support for systems with SME but not SVE was broken. The code that sele
arm64/sme: Fix context switch for SME only systems
When refactoring fpsimd_load() to support keeping SVE enabled over syscalls support for systems with SME but not SVE was broken. The code that selects between loading regular FPSIMD and SVE states was guarded by using system_supports_sve() but is also needed to handle the streaming SVE state in SME only systems where that check will be false. Fix this by also checking for system_supports_sme().
Fixes: a0136be443d5 ("arm64/fpsimd: Load FP state based on recorded data type") Signed-off-by: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20221223-arm64-fix-sme-only-v1-1-938d663f69e5@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
show more ...
|
Revision tags: v6.1.1, v6.0.15, v6.0.14, v6.0.13, v6.1, v6.0.12, v6.0.11, v6.0.10, v5.15.80, v6.0.9, v5.15.79 |
|
#
1192b93b |
| 15-Nov-2022 |
Mark Brown <broonie@kernel.org> |
arm64/fp: Use a struct to pass data to fpsimd_bind_state_to_cpu()
For reasons that are unclear to this reader fpsimd_bind_state_to_cpu() populates the struct fpsimd_last_state_struct that it uses to
arm64/fp: Use a struct to pass data to fpsimd_bind_state_to_cpu()
For reasons that are unclear to this reader fpsimd_bind_state_to_cpu() populates the struct fpsimd_last_state_struct that it uses to store the active floating point state for KVM guests by passing an argument for each member of the structure. As the richness of the architecture increases this is resulting in a function with a rather large number of arguments which isn't ideal.
Simplify the interface by using the struct directly as the single argument for the function, renaming it as we lift the definition into the header. This could be built on further to reduce the work we do adding storage for new FP state in various places but for now it just simplifies this one interface.
Signed-off-by: Mark Brown <broonie@kernel.org> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20221115094640.112848-9-broonie@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
show more ...
|
#
8c845e27 |
| 15-Nov-2022 |
Mark Brown <broonie@kernel.org> |
arm64/sve: Leave SVE enabled on syscall if we don't context switch
The syscall ABI says that the SVE register state not shared with FPSIMD may not be preserved on syscall, and this is the only mecha
arm64/sve: Leave SVE enabled on syscall if we don't context switch
The syscall ABI says that the SVE register state not shared with FPSIMD may not be preserved on syscall, and this is the only mechanism we have in the ABI to stop tracking the extra SVE state for a process. Currently we do this unconditionally by means of disabling SVE for the process on syscall, causing userspace to take a trap to EL1 if it uses SVE again. These extra traps result in a noticeable overhead for using SVE instead of FPSIMD in some workloads, especially for simple syscalls where we can return directly to userspace and would not otherwise need to update the floating point registers. Tests with fp-pidbench show an approximately 70% overhead on a range of implementations when SVE is in use - while this is an extreme and entirely artificial benchmark it is clear that there is some useful room for improvement here.
Now that we have the ability to track the decision about what to save seprately to TIF_SVE we can improve things by leaving TIF_SVE enabled on syscall but only saving the FPSIMD registers if we are in a syscall. This means that if we need to restore the register state from memory (eg, after a context switch or kernel mode NEON) we will drop TIF_SVE and reenable traps for userspace but if we can just return to userspace then traps will remain disabled.
Since our current implementation and hence ABI has the effect of zeroing all the SVE register state not shared with FPSIMD on syscall we replace the disabling of TIF_SVE with a flush of the non-shared register state, this means that there is still some overhead for syscalls when SVE is in use but it is very much reduced.
Signed-off-by: Mark Brown <broonie@kernel.org> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20221115094640.112848-8-broonie@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
show more ...
|
#
bbc6172e |
| 15-Nov-2022 |
Mark Brown <broonie@kernel.org> |
arm64/fpsimd: SME no longer requires SVE register state
Now that we track the type of the stored register state separately to what is active in the task, it is valid to have the FPSIMD register stat
arm64/fpsimd: SME no longer requires SVE register state
Now that we track the type of the stored register state separately to what is active in the task, it is valid to have the FPSIMD register state stored while in streaming mode. Remove the special case handling for SME when setting FPSIMD register state.
Signed-off-by: Mark Brown <broonie@kernel.org> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20221115094640.112848-7-broonie@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
show more ...
|
#
a0136be4 |
| 15-Nov-2022 |
Mark Brown <broonie@kernel.org> |
arm64/fpsimd: Load FP state based on recorded data type
Now that we are recording the type of floating point register state we are saving when we write the register state out to memory we can use th
arm64/fpsimd: Load FP state based on recorded data type
Now that we are recording the type of floating point register state we are saving when we write the register state out to memory we can use that information when we load from memory to decide which format to load, bringing TIF_SVE into line with what we saved rather than relying on TIF_SVE to determine what to load.
The SME state details are already recorded directly in the saved SVCR and handled based on the information there.
Since we are not changing any of the save paths there should be no functional change from this patch, further patches will make use of this to optimise and clarify the code.
Signed-off-by: Mark Brown <broonie@kernel.org> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20221115094640.112848-6-broonie@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
show more ...
|
#
62021cc3 |
| 15-Nov-2022 |
Mark Brown <broonie@kernel.org> |
arm64/fpsimd: Stop using TIF_SVE to manage register saving in KVM
Now that we are explicitly telling the host FP code which register state it needs to save we can remove the manipulation of TIF_SVE
arm64/fpsimd: Stop using TIF_SVE to manage register saving in KVM
Now that we are explicitly telling the host FP code which register state it needs to save we can remove the manipulation of TIF_SVE from the KVM code, simplifying it and allowing us to optimise our handling of normal tasks. Remove the manipulation of TIF_SVE from KVM and instead rely on to_save to ensure we save the correct data for it.
There should be no functional or performance impact from this change.
Signed-off-by: Mark Brown <broonie@kernel.org> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20221115094640.112848-5-broonie@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
show more ...
|