mm/gup: remove unused vmas parameter from get_user_pages()

Patch series "remove the vmas parameter from GUP APIs", v6.

(pin_/get)_user_pages[_remote]() each provide an optional output parameter
for

mm/gup: remove unused vmas parameter from get_user_pages()

Patch series "remove the vmas parameter from GUP APIs", v6.

(pin_/get)_user_pages[_remote]() each provide an optional output parameter
for an array of VMA objects associated with each page in the input range.

These provide the means for VMAs to be returned, as long as mm->mmap_lock
is never released during the GUP operation (i.e. the internal flag
FOLL_UNLOCKABLE is not specified).

In addition, these VMAs can only be accessed with the mmap_lock held and
become invalidated the moment it is released.

The vast majority of invocations do not use this functionality and of
those that do, all but one case retrieve a single VMA to perform checks
upon.

It is not egregious in the single VMA cases to simply replace the
operation with a vma_lookup(). In these cases we duplicate the (fast)
lookup on a slow path already under the mmap_lock, abstracted to a new
get_user_page_vma_remote() inline helper function which also performs
error checking and reference count maintenance.

The special case is io_uring, where io_pin_pages() specifically needs to
assert that the VMAs underlying the range do not result in broken
long-term GUP file-backed mappings.

As GUP now internally asserts that FOLL_LONGTERM mappings are not
file-backed in a broken fashion (i.e. requiring dirty tracking) - as
implemented in "mm/gup: disallow FOLL_LONGTERM GUP-nonfast writing to
file-backed mappings" - this logic is no longer required and so we can
simply remove it altogether from io_uring.

Eliminating the vmas parameter eliminates an entire class of danging
pointer errors that might have occured should the lock have been
incorrectly released.

In addition, the API is simplified and now clearly expresses what it is
intended for - applying the specified GUP flags and (if pinning) returning
pinned pages.

This change additionally opens the door to further potential improvements
in GUP and the possible marrying of disparate code paths.

I have run this series against gup_test with no issues.

Thanks to Matthew Wilcox for suggesting this refactoring!


This patch (of 6):

No invocation of get_user_pages() use the vmas parameter, so remove it.

The GUP API is confusing and caveated. Recent changes have done much to
improve that, however there is more we can do. Exporting vmas is a prime
target as the caller has to be extremely careful to preclude their use
after the mmap_lock has expired or otherwise be left with dangling
pointers.

Removing the vmas parameter focuses the GUP functions upon their primary
purpose - pinning (and outputting) pages as well as performing the actions
implied by the input flags.

This is part of a patch series aiming to remove the vmas parameter
altogether.

Link: https://lkml.kernel.org/r/cover.1684350871.git.lstoakes@gmail.com
Link: https://lkml.kernel.org/r/589e0c64794668ffc799651e8d85e703262b1e9d.1684350871.git.lstoakes@gmail.com
Signed-off-by: Lorenzo Stoakes <lstoakes@gmail.com>
Suggested-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Christian König <christian.koenig@amd.com> (for radeon parts)
Acked-by: Jarkko Sakkinen <jarkko@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Sean Christopherson <seanjc@google.com> (KVM)
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
Cc: Janosch Frank <frankja@linux.ibm.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

show more ...

drm/radeon: stop including swiotlb.h

radeon does not need swiotlb.h, so stop including it.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Remove unnecessary (void*) conversions

No need cast (void*) to (struct radeon_device *)
or (struct radeon_ring *).

Signed-off-by: Su Hui <suhui@nfschina.com>
Signed-off-by: Alex Deucher

drm/radeon: Remove unnecessary (void*) conversions

No need cast (void*) to (struct radeon_device *)
or (struct radeon_ring *).

Signed-off-by: Su Hui <suhui@nfschina.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

show more ...

drm/radeon: handle NULL bo->resource in move callback

The ttm bo now initially has NULL bo->resource, and leaves the driver
the handle that. However it looks like we forgot to handle that for
radeon

drm/radeon: handle NULL bo->resource in move callback

The ttm bo now initially has NULL bo->resource, and leaves the driver
the handle that. However it looks like we forgot to handle that for
radeon. It looks like this will just null-ptr-deref in
radeon_bo_move(), if bo->resource is NULL.

Fix this by calling move_null().

Fixes: 180253782038 ("drm/ttm: stop allocating dummy resources during BO creation")
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230208145319.397235-4-matthew.auld@intel.com
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>

show more ...

drm/ttm: prevent moving of pinned BOs

We have checks for this in the individual drivers move callback, but
it's probably better to generally forbid that on a higher level.

Also stops exporting ttm_

drm/ttm: prevent moving of pinned BOs

We have checks for this in the individual drivers move callback, but
it's probably better to generally forbid that on a higher level.

Also stops exporting ttm_resource_compat() since that's not necessary
any more after removing the extra checks in vmwgfx.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230130120636.63765-6-matthew.auld@intel.com

show more ...

drm/ttm: revert "prevent moving of pinned BOs"

This reverts commit b49323aa35d502b0d9a7950327f30a1a52eae534.

This still seems to break i915.

Signed-off-by: Christian König <christian.koenig@amd.co

drm/ttm: revert "prevent moving of pinned BOs"

This reverts commit b49323aa35d502b0d9a7950327f30a1a52eae534.

This still seems to break i915.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Matthew Auld <matthew.william.auld@gmail.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230125155023.105584-1-christian.koenig@amd.com

show more ...

drm/ttm: prevent moving of pinned BOs

We have checks for this in the individual drivers move callback, but
it's probably better to generally forbid that on a higher level.

Also stops exporting ttm_

drm/ttm: prevent moving of pinned BOs

We have checks for this in the individual drivers move callback, but
it's probably better to generally forbid that on a higher level.

Also stops exporting ttm_resource_compat() since that's not necessary
any more after removing the extra checks in vmwgfx.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230124125726.13323-4-christian.koenig@amd.com

show more ...

drm/ttm: merge ttm_bo_api.h and ttm_bo_driver.h v2

Merge and cleanup the two headers into a single description of the
object API. Also move all the documentation to the implementation and
drop unnec

drm/ttm: merge ttm_bo_api.h and ttm_bo_driver.h v2

Merge and cleanup the two headers into a single description of the
object API. Also move all the documentation to the implementation and
drop unnecessary includes from the header.

No functional change.

v2: minimal checkpatch.pl cleanup

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221125102137.1801-4-christian.koenig@amd.com

show more ...

drm/ttm: rework on ttm_resource to use size_t type

Change ttm_resource structure from num_pages to size_t size in bytes.
v1 -> v2: change PFN_UP(dst_mem->size) to ttm->num_pages
v1 -> v2: change bo-

drm/ttm: rework on ttm_resource to use size_t type

Change ttm_resource structure from num_pages to size_t size in bytes.
v1 -> v2: change PFN_UP(dst_mem->size) to ttm->num_pages
v1 -> v2: change bo->resource->size to bo->base.size at some places
v1 -> v2: remove the local variable
v1 -> v2: cleanup cmp_size_smaller_first()
v2 -> v3: adding missing PFN_UP in ttm_bo_vm_fault_reserved

Signed-off-by: Somalapuram Amaranath <Amaranath.Somalapuram@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221027091237.983582-1-Amaranath.Somalapuram@amd.com
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>

show more ...

drm/radeon: Replace kmap() with kmap_local_page()

The use of kmap() is being deprecated in favor of kmap_local_page().

There are two main problems with kmap(): (1) It comes with an overhead as
the

drm/radeon: Replace kmap() with kmap_local_page()

The use of kmap() is being deprecated in favor of kmap_local_page().

There are two main problems with kmap(): (1) It comes with an overhead as
the mapping space is restricted and protected by a global lock for
synchronization and (2) it also requires global TLB invalidation when the
kmap’s pool wraps and it might block when the mapping space is fully
utilized until a slot becomes available.

With kmap_local_page() the mappings are per thread, CPU local, can take
page faults, and can be called from any context (including interrupts).
It is faster than kmap() in kernels with HIGHMEM enabled. Furthermore,
the tasks can be preempted and, when they are scheduled to run again, the
kernel virtual addresses are restored and still valid.

Therefore, replace kmap() with kmap_local_page() in radeon_ttm_gtt_read().

Cc: "Venkataramanan, Anirudh" <anirudh.venkataramanan@intel.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Christian König <christian.koenig@amd.com>
Suggested-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Fabio M. De Francesco <fmdefrancesco@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

show more ...

drm/radeon: Use TTM builtin resource manager debugfs code

Switch to using the TTM resource manager debugfs helpers. The
functionality is largely the same.
The TTM resource managers need to stay vali

drm/radeon: Use TTM builtin resource manager debugfs code

Switch to using the TTM resource manager debugfs helpers. The
functionality is largely the same.
The TTM resource managers need to stay valid for as long as the
drm debugfs_root is valid.

Signed-off-by: Zack Rusin <zackr@vmware.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "Pan, Xinhui" <Xinhui.Pan@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: amd-gfx@lists.freedesktop.org
Link: https://patchwork.freedesktop.org/patch/msgid/20220412033526.369115-6-zack@kde.org
Reviewed-by: Christian König <christian.koenig@amd.com>

show more ...

drm/radeon: remove resource accounting v2

Use the one provided by TTM instead.

v2: drop new_mem parameter as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Tested-by: Bas Nieuwenhu

drm/radeon: remove resource accounting v2

Use the one provided by TTM instead.

v2: drop new_mem parameter as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Tested-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220214093439.2989-5-christian.koenig@amd.com

show more ...

drm/radeon: use ttm_resource_manager_debug

Instead of calling the debug operation directly.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Link

drm/radeon: use ttm_resource_manager_debug

Instead of calling the debug operation directly.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211124124430.20859-8-christian.koenig@amd.com

show more ...

drm/ttm: s/FLAG_SG/FLAG_EXTERNAL/

It covers more than just ttm_bo_type_sg usage, like with say dma-buf,
since one other user is userptr in amdgpu, and in the future we might
have some more. Hence EX

drm/ttm: s/FLAG_SG/FLAG_EXTERNAL/

It covers more than just ttm_bo_type_sg usage, like with say dma-buf,
since one other user is userptr in amdgpu, and in the future we might
have some more. Hence EXTERNAL is likely a more suitable name.

v2(Christian):
- Rename these to TTM_TT_FLAGS_*
- Fix up all the holes in the flag values

Suggested-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Christian König <christian.koenig@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210929132629.353541-1-matthew.auld@intel.com
Signed-off-by: Christian König <christian.koenig@amd.com>

show more ...

drm/ttm: remove ttm_tt_destroy_common v2

Move the functionality into ttm_tt_fini and ttm_bo_tt_destroy instead.

We don't need this any more since we removed the unbind from the destroy
code paths i

drm/ttm: remove ttm_tt_destroy_common v2

Move the functionality into ttm_tt_fini and ttm_bo_tt_destroy instead.

We don't need this any more since we removed the unbind from the destroy
code paths in the drivers.

Also add a warning to ttm_tt_fini() if we try to fini a still populated TT
object.

v2: instead of reverting the patch move the functionality to different
places.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210728130552.2074-5-christian.koenig@amd.com

show more ...

drm/radeon: unbind in radeon_ttm_tt_unpopulate()

Doing this in radeon_ttm_tt_destroy() is to late.

It turned out that this is not a good idea at all because it leaves pointers
to freed up system me

drm/radeon: unbind in radeon_ttm_tt_unpopulate()

Doing this in radeon_ttm_tt_destroy() is to late.

It turned out that this is not a good idea at all because it leaves pointers
to freed up system memory pages in the GART tables of the drivers.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210728130552.2074-4-christian.koenig@amd.com

show more ...

drm/radeon: Fix NULL dereference when updating memory stats

radeon_ttm_bo_destroy() is attempting to access the resource object to
update memory counters. However, the resource object is already fre

drm/radeon: Fix NULL dereference when updating memory stats

radeon_ttm_bo_destroy() is attempting to access the resource object to
update memory counters. However, the resource object is already freed when
ttm calls this function via the destroy callback. This causes an oops when
a bo is freed:

BUG: kernel NULL pointer dereference, address: 0000000000000010
RIP: 0010:radeon_ttm_bo_destroy+0x2c/0x100 [radeon]
Call Trace:
radeon_bo_unref+0x1a/0x30 [radeon]
radeon_gem_object_free+0x33/0x50 [radeon]
drm_gem_object_release_handle+0x69/0x70 [drm]
drm_gem_handle_delete+0x62/0xa0 [drm]
? drm_mode_destroy_dumb+0x40/0x40 [drm]
drm_ioctl_kernel+0xb2/0xf0 [drm]
drm_ioctl+0x30a/0x3c0 [drm]
? drm_mode_destroy_dumb+0x40/0x40 [drm]
radeon_drm_ioctl+0x49/0x80 [radeon]
__x64_sys_ioctl+0x8e/0xd0

Avoid the issue by updating the counters in the delete_mem_notify callback
instead. Also, fix memory statistic updating in radeon_bo_move() to
identify the source type correctly. The source type needs to be saved
before the move, because the moved from object may be altered by the move.

Fixes: bfa3357ef9ab ("drm/ttm: allocate resource object instead of embedding it v2")
Signed-off-by: Mikel Rychliski <mikel@mikelr.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210624045121.15643-1-mikel@mikelr.com

show more ...

drm/radeon: Fix NULL dereference when updating memory stats

radeon_ttm_bo_destroy() is attempting to access the resource object to
update memory counters. However, the resource object is already fre

drm/radeon: Fix NULL dereference when updating memory stats

radeon_ttm_bo_destroy() is attempting to access the resource object to
update memory counters. However, the resource object is already freed when
ttm calls this function via the destroy callback. This causes an oops when
a bo is freed:

BUG: kernel NULL pointer dereference, address: 0000000000000010
RIP: 0010:radeon_ttm_bo_destroy+0x2c/0x100 [radeon]
Call Trace:
radeon_bo_unref+0x1a/0x30 [radeon]
radeon_gem_object_free+0x33/0x50 [radeon]
drm_gem_object_release_handle+0x69/0x70 [drm]
drm_gem_handle_delete+0x62/0xa0 [drm]
? drm_mode_destroy_dumb+0x40/0x40 [drm]
drm_ioctl_kernel+0xb2/0xf0 [drm]
drm_ioctl+0x30a/0x3c0 [drm]
? drm_mode_destroy_dumb+0x40/0x40 [drm]
radeon_drm_ioctl+0x49/0x80 [radeon]
__x64_sys_ioctl+0x8e/0xd0

Avoid the issue by updating the counters in the delete_mem_notify callback
instead. Also, fix memory statistic updating in radeon_bo_move() to
identify the source type correctly. The source type needs to be saved
before the move, because the moved from object may be altered by the move.

Fixes: bfa3357ef9ab ("drm/ttm: allocate resource object instead of embedding it v2")
Signed-off-by: Mikel Rychliski <mikel@mikelr.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210624045121.15643-1-mikel@mikelr.com

show more ...

drm/ttm: flip over the range manager to self allocated nodes

Start with the range manager to make the resource object the base
class for the allocated nodes.

While at it cleanup a lot of the code a

drm/ttm: flip over the range manager to self allocated nodes

Start with the range manager to make the resource object the base
class for the allocated nodes.

While at it cleanup a lot of the code around that.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210602100914.46246-2-christian.koenig@amd.com

show more ...

drm/ttm: allocate resource object instead of embedding it v2

To improve the handling we want the establish the resource object as base
class for the backend allocations.

v2: add missing error handl

drm/ttm: allocate resource object instead of embedding it v2

To improve the handling we want the establish the resource object as base
class for the backend allocations.

v2: add missing error handling

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210602100914.46246-1-christian.koenig@amd.com

show more ...

drm/ttm: rename bo->mem and make it a pointer

When we want to decouble resource management from buffer management we need to
be able to handle resources separately.

Add a resource pointer and renam

drm/ttm: rename bo->mem and make it a pointer

When we want to decouble resource management from buffer management we need to
be able to handle resources separately.

Add a resource pointer and rename bo->mem so that all code needs to
change to access the pointer instead.

No functional change.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210430092508.60710-4-christian.koenig@amd.com

show more ...

drm/radeon: Implement mmap as GEM object function

Moving the driver-specific mmap code into a GEM object function allows
for using DRM helpers for various mmap callbacks.

This change also allows to

drm/radeon: Implement mmap as GEM object function

Moving the driver-specific mmap code into a GEM object function allows
for using DRM helpers for various mmap callbacks.

This change also allows to support prime-based mmap via DRM's helper
drm_gem_prime_mmap().

Permission checks are implemented by drm_gem_mmap(), with an additional
check for radeon_ttm_tt_has_userptr() in the GEM object function. The
function radeon_verify_access() is now unused and has thus been removed.

As a side effect, radeon_ttm_vm_ops and radeon_ttm_fault() are now
implemented in amdgpu's GEM code.

v3:
* remove unnecessary checks from mmap (Christian)
v2:
* rename radeon_ttm_vm_ops and radeon_ttm_fault() to
radeon_gem_vm_ops and radeon_gem_fault() (Christian)
* fix commit description (Alex)

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210525151055.8174-4-tzimmermann@suse.de

show more ...

drm/radeon: Move AGP data structures into radeon

With the AGP code already duplicated, move over the AGP structures
from the legacy code base in to radeon. The AGP data structures that
are required

drm/radeon: Move AGP data structures into radeon

With the AGP code already duplicated, move over the AGP structures
from the legacy code base in to radeon. The AGP data structures that
are required by radeon are now declared within the driver. The AGP
instance is stored in struct radeon_device.agp.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210507185709.22797-3-tzimmermann@suse.de

show more ...

drm/radeon: Fix size overflow

ttm->num_pages is uint32. Hit overflow when << PAGE_SHIFT directly

Fixes: 230c079fdcf4 ("drm/ttm: make num_pages uint32_t")
Signed-off-by: xinhui pan <xinhui.pan@amd.c

drm/radeon: Fix size overflow

ttm->num_pages is uint32. Hit overflow when << PAGE_SHIFT directly

Fixes: 230c079fdcf4 ("drm/ttm: make num_pages uint32_t")
Signed-off-by: xinhui pan <xinhui.pan@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

show more ...

drm/radeon: avoid potential null pointer access

Leverage the same logic from amdgpu_ttm_tt_unpin_userptr.

Signed-off-by: Guchun Chen <guchun.chen@amd.com>
Reviewed-by: Christian König <christian.ko

drm/radeon: avoid potential null pointer access

Leverage the same logic from amdgpu_ttm_tt_unpin_userptr.

Signed-off-by: Guchun Chen <guchun.chen@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

show more ...