dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the numb

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the number of pages.

Fixes: 4d0564785bb0 ("dma-direct: factor out dma_set_{de,en}crypted helpers")
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the numb

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the number of pages.

Fixes: 4d0564785bb0 ("dma-direct: factor out dma_set_{de,en}crypted helpers")
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the numb

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the number of pages.

Fixes: 4d0564785bb0 ("dma-direct: factor out dma_set_{de,en}crypted helpers")
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the numb

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the number of pages.

Fixes: 4d0564785bb0 ("dma-direct: factor out dma_set_{de,en}crypted helpers")
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the numb

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the number of pages.

Fixes: 4d0564785bb0 ("dma-direct: factor out dma_set_{de,en}crypted helpers")
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the numb

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the number of pages.

Fixes: 4d0564785bb0 ("dma-direct: factor out dma_set_{de,en}crypted helpers")
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the numb

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the number of pages.

Fixes: 4d0564785bb0 ("dma-direct: factor out dma_set_{de,en}crypted helpers")
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the numb

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the number of pages.

Fixes: 4d0564785bb0 ("dma-direct: factor out dma_set_{de,en}crypted helpers")
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the numb

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the number of pages.

Fixes: 4d0564785bb0 ("dma-direct: factor out dma_set_{de,en}crypted helpers")
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the numb

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the number of pages.

Fixes: 4d0564785bb0 ("dma-direct: factor out dma_set_{de,en}crypted helpers")
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the numb

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the number of pages.

Fixes: 4d0564785bb0 ("dma-direct: factor out dma_set_{de,en}crypted helpers")
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the numb

dma-direct: use the correct size for dma_set_encrypted()

commit 3be4562584bba603f33863a00c1c32eecf772ee6 upstream.

The third parameter of dma_set_encrypted() is a size in bytes rather than
the number of pages.

Fixes: 4d0564785bb0 ("dma-direct: factor out dma_set_{de,en}crypted helpers")
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

dma-direct: don't over-decrypt memory

[ Upstream commit 4a37f3dd9a83186cb88d44808ab35b78375082c9 ]

The original x86 sev_alloc() only called set_memory_decrypted() on
memory returned by alloc_pages_

dma-direct: don't over-decrypt memory

[ Upstream commit 4a37f3dd9a83186cb88d44808ab35b78375082c9 ]

The original x86 sev_alloc() only called set_memory_decrypted() on
memory returned by alloc_pages_node(), so the page order calculation
fell out of that logic. However, the common dma-direct code has several
potential allocators, not all of which are guaranteed to round up the
underlying allocation to a power-of-two size, so carrying over that
calculation for the encryption/decryption size was a mistake. Fix it by
rounding to a *number* of pages, rather than an order.

Until recently there was an even worse interaction with DMA_DIRECT_REMAP
where we could have ended up decrypting part of the next adjacent
vmalloc area, only averted by no architecture actually supporting both
configs at once. Don't ask how I found that one out...

Fixes: c10f07aa27da ("dma/direct: Handle force decryption for DMA coherent buffers in common code")
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: David Rientjes <rientjes@google.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

dma-direct: always leak memory that can't be re-encrypted

[ Upstream commit a90cf30437489343b8386ae87b4827b6d6c3ed50 ]

We must never let unencrypted memory go back into the general page pool.
So if

dma-direct: always leak memory that can't be re-encrypted

[ Upstream commit a90cf30437489343b8386ae87b4827b6d6c3ed50 ]

We must never let unencrypted memory go back into the general page pool.
So if we fail to set it back to encrypted when freeing DMA memory, leak
the memory instead and warn the user.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

dma-direct: don't call dma_set_decrypted for remapped allocations

[ Upstream commit 5570449b6876f215d49ac4db9ccce6ff7aa1e20a ]

Remapped allocations handle the encrypted bit through the pgprot passe

dma-direct: don't call dma_set_decrypted for remapped allocations

[ Upstream commit 5570449b6876f215d49ac4db9ccce6ff7aa1e20a ]

Remapped allocations handle the encrypted bit through the pgprot passed
to vmap, so there is no call dma_set_decrypted. Note that this case is
currently entirely theoretical as no valid kernel configuration supports
remapped allocations and memory encryption currently.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

dma-direct: factor out dma_set_{de,en}crypted helpers

[ Upstream commit 4d0564785bb03841e4b5c5b31aa4ecd1eb0d01bb ]

Factor out helpers the make dealing with memory encryption a little less
cumbersom

dma-direct: factor out dma_set_{de,en}crypted helpers

[ Upstream commit 4d0564785bb03841e4b5c5b31aa4ecd1eb0d01bb ]

Factor out helpers the make dealing with memory encryption a little less
cumbersome.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

dma-direct: don't fail on highmem CMA pages in dma_direct_alloc_pages

[ Upstream commit 92826e967535db2eb117db227b1191aaf98e4bb3 ]

When dma_direct_alloc_pages encounters a highmem page it just give

dma-direct: don't fail on highmem CMA pages in dma_direct_alloc_pages

[ Upstream commit 92826e967535db2eb117db227b1191aaf98e4bb3 ]

When dma_direct_alloc_pages encounters a highmem page it just gives up
currently. But what we really should do is to try memory using the
page allocator instead - without this platforms with a global highmem
CMA pool will fail all dma_alloc_pages allocations.

Fixes: efa70f2fdc84 ("dma-mapping: add a new dma_alloc_pages API")
Reported-by: Mark O'Neill <mao@tumblingdice.co.uk>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

dma-direct: factor out a helper for DMA_ATTR_NO_KERNEL_MAPPING allocations

[ Upstream commit d541ae55d538265861ef729a64d2d816d34ef1e2 ]

Split the code for DMA_ATTR_NO_KERNEL_MAPPING allocations int

dma-direct: factor out a helper for DMA_ATTR_NO_KERNEL_MAPPING allocations

[ Upstream commit d541ae55d538265861ef729a64d2d816d34ef1e2 ]

Split the code for DMA_ATTR_NO_KERNEL_MAPPING allocations into a separate
helper to make dma_direct_alloc a little more readable.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: David Rientjes <rientjes@google.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

dma-direct: don't over-decrypt memory

[ Upstream commit 4a37f3dd9a83186cb88d44808ab35b78375082c9 ]

The original x86 sev_alloc() only called set_memory_decrypted() on
memory returned by alloc_pages_

dma-direct: don't over-decrypt memory

[ Upstream commit 4a37f3dd9a83186cb88d44808ab35b78375082c9 ]

The original x86 sev_alloc() only called set_memory_decrypted() on
memory returned by alloc_pages_node(), so the page order calculation
fell out of that logic. However, the common dma-direct code has several
potential allocators, not all of which are guaranteed to round up the
underlying allocation to a power-of-two size, so carrying over that
calculation for the encryption/decryption size was a mistake. Fix it by
rounding to a *number* of pages, rather than an order.

Until recently there was an even worse interaction with DMA_DIRECT_REMAP
where we could have ended up decrypting part of the next adjacent
vmalloc area, only averted by no architecture actually supporting both
configs at once. Don't ask how I found that one out...

Fixes: c10f07aa27da ("dma/direct: Handle force decryption for DMA coherent buffers in common code")
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: David Rientjes <rientjes@google.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

dma-direct: always leak memory that can't be re-encrypted

[ Upstream commit a90cf30437489343b8386ae87b4827b6d6c3ed50 ]

We must never let unencrypted memory go back into the general page pool.
So if

dma-direct: always leak memory that can't be re-encrypted

[ Upstream commit a90cf30437489343b8386ae87b4827b6d6c3ed50 ]

We must never let unencrypted memory go back into the general page pool.
So if we fail to set it back to encrypted when freeing DMA memory, leak
the memory instead and warn the user.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

dma-direct: don't call dma_set_decrypted for remapped allocations

[ Upstream commit 5570449b6876f215d49ac4db9ccce6ff7aa1e20a ]

Remapped allocations handle the encrypted bit through the pgprot passe

dma-direct: don't call dma_set_decrypted for remapped allocations

[ Upstream commit 5570449b6876f215d49ac4db9ccce6ff7aa1e20a ]

Remapped allocations handle the encrypted bit through the pgprot passed
to vmap, so there is no call dma_set_decrypted. Note that this case is
currently entirely theoretical as no valid kernel configuration supports
remapped allocations and memory encryption currently.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

dma-direct: factor out dma_set_{de,en}crypted helpers

[ Upstream commit 4d0564785bb03841e4b5c5b31aa4ecd1eb0d01bb ]

Factor out helpers the make dealing with memory encryption a little less
cumbersom

dma-direct: factor out dma_set_{de,en}crypted helpers

[ Upstream commit 4d0564785bb03841e4b5c5b31aa4ecd1eb0d01bb ]

Factor out helpers the make dealing with memory encryption a little less
cumbersome.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

dma-direct: don't fail on highmem CMA pages in dma_direct_alloc_pages

[ Upstream commit 92826e967535db2eb117db227b1191aaf98e4bb3 ]

When dma_direct_alloc_pages encounters a highmem page it just give

dma-direct: don't fail on highmem CMA pages in dma_direct_alloc_pages

[ Upstream commit 92826e967535db2eb117db227b1191aaf98e4bb3 ]

When dma_direct_alloc_pages encounters a highmem page it just gives up
currently. But what we really should do is to try memory using the
page allocator instead - without this platforms with a global highmem
CMA pool will fail all dma_alloc_pages allocations.

Fixes: efa70f2fdc84 ("dma-mapping: add a new dma_alloc_pages API")
Reported-by: Mark O'Neill <mao@tumblingdice.co.uk>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

dma-direct: factor out a helper for DMA_ATTR_NO_KERNEL_MAPPING allocations

[ Upstream commit d541ae55d538265861ef729a64d2d816d34ef1e2 ]

Split the code for DMA_ATTR_NO_KERNEL_MAPPING allocations int

dma-direct: factor out a helper for DMA_ATTR_NO_KERNEL_MAPPING allocations

[ Upstream commit d541ae55d538265861ef729a64d2d816d34ef1e2 ]

Split the code for DMA_ATTR_NO_KERNEL_MAPPING allocations into a separate
helper to make dma_direct_alloc a little more readable.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: David Rientjes <rientjes@google.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

dma-direct: add support for dma_coherent_default_memory

Add an option to allocate uncached memory for dma_alloc_coherent from
the global dma_coherent_default_memory. This will allow to move
arm-nom

dma-direct: add support for dma_coherent_default_memory

Add an option to allocate uncached memory for dma_alloc_coherent from
the global dma_coherent_default_memory. This will allow to move
arm-nommu (and eventually other platforms) to use generic code for
allocating uncached memory from a pre-populated pool.

Note that this is a different pool from the one that platforms that
can remap at runtime use for GFP_ATOMIC allocations for now, although
there might be opportunities to eventually end up with a common codebase
for the two use cases.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Tested-by: Dillon Min <dillon.minfei@gmail.com>

show more ...