1.. SPDX-License-Identifier: GPL-2.0
2
3=================
4Inline Encryption
5=================
6
7Background
8==========
9
10Inline encryption hardware sits logically between memory and the disk, and can
11en/decrypt data as it goes in/out of the disk. Inline encryption hardware has a
12fixed number of "keyslots" - slots into which encryption contexts (i.e. the
13encryption key, encryption algorithm, data unit size) can be programmed by the
14kernel at any time. Each request sent to the disk can be tagged with the index
15of a keyslot (and also a data unit number to act as an encryption tweak), and
16the inline encryption hardware will en/decrypt the data in the request with the
17encryption context programmed into that keyslot. This is very different from
18full disk encryption solutions like self encrypting drives/TCG OPAL/ATA
19Security standards, since with inline encryption, any block on disk could be
20encrypted with any encryption context the kernel chooses.
21
22
23Objective
24=========
25
26We want to support inline encryption (IE) in the kernel.
27To allow for testing, we also want a crypto API fallback when actual
28IE hardware is absent. We also want IE to work with layered devices
29like dm and loopback (i.e. we want to be able to use the IE hardware
30of the underlying devices if present, or else fall back to crypto API
31en/decryption).
32
33
34Constraints and notes
35=====================
36
37- IE hardware has a limited number of "keyslots" that can be programmed
38  with an encryption context (key, algorithm, data unit size, etc.) at any time.
39  One can specify a keyslot in a data request made to the device, and the
40  device will en/decrypt the data using the encryption context programmed into
41  that specified keyslot. When possible, we want to make multiple requests with
42  the same encryption context share the same keyslot.
43
44- We need a way for upper layers like filesystems to specify an encryption
45  context to use for en/decrypting a struct bio, and a device driver (like UFS)
46  needs to be able to use that encryption context when it processes the bio.
47
48- We need a way for device drivers to expose their inline encryption
49  capabilities in a unified way to the upper layers.
50
51
52Design
53======
54
55We add a struct bio_crypt_ctx to struct bio that can
56represent an encryption context, because we need to be able to pass this
57encryption context from the upper layers (like the fs layer) to the
58device driver to act upon.
59
60While IE hardware works on the notion of keyslots, the FS layer has no
61knowledge of keyslots - it simply wants to specify an encryption context to
62use while en/decrypting a bio.
63
64We introduce a keyslot manager (KSM) that handles the translation from
65encryption contexts specified by the FS to keyslots on the IE hardware.
66This KSM also serves as the way IE hardware can expose its capabilities to
67upper layers. The generic mode of operation is: each device driver that wants
68to support IE will construct a KSM and set it up in its struct request_queue.
69Upper layers that want to use IE on this device can then use this KSM in
70the device's struct request_queue to translate an encryption context into
71a keyslot. The presence of the KSM in the request queue shall be used to mean
72that the device supports IE.
73
74The KSM uses refcounts to track which keyslots are idle (either they have no
75encryption context programmed, or there are no in-flight struct bios
76referencing that keyslot). When a new encryption context needs a keyslot, it
77tries to find a keyslot that has already been programmed with the same
78encryption context, and if there is no such keyslot, it evicts the least
79recently used idle keyslot and programs the new encryption context into that
80one. If no idle keyslots are available, then the caller will sleep until there
81is at least one.
82
83
84blk-mq changes, other block layer changes and blk-crypto-fallback
85=================================================================
86
87We add a pointer to a ``bi_crypt_context`` and ``keyslot`` to
88struct request. These will be referred to as the ``crypto fields``
89for the request. This ``keyslot`` is the keyslot into which the
90``bi_crypt_context`` has been programmed in the KSM of the ``request_queue``
91that this request is being sent to.
92
93We introduce ``block/blk-crypto-fallback.c``, which allows upper layers to remain
94blissfully unaware of whether or not real inline encryption hardware is present
95underneath. When a bio is submitted with a target ``request_queue`` that doesn't
96support the encryption context specified with the bio, the block layer will
97en/decrypt the bio with the blk-crypto-fallback.
98
99If the bio is a ``WRITE`` bio, a bounce bio is allocated, and the data in the bio
100is encrypted stored in the bounce bio - blk-mq will then proceed to process the
101bounce bio as if it were not encrypted at all (except when blk-integrity is
102concerned). ``blk-crypto-fallback`` sets the bounce bio's ``bi_end_io`` to an
103internal function that cleans up the bounce bio and ends the original bio.
104
105If the bio is a ``READ`` bio, the bio's ``bi_end_io`` (and also ``bi_private``)
106is saved and overwritten by ``blk-crypto-fallback`` to
107``bio_crypto_fallback_decrypt_bio``.  The bio's ``bi_crypt_context`` is also
108overwritten with ``NULL``, so that to the rest of the stack, the bio looks
109as if it was a regular bio that never had an encryption context specified.
110``bio_crypto_fallback_decrypt_bio`` will decrypt the bio, restore the original
111``bi_end_io`` (and also ``bi_private``) and end the bio again.
112
113Regardless of whether real inline encryption hardware is used or the
114blk-crypto-fallback is used, the ciphertext written to disk (and hence the
115on-disk format of data) will be the same (assuming the hardware's implementation
116of the algorithm being used adheres to spec and functions correctly).
117
118If a ``request queue``'s inline encryption hardware claimed to support the
119encryption context specified with a bio, then it will not be handled by the
120``blk-crypto-fallback``. We will eventually reach a point in blk-mq when a
121struct request needs to be allocated for that bio. At that point,
122blk-mq tries to program the encryption context into the ``request_queue``'s
123keyslot_manager, and obtain a keyslot, which it stores in its newly added
124``keyslot`` field. This keyslot is released when the request is completed.
125
126When the first bio is added to a request, ``blk_crypto_rq_bio_prep`` is called,
127which sets the request's ``crypt_ctx`` to a copy of the bio's
128``bi_crypt_context``. bio_crypt_do_front_merge is called whenever a subsequent
129bio is merged to the front of the request, which updates the ``crypt_ctx`` of
130the request so that it matches the newly merged bio's ``bi_crypt_context``. In particular, the request keeps a copy of the ``bi_crypt_context`` of the first
131bio in its bio-list (blk-mq needs to be careful to maintain this invariant
132during bio and request merges).
133
134To make it possible for inline encryption to work with request queue based
135layered devices, when a request is cloned, its ``crypto fields`` are cloned as
136well. When the cloned request is submitted, blk-mq programs the
137``bi_crypt_context`` of the request into the clone's request_queue's keyslot
138manager, and stores the returned keyslot in the clone's ``keyslot``.
139
140
141API presented to users of the block layer
142=========================================
143
144``struct blk_crypto_key`` represents a crypto key (the raw key, size of the
145key, the crypto algorithm to use, the data unit size to use, and the number of
146bytes required to represent data unit numbers that will be specified with the
147``bi_crypt_context``).
148
149``blk_crypto_init_key`` allows upper layers to initialize such a
150``blk_crypto_key``.
151
152``bio_crypt_set_ctx`` should be called on any bio that a user of
153the block layer wants en/decrypted via inline encryption (or the
154blk-crypto-fallback, if hardware support isn't available for the desired
155crypto configuration). This function takes the ``blk_crypto_key`` and the
156data unit number (DUN) to use when en/decrypting the bio.
157
158``blk_crypto_config_supported`` allows upper layers to query whether or not the
159an encryption context passed to request queue can be handled by blk-crypto
160(either by real inline encryption hardware, or by the blk-crypto-fallback).
161This is useful e.g. when blk-crypto-fallback is disabled, and the upper layer
162wants to use an algorithm that may not supported by hardware - this function
163lets the upper layer know ahead of time that the algorithm isn't supported,
164and the upper layer can fallback to something else if appropriate.
165
166``blk_crypto_start_using_key`` - Upper layers must call this function on
167``blk_crypto_key`` and a ``request_queue`` before using the key with any bio
168headed for that ``request_queue``. This function ensures that either the
169hardware supports the key's crypto settings, or the crypto API fallback has
170transforms for the needed mode allocated and ready to go. Note that this
171function may allocate an ``skcipher``, and must not be called from the data
172path, since allocating ``skciphers`` from the data path can deadlock.
173
174``blk_crypto_evict_key`` *must* be called by upper layers before a
175``blk_crypto_key`` is freed. Further, it *must* only be called only once
176there are no more in-flight requests that use that ``blk_crypto_key``.
177``blk_crypto_evict_key`` will ensure that a key is removed from any keyslots in
178inline encryption hardware that the key might have been programmed into (or the blk-crypto-fallback).
179
180API presented to device drivers
181===============================
182
183A :c:type:``struct blk_keyslot_manager`` should be set up by device drivers in
184the ``request_queue`` of the device. The device driver needs to call
185``blk_ksm_init`` (or its resource-managed variant ``devm_blk_ksm_init``) on the
186``blk_keyslot_manager``, while specifying the number of keyslots supported by
187the hardware.
188
189The device driver also needs to tell the KSM how to actually manipulate the
190IE hardware in the device to do things like programming the crypto key into
191the IE hardware into a particular keyslot. All this is achieved through the
192struct blk_ksm_ll_ops field in the KSM that the device driver
193must fill up after initing the ``blk_keyslot_manager``.
194
195The KSM also handles runtime power management for the device when applicable
196(e.g. when it wants to program a crypto key into the IE hardware, the device
197must be runtime powered on) - so the device driver must also set the ``dev``
198field in the ksm to point to the `struct device` for the KSM to use for runtime
199power management.
200
201``blk_ksm_reprogram_all_keys`` can be called by device drivers if the device
202needs each and every of its keyslots to be reprogrammed with the key it
203"should have" at the point in time when the function is called. This is useful
204e.g. if a device loses all its keys on runtime power down/up.
205
206If the driver used ``blk_ksm_init`` instead of ``devm_blk_ksm_init``, then
207``blk_ksm_destroy`` should be called to free up all resources used by a
208``blk_keyslot_manager`` once it is no longer needed.
209
210Layered Devices
211===============
212
213Request queue based layered devices like dm-rq that wish to support IE need to
214create their own keyslot manager for their request queue, and expose whatever
215functionality they choose. When a layered device wants to pass a clone of that
216request to another ``request_queue``, blk-crypto will initialize and prepare the
217clone as necessary - see ``blk_crypto_insert_cloned_request`` in
218``blk-crypto.c``.
219
220
221Future Optimizations for layered devices
222========================================
223
224Creating a keyslot manager for a layered device uses up memory for each
225keyslot, and in general, a layered device merely passes the request on to a
226"child" device, so the keyslots in the layered device itself are completely
227unused, and don't need any refcounting or keyslot programming. We can instead
228define a new type of KSM; the "passthrough KSM", that layered devices can use
229to advertise an unlimited number of keyslots, and support for any encryption
230algorithms they choose, while not actually using any memory for each keyslot.
231Another use case for the "passthrough KSM" is for IE devices that do not have a
232limited number of keyslots.
233
234
235Interaction between inline encryption and blk integrity
236=======================================================
237
238At the time of this patch, there is no real hardware that supports both these
239features. However, these features do interact with each other, and it's not
240completely trivial to make them both work together properly. In particular,
241when a WRITE bio wants to use inline encryption on a device that supports both
242features, the bio will have an encryption context specified, after which
243its integrity information is calculated (using the plaintext data, since
244the encryption will happen while data is being written), and the data and
245integrity info is sent to the device. Obviously, the integrity info must be
246verified before the data is encrypted. After the data is encrypted, the device
247must not store the integrity info that it received with the plaintext data
248since that might reveal information about the plaintext data. As such, it must
249re-generate the integrity info from the ciphertext data and store that on disk
250instead. Another issue with storing the integrity info of the plaintext data is
251that it changes the on disk format depending on whether hardware inline
252encryption support is present or the kernel crypto API fallback is used (since
253if the fallback is used, the device will receive the integrity info of the
254ciphertext, not that of the plaintext).
255
256Because there isn't any real hardware yet, it seems prudent to assume that
257hardware implementations might not implement both features together correctly,
258and disallow the combination for now. Whenever a device supports integrity, the
259kernel will pretend that the device does not support hardware inline encryption
260(by essentially setting the keyslot manager in the request_queue of the device
261to NULL). When the crypto API fallback is enabled, this means that all bios with
262and encryption context will use the fallback, and IO will complete as usual.
263When the fallback is disabled, a bio with an encryption context will be failed.
264