1.. _embargoed_hardware_issues:
2
3Embargoed hardware issues
4=========================
5
6Scope
7-----
8
9Hardware issues which result in security problems are a different category
10of security bugs than pure software bugs which only affect the Linux
11kernel.
12
13Hardware issues like Meltdown, Spectre, L1TF etc. must be treated
14differently because they usually affect all Operating Systems ("OS") and
15therefore need coordination across different OS vendors, distributions,
16hardware vendors and other parties. For some of the issues, software
17mitigations can depend on microcode or firmware updates, which need further
18coordination.
19
20.. _Contact:
21
22Contact
23-------
24
25The Linux kernel hardware security team is separate from the regular Linux
26kernel security team.
27
28The team only handles the coordination of embargoed hardware security
29issues.  Reports of pure software security bugs in the Linux kernel are not
30handled by this team and the reporter will be guided to contact the regular
31Linux kernel security team (:ref:`Documentation/admin-guide/
32<securitybugs>`) instead.
33
34The team can be contacted by email at <hardware-security@kernel.org>. This
35is a private list of security officers who will help you to coordinate an
36issue according to our documented process.
37
38The list is encrypted and email to the list can be sent by either PGP or
39S/MIME encrypted and must be signed with the reporter's PGP key or S/MIME
40certificate. The list's PGP key and S/MIME certificate are available from
41the following URLs:
42
43  - PGP: https://www.kernel.org/static/files/hardware-security.asc
44  - S/MIME: https://www.kernel.org/static/files/hardware-security.crt
45
46While hardware security issues are often handled by the affected hardware
47vendor, we welcome contact from researchers or individuals who have
48identified a potential hardware flaw.
49
50Hardware security officers
51^^^^^^^^^^^^^^^^^^^^^^^^^^
52
53The current team of hardware security officers:
54
55  - Linus Torvalds (Linux Foundation Fellow)
56  - Greg Kroah-Hartman (Linux Foundation Fellow)
57  - Thomas Gleixner (Linux Foundation Fellow)
58
59Operation of mailing-lists
60^^^^^^^^^^^^^^^^^^^^^^^^^^
61
62The encrypted mailing-lists which are used in our process are hosted on
63Linux Foundation's IT infrastructure. By providing this service, members
64of Linux Foundation's IT operations personnel technically have the
65ability to access the embargoed information, but are obliged to
66confidentiality by their employment contract. Linux Foundation IT
67personnel are also responsible for operating and managing the rest of
68kernel.org infrastructure.
69
70The Linux Foundation's current director of IT Project infrastructure is
71Konstantin Ryabitsev.
72
73
74Non-disclosure agreements
75-------------------------
76
77The Linux kernel hardware security team is not a formal body and therefore
78unable to enter into any non-disclosure agreements.  The kernel community
79is aware of the sensitive nature of such issues and offers a Memorandum of
80Understanding instead.
81
82
83Memorandum of Understanding
84---------------------------
85
86The Linux kernel community has a deep understanding of the requirement to
87keep hardware security issues under embargo for coordination between
88different OS vendors, distributors, hardware vendors and other parties.
89
90The Linux kernel community has successfully handled hardware security
91issues in the past and has the necessary mechanisms in place to allow
92community compliant development under embargo restrictions.
93
94The Linux kernel community has a dedicated hardware security team for
95initial contact, which oversees the process of handling such issues under
96embargo rules.
97
98The hardware security team identifies the developers (domain experts) who
99will form the initial response team for a particular issue. The initial
100response team can bring in further developers (domain experts) to address
101the issue in the best technical way.
102
103All involved developers pledge to adhere to the embargo rules and to keep
104the received information confidential. Violation of the pledge will lead to
105immediate exclusion from the current issue and removal from all related
106mailing-lists. In addition, the hardware security team will also exclude
107the offender from future issues. The impact of this consequence is a highly
108effective deterrent in our community. In case a violation happens the
109hardware security team will inform the involved parties immediately. If you
110or anyone becomes aware of a potential violation, please report it
111immediately to the Hardware security officers.
112
113
114Process
115^^^^^^^
116
117Due to the globally distributed nature of Linux kernel development,
118face-to-face meetings are almost impossible to address hardware security
119issues.  Phone conferences are hard to coordinate due to time zones and
120other factors and should be only used when absolutely necessary. Encrypted
121email has been proven to be the most effective and secure communication
122method for these types of issues.
123
124Start of Disclosure
125"""""""""""""""""""
126
127Disclosure starts by contacting the Linux kernel hardware security team by
128email. This initial contact should contain a description of the problem and
129a list of any known affected hardware. If your organization builds or
130distributes the affected hardware, we encourage you to also consider what
131other hardware could be affected.
132
133The hardware security team will provide an incident-specific encrypted
134mailing-list which will be used for initial discussion with the reporter,
135further disclosure and coordination.
136
137The hardware security team will provide the disclosing party a list of
138developers (domain experts) who should be informed initially about the
139issue after confirming with the developers  that they will adhere to this
140Memorandum of Understanding and the documented process. These developers
141form the initial response team and will be responsible for handling the
142issue after initial contact. The hardware security team is supporting the
143response team, but is not necessarily involved in the mitigation
144development process.
145
146While individual developers might be covered by a non-disclosure agreement
147via their employer, they cannot enter individual non-disclosure agreements
148in their role as Linux kernel developers. They will, however, agree to
149adhere to this documented process and the Memorandum of Understanding.
150
151The disclosing party should provide a list of contacts for all other
152entities who have already been, or should be, informed about the issue.
153This serves several purposes:
154
155 - The list of disclosed entities allows communication across the
156   industry, e.g. other OS vendors, HW vendors, etc.
157
158 - The disclosed entities can be contacted to name experts who should
159   participate in the mitigation development.
160
161 - If an expert which is required to handle an issue is employed by an
162   listed entity or member of an listed entity, then the response teams can
163   request the disclosure of that expert from that entity. This ensures
164   that the expert is also part of the entity's response team.
165
166Disclosure
167""""""""""
168
169The disclosing party provides detailed information to the initial response
170team via the specific encrypted mailing-list.
171
172From our experience the technical documentation of these issues is usually
173a sufficient starting point and further technical clarification is best
174done via email.
175
176Mitigation development
177""""""""""""""""""""""
178
179The initial response team sets up an encrypted mailing-list or repurposes
180an existing one if appropriate.
181
182Using a mailing-list is close to the normal Linux development process and
183has been successfully used in developing mitigations for various hardware
184security issues in the past.
185
186The mailing-list operates in the same way as normal Linux development.
187Patches are posted, discussed and reviewed and if agreed on applied to a
188non-public git repository which is only accessible to the participating
189developers via a secure connection. The repository contains the main
190development branch against the mainline kernel and backport branches for
191stable kernel versions as necessary.
192
193The initial response team will identify further experts from the Linux
194kernel developer community as needed. Bringing in experts can happen at any
195time of the development process and needs to be handled in a timely manner.
196
197If an expert is employed by or member of an entity on the disclosure list
198provided by the disclosing party, then participation will be requested from
199the relevant entity.
200
201If not, then the disclosing party will be informed about the experts
202participation. The experts are covered by the Memorandum of Understanding
203and the disclosing party is requested to acknowledge the participation. In
204case that the disclosing party has a compelling reason to object, then this
205objection has to be raised within five work days and resolved with the
206incident team immediately. If the disclosing party does not react within
207five work days this is taken as silent acknowledgement.
208
209After acknowledgement or resolution of an objection the expert is disclosed
210by the incident team and brought into the development process.
211
212
213Coordinated release
214"""""""""""""""""""
215
216The involved parties will negotiate the date and time where the embargo
217ends. At that point the prepared mitigations are integrated into the
218relevant kernel trees and published.
219
220While we understand that hardware security issues need coordinated embargo
221time, the embargo time should be constrained to the minimum time which is
222required for all involved parties to develop, test and prepare the
223mitigations. Extending embargo time artificially to meet conference talk
224dates or other non-technical reasons is creating more work and burden for
225the involved developers and response teams as the patches need to be kept
226up to date in order to follow the ongoing upstream kernel development,
227which might create conflicting changes.
228
229CVE assignment
230""""""""""""""
231
232Neither the hardware security team nor the initial response team assign
233CVEs, nor are CVEs required for the development process. If CVEs are
234provided by the disclosing party they can be used for documentation
235purposes.
236
237Process ambassadors
238-------------------
239
240For assistance with this process we have established ambassadors in various
241organizations, who can answer questions about or provide guidance on the
242reporting process and further handling. Ambassadors are not involved in the
243disclosure of a particular issue, unless requested by a response team or by
244an involved disclosed party. The current ambassadors list:
245
246  ============= ========================================================
247  AMD		Tom Lendacky <thomas.lendacky@amd.com>
248  Ampere	Darren Hart <darren@os.amperecomputing.com>
249  ARM		Catalin Marinas <catalin.marinas@arm.com>
250  IBM Power	Anton Blanchard <anton@linux.ibm.com>
251  IBM Z		Christian Borntraeger <borntraeger@de.ibm.com>
252  Intel		Tony Luck <tony.luck@intel.com>
253  Qualcomm	Trilok Soni <tsoni@codeaurora.org>
254
255  Microsoft	James Morris <jamorris@linux.microsoft.com>
256  VMware
257  Xen		Andrew Cooper <andrew.cooper3@citrix.com>
258
259  Canonical	John Johansen <john.johansen@canonical.com>
260  Debian	Ben Hutchings <ben@decadent.org.uk>
261  Oracle	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
262  Red Hat	Josh Poimboeuf <jpoimboe@redhat.com>
263  SUSE		Jiri Kosina <jkosina@suse.cz>
264
265  Amazon
266  Google	Kees Cook <keescook@chromium.org>
267
268  GCC
269  LLVM		Nick Desaulniers <ndesaulniers@google.com>
270  ============= ========================================================
271
272If you want your organization to be added to the ambassadors list, please
273contact the hardware security team. The nominated ambassador has to
274understand and support our process fully and is ideally well connected in
275the Linux kernel community.
276
277Encrypted mailing-lists
278-----------------------
279
280We use encrypted mailing-lists for communication. The operating principle
281of these lists is that email sent to the list is encrypted either with the
282list's PGP key or with the list's S/MIME certificate. The mailing-list
283software decrypts the email and re-encrypts it individually for each
284subscriber with the subscriber's PGP key or S/MIME certificate. Details
285about the mailing-list software and the setup which is used to ensure the
286security of the lists and protection of the data can be found here:
287https://korg.wiki.kernel.org/userdoc/remail.
288
289List keys
290^^^^^^^^^
291
292For initial contact see :ref:`Contact`. For incident specific mailing-lists
293the key and S/MIME certificate are conveyed to the subscribers by email
294sent from the specific list.
295
296Subscription to incident specific lists
297^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
298
299Subscription is handled by the response teams. Disclosed parties who want
300to participate in the communication send a list of potential subscribers to
301the response team so the response team can validate subscription requests.
302
303Each subscriber needs to send a subscription request to the response team
304by email. The email must be signed with the subscriber's PGP key or S/MIME
305certificate. If a PGP key is used, it must be available from a public key
306server and is ideally connected to the Linux kernel's PGP web of trust. See
307also: https://www.kernel.org/signature.html.
308
309The response team verifies that the subscriber request is valid and adds
310the subscriber to the list. After subscription the subscriber will receive
311email from the mailing-list which is signed either with the list's PGP key
312or the list's S/MIME certificate. The subscriber's email client can extract
313the PGP key or the S/MIME certificate from the signature so the subscriber
314can send encrypted email to the list.
315
316