1Embargoed hardware issues 2========================= 3 4Scope 5----- 6 7Hardware issues which result in security problems are a different category 8of security bugs than pure software bugs which only affect the Linux 9kernel. 10 11Hardware issues like Meltdown, Spectre, L1TF etc. must be treated 12differently because they usually affect all Operating Systems ("OS") and 13therefore need coordination across different OS vendors, distributions, 14hardware vendors and other parties. For some of the issues, software 15mitigations can depend on microcode or firmware updates, which need further 16coordination. 17 18.. _Contact: 19 20Contact 21------- 22 23The Linux kernel hardware security team is separate from the regular Linux 24kernel security team. 25 26The team only handles the coordination of embargoed hardware security 27issues. Reports of pure software security bugs in the Linux kernel are not 28handled by this team and the reporter will be guided to contact the regular 29Linux kernel security team (:ref:`Documentation/admin-guide/ 30<securitybugs>`) instead. 31 32The team can be contacted by email at <hardware-security@kernel.org>. This 33is a private list of security officers who will help you to coordinate an 34issue according to our documented process. 35 36The list is encrypted and email to the list can be sent by either PGP or 37S/MIME encrypted and must be signed with the reporter's PGP key or S/MIME 38certificate. The list's PGP key and S/MIME certificate are available from 39the following URLs: 40 41 - PGP: https://www.kernel.org/static/files/hardware-security.asc 42 - S/MIME: https://www.kernel.org/static/files/hardware-security.crt 43 44While hardware security issues are often handled by the affected hardware 45vendor, we welcome contact from researchers or individuals who have 46identified a potential hardware flaw. 47 48Hardware security officers 49^^^^^^^^^^^^^^^^^^^^^^^^^^ 50 51The current team of hardware security officers: 52 53 - Linus Torvalds (Linux Foundation Fellow) 54 - Greg Kroah-Hartman (Linux Foundation Fellow) 55 - Thomas Gleixner (Linux Foundation Fellow) 56 57Operation of mailing-lists 58^^^^^^^^^^^^^^^^^^^^^^^^^^ 59 60The encrypted mailing-lists which are used in our process are hosted on 61Linux Foundation's IT infrastructure. By providing this service, members 62of Linux Foundation's IT operations personnel technically have the 63ability to access the embargoed information, but are obliged to 64confidentiality by their employment contract. Linux Foundation IT 65personnel are also responsible for operating and managing the rest of 66kernel.org infrastructure. 67 68The Linux Foundation's current director of IT Project infrastructure is 69Konstantin Ryabitsev. 70 71 72Non-disclosure agreements 73------------------------- 74 75The Linux kernel hardware security team is not a formal body and therefore 76unable to enter into any non-disclosure agreements. The kernel community 77is aware of the sensitive nature of such issues and offers a Memorandum of 78Understanding instead. 79 80 81Memorandum of Understanding 82--------------------------- 83 84The Linux kernel community has a deep understanding of the requirement to 85keep hardware security issues under embargo for coordination between 86different OS vendors, distributors, hardware vendors and other parties. 87 88The Linux kernel community has successfully handled hardware security 89issues in the past and has the necessary mechanisms in place to allow 90community compliant development under embargo restrictions. 91 92The Linux kernel community has a dedicated hardware security team for 93initial contact, which oversees the process of handling such issues under 94embargo rules. 95 96The hardware security team identifies the developers (domain experts) who 97will form the initial response team for a particular issue. The initial 98response team can bring in further developers (domain experts) to address 99the issue in the best technical way. 100 101All involved developers pledge to adhere to the embargo rules and to keep 102the received information confidential. Violation of the pledge will lead to 103immediate exclusion from the current issue and removal from all related 104mailing-lists. In addition, the hardware security team will also exclude 105the offender from future issues. The impact of this consequence is a highly 106effective deterrent in our community. In case a violation happens the 107hardware security team will inform the involved parties immediately. If you 108or anyone becomes aware of a potential violation, please report it 109immediately to the Hardware security officers. 110 111 112Process 113^^^^^^^ 114 115Due to the globally distributed nature of Linux kernel development, 116face-to-face meetings are almost impossible to address hardware security 117issues. Phone conferences are hard to coordinate due to time zones and 118other factors and should be only used when absolutely necessary. Encrypted 119email has been proven to be the most effective and secure communication 120method for these types of issues. 121 122Start of Disclosure 123""""""""""""""""""" 124 125Disclosure starts by contacting the Linux kernel hardware security team by 126email. This initial contact should contain a description of the problem and 127a list of any known affected hardware. If your organization builds or 128distributes the affected hardware, we encourage you to also consider what 129other hardware could be affected. 130 131The hardware security team will provide an incident-specific encrypted 132mailing-list which will be used for initial discussion with the reporter, 133further disclosure and coordination. 134 135The hardware security team will provide the disclosing party a list of 136developers (domain experts) who should be informed initially about the 137issue after confirming with the developers that they will adhere to this 138Memorandum of Understanding and the documented process. These developers 139form the initial response team and will be responsible for handling the 140issue after initial contact. The hardware security team is supporting the 141response team, but is not necessarily involved in the mitigation 142development process. 143 144While individual developers might be covered by a non-disclosure agreement 145via their employer, they cannot enter individual non-disclosure agreements 146in their role as Linux kernel developers. They will, however, agree to 147adhere to this documented process and the Memorandum of Understanding. 148 149The disclosing party should provide a list of contacts for all other 150entities who have already been, or should be, informed about the issue. 151This serves several purposes: 152 153 - The list of disclosed entities allows communication accross the 154 industry, e.g. other OS vendors, HW vendors, etc. 155 156 - The disclosed entities can be contacted to name experts who should 157 participate in the mitigation development. 158 159 - If an expert which is required to handle an issue is employed by an 160 listed entity or member of an listed entity, then the response teams can 161 request the disclosure of that expert from that entity. This ensures 162 that the expert is also part of the entity's response team. 163 164Disclosure 165"""""""""" 166 167The disclosing party provides detailed information to the initial response 168team via the specific encrypted mailing-list. 169 170From our experience the technical documentation of these issues is usually 171a sufficient starting point and further technical clarification is best 172done via email. 173 174Mitigation development 175"""""""""""""""""""""" 176 177The initial response team sets up an encrypted mailing-list or repurposes 178an existing one if appropriate. 179 180Using a mailing-list is close to the normal Linux development process and 181has been successfully used in developing mitigations for various hardware 182security issues in the past. 183 184The mailing-list operates in the same way as normal Linux development. 185Patches are posted, discussed and reviewed and if agreed on applied to a 186non-public git repository which is only accessible to the participating 187developers via a secure connection. The repository contains the main 188development branch against the mainline kernel and backport branches for 189stable kernel versions as necessary. 190 191The initial response team will identify further experts from the Linux 192kernel developer community as needed. Bringing in experts can happen at any 193time of the development process and needs to be handled in a timely manner. 194 195If an expert is employed by or member of an entity on the disclosure list 196provided by the disclosing party, then participation will be requested from 197the relevant entity. 198 199If not, then the disclosing party will be informed about the experts 200participation. The experts are covered by the Memorandum of Understanding 201and the disclosing party is requested to acknowledge the participation. In 202case that the disclosing party has a compelling reason to object, then this 203objection has to be raised within five work days and resolved with the 204incident team immediately. If the disclosing party does not react within 205five work days this is taken as silent acknowledgement. 206 207After acknowledgement or resolution of an objection the expert is disclosed 208by the incident team and brought into the development process. 209 210 211Coordinated release 212""""""""""""""""""" 213 214The involved parties will negotiate the date and time where the embargo 215ends. At that point the prepared mitigations are integrated into the 216relevant kernel trees and published. 217 218While we understand that hardware security issues need coordinated embargo 219time, the embargo time should be constrained to the minimum time which is 220required for all involved parties to develop, test and prepare the 221mitigations. Extending embargo time artificially to meet conference talk 222dates or other non-technical reasons is creating more work and burden for 223the involved developers and response teams as the patches need to be kept 224up to date in order to follow the ongoing upstream kernel development, 225which might create conflicting changes. 226 227CVE assignment 228"""""""""""""" 229 230Neither the hardware security team nor the initial response team assign 231CVEs, nor are CVEs required for the development process. If CVEs are 232provided by the disclosing party they can be used for documentation 233purposes. 234 235Process ambassadors 236------------------- 237 238For assistance with this process we have established ambassadors in various 239organizations, who can answer questions about or provide guidance on the 240reporting process and further handling. Ambassadors are not involved in the 241disclosure of a particular issue, unless requested by a response team or by 242an involved disclosed party. The current ambassadors list: 243 244 ============= ======================================================== 245 ARM 246 AMD Tom Lendacky <tom.lendacky@amd.com> 247 IBM 248 Intel Tony Luck <tony.luck@intel.com> 249 Qualcomm Trilok Soni <tsoni@codeaurora.org> 250 251 Microsoft Sasha Levin <sashal@kernel.org> 252 VMware 253 Xen Andrew Cooper <andrew.cooper3@citrix.com> 254 255 Canonical Tyler Hicks <tyhicks@canonical.com> 256 Debian Ben Hutchings <ben@decadent.org.uk> 257 Oracle Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> 258 Red Hat Josh Poimboeuf <jpoimboe@redhat.com> 259 SUSE Jiri Kosina <jkosina@suse.cz> 260 261 Amazon 262 Google Kees Cook <keescook@chromium.org> 263 ============= ======================================================== 264 265If you want your organization to be added to the ambassadors list, please 266contact the hardware security team. The nominated ambassador has to 267understand and support our process fully and is ideally well connected in 268the Linux kernel community. 269 270Encrypted mailing-lists 271----------------------- 272 273We use encrypted mailing-lists for communication. The operating principle 274of these lists is that email sent to the list is encrypted either with the 275list's PGP key or with the list's S/MIME certificate. The mailing-list 276software decrypts the email and re-encrypts it individually for each 277subscriber with the subscriber's PGP key or S/MIME certificate. Details 278about the mailing-list software and the setup which is used to ensure the 279security of the lists and protection of the data can be found here: 280https://korg.wiki.kernel.org/userdoc/remail. 281 282List keys 283^^^^^^^^^ 284 285For initial contact see :ref:`Contact`. For incident specific mailing-lists 286the key and S/MIME certificate are conveyed to the subscribers by email 287sent from the specific list. 288 289Subscription to incident specific lists 290^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 291 292Subscription is handled by the response teams. Disclosed parties who want 293to participate in the communication send a list of potential subscribers to 294the response team so the response team can validate subscription requests. 295 296Each subscriber needs to send a subscription request to the response team 297by email. The email must be signed with the subscriber's PGP key or S/MIME 298certificate. If a PGP key is used, it must be available from a public key 299server and is ideally connected to the Linux kernel's PGP web of trust. See 300also: https://www.kernel.org/signature.html. 301 302The response team verifies that the subscriber request is valid and adds 303the subscriber to the list. After subscription the subscriber will receive 304email from the mailing-list which is signed either with the list's PGP key 305or the list's S/MIME certificate. The subscriber's email client can extract 306the PGP key or the S/MIME certificate from the signature so the subscriber 307can send encrypted email to the list. 308 309