1ddaedbbeSThomas GleixnerEmbargoed hardware issues 2ddaedbbeSThomas Gleixner========================= 3ddaedbbeSThomas Gleixner 4ddaedbbeSThomas GleixnerScope 5ddaedbbeSThomas Gleixner----- 6ddaedbbeSThomas Gleixner 7ddaedbbeSThomas GleixnerHardware issues which result in security problems are a different category 8ddaedbbeSThomas Gleixnerof security bugs than pure software bugs which only affect the Linux 9ddaedbbeSThomas Gleixnerkernel. 10ddaedbbeSThomas Gleixner 11ddaedbbeSThomas GleixnerHardware issues like Meltdown, Spectre, L1TF etc. must be treated 12ddaedbbeSThomas Gleixnerdifferently because they usually affect all Operating Systems ("OS") and 13ddaedbbeSThomas Gleixnertherefore need coordination across different OS vendors, distributions, 14ddaedbbeSThomas Gleixnerhardware vendors and other parties. For some of the issues, software 15ddaedbbeSThomas Gleixnermitigations can depend on microcode or firmware updates, which need further 16ddaedbbeSThomas Gleixnercoordination. 17ddaedbbeSThomas Gleixner 18ddaedbbeSThomas Gleixner.. _Contact: 19ddaedbbeSThomas Gleixner 20ddaedbbeSThomas GleixnerContact 21ddaedbbeSThomas Gleixner------- 22ddaedbbeSThomas Gleixner 23ddaedbbeSThomas GleixnerThe Linux kernel hardware security team is separate from the regular Linux 24ddaedbbeSThomas Gleixnerkernel security team. 25ddaedbbeSThomas Gleixner 26ddaedbbeSThomas GleixnerThe team only handles the coordination of embargoed hardware security 27ddaedbbeSThomas Gleixnerissues. Reports of pure software security bugs in the Linux kernel are not 28ddaedbbeSThomas Gleixnerhandled by this team and the reporter will be guided to contact the regular 29ddaedbbeSThomas GleixnerLinux kernel security team (:ref:`Documentation/admin-guide/ 30ddaedbbeSThomas Gleixner<securitybugs>`) instead. 31ddaedbbeSThomas Gleixner 32ddaedbbeSThomas GleixnerThe team can be contacted by email at <hardware-security@kernel.org>. This 33ddaedbbeSThomas Gleixneris a private list of security officers who will help you to coordinate an 34ddaedbbeSThomas Gleixnerissue according to our documented process. 35ddaedbbeSThomas Gleixner 36ddaedbbeSThomas GleixnerThe list is encrypted and email to the list can be sent by either PGP or 37ddaedbbeSThomas GleixnerS/MIME encrypted and must be signed with the reporter's PGP key or S/MIME 38ddaedbbeSThomas Gleixnercertificate. The list's PGP key and S/MIME certificate are available from 39ddaedbbeSThomas Gleixnerhttps://www.kernel.org/.... 40ddaedbbeSThomas Gleixner 41ddaedbbeSThomas GleixnerWhile hardware security issues are often handled by the affected hardware 42ddaedbbeSThomas Gleixnervendor, we welcome contact from researchers or individuals who have 43ddaedbbeSThomas Gleixneridentified a potential hardware flaw. 44ddaedbbeSThomas Gleixner 45ddaedbbeSThomas GleixnerHardware security officers 46ddaedbbeSThomas Gleixner^^^^^^^^^^^^^^^^^^^^^^^^^^ 47ddaedbbeSThomas Gleixner 48ddaedbbeSThomas GleixnerThe current team of hardware security officers: 49ddaedbbeSThomas Gleixner 50ddaedbbeSThomas Gleixner - Linus Torvalds (Linux Foundation Fellow) 51ddaedbbeSThomas Gleixner - Greg Kroah-Hartman (Linux Foundation Fellow) 52ddaedbbeSThomas Gleixner - Thomas Gleixner (Linux Foundation Fellow) 53ddaedbbeSThomas Gleixner 54ddaedbbeSThomas GleixnerOperation of mailing-lists 55ddaedbbeSThomas Gleixner^^^^^^^^^^^^^^^^^^^^^^^^^^ 56ddaedbbeSThomas Gleixner 57ddaedbbeSThomas GleixnerThe encrypted mailing-lists which are used in our process are hosted on 58ddaedbbeSThomas GleixnerLinux Foundation's IT infrastructure. By providing this service Linux 59ddaedbbeSThomas GleixnerFoundation's director of IT Infrastructure security technically has the 60ddaedbbeSThomas Gleixnerability to access the embargoed information, but is obliged to 61ddaedbbeSThomas Gleixnerconfidentiality by his employment contract. Linux Foundation's director of 62ddaedbbeSThomas GleixnerIT Infrastructure security is also responsible for the kernel.org 63ddaedbbeSThomas Gleixnerinfrastructure. 64ddaedbbeSThomas Gleixner 65ddaedbbeSThomas GleixnerThe Linux Foundation's current director of IT Infrastructure security is 66ddaedbbeSThomas GleixnerKonstantin Ryabitsev. 67ddaedbbeSThomas Gleixner 68ddaedbbeSThomas Gleixner 69ddaedbbeSThomas GleixnerNon-disclosure agreements 70ddaedbbeSThomas Gleixner------------------------- 71ddaedbbeSThomas Gleixner 72ddaedbbeSThomas GleixnerThe Linux kernel hardware security team is not a formal body and therefore 73ddaedbbeSThomas Gleixnerunable to enter into any non-disclosure agreements. The kernel community 74ddaedbbeSThomas Gleixneris aware of the sensitive nature of such issues and offers a Memorandum of 75ddaedbbeSThomas GleixnerUnderstanding instead. 76ddaedbbeSThomas Gleixner 77ddaedbbeSThomas Gleixner 78ddaedbbeSThomas GleixnerMemorandum of Understanding 79ddaedbbeSThomas Gleixner--------------------------- 80ddaedbbeSThomas Gleixner 81ddaedbbeSThomas GleixnerThe Linux kernel community has a deep understanding of the requirement to 82ddaedbbeSThomas Gleixnerkeep hardware security issues under embargo for coordination between 83ddaedbbeSThomas Gleixnerdifferent OS vendors, distributors, hardware vendors and other parties. 84ddaedbbeSThomas Gleixner 85ddaedbbeSThomas GleixnerThe Linux kernel community has successfully handled hardware security 86ddaedbbeSThomas Gleixnerissues in the past and has the necessary mechanisms in place to allow 87ddaedbbeSThomas Gleixnercommunity compliant development under embargo restrictions. 88ddaedbbeSThomas Gleixner 89ddaedbbeSThomas GleixnerThe Linux kernel community has a dedicated hardware security team for 90ddaedbbeSThomas Gleixnerinitial contact, which oversees the process of handling such issues under 91ddaedbbeSThomas Gleixnerembargo rules. 92ddaedbbeSThomas Gleixner 93ddaedbbeSThomas GleixnerThe hardware security team identifies the developers (domain experts) who 94ddaedbbeSThomas Gleixnerwill form the initial response team for a particular issue. The initial 95ddaedbbeSThomas Gleixnerresponse team can bring in further developers (domain experts) to address 96ddaedbbeSThomas Gleixnerthe issue in the best technical way. 97ddaedbbeSThomas Gleixner 98ddaedbbeSThomas GleixnerAll involved developers pledge to adhere to the embargo rules and to keep 99ddaedbbeSThomas Gleixnerthe received information confidential. Violation of the pledge will lead to 100ddaedbbeSThomas Gleixnerimmediate exclusion from the current issue and removal from all related 101ddaedbbeSThomas Gleixnermailing-lists. In addition, the hardware security team will also exclude 102ddaedbbeSThomas Gleixnerthe offender from future issues. The impact of this consequence is a highly 103ddaedbbeSThomas Gleixnereffective deterrent in our community. In case a violation happens the 104ddaedbbeSThomas Gleixnerhardware security team will inform the involved parties immediately. If you 105ddaedbbeSThomas Gleixneror anyone becomes aware of a potential violation, please report it 106ddaedbbeSThomas Gleixnerimmediately to the Hardware security officers. 107ddaedbbeSThomas Gleixner 108ddaedbbeSThomas Gleixner 109ddaedbbeSThomas GleixnerProcess 110ddaedbbeSThomas Gleixner^^^^^^^ 111ddaedbbeSThomas Gleixner 112ddaedbbeSThomas GleixnerDue to the globally distributed nature of Linux kernel development, 113ddaedbbeSThomas Gleixnerface-to-face meetings are almost impossible to address hardware security 114ddaedbbeSThomas Gleixnerissues. Phone conferences are hard to coordinate due to time zones and 115ddaedbbeSThomas Gleixnerother factors and should be only used when absolutely necessary. Encrypted 116ddaedbbeSThomas Gleixneremail has been proven to be the most effective and secure communication 117ddaedbbeSThomas Gleixnermethod for these types of issues. 118ddaedbbeSThomas Gleixner 119ddaedbbeSThomas GleixnerStart of Disclosure 120ddaedbbeSThomas Gleixner""""""""""""""""""" 121ddaedbbeSThomas Gleixner 122ddaedbbeSThomas GleixnerDisclosure starts by contacting the Linux kernel hardware security team by 123ddaedbbeSThomas Gleixneremail. This initial contact should contain a description of the problem and 124ddaedbbeSThomas Gleixnera list of any known affected hardware. If your organization builds or 125ddaedbbeSThomas Gleixnerdistributes the affected hardware, we encourage you to also consider what 126ddaedbbeSThomas Gleixnerother hardware could be affected. 127ddaedbbeSThomas Gleixner 128ddaedbbeSThomas GleixnerThe hardware security team will provide an incident-specific encrypted 129ddaedbbeSThomas Gleixnermailing-list which will be used for initial discussion with the reporter, 130ddaedbbeSThomas Gleixnerfurther disclosure and coordination. 131ddaedbbeSThomas Gleixner 132ddaedbbeSThomas GleixnerThe hardware security team will provide the disclosing party a list of 133ddaedbbeSThomas Gleixnerdevelopers (domain experts) who should be informed initially about the 134ddaedbbeSThomas Gleixnerissue after confirming with the developers that they will adhere to this 135ddaedbbeSThomas GleixnerMemorandum of Understanding and the documented process. These developers 136ddaedbbeSThomas Gleixnerform the initial response team and will be responsible for handling the 137ddaedbbeSThomas Gleixnerissue after initial contact. The hardware security team is supporting the 138ddaedbbeSThomas Gleixnerresponse team, but is not necessarily involved in the mitigation 139ddaedbbeSThomas Gleixnerdevelopment process. 140ddaedbbeSThomas Gleixner 141ddaedbbeSThomas GleixnerWhile individual developers might be covered by a non-disclosure agreement 142ddaedbbeSThomas Gleixnervia their employer, they cannot enter individual non-disclosure agreements 143ddaedbbeSThomas Gleixnerin their role as Linux kernel developers. They will, however, agree to 144ddaedbbeSThomas Gleixneradhere to this documented process and the Memorandum of Understanding. 145ddaedbbeSThomas Gleixner 146ddaedbbeSThomas Gleixner 147ddaedbbeSThomas GleixnerDisclosure 148ddaedbbeSThomas Gleixner"""""""""" 149ddaedbbeSThomas Gleixner 150ddaedbbeSThomas GleixnerThe disclosing party provides detailed information to the initial response 151ddaedbbeSThomas Gleixnerteam via the specific encrypted mailing-list. 152ddaedbbeSThomas Gleixner 153ddaedbbeSThomas GleixnerFrom our experience the technical documentation of these issues is usually 154ddaedbbeSThomas Gleixnera sufficient starting point and further technical clarification is best 155ddaedbbeSThomas Gleixnerdone via email. 156ddaedbbeSThomas Gleixner 157ddaedbbeSThomas GleixnerMitigation development 158ddaedbbeSThomas Gleixner"""""""""""""""""""""" 159ddaedbbeSThomas Gleixner 160ddaedbbeSThomas GleixnerThe initial response team sets up an encrypted mailing-list or repurposes 161ddaedbbeSThomas Gleixneran existing one if appropriate. The disclosing party should provide a list 162f56f791fSKees Cookof contacts for all other parties who have already been, or should be, 163ddaedbbeSThomas Gleixnerinformed about the issue. The response team contacts these parties so they 164ddaedbbeSThomas Gleixnercan name experts who should be subscribed to the mailing-list. 165ddaedbbeSThomas Gleixner 166ddaedbbeSThomas GleixnerUsing a mailing-list is close to the normal Linux development process and 167ddaedbbeSThomas Gleixnerhas been successfully used in developing mitigations for various hardware 168ddaedbbeSThomas Gleixnersecurity issues in the past. 169ddaedbbeSThomas Gleixner 170ddaedbbeSThomas GleixnerThe mailing-list operates in the same way as normal Linux development. 171ddaedbbeSThomas GleixnerPatches are posted, discussed and reviewed and if agreed on applied to a 172ddaedbbeSThomas Gleixnernon-public git repository which is only accessible to the participating 173ddaedbbeSThomas Gleixnerdevelopers via a secure connection. The repository contains the main 174ddaedbbeSThomas Gleixnerdevelopment branch against the mainline kernel and backport branches for 175ddaedbbeSThomas Gleixnerstable kernel versions as necessary. 176ddaedbbeSThomas Gleixner 177ddaedbbeSThomas GleixnerThe initial response team will identify further experts from the Linux 178ddaedbbeSThomas Gleixnerkernel developer community as needed and inform the disclosing party about 179ddaedbbeSThomas Gleixnertheir participation. Bringing in experts can happen at any time of the 180ddaedbbeSThomas Gleixnerdevelopment process and often needs to be handled in a timely manner. 181ddaedbbeSThomas Gleixner 182ddaedbbeSThomas GleixnerCoordinated release 183ddaedbbeSThomas Gleixner""""""""""""""""""" 184ddaedbbeSThomas Gleixner 185ddaedbbeSThomas GleixnerThe involved parties will negotiate the date and time where the embargo 186ddaedbbeSThomas Gleixnerends. At that point the prepared mitigations are integrated into the 187ddaedbbeSThomas Gleixnerrelevant kernel trees and published. 188ddaedbbeSThomas Gleixner 189ddaedbbeSThomas GleixnerWhile we understand that hardware security issues need coordinated embargo 190ddaedbbeSThomas Gleixnertime, the embargo time should be constrained to the minimum time which is 191ddaedbbeSThomas Gleixnerrequired for all involved parties to develop, test and prepare the 192ddaedbbeSThomas Gleixnermitigations. Extending embargo time artificially to meet conference talk 193ddaedbbeSThomas Gleixnerdates or other non-technical reasons is creating more work and burden for 194ddaedbbeSThomas Gleixnerthe involved developers and response teams as the patches need to be kept 195ddaedbbeSThomas Gleixnerup to date in order to follow the ongoing upstream kernel development, 196ddaedbbeSThomas Gleixnerwhich might create conflicting changes. 197ddaedbbeSThomas Gleixner 198ddaedbbeSThomas GleixnerCVE assignment 199ddaedbbeSThomas Gleixner"""""""""""""" 200ddaedbbeSThomas Gleixner 201ddaedbbeSThomas GleixnerNeither the hardware security team nor the initial response team assign 202ddaedbbeSThomas GleixnerCVEs, nor are CVEs required for the development process. If CVEs are 203ddaedbbeSThomas Gleixnerprovided by the disclosing party they can be used for documentation 204ddaedbbeSThomas Gleixnerpurposes. 205ddaedbbeSThomas Gleixner 206ddaedbbeSThomas GleixnerProcess ambassadors 207ddaedbbeSThomas Gleixner------------------- 208ddaedbbeSThomas Gleixner 209ddaedbbeSThomas GleixnerFor assistance with this process we have established ambassadors in various 210ddaedbbeSThomas Gleixnerorganizations, who can answer questions about or provide guidance on the 211ddaedbbeSThomas Gleixnerreporting process and further handling. Ambassadors are not involved in the 212ddaedbbeSThomas Gleixnerdisclosure of a particular issue, unless requested by a response team or by 213ddaedbbeSThomas Gleixneran involved disclosed party. The current ambassadors list: 214ddaedbbeSThomas Gleixner 215ddaedbbeSThomas Gleixner ============= ======================================================== 216ddaedbbeSThomas Gleixner ARM 217ddaedbbeSThomas Gleixner AMD 218ddaedbbeSThomas Gleixner IBM 219*38c7a30aSTony Luck Intel Tony Luck <tony.luck@intel.com> 220a8e0abaeSTrilok Soni Qualcomm Trilok Soni <tsoni@codeaurora.org> 221ddaedbbeSThomas Gleixner 2221f493162SSasha Levin Microsoft Sasha Levin <sashal@kernel.org> 223ddaedbbeSThomas Gleixner VMware 22402e740aeSAndrew Cooper Xen Andrew Cooper <andrew.cooper3@citrix.com> 225ddaedbbeSThomas Gleixner 226ddaedbbeSThomas Gleixner Canonical Tyler Hicks <tyhicks@canonical.com> 227ddaedbbeSThomas Gleixner Debian Ben Hutchings <ben@decadent.org.uk> 228ddaedbbeSThomas Gleixner Oracle Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> 229ddaedbbeSThomas Gleixner Red Hat Josh Poimboeuf <jpoimboe@redhat.com> 230ddaedbbeSThomas Gleixner SUSE Jiri Kosina <jkosina@suse.cz> 231ddaedbbeSThomas Gleixner 232ddaedbbeSThomas Gleixner Amazon 233f56f791fSKees Cook Google Kees Cook <keescook@chromium.org> 234f56f791fSKees Cook ============= ======================================================== 235ddaedbbeSThomas Gleixner 236ddaedbbeSThomas GleixnerIf you want your organization to be added to the ambassadors list, please 237ddaedbbeSThomas Gleixnercontact the hardware security team. The nominated ambassador has to 238ddaedbbeSThomas Gleixnerunderstand and support our process fully and is ideally well connected in 239ddaedbbeSThomas Gleixnerthe Linux kernel community. 240ddaedbbeSThomas Gleixner 241ddaedbbeSThomas GleixnerEncrypted mailing-lists 242ddaedbbeSThomas Gleixner----------------------- 243ddaedbbeSThomas Gleixner 244ddaedbbeSThomas GleixnerWe use encrypted mailing-lists for communication. The operating principle 245ddaedbbeSThomas Gleixnerof these lists is that email sent to the list is encrypted either with the 246ddaedbbeSThomas Gleixnerlist's PGP key or with the list's S/MIME certificate. The mailing-list 247ddaedbbeSThomas Gleixnersoftware decrypts the email and re-encrypts it individually for each 248ddaedbbeSThomas Gleixnersubscriber with the subscriber's PGP key or S/MIME certificate. Details 249ddaedbbeSThomas Gleixnerabout the mailing-list software and the setup which is used to ensure the 250ddaedbbeSThomas Gleixnersecurity of the lists and protection of the data can be found here: 251ddaedbbeSThomas Gleixnerhttps://www.kernel.org/.... 252ddaedbbeSThomas Gleixner 253ddaedbbeSThomas GleixnerList keys 254ddaedbbeSThomas Gleixner^^^^^^^^^ 255ddaedbbeSThomas Gleixner 256ddaedbbeSThomas GleixnerFor initial contact see :ref:`Contact`. For incident specific mailing-lists 257ddaedbbeSThomas Gleixnerthe key and S/MIME certificate are conveyed to the subscribers by email 258ddaedbbeSThomas Gleixnersent from the specific list. 259ddaedbbeSThomas Gleixner 260ddaedbbeSThomas GleixnerSubscription to incident specific lists 261ddaedbbeSThomas Gleixner^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 262ddaedbbeSThomas Gleixner 263ddaedbbeSThomas GleixnerSubscription is handled by the response teams. Disclosed parties who want 264ddaedbbeSThomas Gleixnerto participate in the communication send a list of potential subscribers to 265ddaedbbeSThomas Gleixnerthe response team so the response team can validate subscription requests. 266ddaedbbeSThomas Gleixner 267ddaedbbeSThomas GleixnerEach subscriber needs to send a subscription request to the response team 268ddaedbbeSThomas Gleixnerby email. The email must be signed with the subscriber's PGP key or S/MIME 269ddaedbbeSThomas Gleixnercertificate. If a PGP key is used, it must be available from a public key 270ddaedbbeSThomas Gleixnerserver and is ideally connected to the Linux kernel's PGP web of trust. See 271ddaedbbeSThomas Gleixneralso: https://www.kernel.org/signature.html. 272ddaedbbeSThomas Gleixner 273ddaedbbeSThomas GleixnerThe response team verifies that the subscriber request is valid and adds 274ddaedbbeSThomas Gleixnerthe subscriber to the list. After subscription the subscriber will receive 275ddaedbbeSThomas Gleixneremail from the mailing-list which is signed either with the list's PGP key 276ddaedbbeSThomas Gleixneror the list's S/MIME certificate. The subscriber's email client can extract 277ddaedbbeSThomas Gleixnerthe PGP key or the S/MIME certificate from the signature so the subscriber 278ddaedbbeSThomas Gleixnercan send encrypted email to the list. 279ddaedbbeSThomas Gleixner 280