111d810dbSLei YU# PSU firmware update 211d810dbSLei YU 3*f4febd00SPatrick WilliamsAuthor: Lei YU <mine260309@gmail.com> <LeiYU> 411d810dbSLei YU 5*f4febd00SPatrick WilliamsOther contributors: Su Xiao <suxiao@inspur.com> Derek Howard <derekh@us.ibm.com> 611d810dbSLei YU 7*f4febd00SPatrick WilliamsCreated: 2019-06-03 811d810dbSLei YU 911d810dbSLei YU## Problem Description 1011d810dbSLei YU 1111d810dbSLei YUThere is no support in OpenBMC to update the firmware for PSUs. 1211d810dbSLei YU 1311d810dbSLei YU## Background and References 1411d810dbSLei YU 1511d810dbSLei YUIn OpenBMC, there is an existing interface for [software update][1]. 1611d810dbSLei YU 1711d810dbSLei YUThe update process consists of: 18*f4febd00SPatrick Williams 1911d810dbSLei YU1. Uploading an image to the BMC; 2011d810dbSLei YU2. Processing the image to check the version and purpose of the image; 2111d810dbSLei YU3. Verifying and activating the image. 2211d810dbSLei YU 2311d810dbSLei YUCurrently, BMC and BIOS firmware update are supported: 24*f4febd00SPatrick Williams 25*f4febd00SPatrick Williams- [phosphor-bmc-code-mgmt][2] implements BMC code update, and it supports all 2611d810dbSLei YU the above 3 processes. 27*f4febd00SPatrick Williams- [openpower-pnor-code-mgmt][3] implements BIOS code update, and it only 28*f4febd00SPatrick Williams implements "verifying and activating" the image. It shares the function of the 29*f4febd00SPatrick Williams above 1 & 2 processes. 30*f4febd00SPatrick Williams- Both of the above use the same [Software DBus interface][1]. 3111d810dbSLei YU 3211d810dbSLei YUFor PSU firmware update, it is preferred to re-use the same function for the 3311d810dbSLei YUabove 1 & 2. 3411d810dbSLei YU 3511d810dbSLei YU## Requirements 3611d810dbSLei YU 3711d810dbSLei YUThe PSU firmware shall be updated in the below cases: 38*f4febd00SPatrick Williams 3911d810dbSLei YU1. The user manually invokes the APIs to do the update; 4011d810dbSLei YU2. After BMC code update and if there is a newer PSU image in the BMC's 4111d810dbSLei YU filesystem, BMC shall update the PSU firmware; 4211d810dbSLei YU3. When a PSU is replaced and the version is older than the one in BMC's 4311d810dbSLei YU filesystem, BMC shall update the PSU firmware. 4411d810dbSLei YU4. There are cases that a system could use different models of PSUs, and thus 4511d810dbSLei YU different PSU firmware images need to be supported. 4611d810dbSLei YU 4711d810dbSLei YUFor some PSUs, it is risky to do PSU code update while the host is running to 48*f4febd00SPatrick Williamsavoid power loss. This shall be handled by PSU vendor-specific tools, but not in 49*f4febd00SPatrick Williamsthe generic framework. 5011d810dbSLei YU 5111d810dbSLei YUNote: The "vendor-specific" referred below is the PSU vendor-specific. 5211d810dbSLei YU 5311d810dbSLei YUSo the below checks are optional and expected to be handled by vendor-specific 5411d810dbSLei YUtool: 55*f4febd00SPatrick Williams 5611d810dbSLei YU1. If the host is powered off; 5711d810dbSLei YU2. If the redundant PSUs are all connected; 5811d810dbSLei YU3. If the AC input and DC standby output is OK on all the PSUs; 5911d810dbSLei YU 6011d810dbSLei YU## Proposed Design 6111d810dbSLei YU 62*f4febd00SPatrick WilliamsAs described in the above requirements, there are different cases where the PSU 63*f4febd00SPatrick Williamsfirmware is updated: 64*f4febd00SPatrick Williams 65*f4febd00SPatrick Williams- When the APIs are invoked; 66*f4febd00SPatrick Williams- When a new version is updated together with BMC code update; 67*f4febd00SPatrick Williams- When a PSU is replaced with an old version of the firmware. 6811d810dbSLei YU 6911d810dbSLei YU### Update by API 7011d810dbSLei YU 7111d810dbSLei YUThis method is usually used by users who manually update PSU firmware. 7211d810dbSLei YU 73*f4febd00SPatrick WilliamsIt will re-use the current interfaces to upload, verify, and activate the image. 7411d810dbSLei YU 7511d810dbSLei YU1. The "Version" interface needs to be extended: 76*f4febd00SPatrick Williams - Add a new [VersionPurpose][4] for PSU; 77*f4febd00SPatrick Williams - Re-use the existing `ExtendedVersion` as an additional string for 7811d810dbSLei YU vendor-specific purpose, e.g. to indicate the PSU model. 7911d810dbSLei YU2. Re-use the existing functions implemented by [phosphor-bmc-code-mgmt][2] for 8011d810dbSLei YU uploading and processing the image. 81*f4febd00SPatrick Williams - The PSU update image shall be a tarball that consists of a MANIFEST, 8211d810dbSLei YU images, and signatures. 83*f4febd00SPatrick Williams - When the PSU image is uploaded and processed, a `VersionObject` shall be 8411d810dbSLei YU created to indicate the version and its purpose. 8511d810dbSLei YU3. There will be a new service that implements the [Activation][5] interface to 8611d810dbSLei YU update the PSU firmware. 87*f4febd00SPatrick Williams - The service will be started by default when BMC starts; 88*f4febd00SPatrick Williams - On start, the service will check the PSU's existing firmware and create the 89*f4febd00SPatrick Williams `Version` and `Activation` interfaces. 90*f4febd00SPatrick Williams - The service shall watch the interface added on 9111d810dbSLei YU `/xyz/openbmc_project/Software`. 92*f4febd00SPatrick Williams - When a new object with PSU `VersionPurpose` is added, the service will 9311d810dbSLei YU verify the signature of the image; 94*f4febd00SPatrick Williams - The service shall check the `ExtendedVersion` to make sure the image 9511d810dbSLei YU matches the PSU model. 96*f4febd00SPatrick Williams - The service will have a configuration file to describe the PSU model and 9711d810dbSLei YU its related vendor-specific tools. 98*f4febd00SPatrick Williams - The service will find the matched vendor-specific tool to perform the code 99*f4febd00SPatrick Williams update. For example, if a vendor specific tool `foo` is configured in 10011d810dbSLei YU `psu-update@foo.service` which executes `foo psu.bin`, the service will 10111d810dbSLei YU find the `psu-update@foo.service` and start it by systemd, which performs 10211d810dbSLei YU the update. 103*f4febd00SPatrick Williams - When the PSU code update is completed, an informational event log shall be 10411d810dbSLei YU created. 105*f4febd00SPatrick Williams - When the PSU code update is completed, the image, MANIFEST, and optionally 10611d810dbSLei YU the signature will be saved to a pre-defined directory in read-write 10711d810dbSLei YU filesystem for future use, in case a new PSU with old firmware is plugged. 10811d810dbSLei YU4. The vendor-specific tool shall run all the checks it needs to be run, before 10911d810dbSLei YU and after the PSU update, and return a status to the above service to 11011d810dbSLei YU indicate the result. 11111d810dbSLei YU5. When the vendor-specific tool returns errors, the PSU update will be aborted 11211d810dbSLei YU and an error event log shall be created. 11311d810dbSLei YU6. During the update, the service shall set the related sensors to 114*f4febd00SPatrick Williams non-functional, and when the update is done, it shall set the related sensors 115*f4febd00SPatrick Williams back to functional. 11611d810dbSLei YU 11711d810dbSLei YU### Update by new BMC image 11811d810dbSLei YU 11911d810dbSLei YUWhen BMC is updated and a new version of PSU firmware is included, it shall be 120*f4febd00SPatrick Williamsupdated to the PSU. This will be done by the same service described above. 12111d810dbSLei YU 12211d810dbSLei YU1. On start, the service will check the PSU image, model and version in its 12311d810dbSLei YU filesystem, compare with the ones in PSU hardware and decide if PSU firmware 12411d810dbSLei YU update shall be performed. 12511d810dbSLei YU2. There could be two places containing the PSU images: 126*f4febd00SPatrick Williams - The pre-defined directory in read-only filesystem, which is part of BMC 12711d810dbSLei YU image. 128*f4febd00SPatrick Williams - The other pre-defined directory in read-write filesystem, which is the 129*f4febd00SPatrick Williams location for the saved PSU images by API update. Both places shall be 130*f4febd00SPatrick Williams checked and a newer version will be selected to compare with the PSU 131*f4febd00SPatrick Williams hardware. 13211d810dbSLei YU3. If PSU update is needed, the service will find the matched vendor-specific 13311d810dbSLei YU tool to perform the code update. 13411d810dbSLei YU4. The following process will be the same as [Update by API]. 13511d810dbSLei YU 13611d810dbSLei YU### Update on replaced PSU 13711d810dbSLei YU 13811d810dbSLei YUWhen a PSU is replaced, and the firmware version is older than the one in BMC 139*f4febd00SPatrick Williamsfilesystem, it shall be updated. This will be done by the same service described 140*f4febd00SPatrick Williamsabove. 14111d810dbSLei YU 142*f4febd00SPatrick Williams1. On start, the service will subscribe to the PropertiesChanged signal to the 143*f4febd00SPatrick Williams PSU object path to monitor the PSU presence status. (Or maybe subscribe the 144*f4febd00SPatrick Williams InterfacesAdded/Removed signal?) 14511d810dbSLei YU2. When a PSU's presence status is changed from false to true (or the 14611d810dbSLei YU InterfacesAdded event occurs), the service will check the new PSU's model, 14711d810dbSLei YU firmware version to decide if the firmware needs to be updated. 148*f4febd00SPatrick Williams3. If yes, the service will find the matched vendor-specific tool to perform the 149*f4febd00SPatrick Williams code update. 15011d810dbSLei YU4. The following process will be the same as [Update by API]. 15111d810dbSLei YU 15211d810dbSLei YU## Alternatives Considered 15311d810dbSLei YU 15411d810dbSLei YU### General implementation 15511d810dbSLei YU 156*f4febd00SPatrick WilliamsThe PSU firmware update could be implemented by separated recipes that only call 157*f4febd00SPatrick Williamsvendor-specific tools. It will be a bit simpler but loses the unified interface 158*f4febd00SPatrick Williamsprovided by OpenBMC's existing [software update interface][1], and thus it will 159*f4febd00SPatrick Williamsbecome difficult to use a standard API to the PSU firmware update. 16011d810dbSLei YU 16111d810dbSLei YU### VersionPurpose 162*f4febd00SPatrick Williams 163*f4febd00SPatrick WilliamsIt is possible to re-use the `VersionPurpose.Other` to represent the PSU image's 164*f4febd00SPatrick Williamsversion purpose. But that requires additional information about the image, 165*f4febd00SPatrick Williamsotherwise, there is no way to tell if the image is for PSU, or CPLD, or other 166*f4febd00SPatrick Williamsperipherals. A new `VersionPurpose.PSU` is more specific and makes it easier to 167*f4febd00SPatrick Williamsimplement and friendly for the user. 16811d810dbSLei YU 16911d810dbSLei YU### Additional string 170*f4febd00SPatrick Williams 17111d810dbSLei YUThe design proposal uses `ExtendedVersion` as the additional string for 17211d810dbSLei YUvendor-specific purpose, e.g. to indicate the PSU model, so the implementation 173*f4febd00SPatrick Williamscould check and compare if the image matches the PSU model. It is possible to 174*f4febd00SPatrick Williamsmake it optional or remove this additional string, then the implementation will 175*f4febd00SPatrick Williamsnot verify if the image matches the PSU. It could be OK if we trust the user who 176*f4febd00SPatrick Williamsis uploading the correct image, especially the image shall be signed. But it is 177*f4febd00SPatrick Williamsalways risky in case the image does not match the PSU, and cause unintended 178*f4febd00SPatrick Williamsdamage if the incorrect PSU firmware is updated. 17911d810dbSLei YU 18011d810dbSLei YU## Impacts 18111d810dbSLei YU 18211d810dbSLei YUThis design only introduces a new `VersionPurpose` enum into the dbus 183*f4febd00SPatrick Williamsinterfaces. The newly introduced PSU firmware update service will be a new 184*f4febd00SPatrick Williamsservice that implements existing [Activation][5] interface. There will be new 185*f4febd00SPatrick Williamsconfiguration files for the service to: 186*f4febd00SPatrick Williams 187*f4febd00SPatrick Williams- Link the vendor specific tool with PSU models. 188*f4febd00SPatrick Williams- Get the sensors related to the PSU. 189*f4febd00SPatrick Williams- etc. 19011d810dbSLei YU 19111d810dbSLei YUSo the impacts are minimal to existing systems. 19211d810dbSLei YU 19311d810dbSLei YU## Testing 19411d810dbSLei YU 19511d810dbSLei YUIt requires the manual tests to verify the PSU code update process. 196*f4febd00SPatrick Williams 197*f4febd00SPatrick Williams- Verify the PSU code update is done on all PSUs successfully; 198*f4febd00SPatrick Williams- Verify the PSU code update will fail if the vendor-specific tool fails on 19911d810dbSLei YU pre-condition check, of fails on updating PSU. 200*f4febd00SPatrick Williams- Verify the PSU code update is performed after a new BMC image is updated 20111d810dbSLei YU containing a new version of PSU firmware. 202*f4febd00SPatrick Williams- Verify the PSU code update is performed after a PSU with old firmware is 20311d810dbSLei YU plugged in. 20411d810dbSLei YU 205*f4febd00SPatrick Williams[1]: 206*f4febd00SPatrick Williams https://github.com/openbmc/phosphor-dbus-interfaces/tree/master/yaml/xyz/openbmc_project/Software 20711d810dbSLei YU[2]: https://github.com/openbmc/phosphor-bmc-code-mgmt/ 20811d810dbSLei YU[3]: https://github.com/openbmc/openpower-pnor-code-mgmt/ 209*f4febd00SPatrick Williams[4]: 210*f4febd00SPatrick Williams https://github.com/openbmc/phosphor-dbus-interfaces/blob/57b878d048f929643276f1bf7fdf750abc4bde8b/xyz/openbmc_project/Software/Version.interface.yaml#L14 211*f4febd00SPatrick Williams[5]: 212*f4febd00SPatrick Williams https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/yaml/xyz/openbmc_project/Software/Activation.interface.yaml 213