xref: /openbmc/docs/designs/code-update.md (revision 5873e0b9)
1# Code Update Design
2
3Author: Jagpal Singh Gill <paligill@gmail.com>
4
5Created: 4th August 2023
6
7## Problem Description
8
9This section covers the limitations discoverd with
10[phosphor-bmc-code-mgmt](https://github.com/openbmc/phosphor-bmc-code-mgmt)
11
121. Current code update flow is complex as it involves 3 different daemons -
13   Image Manager, Image Updater and Update Service.
142. Update invocation flow has no explicit interface but rather depends upon the
15   discovery of a new file in /tmp/images by Image Manager.
163. Images POSTed via Redfish are downloaded by BMCWeb to /tmp/images which
17   requires write access to filesystem. This poses a security risk.
184. Current design doesn't support parallel upgrades for different firmware
19   ([Issue](https://github.com/openbmc/bmcweb/issues/257)).
20
21## Background and References
22
23- [phosphor-bmc-code-mgmt](https://github.com/openbmc/phosphor-bmc-code-mgmt)
24- [Software DBus Interface](https://github.com/openbmc/phosphor-dbus-interfaces/tree/master/yaml/xyz/openbmc_project/Software)
25- [Code Update Design](https://github.com/openbmc/docs/tree/master/architecture/code-update)
26
27## Requirements
28
291. Able to start an update, given a firmware image and update settings.
30
31- Update settings shall be able to specify when to apply the image, for example
32  immediately or on device reset or on-demand.
33
342. Able to retrieve the update progress and status.
353. Able to produce an interface complaint with
36   [Redfish UpdateService](https://redfish.dmtf.org/schemas/v1/UpdateService.v1_11_3.json)
374. Unprivileged daemons with access to DBus should be able to accept and perform
38   a firmware update.
395. Update request shall respond back immediately, so client can query the status
40   while update is in progress.
416. All errors shall propagate back to the client.
427. Able to support update for different type of hardware components such as
43   CPLD, NIC, BIOS, BIC, PCIe switches, etc.
448. Design shall impose no restriction to choose any specific image format.
459. Able to update multiple hardware components of same type running different
46   firmware images, for example, two instances of CPLDx residing on the board,
47   one performing functionX and other performing functionY and hence running
48   different firmware images.
4910. Able to update multiple components in parallel.
5011. Able to restrict critical system actions, such as reboot for entity under
51    update while the code update is in flight.
52
53## Proposed Design
54
55### Proposed End to End Flow
56
57```mermaid
58sequenceDiagram;
59participant CL as Client
60participant BMCW as BMCWeb
61participant CU as <deviceX>CodeUpdater<br> ServiceName: xyz.openbmc_project.Software.<deviceX>
62
63% Bootstrap Action for CodeUpdater
64note over CU: Get device access info from<br> /xyz/openbmc_project/inventory/system/... path
65note over CU: Swid = <DeviceX>_<RandomId>
66CU ->> CU: Create Interface<br> xyz.openbmc_project.Software.Update<br> at /xyz/openbmc_project/Software/<SwId>
67CU ->> CU: Create Interface<br> xyz.openbmc_project.Software.Version<br> at /xyz/openbmc_project/Software/<SwId>
68CU ->> CU: Create Interface<br>xyz.openbmc_project.Software.Activation<br> at /xyz/openbmc_project/Software/<SwId> <br> with Status = Active
69CU ->> CU: Create functional association <br> from Version to Inventory Item
70
71CL ->> BMCW: HTTP POST: /redfish/v1/UpdateService/update <br> (Image, settings, RedfishTargetURIArray)
72
73loop For every RedfishTargetURI
74  note over BMCW: Map RedfishTargetURI /redfish/v1/UpdateService/FirmwareInventory/<SwId> to<br> Object path /xyz/openbmc_project/software/<SwId>
75  note over BMCW: Get serviceName corresponding to the object path <br>from mapper.
76  BMCW ->> CU: StartUpdate(Image, ApplyTime)
77
78  note over CU: Swid = <DeviceX>_<RandomId>
79  note over CU: ObjectPath = /xyz/openbmc_project/Software/<SwId>
80  CU ->> CU: Create Interface<br>xyz.openbmc_project.Software.Activation<br> at ObjectPath with Status = NotReady
81  CU -->> BMCW: {ObjectPath, Success}
82  CU ->> CU: << Delegate Update for asynchronous processing >>
83
84  par BMCWeb Processing
85      BMCW ->> BMCW: Create Matcher<br>(PropertiesChanged,<br> xyz.openbmc_project.Software.Activation,<br> ObjectPath)
86      BMCW ->> BMCW: Create Matcher<br>(PropertiesChanged,<br> xyz.openbmc_project.Software.ActivationProgress,<br> ObjectPath)
87      BMCW ->> BMCW: Create Task<br> to handle matcher notifications
88      BMCW -->> CL: <TaskNum>
89      loop
90          BMCW --) BMCW: Process notifications<br> and update Task attributes
91          CL ->> BMCW: /redfish/v1/TaskMonitor/<TaskNum>
92          BMCW -->>CL: TaskStatus
93      end
94  and << Asynchronous Update in Progress >>
95      note over CU: Verify Image
96      break Image Verification FAILED
97        CU ->> CU: Activation.Status = Invalid
98        CU --) BMCW: Notify Activation.Status change
99      end
100      CU ->> CU: Activation.Status = Ready
101      CU --) BMCW: Notify Activation.Status change
102
103      CU ->> CU: Create Interface<br> xyz.openbmc_project.Software.Version<br> at ObjectPath
104      CU ->> CU: Create Interface<br>xyz.openbmc_project.Software.ActivationProgress<br> at ObjectPath
105      CU ->> CU: Create Interface<br> xyz.openbmc_project.Software.ActivationBlocksTransition<br> at ObjectPath
106      CU ->> CU: Activation.Status = Activating
107      CU --) BMCW: Notify Activation.Status change
108      note over CU: Start Update
109      loop
110          CU --) BMCW: Notify ActivationProgress.Progress change
111      end
112      note over CU: Finish Update
113      CU ->> CU: Activation.Status = Active
114      CU --) BMCW: Notify Activation.Status change
115      CU ->> CU: Delete Interface<br> xyz.openbmc_project.Software.ActivationBlocksTransition
116      CU ->> CU: Delete Interface<br> xyz.openbmc_project.Software.ActivationProgress
117      alt ApplyTime == Immediate
118          note over CU: Reset Device and<br> update functional association to System Inventory Item
119      else
120          note over CU: Create active association to System Inventory Item
121      end
122  end
123end
124```
125
126- Each upgradable hardware type may have a separate daemon (\<deviceX\> as per
127  above flow) handling its update process and would need to implement the
128  proposed interfaces in next section. This satisfies the
129  [Requirement# 6](#requirements).
130- Since, there would be single daemon handling the update (as compared to
131  three), less hand shaking would be involved and hence addresses the
132  [Issue# 1](#problem-description) and [Requirement# 4](#requirements).
133
134### Proposed D-Bus Interface
135
136The DBus Interface for code update will consist of following -
137
138| Interface Name                                                                                                                                                                                         | Existing/New |                               Purpose                               |
139| :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------: | :-----------------------------------------------------------------: |
140| [xyz.openbmc_project.Software.Update](https://gerrit.openbmc.org/c/openbmc/phosphor-dbus-interfaces/+/65738)                                                                                           |     New      |                       Provides update method                        |
141| [xyz.openbmc_project.Software.Version](https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/yaml/xyz/openbmc_project/Software/Version.interface.yaml)                                       |   Existing   |                        Provides version info                        |
142| [xyz.openbmc_project.Software.Activation](https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/yaml/xyz/openbmc_project/Software/Activation.interface.yaml)                                 |   Existing   |                     Provides activation status                      |
143| [xyz.openbmc_project.Software.ActivationProgress](https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/yaml/xyz/openbmc_project/Software/ActivationProgress.interface.yaml)                 |   Existing   |               Provides activation progress percentage               |
144| [xyz.openbmc_project.Software.ActivationBlocksTransition](https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/yaml/xyz/openbmc_project/Software/ActivationBlocksTransition.interface.yaml) |   Existing   | Signifies barrier for state transitions while update is in progress |
145| [xyz.openbmc_project.Software.RedundancyPriority](https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/yaml/xyz/openbmc_project/Software/RedundancyPriority.interface.yaml)                 |   Existing   |     Provides the redundancy priority for the version interface      |
146
147Introduction of xyz.openbmc_project.Software.Update interface streamlines the
148update invocation flow and hence addresses the [Issue# 2](#problem-description)
149and [Requirement# 1 & 2](#requirements).
150
151#### Association
152
153`running` : A `running` association from xyz.openbmc_project.Inventory.Item to
154xyz.openbmc_project.Software.Version represents the current functional or
155running software version for the associated inventory item. The `ran_on` would
156be the corresponding reverse association.
157
158`activating` : An `activating` association from
159xyz.openbmc_project.Inventory.Item to xyz.openbmc_project.Software.Version
160represents the activated (but not yet run) software version for the associated
161inventory item. There could be more than one active versions for an inventory
162item, for example, in case of A/B redundancy models there are 2 associated
163flash-banks and xyz.openbmc_project.Software.RedundancyPriority interface
164defines the priority for each one.
165
166For A/B redundancy model with staging support, the
167xyz.openbmc_project.Software.Activation.Activations.Staged will help to define
168which software version is currently staged.
169
170The `activated_on` would be the corresponding reverse association.
171
172### Keep images in memory
173
174Images will be kept in memory and passed to \<deviceX>CodeUpdater using a file
175descriptor rather than file path. Implementation needs to monitor appropriate
176memory limits to prevent parallel updates from running BMC out of memory.
177
178### Propagate errors to client
179
180xyz.openbmc_project.Software.Update.StartUpdate return value will propagate any
181errors related to initial setup and image metadata/header parsing back to user.
182Any asynchronous errors which happen during the update process will be notified
183via failed activation status which maps to failed task associated with the
184update. Also, a phosphor-logging event will be created and sent back to client
185via
186[Redfish Log Service](https://redfish.dmtf.org/schemas/v1/LogService.v1_4_0.json).
187
188Another alternative could be to use
189[Redfish Event Services](https://redfish.dmtf.org/schemas/v1/EventService.v1_10_0.json).
190
191### Firmware Image Format
192
193Image parsing will be performed in \<deviceX>CodeUpdater and since
194\<deviceX>CodeUpdater may be a device specific daemon, vendor may choose any
195image format for the firmware image. This fulfills the
196[Requirement# 7](#requirements).
197
198### Multi part Images
199
200A multi part image has multiple component images as part of one image package.
201PLDM image is one such example of multi part image format. Sometimes, for multi
202part devices there is no concrete physical firmware device but firmware device
203itself consists of multiple phsyical components, each of which may have its own
204component image. In such a scenario, \<deviceX>CodeUpdater can create a logical
205inventory item for the firmware device. While performing the firmware device
206update, the client may target the logical firmware device which further knows
207how to update the corresponding child components for supplied component images.
208The user can also update the specific component by providing the image package
209with component as head node. The \<deviceX>CodeUpdater can implement the
210required logic to verify if the supplied image is targeted for itself (and child
211components) or not.
212
213### Update multiple devices of same type
214
215- For same type devices, extend the Dbus path to specify device instance, for
216  example, /xyz/openbmc_project/Software/\<deviceX>\_\<InstanceNum>\_\<SwId>.
217  All the corresponding interfaces can reside on this path and same path will be
218  returned from xyz.openbmc_project.Software.Update.StartUpdate.
219
220This fulfills the [Requirement# 9](#requirements).
221
222### Parallel Upgrade
223
224- Different type hardware components:
225
226  Upgrade for different type hardware components can be handled either by
227  different <deviceX>CodeUpdater daemons or by a single daemon for hardware
228  components with common features, for example, PLDMd may handle update for
229  devices using PLDM specification. Such updates can be invoked in parallel from
230  BMCWeb and tracked via different tasks.
231
232- Similar type hardware component:
233
234  BMCWeb will trigger xyz.openbmc_project.Software.Update.StartUpdate on
235  different D-Bus paths pertaining to each hardware instance. For more details
236  on D-Bus paths refer to
237  [Update multiple devices of same type](#update-multiple-devices-of-same-type).
238
239This fulfills the [Requirement# 9](#requirements).
240
241### Uninterrupted Updates
242
243`ActivationBlocksTransitions` interface will be created on the specific D-Bus
244path for a version update which will help to block any interruptions from
245critical system actions such as reboots. This interface can in turn start and
246stop services such as Boot Guard Service to prevent such interruptions.
247
248Moreover, when a device is being upgraded the sensor scanning for that device
249might need to be disabled. To achieve this, the sensor scanning flow can check
250for existence of `ActivationBlocksTransitions` interface on associated `Version`
251DBus path for the inventory item. If such interface exists, the sensor scanning
252for that device can be skipped by returning back relevant error (such as
253`EBUSY`) to the client. Another alternative is to check for existence of
254`ActivationBlocksTransitions` interface only if sensor scanning times out. This
255won't impact average case performance for sensor scanning but only the worst
256case scenario when device is busy, for example, due to update in progress.
257
258## Alternatives Considered
259
260### Centralized Design with Global Software Manager
261
262Single SoftwareManager which communicates with the BCMWeb, hosts all the
263interfaces such as Version, Activation, Progress for all hardware components
264within the system on different DBus paths. Software Manager keeps list of
265various hardware update services within the system and start them based on
266update request. These on-demand services update the hardware and interfaces
267hosted by Software Manager and exits.
268
269#### Pros
270
271- Most of the DBus interfaces gets implemented by Software Manager and vendors
272  would need to write minimal code to change properties for these interfaces
273  based on status and progress.
274- Under normal operating conditions (no update in flight), only Software Manager
275  will be running.
276
277#### Cons
278
279- Imposes the need of a common image format as Software Manager needs to parse
280  and verify the image for creating interfaces.
281- Limitation in the design, as there is a need to get the current running
282  version from the hardware at system bring up. So, Software Manager would need
283  to start each update daemon at system startup to get the running version.
284
285### Pull model for Status and Progress
286
287The proposed solution uses a push model where status and progress updates are
288asynchronously pushed to BMCWeb. Another alternative would be to use a pull
289model where Update interface can have get methods for status and progress (for
290example, getActivationStatus and getActivationProgress).
291
292#### Pros
293
294- Server doesn't have to maintain a Dbus matcher
295  ([Issue](https://github.com/openbmc/bmcweb/issues/202)).
296- Easier implementation in Server as no asynchronous handlers would be required.
297
298#### Cons
299
300- Server would still need maintain some info so it can map client's task status
301  request to Dbus path for /xyz/openbmc_project/Software/<deviceX> for calling
302  getActivationStatus and getActivationProgress.
303- Aforementioned [issue](https://github.com/openbmc/bmcweb/issues/202) is more
304  of an implementation problem which can be resolved through implementation
305  changes.
306- Currently, activation and progress interfaces are being used in
307  [lot of Servers](#organizational). In future, harmonizing the flow to single
308  one will involve changing the push to pull model in all those places. With the
309  current proposal, the only change will be in update invocation flow.
310
311## Impacts
312
313The introduction of new DBus API will temporarily create two invocation flows
314from Server. Servers (BMCWeb, IPMI, etc) can initially support both the code
315stacks. As all the code update daemons gets moved to the new flow, Servers would
316be changed to only support new API stack. No user-api impact as design adheres
317to Redfish UpdateService.
318
319## Organizational
320
321### Does this design require a new repository?
322
323Yes. There will be a device transport level repositories and multiple
324\<deviceX>CodeUpdater using similar transport layer can reside in same
325repository. For example, all devices using PMBus could have a common repository.
326
327### Who will be the initial maintainer(s) of this repository?
328
329Meta will propose repositories for following devices and `Jagpal Singh Gill` &
330`Patrick Williams` will be the maintainer for them.
331
332- VR Update
333- CPLD Update
334
335### Which repositories are expected to be modified to execute this design?
336
337Requires changes in following repositories to incorporate the new interface for
338update invocation -
339
340| Repository                                                                      | Modification Owner |
341| :------------------------------------------------------------------------------ | :----------------- |
342| [phosphor-bmc-code-mgmt](https://github.com/openbmc/phosphor-bmc-code-mgmt)     | Jagpal Singh Gill  |
343| [BMCWeb](https://github.com/openbmc/bmcweb)                                     | Jagpal Singh Gill  |
344| [phosphor-host-ipmid](https://github.com/openbmc/phosphor-host-ipmid)           | Jagpal Singh Gill  |
345| [pldm](https://github.com/openbmc/pldm/tree/master/fw-update)                   | Jagpal Singh Gill  |
346| [openpower-pnor-code-mgmt](https://github.com/openbmc/openpower-pnor-code-mgmt) | Adriana Kobylak    |
347| [openbmc-test-automation](https://github.com/openbmc/openbmc-test-automation)   | Adriana Kobylak    |
348
349NOTE: For
350[phosphor-psu-code-mgmt](https://github.com/openbmc/phosphor-psu-code-mgmt) code
351seems unused, so not tracking for change.
352
353## Testing
354
355### Unit Testing
356
357All the functional testing of the reference implementation will be performed
358using GTest.
359
360### Integration Testing
361
362The end to end integration testing involving Servers (for example BMCWeb) will
363be covered using openbmc-test-automation.
364