docs/designs/target-fail-monitoring.md

14 [openbmc-systemd.md][1] has a good summary of systemd and the basics of how it
22 cases, the unit which caused the failure will log an error to phosphor-logging
24 ...), there will be no indication to the end user that something failed. It is
25 critical that if a systemd target fails within OpenBMC, the user be notified.
29 critical, but something like fan-control or a power monitoring service, could be
30 very critical. At a minimum, need to ensure the user of the system is informed
37 See the [phosphor-state-manager][2] repository for background information on
48 phosphor-state-manager code already monitors for these signals but only looks
53 an appropriate error to phosphor-logging.
56 systemd defined retries have been executed. For OpenBMC systems, that involves 2
63 - Must be able to monitor any arbitrary systemd target and log a defined error
65 - Must be configurable
66   - Target: Choose any systemd target
67   - Status: Choose one or more status's to monitor for
68   - Error to log: Specify error to log when error is detected
69 - Must be able to take multiple configure files as input on startup
70 - Support a `default` for errors to monitor for
71   - This will be `timeout`,`failed`, and `dependency`
72 - Error will always be the configured one with additional data indicating the
74   - Example:
81 - By default, enable this monitor and logging service for all critical OpenBMC
83   - Critical systemd targets are ones used by phosphor-state-manager
84     - BMC: multi-user.target
85     - Chassis: `obmc-chassis-poweron@0.target, obmc-chassis-poweroff@0.target`
86     - Host: `obmc-host-start@0.target, obmc-host-startmin@0.target`,
87       `obmc-host-shutdown@0.target`, `obmc-host-stop@0.target`,
88       `obmc-host-reboot@0.target`
89   - The errors for these must be defined in phosphor-dbus-interfaces
90 - Limitations:
91   - Fully qualified target name must be input (i.e. no templated / wild card
96 - Must be able to monitor any arbitrary systemd service within OpenBMC
97   - Service(s) to monitor must be configurable
98 - Log an error indicating a service has failed (with service name in log)
99   - `xyz.openbmc_project.State.Error.CriticalServiceFailure`
100 - Collect a BMC dump
101 - Changes BMC state (CurrentBMCState) to indicate a degraded mode of the BMC
102 - Report changed state externally via Redfish managers/bmc state
106 Create a new standalone application in phosphor-state-manager which will load
115       "name": "multi-user.target",
120       "name": "obmc-chassis-poweron@0.target",
141 This application will not register any interfaces on D-Bus but will subscribe to
142 systemd dbus signals for JobRemoved. sdeventplus will be used for all D-Bus
147 logged by this service be put into `phosphor-debug-errors/errors_watch.yaml`.
172 A phosphor-logging event will be logged when one of the units listed above
175 application detects an issue during power on and logs it to phosphor-logging and
177 fail and the code logic defined in this design will also log an error.
180 architecture being defined for phosphor-logging. The user could monitor for that
181 first error within the phosphor-logging plugin architecture and as a defined
183 in phosphor-state-manager generating an error.
188 for each of the reasons defined within this design and the error generated for
191 [1]: https://github.com/openbmc/docs/blob/master/architecture/openbmc-systemd.md
192 [2]: https://github.com/openbmc/phosphor-state-manager