PEL: Adding the support for Systems Key in message registry

The current implementation has the support for adding system specific
callouts with the help of 'System' key in message_registry.json.

Ad

PEL: Adding the support for Systems Key in message registry

The current implementation has the support for adding system specific
callouts with the help of 'System' key in message_registry.json.

Adding one more key named 'Systems' where it can have array of system
names in the form of strings. The 'Systems' key can be used to define
the shared callouts for a group of systems.

A unique callout to a specific system can be added using the existing
System key. If both 'System' and 'Systems' are not present or not
matching with the system name, then the default calloutList will be
taken if configured.

Tested:

The test setup has the following names for the compatible interface.

```
busctl -j get-property xyz.openbmc_project.EntityManager
/xyz/openbmc_project/inventory/system/chassis/Rainier_2U_Chassis
xyz.openbmc_project.Inventory.Decorator.Compatible Names
{
"type" : "as",
"data" : [
"com.ibm.Hardware.Chassis.Model.Rainier2U",
"com.ibm.Hardware.Chassis.Model.Rainier"
]
}
```
The callout section in the message_registry.json for TestError1 is
defined as below.
```
"Callouts": [
{
"Systems": ["com.ibm.Hardware.Chassis.Model.Rainier",
"com.ibm.Hardware.Chassis.Model.Blue_Ridge"],
"CalloutList": [
{"Priority": "medium", "SymbolicFRU": "service_docs"}
]
},
{
"System": "com.ibm.Hardware.Chassis.Model.Rainier",
"CalloutList": [
{"Priority": "high", "Procedure": "BMC0001"}
]
},
{
"CalloutList": [
{ "LocCode": "P0", "Priority": "high" },
{ "LocCode": "P0-C15","Priority": "low" }
]
}
]
```

Leads to PEL callouts section as below:
```
"Callout Section": {
"Callout Count": "2",
"Callouts": [{
"FRU Type": "Maintenance Procedure Required",
"Priority": "Mandatory, replace all with this type
as a unit",
"Procedure": "BMC0001"
}, {
"FRU Type": "Symbolic FRU",
"Priority": "Medium Priority",
"Part Number": "SVCDOCS"
}]
}
```

Signed-off-by: Arya K Padman <aryakpadman@gmail.com>
Change-Id: Iea65816dcb822bb07043897488a6251929548dc7

show more ...

PEL: Use raw procedure names in PEL msg reg

Start using the raw maintenance procedure names in the PEL message
registry, such as BMC0001, instead of the enum values, such as
'bmc_firmware'. This wa

PEL: Use raw procedure names in PEL msg reg

Start using the raw maintenance procedure names in the PEL message
registry, such as BMC0001, instead of the enum values, such as
'bmc_firmware'. This way, the script that generates documentation can
list the procedure in the section for that PEL. This was requested by
the support team.

Note that there is still one user of the enums - the checkstop analysis
code uses them when it creates callouts by passing in them via JSON user
data files.

There are pointers added to the README and schema to find the available
procedures and their descriptions.

Tested:
- The unit tests still work which verify the callout contents when PEL
callouts are created via the message registry.
- Also manually verified PEL procedure callouts look correct when
created from the message registry.

Signed-off-by: Matt Spinler <spinler@us.ibm.com>
Change-Id: I3e8416956120d4dae1ff40592fd0df4f0285d072

show more ...

PEL: Support for CheckstopFlag msg reg field

Similiar to the DeconfigFlag field that was recently added, this one
indicates the PEL is for a hardware checkstop and results in a bit in
SRC hex word 5

PEL: Support for CheckstopFlag msg reg field

Similiar to the DeconfigFlag field that was recently added, this one
indicates the PEL is for a hardware checkstop and results in a bit in
SRC hex word 5 being set.

Signed-off-by: Matt Spinler <spinler@us.ibm.com>
Change-Id: Ib05de7471ad3e32f48e7f20a5c611abc119fe82a

show more ...

PEL: Use uppercase hex in message registry

The code in the Registry class requires the ReasonCode field in the
message registry to use upper case hex digits (A-F) to be able to find a
match to print

PEL: Use uppercase hex in message registry

The code in the Registry class requires the ReasonCode field in the
message registry to use upper case hex digits (A-F) to be able to find a
match to print the description in peltool, so change the message
registry schema to require that. Also fix an entry that had lower case.

Alternatively, the code could be fixed, but with the schema checking
that runs in CI to catch errors this method seems less impactful.

Signed-off-by: Matt Spinler <spinler@us.ibm.com>
Change-Id: I4f26c876575be58232f1408f9357465a3340002a

show more ...

PEL: Support new DeconfigFlag msg reg field

Add a new field to the PEL message registry to indicate that some
hardware associated with a callout was deconfigured.

"DeconfigFlag": true

This results

PEL: Support new DeconfigFlag msg reg field

Add a new field to the PEL message registry to indicate that some
hardware associated with a callout was deconfigured.

"DeconfigFlag": true

This results in bit 6 in the SRC's hex word 5 being set. It is looked
at by the code that creates the periodic PELs when there is a system
running with deconfigured hardware.

Note: For something to be considered deconfigured, it doesn't
necessarily mean that firmware chose to not configure it. It could also
mean that it is broken or missing, at least in the case of power
supplies or fans.

Signed-off-by: Matt Spinler <spinler@us.ibm.com>
Change-Id: I3e657cb52416fc50000e299bf2d583db7b67caf8

show more ...

PEL: Add new error message for system health state

A new error message added that will be created when there are
guard records And/Or Deconfigured records And/Or unresolved pels
with deconfig bit se

PEL: Add new error message for system health state

A new error message added that will be created when there are
guard records And/Or Deconfigured records And/Or unresolved pels
with deconfig bit set.

HMC will subscribe to this error SRC and when it gets notified
it requests for NAG dump which will capture serviceable records
in a JSON file and attach it to the dump.

This error message acts as a reminder for servicing faulty hardware
pending service present in the system.

Tested:
"0x500F05A1": {
"SRC": "BD50F138",
"PLID": "0x500F05A1",
"CreatorID": "BMC",
"Subsystem": "CEC Hardware",
"Commit Time": "04/24/2023 07:47:45",
"Sev": "Predictive Error",
"CompID": "bmc faultlog"
}

"User Header": {
"Section Version": "1",
"Sub-section type": "0",
"Log Committed by": "bmc error logging",
"Subsystem": "CEC Hardware",
"Event Scope": "Entire Platform",
"Event Severity": "Predictive Error",
"Event Type": "Not Applicable",
"Action Flags": [
"Service Action Required",
"Report Externally",
"HMC Call Home"
],
"Host Transmission": "Not Sent",
"HMC Transmission": "Not Sent"
},

"User Data 1": {
"Section Version": "1",
"Sub-section type": "1",
"Created by": "bmc error logging",
"DECONFIG_RECORD_COUNT": "0",
"GUARD_WITH_ASSOC_ERROR_COUNT": "1",
"GUARD_WITH_NO_ASSOC_ERROR_COUNT": "0",
"UNRESOLVED_PEL_WITH_DECONFIG_BIT_COUNT": "0"
}

"Primary SRC": {
"Section Version": "1",
"Sub-section type": "1",
"Created by": "bmc faultlog",
"Callout Section": {
"Callout Count": "1",
"Callouts": [{
"FRU Type": "Maintenance Procedure Required",
"Priority": "Mandatory, replace all with this type as a
unit",
"Procedure": "BMC0008"
}]
}
}
Signed-off-by: Marri Devender Rao <devenrao@in.ibm.com>
Change-Id: I1a046d4416719e479378d7032f32663fc07fd03a

show more ...

PEL: Define JSON schema to capture the journal

This chain of commits is adding the functionality to be able to capture
portions of the journal into PEL UserData sections by defining it in the
messag

PEL: Define JSON schema to capture the journal

This chain of commits is adding the functionality to be able to capture
portions of the journal into PEL UserData sections by defining it in the
message registry entry for an error.

This commit updates the JSON schema for it, and the README.

From the README, examples are:

"JournalCapture": {
"NumLines": 30
}

"JournalCapture":
{
"Sections": [
{
"SyslogID": "phosphor-bmc-state-manager",
"NumLines": 20
},
{
"SyslogID": "phosphor-log-manager",
"NumLines": 15
}
]
}

The first example will capture the previous 30 lines from the journal
into a single UserData section.

The second example will create two UserData sections, the first with the
most recent 20 lines from phosphor-bmc-state-manager, and the second
with 15 lines from phosphor-log-manager.

Signed-off-by: Matt Spinler <spinler@us.ibm.com>
Change-Id: I005a84547b43887a5f746832c68f4b40b6334404

show more ...

PEL:Procedure to correct system VPD issues

This commit adds a new callout procedure "BMC0007" to correct
* system backplane VPD mismatch between hardware value and
backed up value.
* system bac

PEL:Procedure to correct system VPD issues

This commit adds a new callout procedure "BMC0007" to correct
* system backplane VPD mismatch between hardware value and
backed up value.
* system backplane data being blank on both hardware and in
backup.

Test:
Tested that newly added callout procedure has been picked
when there is a system VPD mismatch and when the data is
blank.

peltool -i 0x500024AB
{
"Private Header": {
"Section Version": "1",
"Sub-section type": "0",
"Created by": "0x4000",
"Created at": "12/05/2022 05:58:47",
"Committed at": "12/05/2022 05:58:47",
"Creator Subsystem": "BMC",
"CSSVER": "",
"Platform Log Id": "0x500024AB",
"Entry Id": "0x500024AB",
"BMC Event Log Id": "314"
},
"User Header": {
"Section Version": "1",
"Sub-section type": "0",
"Log Committed by": "0x2000",
"Subsystem": "CEC Hardware - VPD Interface",
"Event Scope": "Entire Platform",
"Event Severity": "Predictive Error",
"Event Type": "Not Applicable",
"Action Flags": [
"Service Action Required",
"Report Externally",
"HMC Call Home"
],
"Host Transmission": "Not Sent",
"HMC Transmission": "Not Sent"
},
"Primary SRC": {
"Section Version": "1",
"Sub-section type": "1",
"Created by": "0x4000",
"SRC Version": "0x02",
"SRC Format": "0x55",
"Virtual Progress SRC": "False",
"I5/OS Service Event Bit": "False",
"Hypervisor Dump Initiated":"False",
"Backplane CCIN": "2E2D",
"Terminate FW Error": "False",
"Deconfigured": "False",
"Guarded": "False",
"Error Details": {
"Message": "A system VPD restoration error"
"occurred."
},
"Valid Word Count": "0x09",
"Reference Code": "BD554008",
"Hex Word 2": "00080055",
"Hex Word 3": "2E2D0010",
"Hex Word 4": "00000000",
"Hex Word 5": "00000000",
"Hex Word 6": "00000000",
"Hex Word 7": "00000000",
"Hex Word 8": "00000000",
"Hex Word 9": "00000000",
"Callout Section": {
"Callout Count": "2",
"Callouts": [{
"FRU Type": "Normal Hardware FRU",
"Priority": "Mandatory, replace all with this"
"type as a unit",
"Location Code": "U78DA.ND0.WZS007H-P0",
"Part Number": "02WG676",
"CCIN": "2E2D",
"Serial Number": "YF33UF19Y007"
}, {
"FRU Type": "Maintenance Procedure Required",
"Priority": "Mandatory, replace all with this"
"type as a unit",
"Procedure": "BMC0007"
}]
}
},
"Extended User Header": {
"Section Version": "1",
"Sub-section type": "0",
"Created by": "0x2000",
"Reporting Machine Type": "9105-22B",
"Reporting Serial Number": "1392BE0",
"FW Released Ver": "",
"FW SubSys Version": "fw1040.00-1.2",
"Common Ref Time": "00/00/0000 00:00:00",
"Symptom Id Len": "20",
"Symptom Id": "BD554008_2E2D0010"
},
"Failing MTMS": {
"Section Version": "1",
"Sub-section type": "0",
"Created by": "0x2000",
"Machine Type Model": "9105-22B",
"Serial Number": "1392BE0"
},
"User Data 0": {
"Section Version": "1",
"Sub-section type": "1",
"Created by": "0x2000",
"BMCLoad": "0.22 0.41 0.33",
"BMCState": "Ready",
"BMCUptime": "0y 0d 0h 13m 6s",
"BootState": "Unspecified",
"ChassisState": "Off",
"FW Version ID": "fw1040.00-1.2-2-g1859836242-dirty",
"HostState": "Off",
"System IM": "50001001"
},
"User Data 1": {
"Section Version": "1",
"Sub-section type": "1",
"Created by": "0x2000",
"CALLOUT_INVENTORY_PATH": "/xyz/openbmc_project/inventory/system"
"/chassis/motherboard",
"DESCRIPTION": "VPD data mismatch on cache and hardware for"
"record: LXR0 and keyword: LX",
"Value on Cache: ": "0x31 0x0 0x4 0x1 0x0 0x30 0x0 0x71 ",
"Value read from EEPROM: ": "0x64 0x65 0x4 0x1 0x0 0x30 0x0 0x71 "
}
}

Signed-off-by: Priyanga Ramasamy <priyanga24@in.ibm.com>
Change-Id: I738542d55781538285cb3747461929195cbd6d48

show more ...

PEL: Add additional power sequencer callouts

Facilitate future code development by adding additional power good and
voltage fault callouts.

Signed-off-by: Jim Wright <jlwright@us.ibm.com>
Change-Id

PEL: Add additional power sequencer callouts

Facilitate future code development by adding additional power good and
voltage fault callouts.

Signed-off-by: Jim Wright <jlwright@us.ibm.com>
Change-Id: Ie56a1dc74bc6b52e44d719a00e59ffcd168ddc07

show more ...

prettier: re-format

Prettier is enabled in openbmc-build-scripts on Markdown, JSON, and YAML
files to have consistent formatting for these file types. Re-run the
formatter on the whole repository.

prettier: re-format

Prettier is enabled in openbmc-build-scripts on Markdown, JSON, and YAML
files to have consistent formatting for these file types. Re-run the
formatter on the whole repository.

Change-Id: I6e8d56540a30898369d7f232d900fd74de2e7bd1
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>

show more ...

PEL: Add OVERTMP symbolic FRU callout

This commit adds the OVERTMP symbolic FRU as a high callout to the
perfloss, warning, and critical high temperature PELs. It uses the
newly introduced Callouts

PEL: Add OVERTMP symbolic FRU callout

This commit adds the OVERTMP symbolic FRU as a high callout to the
perfloss, warning, and critical high temperature PELs. It uses the
newly introduced CalloutsWhenNoADMatch message registry field to handle
doing it for all sensors that aren't the ambient one.

Signed-off-by: Matt Spinler <spinler@us.ibm.com>
Change-Id: I9775daefbf67063dc5bcde2a0ad763eef3e437a5

show more ...

PEL: Add CalloutsWhenNoADMatch msg reg support

There is a new use case where we need to do one callout in the message
registry based on the value of an AdditionalData field value, and
another callou

PEL: Add CalloutsWhenNoADMatch msg reg support

There is a new use case where we need to do one callout in the message
registry based on the value of an AdditionalData field value, and
another callout in all other cases.

To support this, this commit is adding a new 'CalloutsWhenNoADMatch'
field in the PEL message registry that allows one to add callouts when
there is no match on the 'ADValue' field. This behaves like an 'else'
leg to the 'if AdValue == X' structure in the message registry.

Example:
{
"ADName": "PROC_NUM",
"CalloutsWithTheirADValues":
[
{
"ADValue": "0",
"Callouts":
[
// callouts when PROC_NUM == 0
]
},
{
"ADValue": "1",
"Callouts":
[
// callouts when PROC_NUM == 1
]
}
],
"CalloutsWhenNoADMatch": [
{
// callouts when PROC_NUM != 0 or 1
}
]
}

Signed-off-by: Matt Spinler <spinler@us.ibm.com>
Change-Id: Ib8e208ff950a643302e856c7dd2b7474fec61b26

show more ...

PEL: Add min length to 2 msg reg fields

Change the minimum length to 8 for the Message and Description fields in
the Documentation section of the message registry as just a basic check
that they con

PEL: Add min length to 2 msg reg fields

Change the minimum length to 8 for the Message and Description fields in
the Documentation section of the message registry as just a basic check
that they contain something valid.

The message registry is transformed into official documentation that
needs these fields filled in.

Signed-off-by: Matt Spinler <spinler@us.ibm.com>
Change-Id: I0f7c247754a0460576337e61159b23fd02420cd4

show more ...

PEL: Correct callouts for Everest pgood failures

Make necessary corrections to the callouts for pgood related failures on
the Everest system.

Signed-off-by: Jim Wright <jlwright@us.ibm.com>
Change-

PEL: Correct callouts for Everest pgood failures

Make necessary corrections to the callouts for pgood related failures on
the Everest system.

Signed-off-by: Jim Wright <jlwright@us.ibm.com>
Change-Id: Iea42d2bde52e61f9977d860b850917e1ebd08691

show more ...

PEL: Add service procedure to find SUE root cause

An SUE indicates a UE has occurred in another area of hardware. This
procedure instructs service to look for previous PEL for the root cause
of the

PEL: Add service procedure to find SUE root cause

An SUE indicates a UE has occurred in another area of hardware. This
procedure instructs service to look for previous PEL for the root cause
of the SUE attention.

Signed-off-by: Zane Shelley <zshelle@us.ibm.com>
Change-Id: Id9b7e6c65a06e52587ac40464456be42b9a557b1

show more ...

PEL: Remove PowerFault from message entries, registries, and code

The PowerFault flag is deprecated and should be removed in both the
registry and code references. Its functionality is superseded by

PEL: Remove PowerFault from message entries, registries, and code

The PowerFault flag is deprecated and should be removed in both the
registry and code references. Its functionality is superseded by the
critical_system severity level.

Signed-off-by: Mike Capps <mikepcapps@gmail.com>
Change-Id: I1bc1898d9589e0678b5a37b87289c6588859f2ee

show more ...

PEL: Support 'PossibleSubsystems' in msg registry

Add a 'PossibleSubsystems' field to the message registry schema where
someone can list which subsystems they might pass into the PEL_SUBSYSTEM
Addit

PEL: Support 'PossibleSubsystems' in msg registry

Add a 'PossibleSubsystems' field to the message registry schema where
someone can list which subsystems they might pass into the PEL_SUBSYSTEM
AdditionalData property field when creating a PEL.

The PossibleSubsystems value will be used by a off-BMC script that
generates field documentation for SRCs, so it knows what all SRC ASCII
string values are possible for that entry since the subsystem is part of
them. For example, if that field was ['processor', 'memory'], then that
means the possible SRCs are BD10AAAA and BD20AAAA for reason code
'AAAA'.

The Subsystem and PossibleSubsystems fields are mutually exclusive in
the message registry, so code changes had to be made to support the
subsystem field now being optional.

There is now a chance that even though the subsystem is supposed to be
passed in using PEL_SUBSYSTEM, it isn't. In that case, the 'Other'
subsystem will be used, since it seems fairly generic.

Signed-off-by: Matt Spinler <spinler@us.ibm.com>
Change-Id: Ic8087fc46b2a192459b7287186f8972610e84730

show more ...

PEL: Add isolation procedure for power OC

Add in an isolation procedure for an over-current power problem.

Change-Id: I5da1f3e3e9078f4f52fd78469f1fb8948f497e1b
Signed-off-by: Br

PEL: Add isolation procedure for power OC

Add in an isolation procedure for an over-current power problem.

Change-Id: I5da1f3e3e9078f4f52fd78469f1fb8948f497e1b
Signed-off-by: Brandon Wyman <bjwyman@gmail.com>

show more ...

PEL: Add fsi path maintenance procedure

Maintenance procedures are part of the PEL callout section.
This procedure is used to indicate the problem could be related
to the fsi hardwar

PEL: Add fsi path maintenance procedure

Maintenance procedures are part of the PEL callout section.
This procedure is used to indicate the problem could be related
to the fsi hardware path.

Signed-off-by: Jayanth Othayoth <ojayanth@in.ibm.com>
Change-Id: I36ca9f0484e041adcaa542b9ea17b71ab545c6bd

show more ...

PEL: Add SBE code maintenance procedure

Maintenance procedures are part of the PEL callout section. This
procedure is used to indicate the problem is due to code (a bug)
SBE image.

PEL: Add SBE code maintenance procedure

Maintenance procedures are part of the PEL callout section. This
procedure is used to indicate the problem is due to code (a bug)
SBE image.

Signed-off-by: Jayanth Othayoth <ojayanth@in.ibm.com>
Change-Id: I8e1a04b2c8aa83c178990331f181ed026ac62db9

show more ...

PEL: Add next level of support maintenance procedure

Maintenance procedures are part of the PEL callout section.
This procedure is indicates isolation not possible, contact
your next

PEL: Add next level of support maintenance procedure

Maintenance procedures are part of the PEL callout section.
This procedure is indicates isolation not possible, contact
your next level of support.

Signed-off-by: Jayanth Othayoth <ojayanth@in.ibm.com>
Change-Id: If1d481604e411947e47cb7c9c20d79e938c65db3

show more ...

PEL: Add altitude PEL registry entries

These will only be used on systems that use PELs and have the virtual
altitude sensor. The errors come from the sensor-monitor app.

Signed

PEL: Add altitude PEL registry entries

These will only be used on systems that use PELs and have the virtual
altitude sensor. The errors come from the sensor-monitor app.

Signed-off-by: Rashmica Gupta <rashmica.g@gmail.com>
Change-Id: I5fec3ec86a084f8c62f52194f2826c0565be32eb

show more ...

PEL: Remove 'no_vpd_for_fru' maintenance procedure

This procedure was being used in a callout when the location code could
not be found on an inventory item. The code was changed to jus

PEL: Remove 'no_vpd_for_fru' maintenance procedure

This procedure was being used in a callout when the location code could
not be found on an inventory item. The code was changed to just not add
a callout in this case.

This was done for the following reasons:
1) There's no expected reason that the inventory would be missing a
location code for a valid inventory path.
2) There wasn't a way to give a hint to the end user about what the
callout should be in that case. Usually maintenance procedures should
have steps one can take to still do the appropriate replacement.
3) Because of 1), this case indicates a bad inventory path passed in by
the caller. Theoretically there is a way to have a 'bmc_code'
callout, but that wouldn't be appropriate to add in this log since
the intent of this one is for another problem.
3a) There is no way to create a new error log from inside a PEL section,
and I don't think this one case warrants coming up with a way to do
so.
4) The inventory path in question is still being added into a UserData
section so that development can debug it.

Signed-off-by: Matt Spinler <spinler@us.ibm.com>
Change-Id: I0757080f04942807cc34029d8667387db6b959fe

show more ...

PEL: Add FanControllerOffline message reg entry

This error is posted when the fan monitor application cannot find any
fan sensors on D-Bus, so it doesn't know the fan states when a power

PEL: Add FanControllerOffline message reg entry

This error is posted when the fan monitor application cannot find any
fan sensors on D-Bus, so it doesn't know the fan states when a power on
is attempted. It will shut down the system.

The P0-T12 and P0-T17 callouts are connectors for the fan signal cable.
These callouts follow the rules of using a symbolic FRU to identify the
cable for the first endpoint and a 'CBLCONT' symbolic FRU for the other
one.

Signed-off-by: Matt Spinler <spinler@us.ibm.com>
Change-Id: I99eadacf7c9f3a5a1037c9ddbaeeb4f5e11c08e8

show more ...

PEL: Add message registry entry for power loss.

Add PEL message entry for power loss.

Tested:
Power on machine.
Build and install phosphor-log-manager and restart its servic

PEL: Add message registry entry for power loss.

Add PEL message entry for power loss.

Tested:
Power on machine.
Build and install phosphor-log-manager and restart its service.
Delete all existing PEL entries.
Install patched message registry into /etc/phosphor-logging.
Use the APC GUI for the ePDU to turn off power to all power supplies.
Turn power back on, get the PEL entry which shows the blackout entry.

Signed-off-by: Jay Meyer <jaymeyer@us.ibm.com>
Change-Id: Ic7b06c3db7d6356ac19dc46062ee8abc4f9c8723

show more ...