Improve BMC error handling for OCC comm failures

- Delay starting OCC reset until all OCCs have been detected (or
timeout). It will prevent multiple resets from being triggered and to
help detecting

Improve BMC error handling for OCC comm failures

- Delay starting OCC reset until all OCCs have been detected (or
timeout). It will prevent multiple resets from being triggered and to
help detecting when reset is completed (active sensor being set after
reset is complete)
- Wait for PLDM response to OCC reset and HRESET requests and retry if
they fail
- If HRESET returns NOT_READY, collect SBE FFDC and try OCC reset. A
persistent failure will put the system in safe state.

- Prevent overwriting dvfs over-temp filename for p10 and beyond since
that old file is only present in old kernel
- Prevent assert when opening sysfs files. (added catch and then created
an OCC Comm failure PEL, which will force an OCC reset.)
- Check return code after reading sysfs files to confirm success. If
read fails, try reset to recover.

- Updated traces to include which processor/OCC encountered issues.
- Better recovery to close windows that were leaving system in partial
good state.

JIRA: PFES-66
Change-Id: I0b087d0e05bd8562682062e1c662f9e18164a720
Signed-off-by: Chris Cain <cjcain@us.ibm.com>

show more ...

Implement processor throttle dbus properties

- create processor throttle dbus objects for each OCC (processor)
- update throttle properties based on OCC poll response data
or safe mode status.

Th

Implement processor throttle dbus properties

- create processor throttle dbus objects for each OCC (processor)
- update throttle properties based on OCC poll response data
or safe mode status.

Throttle data will be made available via Redfish

NAME TYPE SIGNATURE RESULT/VALUE FLAGS
org.freedesktop.DBus.Introspectable interface - - -
.Introspect method - s -
org.freedesktop.DBus.Peer interface - - -
.GetMachineId method - s -
.Ping method - - -
org.freedesktop.DBus.Properties interface - - -
.Get method ss v -
.GetAll method s a{sv} -
.Set method ssv - -
.PropertiesChanged signal sa{sv}as - -
xyz.openbmc_project.Control.Power.Throttle interface - - -
.ThrottleCauses property as 0 emits-change
.Throttled property b false emits-change

Example of throttled processor (due to a power limit):
as 1 "xyz.openbmc_project.Control.Power.Throttle.ThrottleReasons.PowerLimit"

Change-Id: I0af9d82fab9d694427d0adaa45f4a372d25fbc12
Signed-off-by: Chris Cain <cjcain@us.ibm.com>

show more ...

sdbusplus: use shorter type aliases

The sdbusplus headers provide shortened aliases for many types.
Switch to using them to provide better code clarity and shorter
lines. Possible replacements are

sdbusplus: use shorter type aliases

The sdbusplus headers provide shortened aliases for many types.
Switch to using them to provide better code clarity and shorter
lines. Possible replacements are for:
* bus_t
* exception_t
* manager_t
* match_t
* message_t
* object_t
* slot_t

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I9541d521bf67882215a4a66dce020e38ac2df065

show more ...

Log a PEL for communication, presence mismatch, and safe state errors

Add code to log a PEL in various error scenarios. Refactor some of the
error handling to get the return code out of the driver.

Log a PEL for communication, presence mismatch, and safe state errors

Add code to log a PEL in various error scenarios. Refactor some of the
error handling to get the return code out of the driver.

Signed-off-by: Eddie James <eajames@linux.ibm.com>
Change-Id: Ifd91cfc063718e484ec8886df8357d115c6b41e3

show more ...

Prevent repeatedly reading PLDM sensors when proc garded

If a processor was garded, the OCC sensor would never be enabled.
The prior could would continue to try reading a sensor until it became
enab

Prevent repeatedly reading PLDM sensors when proc garded

If a processor was garded, the OCC sensor would never be enabled.
The prior could would continue to try reading a sensor until it became
enabled.
Code change will stop attempting to read a sensor once a signal or valid
response has been received.

Tested on HW

Change-Id: I9c43840fb25803af4f320ae9de00e896daf190f6
Signed-off-by: Chris Cain <cjcain@us.ibm.com>

show more ...

BMC: Fan control for Idle Power saver

Read IPS active from sysfs and push onto dBus IPS properties
Tested: enable/disable IPS on WEB, occ instrospect ips
Signed-off-by: Sheldon Bailey <baileysh@us.

BMC: Fan control for Idle Power saver

Read IPS active from sysfs and push onto dBus IPS properties
Tested: enable/disable IPS on WEB, occ instrospect ips
Signed-off-by: Sheldon Bailey <baileysh@us.ibm.com>
Change-Id: I12935a4c9f82bf004b1311e477f73ed8b1401a91

show more ...

Wait for OCC Active PDR to become available

occ-control was attempting to communicate with OCCs before the OCC
Active sensor PDR was available. This commit will wait until all PDRs
are available bef

Wait for OCC Active PDR to become available

occ-control was attempting to communicate with OCCs before the OCC
Active sensor PDR was available. This commit will wait until all PDRs
are available before allowing communication with the OCCs.
It will also read the OCC Active sensor to determine the state before
allowing communication.

When Hostboot moves a system to safe mode it will set the PLDM state of
the OCC Active sensor to PLDM_STATE_SET_OPERATIONAL_RUNNING_STATUS_DORMANT
This commit will also handle processing of that DORMANT status.

Change-Id: I348cf14455d9373898deec2ac4a04b3752e81c4a
Signed-off-by: Chris Cain <cjcain@us.ibm.com>

show more ...

sdbusplus: object: don't use 'bool' argument constructor

`sdbusplus::server::object_t` has long had an enum-based parameter for
signal action, but maintained a backwards compatible boolean mapping.

sdbusplus: object: don't use 'bool' argument constructor

`sdbusplus::server::object_t` has long had an enum-based parameter for
signal action, but maintained a backwards compatible boolean mapping.
It is time to remove this boolean to make it more observable which
actions are being used in applications. Map all `true` occurrences to
`action::defer_emit`.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: If65b32e415f0b413df95f637f2c19187bea56590

show more ...

Ignore PLDM message if status objects not created

The OCC status objects are not created until after the host@0-on file
has been removed. This now delays when the objects are created.
This introduce

Ignore PLDM message if status objects not created

The OCC status objects are not created until after the host@0-on file
has been removed. This now delays when the objects are created.
This introduced a window where the host notified occ-control that the
OCCs were running (via PLDM) before the status objects were created
which led to a segfault.
This commit will verify the status object exists when getting the PLDM
callback.

Verified on Rainier

Change-Id: I72404d1b5d89c5259cc9f857c8087096644ef318
Signed-off-by: Chris Cain <cjcain@us.ibm.com>

show more ...

openpower-occ-control:failure to read OCC state.

Failure to read OCC state set the OCCs sensors to Nan/Not Functional
Tested: cronus error inject on OCC with and without OCC resets.
Signed-off-by: S

openpower-occ-control:failure to read OCC state.

Failure to read OCC state set the OCCs sensors to Nan/Not Functional
Tested: cronus error inject on OCC with and without OCC resets.
Signed-off-by: Sheldon Bailey <baileysh@us.ibm.com>
Change-Id: I2a6bb6a431f09ea816979b3a482b54a28e21db53
Signed-off-by: Sheldon Bailey <baileysh@us.ibm.com>

show more ...

Handle file errors when device path gets unbound

occ-control was asserting when the hwmon path would get removed due to
device being unbound. This change will gracefully handle the path
getting remo

Handle file errors when device path gets unbound

occ-control was asserting when the hwmon path would get removed due to
device being unbound. This change will gracefully handle the path
getting removed / added back.

Tested on Raininer by unbind/bind of devices:
echo occ-hwmon.2 > /sys/bus/platform/drivers/occ-hwmon/unbind
echo occ-hwmon.2 > /sys/bus/platform/drivers/occ-hwmon/bind

Change-Id: I46fd2c2c54868ffb8183d3dc49cd0c2751165d3b
Signed-off-by: Chris Cain <cjcain@us.ibm.com>

show more ...

Monitor for power cap changes when occ-control started

The object that is monitoring for power cap changes needs to get created
when occ-control is started so that changes are not missed.
Added vali

Monitor for power cap changes when occ-control started

The object that is monitoring for power cap changes needs to get created
when occ-control is started so that changes are not missed.
Added validation check to ensure that the value sent to the OCC matches
the user configured value on dbus.

Verified on Raininer

Change-Id: I0d77f4569e5459ff58d6fc72147153133d2104a3
Signed-off-by: Chris Cain <cjcain@us.ibm.com>

show more ...

Expose power cap min/max on dbus

The min and max power cap values will be read from sysfs files and then
the data will be put on the dbus. This will allow users to know the
valid range of power cap

Expose power cap min/max on dbus

The min and max power cap values will be read from sysfs files and then
the data will be put on the dbus. This will allow users to know the
valid range of power cap available.

The PowerCap object was moved from Manager to Status.

Change-Id: I5196cc8645f84c31a5282cf844109bae47b09bf7
Signed-off-by: Chris Cain <cjcain@us.ibm.com>

show more ...

Read default power mode & IPS parameters from DBus

If the power mode or idle power saver parameters have never been set,
they will be read from a set of default parameters that are on DBus.
This all

Read default power mode & IPS parameters from DBus

If the power mode or idle power saver parameters have never been set,
they will be read from a set of default parameters that are on DBus.
This allows individual systems to have different default settings.
Mode/IPS parameters moved from Settings to OCC.Control

Code verified on Everest hardware with multiple scenarios including
host reboots, BMC reboots, OCC resets.

Change-Id: Id02613455db7f382791ff37c7dc420dbea1e0906
Signed-off-by: Chris Cain <cjcain@us.ibm.com>

show more ...

Add support for OEM Power Modes

- Allow mode to be set via PassThrough interface
- Allow non-customer OEM power modes to be persisted
- Persist any OEM power mode settings
- moved mode related code

Add support for OEM Power Modes

- Allow mode to be set via PassThrough interface
- Allow non-customer OEM power modes to be persisted
- Persist any OEM power mode settings
- moved mode related code from Status to PowerMode object
- merged PowerIPS into PowerMode object

Tested on Everest and Rainier.
Setting mode through PassThrough/ce-login:
busctl call org.open_power.OCC.Control /org/open_power/control/occ0 org.open_power.OCC.PassThrough SetMode yq 11 3600
Trace (via PassThrough interface)
openpower-occ-control[4440]: PassThrough::setMode() Setting Power Mode 11 (data: 3600)
openpower-occ-control[4440]: PowerMode::sendModeChange: SET_MODE(11,3600) command to OCC0 (9 bytes)
Trace (setting mode via GUI/Redfish):
openpower-occ-control[4440]: Power Mode Change Requested: xyz.openbmc_project.Control.Power.Mode.PowerMode.MaximumPerformance
openpower-occ-control[4440]: PowerMode::sendModeChange: SET_MODE(12,0) command to OCC0 (9 bytes)
Verified when system in any OEM mode that Redfish also reports OEM
Verified all modes are persisted across PM Complex resets and reboots

Change-Id: Idd0be05cb6fd74dbd0776145f212c49addd1c365
Signed-off-by: Chris Cain <cjcain@us.ibm.com>

show more ...

Fix some warnings by cppcheck

Warning message:
../../../occ_events.hpp:16:9: warning: Value stored to 'event' is never read [deadcode.DeadStores]
event = sd_event_unref(event);
^

Fix some warnings by cppcheck

Warning message:
../../../occ_events.hpp:16:9: warning: Value stored to 'event' is never read [deadcode.DeadStores]
event = sd_event_unref(event);
^ ~~~~~~~~~~~~~~~~~~~~~
../../../occ_events.hpp:26:9: warning: Value stored to 'eventSource' is never read [deadcode.DeadStores]
eventSource = sd_event_source_unref(eventSource);

occ_dbus.cpp:34:47: performance: Searching before insertion is not necessary. [stlFindInsert]
path, std::make_unique<SensorIntf>(utils::getBus(), path.c_str()));
^
occ_dbus.cpp:61:47: performance: Searching before insertion is not necessary. [stlFindInsert]
path, std::make_unique<SensorIntf>(utils::getBus(), path.c_str()));
^
occ_dbus.cpp:88:47: performance: Searching before insertion is not necessary. [stlFindInsert]
path, std::make_unique<SensorIntf>(utils::getBus(), path.c_str()));
^
occ_dbus.cpp:115:47: performance: Searching before insertion is not necessary. [stlFindInsert]
path, std::make_unique<SensorIntf>(utils::getBus(), path.c_str()));
^
occ_dbus.cpp:158:80: performance: Searching before insertion is not necessary. [stlFindInsert]
operationalStatus.emplace(path, std::make_unique<OperationalStatusIntf>(
^
occ_pass_through.hpp:42:5: style: Class 'PassThrough' has a constructor with 1 argument that is not explicit. [noExplicitConstructor]
PassThrough(const char* path);
^
powercap.hpp:41:5: style: Class 'PowerCap' has a constructor with 1 argument that is not explicit. [noExplicitConstructor]
PowerCap(Status& occStatus,
^
occ_manager.hpp:77:5: style: Struct 'Manager' has a constructor with 1 argument that is not explicit. [noExplicitConstructor]
Manager(EventPtr& event) :
^
occ_status.hpp:307:26: style: Local variable 'path' shadows outer variable [shadowVariable]
auto path = fs::path(estimatedPath);
^
occ_status.hpp:207:17: note: Shadowed declaration
std::string path;
^
occ_status.hpp:307:26: note: Shadow variable
auto path = fs::path(estimatedPath);
^
pldm.hpp:55:14: warning: Member variable 'Interface::OCCSensorOffset' is not initialized in the constructor. [uninitMemberVar]
explicit Interface(
^
pldm.hpp:55:14: warning: Member variable 'Interface::SBESensorOffset' is not initialized in the constructor. [uninitMemberVar]
explicit Interface(
^
powermode.hpp:60:5: style: Class 'PowerMode' has a constructor with 1 argument that is not explicit. [noExplicitConstructor]
PowerMode(Status& occStatus) :
^
powermode.hpp:96:5: style: Class 'PowerIPS' has a constructor with 1 argument that is not explicit. [noExplicitConstructor]
PowerIPS(Status& occStatus) :
^
occ_manager.cpp:321:24: style: The scope of the variable 'path' can be reduced. [variableScope]
constexpr auto path = "/org/openpower/dump";
^
occ_manager.cpp:322:24: style: The scope of the variable 'interface' can be reduced. [variableScope]
constexpr auto interface = "xyz.openbmc_project.Dump.Create";
^
occ_manager.cpp:323:24: style: The scope of the variable 'function' can be reduced. [variableScope]
constexpr auto function = "CreateDump";
^
pldm.cpp:78:16: style: Local variable 'pdr' shadows outer variable [shadowVariable]
for (auto& pdr : pdrs)
^
pldm.cpp:48:10: note: Shadowed declaration
auto pdr =
^
pldm.cpp:78:16: note: Shadow variable
for (auto& pdr : pdrs)
^
pldm.cpp:248:16: style: Local variable 'pdr' shadows outer variable [shadowVariable]
for (auto& pdr : pdrs)
^
pldm.cpp:221:10: note: Shadowed declaration
auto pdr =
^
pldm.cpp:248:16: note: Shadow variable
for (auto& pdr : pdrs)
^
powermode.cpp:28:22: style: Variable 'pmode' is assigned a value that is never used. [unreadVariable]
SysPwrMode pmode = SysPwrMode::NO_CHANGE;

Tested: Use cppcheck to build successfully and eliminate the above
warning message.

Signed-off-by: George Liu <liuxiwei@inspur.com>
Change-Id: I72751dc94db684fc558b4bb57b1924be64ce6760

show more ...

Clean up error cases on boot or when app restarted

- Detect when no or duplicate masters are found (force reset)
- Prevent sending commands to OCC when OCC is disabled
- Detect OCC state changes to

Clean up error cases on boot or when app restarted

- Detect when no or duplicate masters are found (force reset)
- Prevent sending commands to OCC when OCC is disabled
- Detect OCC state changes to SAFE (after 60 sec force reset)

Tested on Everest and Rainier hardware

Signed-off-by: Chris Cain <cjcain@us.ibm.com>
Change-Id: I490f182405e11da207b42a0607a532566479bfd9

show more ...

Send ambient and altitude to OCC

After the OCCs go active or anytime the ambient temperature changes the
ambient temperature and alititude will get sent to each OCC.
The altitude is only read once.

Send ambient and altitude to OCC

After the OCCs go active or anytime the ambient temperature changes the
ambient temperature and alititude will get sent to each OCC.
The altitude is only read once. If the altitude was not valid,
another attempt to read it will be done when the ambient temperature
is going to be re-sent.

Tested on Everest and Rainier systems

Change-Id: Icd64c50c05469bc985cfcaa3fcc74c3db5b41429
Signed-off-by: Chris Cain <cjcain@us.ibm.com>

show more ...

Add SBE timeout handling

Add an Error instance to detect SBE timeouts in the driver.
Refactor the PLDM code to handle another requeset type (the HRESET) and
another type of sensor event (success or

Add SBE timeout handling

Add an Error instance to detect SBE timeouts in the driver.
Refactor the PLDM code to handle another requeset type (the HRESET) and
another type of sensor event (success or failure of the HRESET).
Handle the timeout and the PLDM event state change in the Manager by
changing the SBE state and requesting an SBE dump if necessary.

Signed-off-by: Eddie James <eajames@linux.ibm.com>
Change-Id: I30d8f9de1914d9808ddb7b547cc57085a5e5779e

show more ...

Add FFDC collection

Add a class to watch for the presence of SBEFIFO FFDC reported up
through the OCC hwmon driver.

Signed-off-by: Eddie James <eajames@linux.ibm.com>
Change-Id: Ia8bf6eddb037aec547

Add FFDC collection

Add a class to watch for the presence of SBEFIFO FFDC reported up
through the OCC hwmon driver.

Signed-off-by: Eddie James <eajames@linux.ibm.com>
Change-Id: Ia8bf6eddb037aec547b72ecce39c1a977d9374bc

show more ...

Use Idle Power Saver parameters from DBus

Allows users to enable and update the IPS parameters instead of
using hardcoded values.

Change-Id: I9010c4b4d3dbdf130a4a778f71c87279681a9f1a
Signed-off-by:

Use Idle Power Saver parameters from DBus

Allows users to enable and update the IPS parameters instead of
using hardcoded values.

Change-Id: I9010c4b4d3dbdf130a4a778f71c87279681a9f1a
Signed-off-by: Chris Cain <cjcain@us.ibm.com>

show more ...

Add meson build

This commit is to add meson build.
and later, we will remove Autotools and replace it with meson build.

Tested: built openpower-occ-control successfully and Unit test passes.

Signe

Add meson build

This commit is to add meson build.
and later, we will remove Autotools and replace it with meson build.

Tested: built openpower-occ-control successfully and Unit test passes.

Signed-off-by: George Liu <liuxiwei@inspur.com>
Change-Id: I7f5fbc7150194a78f9b36bb256613fc70b834130

show more ...

Update clang-format

refer: https://github.com/openbmc/docs/blob/master/style/cpp/.clang-format

Tested: built openpower-occ-control successfully.

Signed-off-by: George Liu <liuxiwei@inspur.com>
Cha

Update clang-format

refer: https://github.com/openbmc/docs/blob/master/style/cpp/.clang-format

Tested: built openpower-occ-control successfully.

Signed-off-by: George Liu <liuxiwei@inspur.com>
Change-Id: I136870023229ff906d471dd4d8debfd404812ff1

show more ...

Read proc temps and dimm temps

This commit uses openpower-occ-control to monitor the temperature and
power sensors, and create sensors on D-BUS. In the loop, read the
sensor values every 1 seconds.

Read proc temps and dimm temps

This commit uses openpower-occ-control to monitor the temperature and
power sensors, and create sensors on D-BUS. In the loop, read the
sensor values every 1 seconds.

Tested:
I use virtual data to get sensor values:
busctl tree org.open_power.OCC.Control
|-/org
| `-/org/open_power
| `-/org/open_power/control
| |-/org/open_power/control/occ0
| `-/org/open_power/control/occ1
`-/xyz
`-/xyz/openbmc_project
`-/xyz/openbmc_project/sensors
|-/xyz/openbmc_project/sensors/power
| |-/xyz/openbmc_project/sensors/power/p0_mem_2_power
| |-/xyz/openbmc_project/sensors/power/p0_mem_power
| |-/xyz/openbmc_project/sensors/power/p0_power
| |-/xyz/openbmc_project/sensors/power/p1_mem_power
| |-/xyz/openbmc_project/sensors/power/p1_power
| |-/xyz/openbmc_project/sensors/power/p2_mem_power
| |-/xyz/openbmc_project/sensors/power/p2_power
| |-/xyz/openbmc_project/sensors/power/p3_mem_power
| |-/xyz/openbmc_project/sensors/power/p3_power
| `-/xyz/openbmc_project/sensors/power/total_power
`-/xyz/openbmc_project/sensors/temperature
|-/xyz/openbmc_project/sensors/temperature/dimm5_dram_temp
|-/xyz/openbmc_project/sensors/temperature/dimm9_dram_temp
|-/xyz/openbmc_project/sensors/temperature/proc0_core2_temp
|-/xyz/openbmc_project/sensors/temperature/proc0_core3_temp
|-/xyz/openbmc_project/sensors/temperature/proc1_core2_temp
|-/xyz/openbmc_project/sensors/temperature/proc1_core3_temp
|-/xyz/openbmc_project/sensors/temperature/vrm_vdd0_temp
`-/xyz/openbmc_project/sensors/temperature/vrm_vdd1_temp

busctl introspect org.open_power.OCC.Control
/xyz/openbmc_project/sensors/temperature/proc0_core3_temp
NAME TYPE SIGNATURE RESULT/VALUE
org.freedesktop.DBus.Introspectable interface - -
.Introspect method - s
org.freedesktop.DBus.Peer interface - -
.GetMachineId method - s
.Ping method - -
org.freedesktop.DBus.Properties interface - -
.Get method ss v
.GetAll method s a{sv}
.Set method ssv -
.PropertiesChanged signal sa{sv}as -
xyz.openbmc_project.Sensor.Value interface - -
.MaxValue property d 0
.MinValue property d 0
.Unit property s "xyz.openbmc...
.Value property d 49
xyz.openbmc_project.State.Decorator.OperationalStatus interface - -
.Functional property b true

busctl introspect org.open_power.OCC.Control
/xyz/openbmc_project/sensors/power/total_power
NAME TYPE SIGNATURE RESULT/VALUE
org.freedesktop.DBus.Introspectable interface - -
.Introspect method - s
org.freedesktop.DBus.Peer interface - -
.GetMachineId method - s
.Ping method - -
org.freedesktop.DBus.Properties interface - -
.Get method ss v
.GetAll method s a
.Set method ssv -
.PropertiesChanged signal sa{sv}as -
xyz.openbmc_project.Sensor.Value interface - -
.MaxValue property d 0
.MinValue property d 0
.Unit property s "xyz.openbmc...
.Value property d 83
xyz.openbmc_project.State.Decorator.OperationalStatus interface - -
.Functional property b true

Signed-off-by: Chicago Duan <duanzhijia01@inspur.com>
Change-Id: Iff30ab51745dab500fa19aa4c35b07e0052ac665

show more ...

Send OCC mode change and IPS parms after reaching active state

Change is only applicable for POWER10 PowerVM systems.
Send mode change requests to OCC when PowerMode is changed.
Retry OCC command on

Send OCC mode change and IPS parms after reaching active state

Change is only applicable for POWER10 PowerVM systems.
Send mode change requests to OCC when PowerMode is changed.
Retry OCC command once, if a valid response is not received.

Tested: built openpower-occ-control successfully and verified
functionality on hardware.

Change-Id: If375f8abfdb2ad82937b8744096c2ccc471263d2
Signed-off-by: Chris Cain <cjcain@us.ibm.com>

show more ...