1d070b7d7SDeepak Kodihalli# PLDM stack on OpenBMC 2d070b7d7SDeepak Kodihalli 3d070b7d7SDeepak KodihalliAuthor: Deepak Kodihalli <dkodihal@linux.vnet.ibm.com> <dkodihal> 44134f4f1SDeepak Kodihalli 5d070b7d7SDeepak KodihalliCreated: 2019-01-22 6d070b7d7SDeepak Kodihalli 7d070b7d7SDeepak Kodihalli## Problem Description 8f4febd00SPatrick Williams 9d070b7d7SDeepak KodihalliOn OpenBMC, in-band IPMI is currently the primary industry-standard means of 10d070b7d7SDeepak Kodihallicommunication between the BMC and the Host firmware. We've started hitting some 11d070b7d7SDeepak Kodihalliinherent limitations of IPMI on OpenPOWER servers: a limited number of sensors, 12d070b7d7SDeepak Kodihalliand a lack of a generic control mechanism (sensors are a generic monitoring 13d070b7d7SDeepak Kodihallimechanism) are the major ones. There is a need to improve upon the communication 14d070b7d7SDeepak Kodihalliprotocol, but at the same time inventing a custom protocol is undesirable. 15d070b7d7SDeepak Kodihalli 16d070b7d7SDeepak KodihalliThis design aims to employ Platform Level Data Model (PLDM), a standard 17d070b7d7SDeepak Kodihalliapplication layer communication protocol defined by the DMTF. PLDM draws inputs 18d070b7d7SDeepak Kodihallifrom IPMI, but it overcomes most of the latter's limitations. PLDM is also 19d070b7d7SDeepak Kodihallidesigned to run on standard transport protocols, for e.g. MCTP (also designed by 20d070b7d7SDeepak Kodihallithe DMTF). MCTP provides for a common transport layer over several physical 21d070b7d7SDeepak Kodihallichannels, by defining hardware bindings. The solution of PLDM over MCTP also 22d070b7d7SDeepak Kodihallihelps overcome some of the limitations of the hardware channels that IPMI uses. 23d070b7d7SDeepak Kodihalli 24d070b7d7SDeepak KodihalliPLDM's purpose is to enable all sorts of "inside the box communication": BMC - 25d070b7d7SDeepak KodihalliHost, BMC - BMC, BMC - Network Controller and BMC - Other (for e.g. sensor) 26d070b7d7SDeepak Kodihallidevices. 27d070b7d7SDeepak Kodihalli 28d070b7d7SDeepak Kodihalli## Background and References 29f4febd00SPatrick Williams 30d070b7d7SDeepak KodihalliPLDM is designed to be an effective interface and data model that provides 31d070b7d7SDeepak Kodihalliefficient access to low-level platform inventory, monitoring, control, event, 32d070b7d7SDeepak Kodihalliand data/parameters transfer functions. For example, temperature, voltage, or 33d070b7d7SDeepak Kodihallifan sensors can have a PLDM representation that can be used to monitor and 34d070b7d7SDeepak Kodihallicontrol the platform using a set of PLDM messages. PLDM defines data 35d070b7d7SDeepak Kodihallirepresentations and commands that abstract the platform management hardware. 36d070b7d7SDeepak Kodihalli 37f4febd00SPatrick WilliamsPLDM groups commands under broader functions, and defines separate 38f4febd00SPatrick Williamsspecifications for each of these functions (also called PLDM "Types"). The 39f4febd00SPatrick Williamscurrently defined Types (and corresponding specs) are : PLDM base (with 40d070b7d7SDeepak Kodihalliassociated IDs and states specs), BIOS, FRU, Platform monitoring and control, 41d070b7d7SDeepak KodihalliFirmware Update and SMBIOS. All these specifications are available at: 42d070b7d7SDeepak Kodihalli 43d070b7d7SDeepak Kodihallihttps://www.dmtf.org/standards/pmci 44d070b7d7SDeepak Kodihalli 45d070b7d7SDeepak KodihalliSome of the reasons PLDM sounds promising (some of these are advantages over 46d070b7d7SDeepak KodihalliIPMI): 47d070b7d7SDeepak Kodihalli 48d070b7d7SDeepak Kodihalli- Common in-band communication protocol. 49d070b7d7SDeepak Kodihalli 50d070b7d7SDeepak Kodihalli- Already existing PLDM Type specifications that cover the most common 51d070b7d7SDeepak Kodihalli communication requirements. Up to 64 PLDM Types can be defined (the last one 52d070b7d7SDeepak Kodihalli is OEM). At the moment, 6 are defined. Each PLDM type can house up to 256 PLDM 53d070b7d7SDeepak Kodihalli commands. 54d070b7d7SDeepak Kodihalli 55d070b7d7SDeepak Kodihalli- PLDM sensors are 2 bytes in length. 56d070b7d7SDeepak Kodihalli 57d070b7d7SDeepak Kodihalli- PLDM introduces the concept of effecters - a control mechanism. Both sensors 58d070b7d7SDeepak Kodihalli and effecters are associated to entities (similar to IPMI, entities can be 59d070b7d7SDeepak Kodihalli physical or logical), where sensors are a mechanism for monitoring and 60d070b7d7SDeepak Kodihalli effecters are a mechanism for control. Effecters can be numeric or state 61d070b7d7SDeepak Kodihalli based. PLDM defines commonly used entities and their IDs, but there 8K slots 62d070b7d7SDeepak Kodihalli available to define OEM entities. 63d070b7d7SDeepak Kodihalli 64d070b7d7SDeepak Kodihalli- A very active PLDM related working group in the DMTF. 65d070b7d7SDeepak Kodihalli 66d070b7d7SDeepak KodihalliThe plan is to run PLDM over MCTP. MCTP is defined in a spec of its own, and a 67d070b7d7SDeepak Kodihalliproposal on the MCTP design is in discussion already. There's going to be an 68d070b7d7SDeepak Kodihalliintermediate PLDM over MCTP binding layer, which lets us send PLDM messages over 69d070b7d7SDeepak KodihalliMCTP. This is defined in a spec of its own, and the design for this binding will 70d070b7d7SDeepak Kodihallibe proposed separately. 71d070b7d7SDeepak Kodihalli 72d070b7d7SDeepak Kodihalli## Requirements 73f4febd00SPatrick Williams 74f4febd00SPatrick WilliamsHow different BMC applications make use of PLDM messages is outside the scope of 75f4febd00SPatrick Williamsthis requirements doc. The requirements listed here are related to the PLDM 76d070b7d7SDeepak Kodihalliprotocol stack and the request/response model: 77d070b7d7SDeepak Kodihalli 78d070b7d7SDeepak Kodihalli- Marshalling and unmarshalling of PLDM messages, defined in various PLDM Type 79d070b7d7SDeepak Kodihalli specs, must be implemented. This can of course be staged based on the need of 80d070b7d7SDeepak Kodihalli specific Types and functions. Since this is just encoding and decoding PLDM 81d070b7d7SDeepak Kodihalli messages, this can be a library that could shared between the BMC, and other 82d070b7d7SDeepak Kodihalli firmware stacks. The specifics of each PLDM Type (such as FRU table 83d070b7d7SDeepak Kodihalli structures, sensor PDR structures, etc) are implemented by this lib. 84d070b7d7SDeepak Kodihalli 85d070b7d7SDeepak Kodihalli- Mapping PLDM concepts to native OpenBMC concepts must be implemented. For 86d070b7d7SDeepak Kodihalli e.g.: mapping PLDM sensors to phosphor-hwmon hosted D-Bus objects, mapping 87d070b7d7SDeepak Kodihalli PLDM FRU data to D-Bus objects hosted by phosphor-inventory-manager, etc. The 88d070b7d7SDeepak Kodihalli mapping shouldn't be restrictive to D-Bus alone (meaning it shouldn't be 89d070b7d7SDeepak Kodihalli necessary to put objects on the Bus just to serve PLDM requests, a problem 90d070b7d7SDeepak Kodihalli that exists with phosphor-host-ipmid today). Essentially these are platform 91d070b7d7SDeepak Kodihalli specific PLDM message handlers. 92d070b7d7SDeepak Kodihalli 93d070b7d7SDeepak Kodihalli- The BMC should be able to act as a PLDM responder as well as a PLDM requester. 94d070b7d7SDeepak Kodihalli As a PLDM requester, the BMC can monitor/control other devices. As a PLDM 95d070b7d7SDeepak Kodihalli responder, the BMC can react to PLDM messages directed to it via requesters in 96d070b7d7SDeepak Kodihalli the platform. 97d070b7d7SDeepak Kodihalli 98d070b7d7SDeepak Kodihalli- As a PLDM requester, the BMC must be able to discover other PLDM enabled 99d070b7d7SDeepak Kodihalli components in the platform. 100d070b7d7SDeepak Kodihalli 101d070b7d7SDeepak Kodihalli- As a PLDM requester, the BMC must be able to send simultaneous messages to 102d070b7d7SDeepak Kodihalli different responders. 103d070b7d7SDeepak Kodihalli 104d070b7d7SDeepak Kodihalli- As a PLDM requester, the BMC must be able to handle out of order responses. 105d070b7d7SDeepak Kodihalli 106d070b7d7SDeepak Kodihalli- As a PLDM responder, the BMC may simultaneously respond to messages from 107d070b7d7SDeepak Kodihalli different requesters, but the spec doesn't mandate this. In other words the 108d070b7d7SDeepak Kodihalli responder could be single-threaded. 109d070b7d7SDeepak Kodihalli 110d070b7d7SDeepak Kodihalli- It should be possible to plug-in OEM PLDM types/functions into the PLDM stack. 111d070b7d7SDeepak Kodihalli 112ed331697SGilbert Chen- As a PLDM sensor monitoring daemon, the BMC must be able to enumerate and 113ed331697SGilbert Chen monitor the static or self-described(with PDRs) PLDM sensors in satellite 114ed331697SGilbert Chen Management Controller, on board device or PCIe add-on card. 115ed331697SGilbert Chen 116d070b7d7SDeepak Kodihalli## Proposed Design 117f4febd00SPatrick Williams 118d070b7d7SDeepak KodihalliThis document covers the architectural, interface, and design details. It 119d070b7d7SDeepak Kodihalliprovides recommendations for implementations, but implementation details are 120d070b7d7SDeepak Kodihallioutside the scope of this document. 121d070b7d7SDeepak Kodihalli 122d070b7d7SDeepak KodihalliThe design aims at having a single PLDM daemon serve both the requester and 123f4febd00SPatrick Williamsresponder functions, and having transport specific endpoints to communicate on 124f4febd00SPatrick Williamsdifferent channels. 125d070b7d7SDeepak Kodihalli 126d070b7d7SDeepak KodihalliThe design enables concurrency aspects of the requester and responder functions, 127d070b7d7SDeepak Kodihallibut the goal is to employ asynchronous IO and event loops, instead of multiple 128d070b7d7SDeepak Kodihallithreads, wherever possible. 129d070b7d7SDeepak Kodihalli 130d070b7d7SDeepak KodihalliThe following are high level structural elements of the design: 131d070b7d7SDeepak Kodihalli 132d070b7d7SDeepak Kodihalli### PLDM encode/decode libraries 133d070b7d7SDeepak Kodihalli 134d070b7d7SDeepak KodihalliThis library would take a PLDM message, decode it and extract the different 135d070b7d7SDeepak Kodihallifields of the message. Conversely, given a PLDM Type, command code, and the 136d070b7d7SDeepak Kodihallicommand's data fields, it would make a PLDM message. The thought is to design 137d070b7d7SDeepak Kodihallithis as a common library, that can be used by the BMC and other firmware stacks, 138d070b7d7SDeepak Kodihallibecause it's the encode/decode and protocol piece (and not the handling of a 139d070b7d7SDeepak Kodihallimessage). 140d070b7d7SDeepak Kodihalli 141d070b7d7SDeepak Kodihalli### PLDM provider libraries 142d070b7d7SDeepak Kodihalli 143d070b7d7SDeepak KodihalliThese libraries would implement the platform specific handling of incoming PLDM 144d070b7d7SDeepak Kodihallirequests (basically helping with the PLDM responder implementation, see next 145d070b7d7SDeepak Kodihallibullet point), so for instance they would query D-Bus objects (or even something 146d070b7d7SDeepak Kodihallilike a JSON file) to fetch platform specific information to respond to the PLDM 147d070b7d7SDeepak Kodihallimessage. They would link with the encode/decode lib. 148d070b7d7SDeepak Kodihalli 149d070b7d7SDeepak KodihalliIt should be possible to plug-in a provider library, that lets someone add 150d070b7d7SDeepak Kodihallifunctionality for new PLDM (standard as well as OEM) Types. The libraries would 151d070b7d7SDeepak Kodihalliimplement a "register" API to plug-in handlers for specific PLDM messages. 152d070b7d7SDeepak KodihalliSomething like: 153d070b7d7SDeepak Kodihalli 154f4febd00SPatrick Williamstemplate <typename Handler, typename... args> auto register(uint8_t type, 155f4febd00SPatrick Williamsuint8_t command, Handler handler); 156d070b7d7SDeepak Kodihalli 157d070b7d7SDeepak KodihalliThis allows for providing a strongly-typed C++ handler registration scheme. It 158d070b7d7SDeepak Kodihalliwould also be possible to validate the parameters passed to the handler at 159d070b7d7SDeepak Kodihallicompile time. 160d070b7d7SDeepak Kodihalli 161d070b7d7SDeepak Kodihalli### Request/Response Model 162d070b7d7SDeepak Kodihalli 163f4febd00SPatrick WilliamsThe PLDM daemon links with the encode/decode and provider libs. The daemon would 164f4febd00SPatrick Williamshave to implement the following functions: 165d070b7d7SDeepak Kodihalli 166d070b7d7SDeepak Kodihalli#### Receiver/Responder 167f4febd00SPatrick Williams 168d070b7d7SDeepak KodihalliThe receiver wakes up on getting notified of incoming PLDM messages (via D-Bus 169d070b7d7SDeepak Kodihallisignal or callback from the transport layer) from a remote PLDM device. If the 170d070b7d7SDeepak Kodihallimessage type is "Request" it would route them to a PLDM provider library. Via 171d070b7d7SDeepak Kodihallithe library, asynchronous D-Bus calls (using sdbusplus-asio) would be made, so 172d070b7d7SDeepak Kodihallithat the receiver can register a handler for the D-Bus response, instead of 173d070b7d7SDeepak Kodihallihaving to wait for the D-Bus response. This way it can go back to listening for 174d070b7d7SDeepak Kodihalliincoming PLDM messages. 175d070b7d7SDeepak Kodihalli 176d070b7d7SDeepak KodihalliIn the D-Bus response handler, the receiver will send out the PLDM response 177d070b7d7SDeepak Kodihallimessage via the transport's send message API. If the transport's send message 178d070b7d7SDeepak KodihalliAPI blocks for a considerably long duration, then it would have to be run in a 179d070b7d7SDeepak Kodihallithread of it's own. 180d070b7d7SDeepak Kodihalli 181d070b7d7SDeepak KodihalliIf the incoming PLDM message is of type "Response", then the receiver emits a 182f4febd00SPatrick WilliamsD-Bus signal pointing to the response message. Any time the message is too large 183f4febd00SPatrick Williamsto fit in a D-Bus payload, the message is written to a file, and a read-only 184f4febd00SPatrick Williamsfile descriptor pointing to that file is contained in the D-Bus signal. 185d070b7d7SDeepak Kodihalli 186d070b7d7SDeepak Kodihalli#### Requester 187f4febd00SPatrick Williams 188d070b7d7SDeepak KodihalliDesigning the BMC as a PLDM requester is interesting. We haven't had this with 189d070b7d7SDeepak KodihalliIPMI, because the BMC was typically an IPMI server. PLDM requester functions 190d070b7d7SDeepak Kodihalliwill be spread across multiple OpenBMC applications (instead of a single big 191d070b7d7SDeepak Kodihallirequester app) - based on the responder they're talking to and the high level 192d070b7d7SDeepak Kodihallifunction they implement. For example, there could be an app that lets the BMC 193f4febd00SPatrick Williamsupgrade firmware for other devices using PLDM - this would be a generic app in 194f4febd00SPatrick Williamsthe sense that the same set of commands might have to be run irrespective of the 195f4febd00SPatrick Williamsdevice on the other side. There could also be an app that does fan control on a 196f4febd00SPatrick Williamsremote device, based on sensors from that device and algorithms specific to that 197f4febd00SPatrick Williamsdevice. 198d070b7d7SDeepak Kodihalli 199e70b2ba6SDeepak Kodihalli##### Proposed requester design 200d070b7d7SDeepak Kodihalli 201e70b2ba6SDeepak KodihalliA requester app/flow comprises of the following : 202e70b2ba6SDeepak Kodihalli 203e70b2ba6SDeepak Kodihalli- Linkage with a PLDM encode/decode library, to be able to pack PLDM requests 204e70b2ba6SDeepak Kodihalli and unpack PLDM responses. 205e70b2ba6SDeepak Kodihalli 206e70b2ba6SDeepak Kodihalli- A D-Bus API to generate a unique PLDM instance id. The id needs to be unique 207f4febd00SPatrick Williams across all outgoing PLDM messages (from potentially different processes). This 208f4febd00SPatrick Williams needs to be on D-Bus because the id needs to be unique across PLDM requester 209f4febd00SPatrick Williams app processes. 210e70b2ba6SDeepak Kodihalli 211e70b2ba6SDeepak Kodihalli- A requester client API that provides blocking and non-blocking functions to 212e70b2ba6SDeepak Kodihalli transfer a PLDM request message and to receive the corresponding response 213f4febd00SPatrick Williams message, over MCTP (the blocking send() will return a PLDM response). This 214f4febd00SPatrick Williams will be a thin wrapper over the socket API provided by the mctp demux daemon. 215f4febd00SPatrick Williams This will provide APIs for common tasks so that the same may not be 216f4febd00SPatrick Williams re-implemented in each PLDM requester app. This set of API will be built into 217f4febd00SPatrick Williams the encode/decode library (so libpldm would house encode/decode APIs, and 218e70b2ba6SDeepak Kodihalli based on a compile time flag, the requester APIs as well). A PLDM requester 219e70b2ba6SDeepak Kodihalli app can choose to not use the client requester APIs, and instead can directly 220e70b2ba6SDeepak Kodihalli talk to the MCTP demux daemon. 221e70b2ba6SDeepak Kodihalli 222e70b2ba6SDeepak Kodihalli##### Proposed requester design - flow diagrams 223e70b2ba6SDeepak Kodihalli 224e70b2ba6SDeepak Kodihallia) With blocking API 225e70b2ba6SDeepak Kodihalli 2267c8847e9SDeepak Kodihalli``` 227e70b2ba6SDeepak Kodihalli+---------------+ +----------------+ +----------------+ +-----------------+ 228e70b2ba6SDeepak Kodihalli|BMC requester/ | |PLDM requester | |PLDM responder | |PLDM Daemon | 229e70b2ba6SDeepak Kodihalli|client app | |lib (part of | | | | | 230e70b2ba6SDeepak Kodihalli| | |libpldm) | | | | | 231e70b2ba6SDeepak Kodihalli+-------+-------+ +-------+--------+ +--------+-------+ +---------+-------+ 232e70b2ba6SDeepak Kodihalli | | | | 233e70b2ba6SDeepak Kodihalli |App starts | | | 234e70b2ba6SDeepak Kodihalli | | | | 235e70b2ba6SDeepak Kodihalli +------------------------------->setup connection with | | 236e70b2ba6SDeepak Kodihalli |init(non_block=false) |MCTP daemon | | 237e70b2ba6SDeepak Kodihalli | | | | 238e70b2ba6SDeepak Kodihalli +<-------+return_code+----------+ | | 239e70b2ba6SDeepak Kodihalli | | | | 240e70b2ba6SDeepak Kodihalli | | | | 241e70b2ba6SDeepak Kodihalli | | | | 242e70b2ba6SDeepak Kodihalli +------------------------------>+ | | 243e70b2ba6SDeepak Kodihalli |encode_pldm_cmd(cmd code, args)| | | 244e70b2ba6SDeepak Kodihalli | | | | 245e70b2ba6SDeepak Kodihalli +<----+returns pldm_msg+--------+ | | 246e70b2ba6SDeepak Kodihalli | | | | 247e70b2ba6SDeepak Kodihalli | | | | 248e70b2ba6SDeepak Kodihalli |----------------------------------------------------------------------------------------------->| 249e70b2ba6SDeepak Kodihalli |DBus.getPLDMInstanceId() | | | 250e70b2ba6SDeepak Kodihalli | | | | 251e70b2ba6SDeepak Kodihalli |<-------------------------returns PLDM instance id----------------------------------------------| 252e70b2ba6SDeepak Kodihalli | | | | 253e70b2ba6SDeepak Kodihalli +------------------------------>+ | | 254e70b2ba6SDeepak Kodihalli |send_msg(mctp_eids, pldm_msg) +----------------------------->+ | 255e70b2ba6SDeepak Kodihalli | |write msg to MCTP socket | | 256e70b2ba6SDeepak Kodihalli | +----------------------------->+ | 257e70b2ba6SDeepak Kodihalli | |call blocking recv() on socket| | 258e70b2ba6SDeepak Kodihalli | | | | 259e70b2ba6SDeepak Kodihalli | +<-+returns pldm_response+-----+ | 260e70b2ba6SDeepak Kodihalli | | | | 261e70b2ba6SDeepak Kodihalli | +----+ | | 262e70b2ba6SDeepak Kodihalli | | | verify eids, instance id| | 263e70b2ba6SDeepak Kodihalli | +<---+ | | 264e70b2ba6SDeepak Kodihalli | | | | 265e70b2ba6SDeepak Kodihalli +<--+returns pldm_response+-----+ | | 266e70b2ba6SDeepak Kodihalli | | | | 267e70b2ba6SDeepak Kodihalli | | | | 268e70b2ba6SDeepak Kodihalli | | | | 269e70b2ba6SDeepak Kodihalli +------------------------------>+ | | 270e70b2ba6SDeepak Kodihalli |decode_pldm_cmd(pldm_resp, | | | 271e70b2ba6SDeepak Kodihalli | output args) | | | 272e70b2ba6SDeepak Kodihalli | | | | 273e70b2ba6SDeepak Kodihalli +------------------------------>+ | | 274e70b2ba6SDeepak Kodihalli |close_connection() | | | 275e70b2ba6SDeepak Kodihalli + + + + 2767c8847e9SDeepak Kodihalli``` 277e70b2ba6SDeepak Kodihalli 278e70b2ba6SDeepak Kodihallib) With non-blocking API 279e70b2ba6SDeepak Kodihalli 2807c8847e9SDeepak Kodihalli``` 281e70b2ba6SDeepak Kodihalli+---------------+ +----------------+ +----------------+ +---------------+ 282e70b2ba6SDeepak Kodihalli|BMC requester/ | |PLDM requester | |PLDM responder | |PLDM daemon | 283e70b2ba6SDeepak Kodihalli|client app | |lib (part of | | | | | 284e70b2ba6SDeepak Kodihalli| | |libpldm) | | | | | 285e70b2ba6SDeepak Kodihalli+-------+-------+ +-------+--------+ +--------+-------+ +--------+------+ 286e70b2ba6SDeepak Kodihalli | | | | 287e70b2ba6SDeepak Kodihalli |App starts | | | 288e70b2ba6SDeepak Kodihalli | | | | 289e70b2ba6SDeepak Kodihalli +------------------------------->setup connection with | | 290e70b2ba6SDeepak Kodihalli |init(non_block=true |MCTP daemon | | 291e70b2ba6SDeepak Kodihalli | int* o_mctp_fd) | | | 292e70b2ba6SDeepak Kodihalli | | | | 293e70b2ba6SDeepak Kodihalli +<-------+return_code+----------+ | | 294e70b2ba6SDeepak Kodihalli | | | | 295e70b2ba6SDeepak Kodihalli | | | | 296e70b2ba6SDeepak Kodihalli | | | | 297e70b2ba6SDeepak Kodihalli +------------------------------>+ | | 298e70b2ba6SDeepak Kodihalli |encode_pldm_cmd(cmd code, args)| | | 299e70b2ba6SDeepak Kodihalli | | | | 300e70b2ba6SDeepak Kodihalli +<----+returns pldm_msg+--------+ | | 301e70b2ba6SDeepak Kodihalli | | | | 302e70b2ba6SDeepak Kodihalli |-------------------------------------------------------------------------------------------->| 303e70b2ba6SDeepak Kodihalli |DBus.getPLDMInstanceId() | | | 304e70b2ba6SDeepak Kodihalli | | | | 305e70b2ba6SDeepak Kodihalli |<-------------------------returns PLDM instance id-------------------------------------------| 306e70b2ba6SDeepak Kodihalli | | | | 307e70b2ba6SDeepak Kodihalli | | | | 308e70b2ba6SDeepak Kodihalli +------------------------------>+ | | 309e70b2ba6SDeepak Kodihalli |send_msg(eids, pldm_msg, +----------------------------->+ | 310e70b2ba6SDeepak Kodihalli | non_block=true) |write msg to MCTP socket | | 311e70b2ba6SDeepak Kodihalli | +<---+return_code+-------------+ | 312e70b2ba6SDeepak Kodihalli +<-+returns rc, doesn't block+--+ | | 313e70b2ba6SDeepak Kodihalli | | | | 314e70b2ba6SDeepak Kodihalli +------+ | | | 315e70b2ba6SDeepak Kodihalli | |Add EPOLLIN on mctp_fd | | | 316e70b2ba6SDeepak Kodihalli | |to self.event_loop | | | 317e70b2ba6SDeepak Kodihalli +<-----+ | | | 318e70b2ba6SDeepak Kodihalli | + | | 319e70b2ba6SDeepak Kodihalli +<----------------------+PLDM response msg written to mctp_fd+-+ | 320e70b2ba6SDeepak Kodihalli | + | | 321e70b2ba6SDeepak Kodihalli +------+EPOLLIN on mctp_fd | | | 322e70b2ba6SDeepak Kodihalli | |received | | | 323e70b2ba6SDeepak Kodihalli | | | | | 324e70b2ba6SDeepak Kodihalli +<-----+ | | | 325e70b2ba6SDeepak Kodihalli | | | | 326e70b2ba6SDeepak Kodihalli +------------------------------>+ | | 327e70b2ba6SDeepak Kodihalli |decode_pldm_cmd(pldm_response) | | | 328e70b2ba6SDeepak Kodihalli | | | | 329e70b2ba6SDeepak Kodihalli +------------------------------>+ | | 330e70b2ba6SDeepak Kodihalli |close_connection() | | | 331e70b2ba6SDeepak Kodihalli + + + + 3327c8847e9SDeepak Kodihalli``` 333e70b2ba6SDeepak Kodihalli 334e70b2ba6SDeepak Kodihalli##### Alternative to the proposed requester design 335e70b2ba6SDeepak Kodihalli 336e70b2ba6SDeepak Kodihallia) Define D-Bus interfaces to send and receive PLDM messages : 337e70b2ba6SDeepak Kodihalli 338e70b2ba6SDeepak Kodihalli``` 339e70b2ba6SDeepak Kodihallimethod sendPLDM(uint8 mctp_eid, uint8 msg[]) 340e70b2ba6SDeepak Kodihalli 341e70b2ba6SDeepak Kodihallisignal recvPLDM(uint8 mctp_eid, uint8 pldm_instance_id, uint8 msg[]) 342e70b2ba6SDeepak Kodihalli``` 343e70b2ba6SDeepak Kodihalli 344e70b2ba6SDeepak KodihalliPLDM requester apps can then invoke the above applications. While this 345e70b2ba6SDeepak Kodihallisimplifies things for the user, it has two disadvantages : 346f4febd00SPatrick Williams 347e70b2ba6SDeepak Kodihalli- the app implementing such an interface could be a single point of failure, 348e70b2ba6SDeepak Kodihalli plus sending messages concurrently would be a challenge. 349e70b2ba6SDeepak Kodihalli- the message payload could be large (several pages), and copying the same for 350e70b2ba6SDeepak Kodihalli D-Bus transfers might be undesirable. 351d070b7d7SDeepak Kodihalli 352d070b7d7SDeepak Kodihalli### Multiple transport channels 353f4febd00SPatrick Williams 354d070b7d7SDeepak KodihalliThe PLDM daemon might have to talk to remote PLDM devices via different 355d070b7d7SDeepak Kodihallichannels. While a level of abstraction might be provided by MCTP, the PLDM 356f4febd00SPatrick Williamsdaemon would have to implement a D-Bus interface to target a specific transport 357f4febd00SPatrick Williamschannel, so that requester apps on the BMC can send messages over that 358f4febd00SPatrick Williamstransport. Also, it should be possible to plug-in platform specific D-Bus 359d070b7d7SDeepak Kodihalliobjects that implement an interface to target a platform specific transport. 360d070b7d7SDeepak Kodihalli 361a7ee8e1aSDeepak Kodihalli### Processing PLDM FRU information sent down by the host firmware 362a7ee8e1aSDeepak Kodihalli 363a7ee8e1aSDeepak KodihalliNote: while this is specific to the host BMC communication, most of this might 364a7ee8e1aSDeepak Kodihalliapply to processing PLDM FRU information received from a device connected to the 365a7ee8e1aSDeepak KodihalliBMC as well. 366a7ee8e1aSDeepak Kodihalli 367a7ee8e1aSDeepak KodihalliThe requirement is for the BMC to consume PLDM FRU information received from the 368a7ee8e1aSDeepak Kodihallihost firmware and then have the same exposed via Redfish. An example can be the 369a7ee8e1aSDeepak Kodihallihost firmware sending down processor and core information via PLDM FRU commands, 370a7ee8e1aSDeepak Kodihalliand the BMC making this information available via the Processor and 371a7ee8e1aSDeepak KodihalliProcessorCollection schemas. 372a7ee8e1aSDeepak Kodihalli 373*d886ce89SThu NguyenThis design is built around the pldmd on the BMC: 374a7ee8e1aSDeepak Kodihalli 375a7ee8e1aSDeepak Kodihalli- The pldmd asks the host firmware's PLDM stack for the host's FRU record table, 376a7ee8e1aSDeepak Kodihalli by sending it the PLDM GetFRURecordTable command. The pldmd should send this 377a7ee8e1aSDeepak Kodihalli command if the host indicates support for the PLDM FRU spec. The pldmd 378a7ee8e1aSDeepak Kodihalli receives a PLDM FRU record table from the host firmware ( 379a7ee8e1aSDeepak Kodihalli www.dmtf.org/sites/default/files/standards/documents/DSP0257_1.0.0.pdf). The 380*d886ce89SThu Nguyen daemon parses the FRU record table and hosts the PLDM FRU information on 381*d886ce89SThu Nguyen D-Bus. 382a7ee8e1aSDeepak Kodihalli 383*d886ce89SThu Nguyen- Pldmd will also host the FRU inventory D-Bus from the xyz.openbmc_project. 384*d886ce89SThu Nguyen Inventory namespace can be applied on these objects, by converting PLDM FRU 385*d886ce89SThu Nguyen property values into xyz.openbmc_project.Inventory.Decorator.Asset, 386*d886ce89SThu Nguyen xyz.openbmc_project.Inventory.Decorator.Revision, 387*d886ce89SThu Nguyen xyz.openbmc_project.Inventory.Decorator.AssetTag and 388*d886ce89SThu Nguyen xyz.openbmc_project.Inventory.Decorator.Compatible interfaces property values, 389*d886ce89SThu Nguyen such as Part Number, Serial Number, Version, Names and AssetTag. Bmcweb can 390*d886ce89SThu Nguyen find these FRU inventory objects based on D-Bus interfaces, as it does today. 391a7ee8e1aSDeepak Kodihalli 392ed331697SGilbert Chen### MCTP endpoint discovery 393ed331697SGilbert Chen 394ed331697SGilbert Chen`pldmd` (PLDM daemon) utilizes the 395ed331697SGilbert Chen[MCTP D-Bus interfaces](https://github.com/openbmc/phosphor-dbus-interfaces/tree/master/yaml/xyz/openbmc_project/MCTP) 396ed331697SGilbert Chento enumerate all MCTP endpoints in the system. The MCTP D-Bus interface 397ed331697SGilbert Chenimplements the `SupportedMessageTypes` to have which Message type supported by 398ed331697SGilbert Cheneach endpoint. `pldmd` watches the `InterfacesAdded` D-Bus signals from `mctpd`. 399ed331697SGilbert ChenIt also matches the `InterfaceRemoved` D-Bus signals to find the removed 400ed331697SGilbert Chenendpoint EIDs from `mctpd`. 401ed331697SGilbert Chen 402ed331697SGilbert Chen### Terminus management and discovery 403ed331697SGilbert Chen 404ed331697SGilbert Chen`pldmd` will maintain a terminus table to manage the PLDM terminus in system. 405ed331697SGilbert ChenWhen `pldmd` received the updated EID table from MCTP D-Bus interface, `pldmd` 406ed331697SGilbert Chenshould check if the EID support PLDM message type (0x01) and then adds the EID 407ed331697SGilbert Chenwhich is not in the terminus table yet. When the terminus EID is removed from 408ed331697SGilbert ChenMCTP D-Bus interface, `pldmd` should also clean up the removed endpoint from the 409ed331697SGilbert Chenterminus table. 410ed331697SGilbert Chen 411ed331697SGilbert ChenFor each of terminus in the table, `pldmd` will go through the below steps: 412ed331697SGilbert Chen 413ed331697SGilbert Chen- Terminus initialization 414ed331697SGilbert Chen- Terminus discovery 415ed331697SGilbert Chen- Terminus monitor and control 416ed331697SGilbert Chen 417ed331697SGilbert ChenAll of the added D-Bus object paths and D-Bus interfaces, Monitoring/Controlling 418ed331697SGilbert Chentasks of the terminus will be removed when it is removed from the terminus 419ed331697SGilbert Chentable. 420ed331697SGilbert Chen 421ed331697SGilbert Chen#### Terminus initialization 422ed331697SGilbert Chen 423ed331697SGilbert ChenEach terminus in PLDM interface is identified by terminus ID (TID). This TID is 424ed331697SGilbert Chenan unique number `TID#`. When a new terminus is added to terminus table, `pldmd` 425ed331697SGilbert Chenshould send `GetTID` to get the `TID#`. When the received `TID#` is already 426ed331697SGilbert Chenexisting in TID pool, `pldmd` will call the `SetTID` command to assign a new TID 427ed331697SGilbert Chenfor the terminus. 428ed331697SGilbert Chen 429ed331697SGilbert ChenBeside the `TID#`, terminus can also have `$TerminusName` or `$DeviceName` which 430ed331697SGilbert Chencan be encoded in the Terminus's `Entity Auxiliary Names PDR` (section 28.18 of 431ed331697SGilbert ChenDSP0248 1.2.1) or in the MCTP Entity-manager endpoint EID configuration file 432ed331697SGilbert Chen[Entity-Manager EID configuration](https://github.com/openbmc/entity-manager/blob/master/configurations/yosemite4_floatingfalls.json#L7). 433ed331697SGilbert Chen 434ed331697SGilbert ChenBecause the `$TerminusName` will be included in the Terminus' sensors, effecters 435ed331697SGilbert Chenand states D-Bus object paths, so the EM EID configuration or Terminus's 436ed331697SGilbert Chen`Entity Auxiliary Names PDR` are recommended to be included for the Terminus 437ed331697SGilbert Chensupport sensors, effecters or status. When the EM EID configuration is not 438ed331697SGilbert Chenavailable, the `Entity Auxiliary name PDR` should be added, so all sensors don't 439ed331697SGilbert Chenhave the terminus number `TID#` in it anyhow. 440ed331697SGilbert Chen 441ed331697SGilbert Chen#### Teminus Discovery 442ed331697SGilbert Chen 443ed331697SGilbert ChenAfter the TID assignment steps, `pldmd` should go through `Terminus Discovery` 444ed331697SGilbert Chensteps: 445ed331697SGilbert Chen 446ed331697SGilbert Chen- Send `GetPLDMType` and `GetPLDMVersions` commands to the terminus to record 447ed331697SGilbert Chen the supported PLDM type message/version. 448ed331697SGilbert Chen- If the terminus supports `GetPDRs` command type, `pldmd` will send that 449ed331697SGilbert Chen command to get the terminus PDRs. Based on the retrieved PDRs, `pldmd` will 450ed331697SGilbert Chen collect: 451ed331697SGilbert Chen 452ed331697SGilbert Chen - The association between the entities in the system using 453ed331697SGilbert Chen `Entity Association PDR` (section 28.17 of DSP0248 1.2.1). 454ed331697SGilbert Chen - The entity names using `Entity Auxiliary Names PDR` (Section 28.18 of 455ed331697SGilbert Chen DSP0248 1.2.1). 456ed331697SGilbert Chen - The sensor/effecter/state info in the entities of terminus 457ed331697SGilbert Chen sensors/effecter/state PDRs (section 28.4, 28.6, 28.8, 28.11, 28.14, 28.15, 458ed331697SGilbert Chen 28.25, etc. of DSP0248 1.2.1). 459ed331697SGilbert Chen - The Fru info using FRU PDRs (section 28.22 of DSP0248 1.2.1). 460ed331697SGilbert Chen - The other info using the others PDRs in section 28.x of DSP0248 1.2.1. 461ed331697SGilbert Chen 462ed331697SGilbert Chen The above PDRs can also be configured in the JSON configuration files. When 463ed331697SGilbert Chen the `PDR configuration` is available, the `pldmd` daemon will bypass `GetPDRs` 464ed331697SGilbert Chen steps and read those files to collect that info. The template of the 465ed331697SGilbert Chen configuration files can follow the current format of 466ed331697SGilbert Chen [PDRs configuration files](https://github.com/openbmc/pldm/tree/master/configurations) 467ed331697SGilbert Chen 468ed331697SGilbert Chen After this step, `pldmd` will have the list of `$sensorAuxNames` from the 469ed331697SGilbert Chen Terminus's PDRs or the Terminus' EM JSON configuration files. This 470ed331697SGilbert Chen `$sensorAuxNames` will be included in the PLDM Sensors D-Bus object paths in 471ed331697SGilbert Chen `Sensor creating and monitor` section. 472ed331697SGilbert Chen 473ed331697SGilbert Chen- The `pldmd` then creates the Terminus inventory, sensors, effecters D-Bus 474ed331697SGilbert Chen object paths. 475ed331697SGilbert Chen- At the final steps of `terminus discovery`, `pldmd` will send 476ed331697SGilbert Chen `SetEventReceiver` notifies about the readiness of the BMC for the event 477ed331697SGilbert Chen messages from the terminus. 478ed331697SGilbert Chen 479ed331697SGilbert Chen#### Terminus monitor and control 480ed331697SGilbert Chen 481ed331697SGilbert ChenAfter finishing the discovery steps, the daemon will start monitoring the 482ed331697SGilbert Chensensors, response for the events from terminus and handle the terminus control 483ed331697SGilbert Chenaction from the user. 484ed331697SGilbert Chen 485ed331697SGilbert Chen### Sensor creating and monitor 486ed331697SGilbert Chen 487ed331697SGilbert ChenTo find out all sensors from PLDM terminus, `pldmd` should retrieve all the 488ed331697SGilbert ChenSensor PDRs by PDR Repository commands (`GetPDRRepositoryInfo`, `GetPDR`) for 489ed331697SGilbert Chenthe necessary parameters (e.g., `sensorID#`, `$SensorAuxName`, unit, etc.). 490ed331697SGilbert Chen`pldmd` can use libpldm encode/decode APIs 491ed331697SGilbert Chen(`encode_get_pdr_repository_info_req()`, 492ed331697SGilbert Chen`decode_get_pdr_repository_info_resp()`, `encode_get_pdr_req()`, 493ed331697SGilbert Chen`decode_get_pdr_resp()`) to build the commands message and then sends it to PLDM 494ed331697SGilbert Chenterminus. 495ed331697SGilbert Chen 496ed331697SGilbert ChenRegarding to the static device described in section 8.3.1 of DSP0248 1.2.1, the 497ed331697SGilbert Chendevice uses PLDM for access only and doesn't support PDRs. The PDRs for the 498ed331697SGilbert Chendevice needs to be encoded by Platform specific PDR JSON file by the platform 499ed331697SGilbert Chendeveloper. `pldmd` will generate these sensor PDRs encoded by JSON files and 500ed331697SGilbert Chenparse them as the same as the PDRs fetched by PLDM terminus. 501ed331697SGilbert Chen 502ed331697SGilbert Chen`pldmd` should expose the found PLDM sensor to D-Bus object path 503ed331697SGilbert Chen`/xyz/openbmc*project/sensors/<sensor_type>/SensorName`. The format of 504ed331697SGilbert Chen`sensorName` can be `$TerminusName_$SensorAuxName` or `$TerminusName_SensorID#`. 505ed331697SGilbert Chen`$SensorAuxName` will be included in the `sensorName` whenever they exist. For 506ed331697SGilbert Chenexposing sensor status to D-Bus, `pldmd` should implement following D-Bus 507ed331697SGilbert Cheninterfaces to the D-Bus object path of PLDM sensor. The EM EID configuration or 508ed331697SGilbert Chenthe Terminus' `Entity Auxiliary name PDR` will provide `$TerminusName`. And 509ed331697SGilbert Chen`$SensorAuxName` can be found in the EM EID sensor configuration or the sensor 510ed331697SGilbert ChenPDRs. 511ed331697SGilbert Chen 512ed331697SGilbert Chen- [xyz.openbmc_project.Sensor.Value](https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/yaml/xyz/openbmc_project/Sensor/Value.interface.yaml), 513ed331697SGilbert Chen the interface exposes the sensor reading unit, value, Max/Min Value. 514ed331697SGilbert Chen 515ed331697SGilbert Chen- [xyz.openbmc_project.State.Decorator.OperationalStatus](https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/yaml/xyz/openbmc_project/State/Decorator/OperationalStatus.interface.yaml), 516ed331697SGilbert Chen the interface exposes the sensor status which is functional or not. 517ed331697SGilbert Chen 518ed331697SGilbert ChenAfter doing the discovery of PLDM sensors, `pldmd` should initialize all found 519ed331697SGilbert Chensensors by necessary commands (e.g., `SetNumericSensorEnable`, 520ed331697SGilbert Chen`SetSensorThresholds`, `SetSensorHysteresis and `InitNumericSensor`) and then 521ed331697SGilbert Chenstart to update the sensor status to D-bus objects by polling or async event 522ed331697SGilbert Chenmethod depending on the capability of PLDM terminus. 523ed331697SGilbert Chen 524ed331697SGilbert Chen`pldmd` should update the value property of `Sensor.Value` D-Bus interface after 525ed331697SGilbert Chengetting the response of `GetSensorReading` command successfully. If `pldmd` 526ed331697SGilbert Chenfailed to get the response from PLDM terminus or the completion code returned by 527ed331697SGilbert ChenPLDM terminus is not `PLDM_SUCCESS`, the Functional property of 528ed331697SGilbert Chen`State.Decorator.OperationalStatus` D-Bus interface should be updated to false. 529ed331697SGilbert Chen 530ed331697SGilbert Chen#### Polling v.s. Async method 531ed331697SGilbert Chen 532ed331697SGilbert ChenFor each terminus, `pldmd` maintains a list to poll the Terminus' sensors and 533ed331697SGilbert Chenexposes the status to D-Bus. `pldmd` has a polling timer with the configurable 534ed331697SGilbert Cheninterval to update the PLDM sensors of the terminus periodically. The PLDM 535ed331697SGilbert Chensensor in list has a `updateTime` which is initialized to the value of the 536ed331697SGilbert Chendefined `updateInterval` in sensor PDRs. Upon the polling timer timeout, the 537ed331697SGilbert Chenterminus' sensors will be read using `GetSensorReading` command. The read 538ed331697SGilbert Chencondition is the `elapsed time` from the `last read timestamp` to 539ed331697SGilbert Chen`current timestamp` is more than the sensor's `updateTime`. `pldmd` should have 540ed331697SGilbert ChenAPIs to be paused and resumed by other task (e.g. pausing sensor polling during 541ed331697SGilbert Chenfirmware updating to maximum bandwidth). 542ed331697SGilbert Chen 543ed331697SGilbert ChenTo enable async event method for a sensor to update its status to `pldmd`, 544ed331697SGilbert Chen`pldmd` needs to implement the responder of `PlatformEventMessage` command 545ed331697SGilbert Chendescribed in 13.1 PLDM Event Message of 546ed331697SGilbert Chen[DSP0248 1.2.1](https://www.dmtf.org/sites/default/files/standards/documents/DSP0248_1.2.1.pdf). 547ed331697SGilbert Chen`pldmd` checks the response of `EventMessageSupported` command from PLDM 548ed331697SGilbert Chenterminus to identify if it can generate events. A PLDM sensor can work in event 549ed331697SGilbert Chenaync method if the `updateInterval` of all sensors in the same PLDM terminus are 550ed331697SGilbert Chenlonger than final polling time. Before `pldmd` starts to receive async event 551ed331697SGilbert Chenfrom PLDM terminus, `pldmd` should remove the sensor from poll list and then 552ed331697SGilbert Chensend necessary commands (e.g., `EventMessageBufferSize` and `SetEventReceiver`) 553ed331697SGilbert Chento PLDM terminus for the initialization. 554ed331697SGilbert Chen 555d070b7d7SDeepak Kodihalli## Alternatives Considered 556f4febd00SPatrick Williams 557f4febd00SPatrick WilliamsContinue using IPMI, but start making more use of OEM extensions to suit the 558f4febd00SPatrick Williamsrequirements of new platforms. However, given that the IPMI standard is no 559f4febd00SPatrick Williamslonger under active development, we would likely end up with a large amount of 560f4febd00SPatrick Williamsplatform-specific customisations. This also does not solve the hardware channel 561f4febd00SPatrick Williamsissues in a standard manner. On OpenPOWER hardware at least, we've started to 562f4febd00SPatrick Williamshit some of the limitations of IPMI (for example, we have need for >255 563f4febd00SPatrick Williamssensors). 564d070b7d7SDeepak Kodihalli 565d070b7d7SDeepak Kodihalli## Impacts 566f4febd00SPatrick Williams 567d070b7d7SDeepak KodihalliDevelopment would be required to implement the PLDM protocol, the 568d070b7d7SDeepak Kodihallirequest/response model, and platform specific handling. Low level design is 569d070b7d7SDeepak Kodihallirequired to implement the protocol specifics of each of the PLDM Types. Such low 570d070b7d7SDeepak Kodihallilevel design is not included in this proposal. 571d070b7d7SDeepak Kodihalli 572d070b7d7SDeepak KodihalliDesign and development needs to involve the firmware stacks of management 573d070b7d7SDeepak Kodihallicontrollers and management devices of a platform management subsystem. 574d070b7d7SDeepak Kodihalli 575d070b7d7SDeepak Kodihalli## Testing 576f4febd00SPatrick Williams 577d070b7d7SDeepak KodihalliTesting can be done without having to depend on the underlying transport layer. 578d070b7d7SDeepak Kodihalli 579d070b7d7SDeepak KodihalliThe responder function can be tested by mocking a requester and the transport 580d070b7d7SDeepak Kodihallilayer: this would essentially test the protocol handling and platform specific 581d070b7d7SDeepak Kodihallihandling. The requester function can be tested by mocking a responder: this 582d070b7d7SDeepak Kodihalliwould test the instance id handling and the send/receive functions. 583d070b7d7SDeepak Kodihalli 584d070b7d7SDeepak KodihalliAPIs from the shared libraries can be tested via fuzzing. 585ed331697SGilbert Chen 586ed331697SGilbert ChenThe APIs to parse PDRs from PLDM terminus can be tested by a mocking responder. 587ed331697SGilbert ChenA sample JSON file is provided to test the APIs for mocking PDRs for static PLDM 588ed331697SGilbert Chensensors. 589