xref: /openbmc/docs/architecture/ipmi-architecture.md (revision f4febd002df578bad816239b70950f84ea4567e8)

# IPMI Architecture

IPMI is made up of commands and channels. Commands are provided by providers and
channels are processes called bridges. Commands are received from external
sources (system admin users) via the bridges and routed to the providers'
command handlers via the main IPMI daemon.

## High-Level Overview

IPMI is all about commands and responses. Channels provide a mechanism for
transporting the data, each with a slightly different protocol and transport
layer, but ultimately, the highest level data is a raw IPMI command consisting
of a NetFn/LUN, Command, and optional data. Each response is likewise a
Completion Code and optional data. So the first step is to break apart channels
and the IPMI queue.

```
                                                     /------------------\
    /----------------------------\                   |                  |
    |      KCS/BT - Host         | <-All IPMI cmds-> |                  |
    |                            |                   |                  |
    \----------------------------/                   |   IPMI Daemon    |
                                                     |    (ipmid)       |
                                                     |                  |
   /-----------------------------\                   |                  |
   |            LAN - RMCP+      |                   |                  |
   | /--------------------------\|                   |                  |
   | |*Process the Session and  || <-All IPMI cmds-> |                  |
   | | SOL commands.            ||  except session   |                  |
   | |*Create Session Objs      ||  and SOL cmds     |                  |
   | \--------------------------/|                   |                  |
   \-----------------------------/                   \------------------/
                :                                             ^     ^
                :                                             |     |
                :                                             |     |
     /-------------------------\                              |     |
     | Active session/SOL Objs | <---------Query the session-/      |
     | - Properties            |           and SOL data via Dbus    |
     \-------------------------/                                    |
                                                                    V
                                                     ---------------------\
                                                     | D-Bus services     |
                                                     | ------------------ |
                                                     | Do work on behalf  |
                                                     | of IPMI commands   |
                                                     ---------------------/
```

The IPMI messages that get passed to and from the IPMI daemon (ipmid) are
basically the equivalent of ipmitool's "raw" commands with a little more
information about the message.

```less
Message Data:
  byte:LUN - LUN from netFn/LUN pair (0-3, as per the IPMI spec)
  byte:netFn - netFn from netFn/LUN pair (as per the IPMI spec)
  byte:cmd - IPMI command ID (as per the IPMI spec)
  array<byte>:data - optional command data (as per the IPMI spec)
  dict<string:variant>:options - optional additional meta-data
    "userId":int - IPMI user ID of caller (0 for session-less channels)
    "privilege": enum:privilege - ADMIN, USER, OPERATOR, CALLBACK;
      must be less than or equal to the privilege of the user and less
      than or equal to the max privilege of this channel
```

```less
Response Data:
  byte:CC - IPMI completion code
  array<byte>:data - optional response data
```

A channel bridge, upon receiving a new IPMI command, will extract the necessary
fields from whatever transport format (RMCP+, IPMB, KCS, etc.) For session-based
channels (RMCP+) the bridge is responsible for establishing the session with
credentials and determining the maximum privilege available for this session.
The bridge then takes the extracted command, data, and possibly user and
privilge information, and encodes them in a D-Bus method call to send to the
IPMI daemon, ipmid. The daemon will take the message, and attempt to find an
appropriate handler in its handler tables. If a handler is found, it will
attempt to extract the required parameters for the handler and pass them along.
The handler will return a tuple of response parameters, which will get packed
back into a D-Bus response message and sent back to the calling channel's
bridge. The bridge will then re-package the response into its transport protocol
and send it off.

The next part is to provide a higher-level, strongly-typed, modern C++ mechanism
for registering handlers. Each handler will specify exactly what arguments are
needed for the command and what types will be returned in the response. This
way, the ipmid queue can unpack requests and pack responses in a safe manner.
Because the handler packing and unpacking code is templated, it is written
mostly in headers.

## Details and Implementation

For session-less channels (like BT, KCS, and IPMB), the only privilege check
will be to see that the requested privilege is less than or equal to the
channel's maximum privilege. If the channel has a session and authenticates
users, the privilege must be less than or equal to the channel's maximum
privilege and the user's maximum privilege.

Ipmid takes the LUN/netFN/Cmd tuple and looks up the corresponding handler
function. If the requested privilege is less than or equal to the required
privilege for the given registered command, the request may proceed. If any of
these checks fail, ipmid returns with _Insufficient Privilege_.

At this point, the IPMI command is run through the filter hooks. The default
hook is ACCEPT, where the command just passes onto the execution phase. But OEMs
and providers can register hooks that would ultimately block IPMI commands from
executing, much like the IPMI 2.0 Spec's Firmware Firewall. The hook would be
passed in the context of the IPMI call and the raw content of the call and has
the opportunity to return any valid IPMI completion code. Any non-zero
completion code would prevent the command from executing and would be returned
to the caller.

The network channel bridges (netipmid), executing one process per interface,
handle session (RMCP+) and SOL commands and responses to those commands. Get/Set
SOL configuration, Get session info, close session commands can also be
requested through KCS/BT interface, ipmid daemon must also need details about
session and SOL. In order to maintain sync between ipmid and netipmid daemon,
session and SOL are exposed in D-Bus, which ipmid can query and respond to
commands issued through host interface (KCS/BT).

The next phase is parameter unpacking and validation. This is done by
compiler-generated code with variadic templates at handler registration time.
The registration function is a templated function that allows any type of
handler to be passed in so that the types of the handler can be extracted and
unpacked.

This is done in the core IPMI library (from a high level) like this:

```cpp
template <typename Handler>
class IpmiHandler
{
  public:
    explicit IpmiHandler(Handler&& handler) :
        handler_(std::forward<Handler>(handler))
    {
    }
    message::Response::ptr call(message::Request::ptr request)
    {
        message::Response::ptr response = request->makeResponse();
        // < code to deduce UnpackArgsType and ResultType from Handler >
        UnpackArgsType unpackArgs;
        ipmi::Cc unpackError = request->unpack(unpackArgs);
        if (unpackError != ipmi::ccSuccess)
        {
            response->cc = unpackError;
            return response;
        }
        ResultType result = std::apply(handler_, unpackArgs);
        response->cc = std::get<0>(result);
        auto payload = std::get<1>(result);
        response->pack(*payload);
        return response;
    }
  private:
    Handler handler_;
};
 ...
 namespace ipmi {
   constexpr NetFn netFnApp = 0x06;
 namespace app {
   constexpr Cmd cmdSetUserAccessCommand = 0x43;
 }
  class Context {
     NetFn netFn;
     Cmd cmd;
     int channel;
     int userId;
     Privilege priv;
     boost::asio::yield_context* yield; // for async operations
   };
 }
```

While some IPMI handlers would look like this:

```cpp
ipmi::RspType<> setUserAccess(
    std::shared_ptr<ipmi::Context> context,
    uint4_t channelNumber,
    bool ipmiEnable,
    bool linkAuth,
    bool callbackRestricted,
    bool changeBit,
    uint6_t userId,
    uint2_t reserved1,
    uint4_t privLimit,
    uint4_t reserved2,
    std::optional<uint4_t> userSessionLimit,
    std::optional<uint4_t> reserved3) {
  ...
  return ipmi::ResponseSuccess();
}

ipmi::RspType<uint8_t, // max user IDs
              uint6_t, // count of enabled user IDs
              uint2_t, // user ID enable status
              uint8_t, // count of fixed user IDs
              uint4_t  // user privilege for given channel
              bool,    // ipmi messaging enabled
              bool,    // link authentication enabled
              bool,    // callback access
              bool,    // reserved bit
              >
    getUserAccess(std::shared_ptr<ipmi::Context> context, uint8_t channelNumber,
                  uint8_t userId)
{
    if (<some error condition>)
    {
        return ipmi::response(ipmi::ccParmOutOfRange);
    }
    // code to get
    const auto& [usersMax, status, usersEnabled, usersFixed, access, priv] =
        getSdBus()->yield_method_call(*context->yield, ...);
    return ipmi::responseSuccess(usersMax, usersEnabled, status, usersFixed,
                                 priv, access.messaging, access.linkAuth,
                                 access.callback, false);
}

void providerInitFn()
{
    ipmi::registerHandler(ipmi::prioOpenBmcBase, ipmi::netFnApp,
                          ipmi::app::cmdSetUserAccessCommand,
                          ipmi::Privilege::Admin,
                          setUserAccess);
    ipmi::registerHandler(ipmi::prioOpenBmcBase, ipmi::netFnApp,
                          ipmi::app::cmdGetUserAccessCommand,
                          ipmi::Privilege::Operator,
                          getUserAccess);
}
```

Ipmid providers are all executed as boost::asio::coroutines. This means that
they can take advantage of any of the boost::asio async method calls in a way
that looks like synchronous code, but will execute asynchronously by yielding
control of the processor to the run loop via the yield_context object. Use the
yield_context object by passing it as a 'callback' which will then cause the
calling coroutine to yield the processor until its IO is ready, at which point
it will 'return' much like a synchronous call.

Ideally, all handlers would take advantage of the asynchronous capabilities of
ipmid via the boost::asio::yield_context. This means that the queue could have
multiple in-flight calls that are waiting on another D-Bus call to return. With
asynchronous calls, this will not block the rest of the queue's operation, The
sdbusplus::asio::connection that is available to all providers via the
getSdBus() function provides yield_method_call() which is an asynchronous D-Bus
call mechanism that 'looks' like a synchronous call. It is important that any
global data that an asynchronous handler uses is protected as if the handler is
multi-threaded. Since many of the resources used in IPMI handlers are actually
D-Bus objects, this is not likely a common issue because of the serialization
that happens via the D-Bus calls.

Using templates, it is possible to extract the return type and argument types of
the handlers and use that to unpack (and validate) the arguments from the
incoming request and then pack the result back into a vector of bytes to send
back to the caller. The deserializer will keep track of the number of bits it
has unpacked and then compare it with the total number of bits that the method
is requesting. In the example, we are assuming that the non-full-byte integers
are packed bits in the message in most-significant-bit first order (same order
the specification describes them in). Optional arguments can be used easily with
C++17's std::optional.

Some types that are supported are as follows:

- standard integer types (uint8_t, uint16_t, uint32_t, int, long, etc.)
- bool (extracts a single bit, same as uint1_t)
- multi-precision integers (uint<N>)
- std::bitset<N>
- std::optional<T> - extracts T if there are enough remaining bits/bytes
- std::array<T, N> - extracts N elements of T if possible
- std::vector<T> - extracts elements of T from the remaining bytes
- any additional types can be created by making a pack/unpack template

For partial byte types, the least-significant bits of the next full byte are
extracted first. While this is opposite of the order the IPMI specification is
written, it makes the code simple. Multi-byte fields are extracted as LSByte
first (little-endian) to match the IPMI specification.

When returning partial byte types, they are also packed into the reply in the
least-significant bit first order. As each byte fills up, the full byte gets
pushed onto the byte stream. For examples of how the packing and unpacking is
used, see the unit test cases in test/message/pack.cpp and
test/message/unpack.cpp.

As an example this is how a bitset is unpacked

```cpp
/** @brief Specialization of UnpackSingle for std::bitset<N>
 */
template <size_t N>
struct UnpackSingle<std::bitset<N>>
{
    static int op(Payload& p, std::bitset<N>& t)
    {
        static_assert(N <= (details::bitStreamSize - CHAR_BIT));
        size_t count = N;
        // acquire enough bits in the stream to fulfill the Payload
        if (p.fillBits(count))
        {
            return -1;
        }
        fixed_uint_t<details::bitStreamSize> bitmask = ((1 << count) - 1);
        t |= (p.bitStream & bitmask).convert_to<unsigned long long>();
        p.bitStream >>= count;
        p.bitCount -= count;
        return 0;
    }
};
```

If a handler needs to unpack a variable payload, it is possible to request the
Payload parameter. When the Payload parameter is present, any remaining,
unpacked bytes are available for inspection. Using the same unpacking calls that
were used to unpack the other parameters, the remaining parameters can be
manually unpacked. This is helpful for multi-function commands like Set LAN
Configuration Parameters, where the first two bytes are fixed, but the remaining
3:N bytes vary based on the selected parameter.