designs/mctp/mctp-kernel.md

1 # OpenBMC in-kernel MCTP
8 This document describes a kernel-based implementation of MCTP infrastructure,
9 providing a sockets-based API for MCTP communication within an OpenBMC-based
12 ## Requirements for a kernel implementation
14 - The MCTP messaging API should be an obvious application of the existing POSIX
17 - Configuration should be simple for a straightforward MCTP endpoint: a single
20 - Infrastructure should be flexible enough to allow for more complex MCTP
23   - each MCTP network (as defined by section 3.2.31 of DSP0236) may consist of
26   - multiple distinct (ie., non-bridged) networks, possibly containing
29   - multiple local EIDs on a single interface, and
31   - customisable routing/bridging configurations within a network.
37 - An interface for userspace applications to send and receive MCTP messages: A
40 - Infrastructure for control and configuration of the MCTP network(s),
44 - Kernel drivers for physical interface bindings.
50 Higher-level protocols (such as PLDM) are implemented in userspace, through the
52 Protocol implementation (DSP0236, section 11) - MCTP endpoints will typically
56 process. MCTP endpoints that require more than just single-endpoint
63 the namespace for these interfaces is separate from other network interfaces -
72 (as defined by DSP0236, section 3.2.47), likely connected to a specific hardware
73 device. This is represented as a `struct netdevice`, and has a user-visible name
74 and index (`ifindex`). Non-hardware-attached interfaces are permitted, to allow
77 A network defines a unique address space for MCTP endpoints by endpoint-ID
78 (described by DSP0236, section 3.2.31). A network has a user-visible identifier
100 the domain. Currently, only a `SOCK_DGRAM` socket type is defined.
139     /* MCTP type values. Only the least-significant 7 bits of
146     /* MCTP-spec-defined fields */
157 The following sections describe the MCTP-specific behaviours of the standard
187 `MCTP_TAG_OWNER` is set, the 3 least-significant bits of `smctp_tag` are not
189 [Tag behaviour for transmitted messages](#tag-behaviour-for-transmitted-messages)
194 incoming packets from any locally-connected network. A specific network value
202 7 bits of the type is matched on incoming messages (ie., the most-significant IC
228 The `smctp_network` field may specify a locally-attached network, or the value
230 This is guaranteed to work for single-network configurations, but may require
246 [Tag behaviour for transmitted messages](#tag-behaviour-for-transmitted-messages)
270     /* arbitrary message to send, with message-type header */
272     memcpy(buf + 1, "hello, world!", sizeof(buf) - 1);
288 buffer, and the most-significant bit of the message type byte must be 1.
306 message buffer to be specified as a scatter-gather list. At present no ancillary
308 defined.
312 destination. The (destination-eid,tag) tuple acts as an implicit local socket
364 `getpeername()` may be used to retrieve a kernel-allocated tag value.
371 The following socket options are defined for MCTP sockets:
380 This as defined as:
412 non-connected sockets when transferring messages to multiple different
423 This should be based on the reply timeout for any upper-level protocol.
426 allocation by this socket. The kernel may also reject subsequent tag-allocation
433 - the socket is sending messages with TO=1 (ie, is a requester); and
434 - messages are sent to multiple different destination EIDs from the one socket.
439 not used in `SOCK_DGRAM`-type sockets:
441 - `listen()`
442 - `accept()`
443 - `ioctl()`
444 - `shutdown()`
445 - `mmap()`
449 These examples cover three general use-cases:
451 - **requester**: sends requests to a particular (EID, type) target, and receives
456 - **responder**: receives all locally-addressed messages of a specific
457   message-type, and responds to the requester immediately.
461 - **controller**: a specific service for a bus owner; may send broadcast
463   low-level packet data.
469 "Client"-side implementation to send requests to a responder, and receive a
511             /* ensure we're nul-terminated */
512             msg.data[sizeof(msg.data)-1] = '\0';
522 "Server"-side implementation to receive requests and respond. Like the client,
563                     /* Reply to requester; this macro just clears the TO-bit.
565                      * so use those as-is.
657 #### Implementation notes
667 The `smctp_network` field specifies a locally defined network identifier. To
668 simplify situations where there is only one network defined, the special value
672 This selection is entirely user-configured; one specific network may be defined
688 forwarded if their destination EID is non-local, and matches a route for another
697 On every message sent with the tag-owner bit set ("TO" in DSP0236), the kernel
699 EID, source EID, tag-owner, tag) tuple. The tag value is 3 bits in size.
707 - the socket is closed
708 - a new message is sent to a new destination EID
709 - an implementation-defined timeout expires
716 The implementation-defined timeout value shall be chosen to reasonably cover
725 - the socket is closed
726 - the remote address is changed through another call to `connect()`.
734 particular tag value must be currently-unused across all EIDs to allow a
744 ##### MCTP Control Protocol implementation
747 implementation would exist as a userspace process, `mctpd`. This process is
755 route table via a netlink interface - the same as that implemented for the
756 [Utility and configuration interfaces](#utility-and-configuration-interfaces).
758 #### Neighbour and routing implementation
760 The packet-transmission behaviour of the MCTP infrastructure relies on a single
765 | --------- | --------- | ---------------- | ------ | --- | ----- | ------ |
769 - From userspace, via a netlink interface (see the
770   [Utility and configuration interfaces](#utility-and-configuration-interfaces)
773 - Directly within the kernel, when basic neighbour information is discovered.
774   Kernel-originated routes are marked as such in the flags field, and have a
777 Kernel-discovered routing information can originate from two sources:
779 - physical-to-EID mappings discovered through received packets
781 - explicit endpoint physical-address resolution requests
787 kernel-originated route into the routing table.
789 This is the only kernel-internal usage of MCTP Control Protocol messages.
804     mctp link set <link> network <network-id>
806     mctp link set <link> bus-owner <hwaddr>
812     mctp network create <network-id>
813     mctp network set <network-id> forwarding <on|off>
814     mctp network set <network-id> default [<true|false>]
815     mctp network delete <network-id>
828 …mctp route add net <network-id> eid <eid|eid-range> via <link> [hwaddr <addr>] [mtu <mtu>] [metric…
829 …mctp route del net <network-id> eid <eid|eid-range> via <link> [hwaddr <addr>] [mtu <mtu>] [metric…
830     mctp route show [net <network-id>]
839 A set of netlink message formats will be defined to support these control
844 ### Including message-type byte in send/receive buffers
855 However, the exact format of the MCTP message payload is not precisely defined
858 defined by the type specification, not DSP0236. The kernel would need knowledge
861 Therefore, we transfer the message payload as-is to userspace, without any
864 ### MCTP message-type specification: using `sockaddr_mctp.smctp_type` rather than protocol
866 This design specifies message-types to be passed in the `smctp_type` field of
874 The `smctp_type` implementation was chosen as it better matches the "addressing"
883 Future additions that perform protocol-specific message handling, and so alter
904 brief overview of the more-feasible of those, and why they were rejected:
911 - needlessly consumes a tag on non-tag-owning sockets (ie, those which send with
912   TO=0 - responders); and
914 - limits us to 8 sockets per network.
919 and only allocate a tag for the (short-lived) task of packetising a message, and
927 - limit the system to one socket of each message type (which, for example,
930 - forward all incoming messages of a specific message-type to all sockets
941 requests - more than one response may be valid for a given request message. For
942 example, in response to a request, a NVMe-MI implementation may send an
943 in-progress reply before the final reply. In this case, we would release the tag