1# Management Component Transport Protocol (MCTP) LPC Transport Binding Specification for ASPEED BMC Systems 2 3## Scope 4 5This design provides an efficient method to transfer MCTP packets between the 6host and BMC over the LPC bus on ASPEED BMC platforms. 7 8## Normative References 9 10The following referenced documents are indispensable for the application of 11this document. 12 13DMTF DSP0236, Management Component Transport Protocol (MCTP) Base Specification 141.0, 15http://www.dmtf.org/standards/published_documents/DSP0236_1.0.pdf 16 17Intel (R) Low Pin Count (LPC) Interface Specification 1.1, 18https://www.intel.com/content/dam/www/program/design/us/en/documents/low-pin-count-interface-specification.pdf 19 20IPMI Consortium, Intelligent Platform Management Interface Specification, v1.5 21Revision 1.1 February 20, 2002, 22http://download.intel.com/design/servers/ipmi/IPMIv1_5rev1_1.pdf 23 24## Terms and Definitions 25 26### Baseline Transmission Unit (BTU) 27 28Defined by the MCTP base specification as the smallest maximum packet size all 29MCTP-compliant endpoints must accept. 30 31### Keyboard Controller Style Interface (KCS) 32 33A set of bit definitions, and operation of the registers typically used in 34keyboard microcontrollers and embedded controllers. The term "Keyboard 35Controller Style" reflects that the register definition was originally used as 36the legacy "8742" keyboard controller interface in PC architecture computer 37systems. This interface is available built-in to several commercially 38available microcontrollers. Data is transferred across the KCS interface using 39a per-byte handshake. 40 41### Low Pin Count (LPC) 42 43A bus specification that implements ISA bus in a reduced physical form while 44extending ISA's capabilities. 45 46### LPC Firmware Cycles (LPC FW) 47 48LPC firmware cycles allow separate boot BIOS firmware memory cycles and 49application memory cycles with respect to the LPC bus. The ASPEED BMCs allow 50remapping of the LPC firmware cycles onto arbitrary regions of the BMC's 51physical address space, including RAM. 52 53### Maximum Transmission Unit (MTU) 54 55The largest payload the link will accept for a packet. The Maximum Transmission 56Unit represents a value that is at least as large as the BTU. Negotiation of 57MTU values larger than the BTU may improve throughput for data-intensive 58transfers. 59 60## Conventions 61 62Where unspecified, state, command and sequence descriptions apply to all 63versions of the protocol unless marked otherwise. 64 65## MCTP over LPC Transport 66 67### Binding Background and Evolution 68 69Version 1 of the binding provides a simple specification for duplex exchange of 70a single packet. The motivation for the design was to demonstrate the potential 71of the binding concept while keeping the design simple. 72 73Version 2 of the binding builds on v1, increasing throughput by providing a 74negotiation strategy for the BMC and the host to agree on a maximum 75transmission unit larger than the baseline transmission unit defined by the 76MCTP base specification. 77 78### Concepts 79 80The basic components used for the transfer are: 81 82* A window of the LPC FW address space, where reads and writes are forwarded to 83 BMC memory, using the LPC2AHB hardware 84* An interrupt mechanism using the IPMI KCS interface 85 86In order to transfer a packet, either side of the channel (BMC or host) will: 87 881. Write the packet to the LPC FW window 89 * The BMC will perform writes by writing to the memory backing the LPC 90 window 91 * The host will perform writes by writing to the LPC bus, at predefined 92 addresses 932. Trigger an interrupt on the remote side, by writing to the KCS data buffer 94 95On this indication, the remote side will: 96 971. Read from the KCS status register, which shows that the single-byte KCS data 98 buffer is full 992. Read the MCTP packet from the LPC FW window 1003. Read from the KCS buffer, to clear the 'buffer full' state. 101 102### Scope 103 104The document limits itself to describing the operation of the binding protocol. 105The following issues of protocol ABI are considered out of scope: 106 1071. The LPC IO address and Serial IRQ parameters of the KCS device 1082. The concrete location of the control region in the LPC FW address space 109 110### LPC FW Window Layout 111 112The window of BMC-memory-backed LPC FW address space has a predefined format, 113consisting of: 114 115* A control descriptor, describing static data about the rest of the window 116* A receive area for BMC-to-host packets 117* A transmit area, for host-to-BMC packets 118 119The control descriptor contains a version, and offset and size data for the 120transmit and receive areas. These offsets are relative to the start of the LPC 121FW window. 122 123Full definition of the control area is defined below, and it will be the base 124for all future versions. 125 126``` 127struct mctp_lpcmap_hdr { 128 uint32_t magic; 129 130 uint16_t bmc_ver_min; 131 uint16_t bmc_ver_cur; 132 uint16_t host_ver_min; 133 uint16_t host_ver_cur; 134 uint16_t negotiated_ver; 135 uint16_t pad0; 136 137 uint32_t rx_offset; 138 uint32_t rx_size; 139 uint32_t tx_offset; 140 uint32_t tx_size; 141} __attribute__((packed)); 142``` 143 144Where the magic value marking the beginning of the control area is the ASCII 145encoding of "MCTP": 146 147``` 148#define LPC_MAGIC 0x4d435450 149``` 150 151The transmit and receive areas contain a length field, followed by the actual 152MCTP packet to be transferred. At version 1, only a single MCTP packet is 153present in the Rx and Tx areas. This may change for future versions of the 154protocol. 155 156All control data is in big-endian format. MCTP packet data is transferred 157exactly as is presented, and no data escaping is performed. 158 159#### Negotiation of the Maximum Transmission Unit 160 161Version 1 of the protocol offers no mechanism for negotiation of the maximum 162transmission unit. The Rx and Tx buffers must be sized to accommodate packets 163up to the Baseline Transmission Unit, and the implementation assumes that the 164MTU is set to the BTU regardless of the values of `rx_size` and `tx_size`. 165 166Version 2 of the protocol exploits the `rx_size` and `tx_size` fields in the 167control region to negotiate the link MTU. Note that at the time that the MTU is 168under negotiation the protocol version has not been finalised, so the process 169is necessarily backwards-compatible. 170 171The relevant property that each endpoint must control is the MTU of packets it 172will receive, as this governs how the remote endpoint's packetisation impacts 173memory pressure at the local endpoint. As such while the BMC MUST populate 174`rx_size` for backwards compatibility with version 1, the host MAY write 175`rx_size` without regard for its current value if the host supports version 2. 176The BMC controls the value of `tx_size`, and MAY choose to adjust it in 177response to the host's proposed `rx_size` value. As such, when `Channel Active` 178is set by the BMC, the host MUST read both `rx_size` and `tx_size` in response 179to ensure both the BMC and the host have a consistent understanding of the MTU 180in each direction. It is convention for `rx_size` and `tx_size` to be set to 181the same value by the BMC as part of finalising the channel, though it is not 182invalid to have asymmetric MTUs. 183 184For all protocol versions, the following properties must be upheld for the Rx 185and Tx buffers to be considered valid: 186 187* Intersect neither eachother nor the control region 188* Not extend beyond the window allocated to MCTP in the LPC FW address space 189* Must accommodate at least BTU-sized payloads 190 191The BMC MAY choose to fail channel initialisation if these properties are 192violated in the negotiation process. 193 194### KCS Control 195 196The KCS hardware on the ASPEED BMCs is used as a method of indicating, to the 197remote side, that a packet is ready to be transferred through the LPC FW 198mapping. 199 200The KCS hardware consists of two single-byte buffers: the Output Data Register 201(ODR) and the Input Data Register (IDR). The ODR is written by the BMC and read 202by the host. The IDR is the obverse. 203 204The KCS unit also contains a status register, allowing both host and BMC to 205determine if there is data in the ODR or IDR. These are single-bit flags, 206designated Input/Output Buffer Full (IBF/OBF), and are automatically set by 207hardware when data has been written to the corresponding ODR/IDR buffer (and 208cleared when data has been read). 209 210While the IBF and OBF flags are managed in hardware, the remaining 211software-defined bits in the status register are used to carry other required 212protocol state. A problematic feature of the KCS status register is described 213in the IPMI specification, which states that an interrupt may be triggered on 214writes to the KCS status register but hardware implementations are not required 215to do so. Comparatively, writes to the data registers must set the 216corresponding buffer-full flag and invoke an interrupt. 217 218To ensure interrupts are generated for status updates, we exploit the OBF 219interrupt to signal a status update by writing a dummy command to ODR after 220updating the status register, as outlined below. 221 222 223#### KCS Status Register Layout 224 225| Bit | Managed By | Description | 226|-----|------------|-------------| 227| 7 | Software | (MSB) BMC Active | 228| 6 | Software | Channel active, version negotiated | 229| 5 | Software | Unused | 230| 4 | Software | Unused | 231| 3 | Hardware | Command / Data | 232| 2 | Software | Unused | 233| 1 | Hardware | Input Buffer Full | 234| 0 | Hardware | (LSB) Output Buffer Full | 235 236#### KCS Data Register Commands 237 238| Command | Description | 239|---------|-------------| 240| 0x00 | Initialise | 241| 0x01 | Tx Begin | 242| 0x02 | Rx Complete | 243| 0xff | Dummy Value | 244 245### KCS Status and Control Sequences 246 247The KCS status flags and command set govern the state of the protocol, defining 248the ability to send and receive packets on the LPC bus. 249 250#### Host Command to BMC Sequence 251 252The host sends commands to the BMC to signal channel initialisation, begin 253transmission of a packet, or to complete reception of a packet. 254 255| Step | Description | 256|------|---------------------------------------------------------| 257| 1 | The host writes a command value to IDR | 258| 2 | The hardware sets IBF, which triggers a BMC interrupt | 259| 3 | The BMC reads the status register for IBF | 260| 4 | If IBF is set, the BMC reads the host command from IDR | 261| 5 | The interrupt is acknowledged by the data register read | 262 263#### BMC Command to Host Sequence 264 265The BMC sends commands to the host to begin transmission of a packet or to 266complete reception of a packet. 267 268| Step | Description | 269|------|---------------------------------------------------------| 270| 1 | The BMC writes a command value to ODR | 271| 2 | The hardware sets OBF, which triggers a host interrupt | 272| 3 | The host reads the status register for OBF | 273| 4 | If OBF is set, the host reads the BMC command from ODR | 274| 5 | The interrupt is acknowledged by the data register read | 275 276#### BMC Status Update Sequence 277 278The BMC sends status updates to the host to signal loss of function, loss of 279channel state, or the presence of a command in the KCS data register. 280 281| Step | Description | 282|------|----------------------------------------------------------------| 283| 1 | The BMC writes the status value to the status register | 284| 2 | The BMC writes the dummy command to ODR | 285| 3 | The hardware sets OBF, which triggers a host interrupt | 286| 4 | If OBF is set, the host reads the BMC command from ODR | 287| 5 | The interrupt is acknowledged by the data register read | 288| 6 | The host observes the command is the dummy command | 289| 7 | The host reads the status register to capture the state change | 290 291#### LPC Window Ownership and Synchronisation 292 293Because the LPC FW window is shared between the host and the BMC we need 294strict rules on which entity is allowed to access it at specific times. 295 296Firstly, we have rules for modification: 297 298* The control data is only written during initialisation. The control area 299 is never modified once the channel is active. 300* Only the BMC may write to the Rx buffer described in the control area 301* Only the host may write to the Tx buffer described in the control area 302 303During packet transmission, the follow sequence occurs: 304 305* The Tx side writes the packet to its Tx buffer 306* The Tx side sends a `Tx Begin` message, indicating that the buffer ownership 307 is transferred 308* The Rx side now owns the buffer, and reads the message from its Rx area 309* The Rx side sends a `Rx Complete` once done, indicating that the buffer 310 ownership is transferred back to the Tx side. 311 312### LPC Binding Operation 313 314The binding operation is not symmetric as the BMC is the only side that can 315drive the status register. Each side's initialisation sequence is outlined 316below. 317 318The sequences below contain steps where the BMC updates the channel status and 319where commands are sent between the BMC and the host. The act of updating 320status or sending a command invokes the behaviour outlined in [KCS 321Control](#kcs-control). 322 323The packet transmission sequences assume that `BMC Active` and `Channel Active` 324are set. 325 326#### BMC Initialisation Sequence 327 328| Step | Description | 329|------|------------------------------------------| 330| 1 | The BMC initialises the control area: magic value, BMC versions and buffer parameters | 331| 2 | The BMC sets the status to `BMC Active` | 332 333#### Host Initialisation Sequence 334 335| Step | v1 | v2 | Description | 336|------|----|----|------------------------------------------------| 337| 1 | ✓ | ✓ | The host waits for the `BMC Active` state | 338| 2 | ✓ | ✓ | The host populates the its version fields | 339| 3 | | ✓ | The host derives and writes to `rx_size` the packet size associated with its desired MTU | 340| 4 | ✓ | ✓ | The host sends the `Initialise` command | 341| 5 | ✓ | ✓ | The BMC observes the `Initialise` command | 342| 6 | ✓ | ✓ | The BMC calculates and writes `negotiated_ver` | 343| 7 | | ✓ | The BMC calculates the MTUs and updates neither, one or both of `rx_size` and `tx_size` | 344| 8 | ✓ | ✓ | The BMC sets the status to `Channel Active` | 345| 9 | ✓ | ✓ | The host observes that `Channel Active` is set | 346| 10 | ✓ | ✓ | The host reads the negotiated version | 347| 11 | | ✓ | The host reads both `rx_size` and `tx_size` to derive the negotiated MTUs | 348 349#### Host Packet Transmission Sequence 350 351| Step | Description | 352|------|--------------------------------------------------------------| 353| 1 | The host waits on any previous `Rx Complete` message | 354| 3 | The host writes the packet to its Tx area (BMC Rx area) | 355| 4 | The host sends the `Tx Begin` command, transferring ownership of its Tx buffer to the BMC | 356| 5 | The BMC observes the `Tx Begin` command | 357| 6 | The BMC reads the packet from the its Rx area (host Tx area) | 358| 7 | The BMC sends the `Rx Complete` command, transferring ownership of its Rx buffer to the host | 359| 8 | The host observes the `Rx Complete` command | 360 361#### BMC Packet Transmission Sequence 362 363| Step | Description | 364|------|---------------------------------------------------------------| 365| 1 | The BMC waits on any previous `Rx Complete` message | 366| 2 | The BMC writes the packet to its Tx area (host Rx area) | 367| 3 | The BMC sends the `Tx Begin` command, transferring ownership of its Tx buffer to the host | 368| 8 | The host observes the `Tx Begin` command | 369| 9 | The host reads the packet from the host Rx area (BMC Tx area) | 370| 10 | The host sends the `Rx Complete` command, transferring ownership of its Rx buffer to the BMC | 371| 15 | The BMC observes the `Rx Complete` command | 372 373## Implementation Notes 374 375On the BMC the initial prototype implementation makes use of the following 376components: 377 378* An LPC KCS device exposed by a [binding-specific kernel driver][mctp-driver] 379* The reserved memory mapped by the LPC2AHB bridge via the [aspeed-lpc-ctrl 380 driver][aspeed-lpc-ctrl] 381* The astlpc binding found in [libmctp][libmctp] 382 383[mctp-driver]: https://github.com/openbmc/linux/commit/9a3b539a175cf4fe1f8fc2997e8a91abec25c37f 384[aspeed-lpc-ctrl]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/soc/aspeed/aspeed-lpc-ctrl.c?h=v5.7 385[libmctp]: https://github.com/openbmc/libmctp 386 387From the host side, the LPC Firmware and KCS IO cycles are driven by 388free-standing firmware. Some firmwares exploit [libmctp][libmctp] by 389implementing the driver hooks for direct access to the LPC devices. 390 391## Alternatives Considered 392 393### The KCS MCTP Binding (DSP0254) 394 395The KCS hardware (used as the full transfer channel) can be used to transfer 396arbitrarily-sized MCTP messages. However, there are much larger overheads in 397synchronisation between host and BMC for every byte transferred. 398 399### The MCTP Serial Binding (DSP0253) 400 401We could use the VUART hardware to transfer the MCTP packets according to the 402existing MCTP Serial Binding. However, the VUART device is already used for 403console data. Multiplexing both MCTP and console would be an alternative, but 404the complexity introduced would make low-level debugging both more difficult 405and less reliable. 406 407### The BT interface 408 409The BT interface allows for block-at-time transfers. However, the BT buffer 410size is only 64 bytes on the AST2500 hardware, which does not allow us to 411comply with the MCTP Base Specification (DSP0236) that requires a 64-byte 412payload size as the minimum. The 64-byte BT buffer does not allow for MCTP and 413transport headers. 414 415Additionally, we would like to develop the MCTP channel alongside the existing 416IPMI interfaces, to allow a gradual transition from IPMI to MCTP. As the BT 417channel is already used on OpenPOWER systems for IPMI transfers, we would not 418be able to support both in parallel. 419 420### Using the AST2500 LPC Mailbox 421 422This would require enabling the SuperIO interface, which allows the host to 423access the entire BMC address space, and so introduces security 424vulnerabilities. 425