1# Management Component Transport Protocol (MCTP) LPC Transport Binding Specification for ASPEED BMC Systems 2 3## Scope 4 5This design provides an efficient method to transfer MCTP packets between the 6host and BMC over the LPC bus on ASPEED BMC platforms. 7 8## References 9 10The following referenced documents are indispensable for the application of this 11document. 12 131. DMTF DSP0236, Management Component Transport Protocol (MCTP) Base 14 Specification 1.0, 15 <http://www.dmtf.org/standards/published_documents/DSP0236_1.0.pdf> 16 172. Intel (R) Low Pin Count (LPC) Interface Specification 1.1, 18 <https://www.intel.com/content/dam/www/program/design/us/en/documents/low-pin-count-interface-specification.pdf> 19 203. IPMI Consortium, Intelligent Platform Management Interface Specification, 21 v1.5 Revision 1.1 February 20, 2002, 22 <http://download.intel.com/design/servers/ipmi/IPMIv1_5rev1_1.pdf> 23 24## Definitions 25 26### BTU: Baseline Transmission Unit 27 28Defined by the MCTP base specification as the smallest maximum packet size all 29MCTP-compliant endpoints must accept. 30 31### IBF: Input Buffer Full 32 33A hardware-defined flag bit in a KCS device's Status Register (STR). The IBF 34flag indicates that a value has been written by the host to the corresponding 35Input Data Register (IDR). 36 37### IDR: Input Data Register 38 39One of the three register interfaces exposed by a KCS device. The IDR is a one 40byte buffer which is written by the host and read by the BMC. 41 42### KCS: Keyboard-Controller-Style 43 44A set of bit definitions and operation of the registers typically used in 45keyboard microcontrollers and embedded controllers. The term "Keyboard 46Controller Style" reflects that the register definition was originally used as 47the legacy "8742" keyboard controller interface in PC architecture computer 48systems. This interface is available built-in to several commercially available 49microcontrollers. Data is transferred across the KCS interface using a per-byte 50handshake. 51 52### LPC Bus: Low Pin Count Bus 53 54A bus specification that implements ISA bus in a reduced physical form while 55extending ISA's capabilities. 56 57### LPC FW: LPC Firmware Cycles 58 59LPC firmware cycles allow separate boot BIOS firmware memory cycles and 60application memory cycles with respect to the LPC bus. The ASPEED BMCs allow 61remapping of the LPC firmware cycles onto arbitrary regions of the BMC's 62physical address space, including RAM. 63 64### MTU: Maximum Transmission Unit 65 66The largest payload the link will accept for a packet. The Maximum Transmission 67Unit represents a value that is at least as large as the BTU. Negotiation of MTU 68values larger than the BTU may improve throughput for data-intensive transfers. 69 70### OBF: Output Buffer Full 71 72A hardware-defined flag bit in a KCS device's Status Register (STR). The OBF 73flag indicates that a value has been written by the BMC to the corresponding 74Output Data Register (ODR). 75 76### ODR: Output Data Register 77 78One of the three register interfaces exposed by a KCS device. The ODR is a one 79byte buffer which is written by the BMC and read by the host. 80 81### STR: Status Register 82 83One of the three register interfaces exposed by a KCS device. STR is a 84BMC-controlled, eight-bit register exposed to both the BMC and the host for 85indication of IBF and OBF events on the input (IDR) and output (ODR) buffers. 86Bits that are not defined by hardware can be software-controlled in a manner 87defined by a platform-specific ABI. 88 89## Conventions 90 91Where unspecified, state, command and sequence descriptions apply to all 92versions of the protocol unless marked otherwise. 93 94## MCTP over LPC Transport 95 96### Concepts 97 98The basic components used for the transfer are: 99 100- An interrupt mechanism using the IPMI KCS interface 101- A window of the LPC FW address space, where reads and writes are forwarded to 102 BMC memory, using the LPC2AHB hardware 103 104In order to transfer a packet, either side of the channel (BMC or host) will: 105 1061. Write the packet to the LPC FW window 107 - The BMC will perform writes by writing to the memory backing the LPC window 108 - The host will perform writes by writing to the LPC bus, at predefined 109 addresses 1102. Trigger an interrupt on the remote side, by writing to the KCS data buffer 111 112On this indication, the remote side will: 113 1141. Read from the KCS status register, which shows that the single-byte KCS data 115 buffer is full 1162. Read the provided command from the KCS data buffer, acknowledging the 117 interrupt 1183. Read the MCTP packet from the LPC FW window 119 120### Scope 121 122The document limits itself to describing the operation of the binding protocol. 123The following issues of protocol ABI are considered out of scope: 124 1251. The LPC IO address and Serial IRQ parameters of the KCS device 1262. The concrete location of the control region in the LPC FW address space 127 128### KCS Interface 129 130The KCS hardware on the ASPEED BMCs is used as a method of indicating, to the 131remote side, that a packet is ready to be transferred through the LPC FW 132mapping. 133 134The KCS hardware consists of two single-byte buffers: the Output Data Register 135(ODR) and the Input Data Register (IDR). The ODR is written by the BMC and read 136by the host. The IDR is the obverse. 137 138The KCS unit also contains a status register (STR), allowing both host and BMC 139to determine if there is data in the ODR or IDR. These are single-bit flags, 140designated Input/Output Buffer Full (IBF/OBF), and are automatically set by 141hardware when data has been written to the corresponding ODR/IDR buffer (and 142cleared when data has been read). 143 144While the IBF and OBF flags are managed in hardware, the remaining 145software-defined bits in the status register are used to carry other required 146protocol state. A problematic feature of the KCS status register is described in 147the IPMI specification, which states that an interrupt may be triggered on 148writes to the KCS status register but hardware implementations are not required 149to do so. Comparatively, writes to the data registers must set the corresponding 150buffer-full flag and invoke an interrupt. 151 152To ensure interrupts are generated for status updates, we exploit the OBF 153interrupt to signal a status update by writing a dummy command to ODR after 154updating the status register, as outlined below. 155 156### LPC FW Window 157 158The window of BMC-memory-backed LPC FW address space has a predefined format, 159consisting of: 160 161- A control descriptor, describing static data about the rest of the window 162- A receive area for BMC-to-host packets 163- A transmit area, for host-to-BMC packets 164 165The control descriptor contains a version, and offset and size data for the 166transmit and receive areas. These offsets are relative to the start of the LPC 167FW window. 168 169Full definition of the control area is defined below, and it will be the base 170for all future versions. 171 172```c 173struct mctp_lpcmap_hdr { 174 uint32_t magic; 175 176 uint16_t bmc_ver_min; 177 uint16_t bmc_ver_cur; 178 uint16_t host_ver_min; 179 uint16_t host_ver_cur; 180 uint16_t negotiated_ver; 181 uint16_t pad0; 182 183 uint32_t rx_offset; 184 uint32_t rx_size; 185 uint32_t tx_offset; 186 uint32_t tx_size; 187} __attribute__((packed)); 188``` 189 190The magic value marking the beginning of the control area is the ASCII encoding 191of "MCTP": 192 193```c 194#define LPC_MAGIC 0x4d435450 195``` 196 197All medium-specific metadata is in big-endian format. This includes: 198 1991. Control area data 2002. Medium-specific packet header fields 2013. Medium-specific packet trailer fields 202 203MCTP packet data is transferred exactly as is presented, and no data escaping is 204performed. 205 206In all versions of the protocol, the transmit and receive areas contain a 207medium-specific header comprising a 32-bit payload length field, followed 208immediately by the MCTP packet data to be transferred. The full MCTP packet, 209including MCTP header, is considered to be the payload for the purpose of the 210header's length field. 211 212```c 213struct mctp_lpcbuf_hdr { 214 uint32_t length; 215} __attribute__((packed)); 216``` 217 218A medium-specific packet trailer must immediately follow the payload. The length 219of the trailer is not accounted for in the length field of the medium-specific 220packet header: The length of the trailer is implied by the negotiated protocol 221version. 222 223For protocol versions 1 and 2, the medium-specific trailer length is zero. 224 225For protocol version 3, the medium-specific trailer comprises a CRC-32 checksum 226of the payload. 227 228```c 229struct mctp_lpcbuf_tlr { 230 uint32_t crc32; 231} __attribute__((packed)); 232``` 233 234Where the CRC-32 implementation is defined by the following characteristics (or 235equivalent): 236 2371. The polynomial 238 `x^32 + x^26 + x^23 + x^22 + x^16 + x^12 + x^11 + x^10 + x^8 + x^7 + x^5 + x^4 + x^2 + x + 1` 2392. Initialising the remainder state to `2^32 - 1` 2403. Incrementally shifting and `XOR`ing data bytes through the reversed 241 polynomial representation `0xEDB88320` 2424. `XOR`ing the calculated remainder with `2^32 - 1` 243 244For all defined versions, only a single MCTP packet is present in the Rx and Tx 245areas. This may change for future versions of the protocol. 246 247#### Negotiation of the Maximum Transmission Unit 248 249Version 1 of the protocol offers no mechanism for negotiation of the maximum 250transmission unit. The Rx and Tx buffers must be sized to accommodate packets up 251to the Baseline Transmission Unit, and the implementation assumes that the MTU 252is set to the BTU regardless of the values of `rx_size` and `tx_size`. 253 254Version 2 of the protocol exploits the `rx_size` and `tx_size` fields in the 255control region to negotiate the link MTU. Note that at the time that the MTU is 256under negotiation the protocol version has not been finalised, so the process is 257necessarily backwards-compatible. 258 259The relevant property that each endpoint must control is the MTU of packets it 260will receive, as this governs how the remote endpoint's packetisation impacts 261memory pressure at the local endpoint. As such while the BMC MUST populate 262`rx_size` for backwards compatibility with version 1, the host MAY write 263`rx_size` without regard for its current value if the host supports version 2. 264The BMC controls the value of `tx_size`, and MAY choose to adjust it in response 265to the host's proposed `rx_size` value. As such, when `Channel Active` is set by 266the BMC, the host MUST read both `rx_size` and `tx_size` in response to ensure 267both the BMC and the host have a consistent understanding of the MTU in each 268direction. It is convention for `rx_size` and `tx_size` to be set to the same 269value by the BMC as part of finalising the channel, though it is not invalid to 270have asymmetric MTUs. 271 272For all protocol versions, the following properties must be upheld for the Rx 273and Tx buffers to be considered valid: 274 275- Intersect neither eachother nor the control region 276- Not extend beyond the window allocated to MCTP in the LPC FW address space 277- Must accommodate at least BTU-sized payloads 278 279The BMC MAY choose to fail channel initialisation if these properties are 280violated in the negotiation process. 281 282### KCS Status and Control Sequences 283 284The KCS status flags and command set govern the state of the protocol, defining 285the ability to send and receive packets on the LPC bus. 286 287#### KCS Status Register Layout 288 289| Bit | Managed By | Description | 290| ------- | ---------- | ---------------------------------- | 291| 7 (MSB) | Software | BMC Active | 292| 6 | Software | Channel active, version negotiated | 293| 5 | Software | Unused | 294| 4 | Software | Unused | 295| 3 | Hardware | Command / Data | 296| 2 | Software | Unused | 297| 1 | Hardware | Input Buffer Full | 298| 0 (LSB) | Hardware | Output Buffer Full | 299 300#### KCS Data Register Commands 301 302| Command | Description | 303| ------- | ----------- | 304| 0x00 | Initialise | 305| 0x01 | Tx Begin | 306| 0x02 | Rx Complete | 307| 0xff | Dummy Value | 308 309#### Host Command to BMC Sequence 310 311The host sends commands to the BMC to signal channel initialisation, begin 312transmission of a packet, or to complete reception of a packet. 313 314| Step | Description | 315| ---- | ------------------------------------------------------- | 316| 1 | The host writes a command value to IDR | 317| 2 | The hardware sets IBF, which triggers a BMC interrupt | 318| 3 | The BMC reads the status register for IBF | 319| 4 | If IBF is set, the BMC reads the host command from IDR | 320| 5 | The interrupt is acknowledged by the data register read | 321 322#### BMC Command to Host Sequence 323 324The BMC sends commands to the host to begin transmission of a packet or to 325complete reception of a packet. 326 327| Step | Description | 328| ---- | ------------------------------------------------------- | 329| 1 | The BMC writes a command value to ODR | 330| 2 | The hardware sets OBF, which triggers a host interrupt | 331| 3 | The host reads the status register for OBF | 332| 4 | If OBF is set, the host reads the BMC command from ODR | 333| 5 | The interrupt is acknowledged by the data register read | 334 335#### BMC Status Update Sequence 336 337The BMC sends status updates to the host to signal loss of function, loss of 338channel state, or the presence of a command in the KCS data register. 339 340| Step | Description | 341| ---- | -------------------------------------------------------------- | 342| 1 | The BMC writes the status value to the status register | 343| 2 | The BMC writes the dummy command to ODR | 344| 3 | The hardware sets OBF, which triggers a host interrupt | 345| 4 | If OBF is set, the host reads the BMC command from ODR | 346| 5 | The interrupt is acknowledged by the data register read | 347| 6 | The host observes the command is the dummy command | 348| 7 | The host reads the status register to capture the state change | 349 350#### LPC Window Ownership and Synchronisation 351 352Because the LPC FW window is shared between the host and the BMC we need strict 353rules on which entity is allowed to access it at specific times. 354 355Firstly, we have rules for modification: 356 357- The control data is only written during initialisation. The control area is 358 never modified once the channel is active. 359- Only the BMC may write to the Rx buffer described in the control area 360- Only the host may write to the Tx buffer described in the control area 361 362During packet transmission, the follow sequence occurs: 363 3641. The Tx side writes the packet to its Tx buffer 3652. The Tx side sends a `Tx Begin` message, indicating that the buffer ownership 366 is transferred 3673. The Rx side now owns the buffer, and reads the message from its Rx area 3684. The Rx side sends a `Rx Complete` once done, indicating that the buffer 369 ownership is transferred back to the Tx side. 370 371### LPC Binding Operation 372 373The binding operation is not symmetric as the BMC is the only side that can 374drive the status register. Each side's initialisation sequence is outlined 375below. 376 377The sequences below contain steps where the BMC updates the channel status and 378where commands are sent between the BMC and the host. The act of updating status 379or sending a command invokes the behaviour outlined in 380[KCS Control](#kcs-control). 381 382The packet transmission sequences assume that `BMC Active` and `Channel Active` 383are set. 384 385#### BMC Initialisation Sequence 386 387| Step | Description | 388| ---- | ------------------------------------------------------------------------------------- | 389| 1 | The BMC initialises the control area: magic value, BMC versions and buffer parameters | 390| 2 | The BMC sets the status to `BMC Active` | 391 392#### Host Initialisation Sequence 393 394| Step | v1 | v2 | v3 | Description | 395| ---- | --- | --- | --- | ---------------------------------------------------------------------------------------- | 396| 1 | ✓ | ✓ | ✓ | The host waits for the `BMC Active` state | 397| 2 | ✓ | ✓ | ✓ | The host populates the its version fields | 398| 3 | | ✓ | ✓ | The host derives and writes to `rx_size` the packet size associated with its desired MTU | 399| 4 | ✓ | ✓ | ✓ | The host sends the `Initialise` command | 400| 5 | ✓ | ✓ | ✓ | The BMC observes the `Initialise` command | 401| 6 | ✓ | ✓ | ✓ | The BMC calculates and writes `negotiated_ver` | 402| 7 | | ✓ | ✓ | The BMC calculates the MTUs and updates neither, one or both of `rx_size` and `tx_size` | 403| 8 | ✓ | ✓ | ✓ | The BMC sets the status to `Channel Active` | 404| 9 | ✓ | ✓ | ✓ | The host observes that `Channel Active` is set | 405| 10 | ✓ | ✓ | ✓ | The host reads the negotiated version | 406| 11 | | ✓ | ✓ | The host reads both `rx_size` and `tx_size` to derive the negotiated MTUs | 407 408#### Host Packet Transmission Sequence 409 410| Step | v1 | v2 | v3 | Description | 411| ---- | --- | --- | --- | ---------------------------------------------------------------------------------------------- | 412| 1 | | | ✓ | The host calculates the CRC-32 over the packet data | 413| 2 | ✓ | ✓ | ✓ | The host waits on any previous `Rx Complete` message | 414| 3 | ✓ | ✓ | ✓ | The host writes the packet data and medium-specific metadata to its Tx area (BMC Rx area) | 415| 4 | ✓ | ✓ | ✓ | The host sends the `Tx Begin` command, transferring ownership of its Tx buffer to the BMC | 416| 5 | ✓ | ✓ | ✓ | The BMC observes the `Tx Begin` command | 417| 6 | ✓ | ✓ | ✓ | The BMC reads the packet data and medium-specific metadata from the its Rx area (host Tx area) | 418| 7 | ✓ | ✓ | ✓ | The BMC sends the `Rx Complete` command, transferring ownership of its Rx buffer to the host | 419| 8 | ✓ | ✓ | ✓ | The host observes the `Rx Complete` command | 420| 9 | | | ✓ | The BMC validates the provided CRC-32 over the packet data | 421 422#### BMC Packet Transmission Sequence 423 424| Step | v1 | v2 | v3 | Description | 425| ---- | --- | --- | --- | ----------------------------------------------------------------------------------------------- | 426| 1 | | | ✓ | The BMC calculates the CRC-32 over the packet data | 427| 2 | ✓ | ✓ | ✓ | The BMC waits on any previous `Rx Complete` message | 428| 3 | ✓ | ✓ | ✓ | The BMC writes the packet data and medium-specific metadata to its Tx area (host Rx area) | 429| 4 | ✓ | ✓ | ✓ | The BMC sends the `Tx Begin` command, transferring ownership of its Tx buffer to the host | 430| 5 | ✓ | ✓ | ✓ | The host observes the `Tx Begin` command | 431| 6 | ✓ | ✓ | ✓ | The host reads the packet data and medium-specific metadata from the host Rx area (BMC Tx area) | 432| 7 | ✓ | ✓ | ✓ | The host sends the `Rx Complete` command, transferring ownership of its Rx buffer to the BMC | 433| 8 | ✓ | ✓ | ✓ | The BMC observes the `Rx Complete` command | 434| 9 | | | ✓ | The host validates the provided CRC-32 over the packet data | 435 436## Implementation Notes 437 438On the BMC the initial prototype implementation makes use of the following 439components: 440 441- An LPC KCS device exposed by a [binding-specific kernel driver][mctp-driver] 442- The reserved memory mapped by the LPC2AHB bridge via the [aspeed-lpc-ctrl 443 driver][aspeed-lpc-ctrl] 444- The astlpc binding found in [libmctp][libmctp] 445 446[mctp-driver]: 447 https://github.com/openbmc/linux/commit/9a3b539a175cf4fe1f8fc2997e8a91abec25c37f 448[aspeed-lpc-ctrl]: 449 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/soc/aspeed/aspeed-lpc-ctrl.c?h=v5.7 450[libmctp]: https://github.com/openbmc/libmctp 451 452From the host side, the LPC Firmware and KCS IO cycles are driven by 453free-standing firmware. Some firmwares exploit [libmctp][libmctp] by 454implementing the driver hooks for direct access to the LPC devices. 455 456## Alternatives Considered 457 458### The KCS MCTP Binding (DSP0254) 459 460The KCS hardware (used as the full transfer channel) can be used to transfer 461arbitrarily-sized MCTP messages. However, there are much larger overheads in 462synchronisation between host and BMC for every byte transferred. 463 464### The MCTP Serial Binding (DSP0253) 465 466We could use the VUART hardware to transfer the MCTP packets according to the 467existing MCTP Serial Binding. However, the VUART device is already used for 468console data. Multiplexing both MCTP and console would be an alternative, but 469the complexity introduced would make low-level debugging both more difficult and 470less reliable. 471 472### The BT interface 473 474The BT interface allows for block-at-time transfers. However, the BT buffer size 475is only 64 bytes on the AST2500 hardware, which does not allow us to comply with 476the MCTP Base Specification (DSP0236) that requires a 64-byte payload size as 477the minimum. The 64-byte BT buffer does not allow for MCTP and transport 478headers. 479 480Additionally, we would like to develop the MCTP channel alongside the existing 481IPMI interfaces, to allow a gradual transition from IPMI to MCTP. As the BT 482channel is already used on OpenPOWER systems for IPMI transfers, we would not be 483able to support both in parallel. 484 485### Using the AST2500 LPC Mailbox 486 487This would require enabling the SuperIO interface, which allows the host to 488access the entire BMC address space, and so introduces security vulnerabilities. 489