1# Management Component Transport Protocol (MCTP) LPC Transport Binding Specification for ASPEED BMC Systems 2 3## Scope 4 5This design provides an efficient method to transfer MCTP packets between the 6host and BMC over the LPC bus on ASPEED BMC platforms. 7 8## References 9 10The following referenced documents are indispensable for the application of 11this document. 12 131. DMTF DSP0236, Management Component Transport Protocol (MCTP) Base 14Specification 1.0, 15http://www.dmtf.org/standards/published_documents/DSP0236_1.0.pdf 16 172. Intel (R) Low Pin Count (LPC) Interface Specification 1.1, 18https://www.intel.com/content/dam/www/program/design/us/en/documents/low-pin-count-interface-specification.pdf 19 203. IPMI Consortium, Intelligent Platform Management Interface Specification, 21v1.5 Revision 1.1 February 20, 2002, 22http://download.intel.com/design/servers/ipmi/IPMIv1_5rev1_1.pdf 23 24## Definitions 25 26**BTU: Baseline Transmission Unit** 27 28Defined by the MCTP base specification as the smallest maximum packet size all 29MCTP-compliant endpoints must accept. 30 31**IBF: Input Buffer Full** 32 33A hardware-defined flag bit in a KCS device's Status Register (STR). The IBF 34flag indicates that a value has been written by the host to the corresponding 35Input Data Register (IDR). 36 37**IDR: Input Data Register** 38 39One of the three register interfaces exposed by a KCS device. The IDR is a one 40byte buffer which is written by the host and read by the BMC. 41 42**KCS: Keyboard-Controller-Style** 43 44A set of bit definitions and operation of the registers typically used in 45keyboard microcontrollers and embedded controllers. The term "Keyboard 46Controller Style" reflects that the register definition was originally used as 47the legacy "8742" keyboard controller interface in PC architecture computer 48systems. This interface is available built-in to several commercially 49available microcontrollers. Data is transferred across the KCS interface using 50a per-byte handshake. 51 52**LPC Bus: Low Pin Count Bus** 53 54A bus specification that implements ISA bus in a reduced physical form while 55extending ISA's capabilities. 56 57**LPC FW: LPC Firmware Cycles** 58 59LPC firmware cycles allow separate boot BIOS firmware memory cycles and 60application memory cycles with respect to the LPC bus. The ASPEED BMCs allow 61remapping of the LPC firmware cycles onto arbitrary regions of the BMC's 62physical address space, including RAM. 63 64**MTU: Maximum Transmission Unit** 65 66The largest payload the link will accept for a packet. The Maximum Transmission 67Unit represents a value that is at least as large as the BTU. Negotiation of 68MTU values larger than the BTU may improve throughput for data-intensive 69transfers. 70 71**OBF: Output Buffer Full** 72 73A hardware-defined flag bit in a KCS device's Status Register (STR). The OBF 74flag indicates that a value has been written by the BMC to the corresponding 75Output Data Register (ODR). 76 77**ODR: Output Data Register** 78 79One of the three register interfaces exposed by a KCS device. The ODR is a 80one byte buffer which is written by the BMC and read by the host. 81 82**STR: Status Register** 83 84One of the three register interfaces exposed by a KCS device. STR is a 85BMC-controlled, eight-bit register exposed to both the BMC and the host for 86indication of IBF and OBF events on the input (IDR) and output (ODR) buffers. 87Bits that are not defined by hardware can be software-controlled in a manner 88defined by a platform-specific ABI. 89 90## Conventions 91 92Where unspecified, state, command and sequence descriptions apply to all 93versions of the protocol unless marked otherwise. 94 95## MCTP over LPC Transport 96 97### Concepts 98 99The basic components used for the transfer are: 100 101* An interrupt mechanism using the IPMI KCS interface 102* A window of the LPC FW address space, where reads and writes are forwarded to 103 BMC memory, using the LPC2AHB hardware 104 105In order to transfer a packet, either side of the channel (BMC or host) will: 106 1071. Write the packet to the LPC FW window 108 * The BMC will perform writes by writing to the memory backing the LPC 109 window 110 * The host will perform writes by writing to the LPC bus, at predefined 111 addresses 1122. Trigger an interrupt on the remote side, by writing to the KCS data buffer 113 114On this indication, the remote side will: 115 1161. Read from the KCS status register, which shows that the single-byte KCS data 117 buffer is full 1182. Read the provided command from the KCS data buffer, acknowledging the 119 interrupt 1203. Read the MCTP packet from the LPC FW window 121 122### Scope 123 124The document limits itself to describing the operation of the binding protocol. 125The following issues of protocol ABI are considered out of scope: 126 1271. The LPC IO address and Serial IRQ parameters of the KCS device 1282. The concrete location of the control region in the LPC FW address space 129 130### KCS Interface 131 132The KCS hardware on the ASPEED BMCs is used as a method of indicating, to the 133remote side, that a packet is ready to be transferred through the LPC FW 134mapping. 135 136The KCS hardware consists of two single-byte buffers: the Output Data Register 137(ODR) and the Input Data Register (IDR). The ODR is written by the BMC and read 138by the host. The IDR is the obverse. 139 140The KCS unit also contains a status register (STR), allowing both host and BMC 141to determine if there is data in the ODR or IDR. These are single-bit flags, 142designated Input/Output Buffer Full (IBF/OBF), and are automatically set by 143hardware when data has been written to the corresponding ODR/IDR buffer (and 144cleared when data has been read). 145 146While the IBF and OBF flags are managed in hardware, the remaining 147software-defined bits in the status register are used to carry other required 148protocol state. A problematic feature of the KCS status register is described 149in the IPMI specification, which states that an interrupt may be triggered on 150writes to the KCS status register but hardware implementations are not required 151to do so. Comparatively, writes to the data registers must set the 152corresponding buffer-full flag and invoke an interrupt. 153 154To ensure interrupts are generated for status updates, we exploit the OBF 155interrupt to signal a status update by writing a dummy command to ODR after 156updating the status register, as outlined below. 157 158### LPC FW Window 159 160The window of BMC-memory-backed LPC FW address space has a predefined format, 161consisting of: 162 163* A control descriptor, describing static data about the rest of the window 164* A receive area for BMC-to-host packets 165* A transmit area, for host-to-BMC packets 166 167The control descriptor contains a version, and offset and size data for the 168transmit and receive areas. These offsets are relative to the start of the LPC 169FW window. 170 171Full definition of the control area is defined below, and it will be the base 172for all future versions. 173 174```c 175struct mctp_lpcmap_hdr { 176 uint32_t magic; 177 178 uint16_t bmc_ver_min; 179 uint16_t bmc_ver_cur; 180 uint16_t host_ver_min; 181 uint16_t host_ver_cur; 182 uint16_t negotiated_ver; 183 uint16_t pad0; 184 185 uint32_t rx_offset; 186 uint32_t rx_size; 187 uint32_t tx_offset; 188 uint32_t tx_size; 189} __attribute__((packed)); 190``` 191 192The magic value marking the beginning of the control area is the ASCII encoding 193of "MCTP": 194 195```c 196#define LPC_MAGIC 0x4d435450 197``` 198 199All control data is in big-endian format. MCTP packet data is transferred 200exactly as is presented, and no data escaping is performed. 201 202The transmit and receive areas contain a length field, followed by the actual 203MCTP packet to be transferred. At version 1, only a single MCTP packet is 204present in the Rx and Tx areas. This may change for future versions of the 205protocol. 206 207#### Negotiation of the Maximum Transmission Unit 208 209Version 1 of the protocol offers no mechanism for negotiation of the maximum 210transmission unit. The Rx and Tx buffers must be sized to accommodate packets 211up to the Baseline Transmission Unit, and the implementation assumes that the 212MTU is set to the BTU regardless of the values of `rx_size` and `tx_size`. 213 214Version 2 of the protocol exploits the `rx_size` and `tx_size` fields in the 215control region to negotiate the link MTU. Note that at the time that the MTU is 216under negotiation the protocol version has not been finalised, so the process 217is necessarily backwards-compatible. 218 219The relevant property that each endpoint must control is the MTU of packets it 220will receive, as this governs how the remote endpoint's packetisation impacts 221memory pressure at the local endpoint. As such while the BMC MUST populate 222`rx_size` for backwards compatibility with version 1, the host MAY write 223`rx_size` without regard for its current value if the host supports version 2. 224The BMC controls the value of `tx_size`, and MAY choose to adjust it in 225response to the host's proposed `rx_size` value. As such, when `Channel Active` 226is set by the BMC, the host MUST read both `rx_size` and `tx_size` in response 227to ensure both the BMC and the host have a consistent understanding of the MTU 228in each direction. It is convention for `rx_size` and `tx_size` to be set to 229the same value by the BMC as part of finalising the channel, though it is not 230invalid to have asymmetric MTUs. 231 232For all protocol versions, the following properties must be upheld for the Rx 233and Tx buffers to be considered valid: 234 235* Intersect neither eachother nor the control region 236* Not extend beyond the window allocated to MCTP in the LPC FW address space 237* Must accommodate at least BTU-sized payloads 238 239The BMC MAY choose to fail channel initialisation if these properties are 240violated in the negotiation process. 241 242### KCS Status and Control Sequences 243 244The KCS status flags and command set govern the state of the protocol, defining 245the ability to send and receive packets on the LPC bus. 246 247#### KCS Status Register Layout 248 249| Bit | Managed By | Description | 250|----------|------------|-------------| 251| 7 (MSB) | Software | BMC Active | 252| 6 | Software | Channel active, version negotiated | 253| 5 | Software | Unused | 254| 4 | Software | Unused | 255| 3 | Hardware | Command / Data | 256| 2 | Software | Unused | 257| 1 | Hardware | Input Buffer Full | 258| 0 (LSB) | Hardware | Output Buffer Full | 259 260#### KCS Data Register Commands 261 262| Command | Description | 263|---------|-------------| 264| 0x00 | Initialise | 265| 0x01 | Tx Begin | 266| 0x02 | Rx Complete | 267| 0xff | Dummy Value | 268 269#### Host Command to BMC Sequence 270 271The host sends commands to the BMC to signal channel initialisation, begin 272transmission of a packet, or to complete reception of a packet. 273 274| Step | Description | 275|------|---------------------------------------------------------| 276| 1 | The host writes a command value to IDR | 277| 2 | The hardware sets IBF, which triggers a BMC interrupt | 278| 3 | The BMC reads the status register for IBF | 279| 4 | If IBF is set, the BMC reads the host command from IDR | 280| 5 | The interrupt is acknowledged by the data register read | 281 282#### BMC Command to Host Sequence 283 284The BMC sends commands to the host to begin transmission of a packet or to 285complete reception of a packet. 286 287| Step | Description | 288|------|---------------------------------------------------------| 289| 1 | The BMC writes a command value to ODR | 290| 2 | The hardware sets OBF, which triggers a host interrupt | 291| 3 | The host reads the status register for OBF | 292| 4 | If OBF is set, the host reads the BMC command from ODR | 293| 5 | The interrupt is acknowledged by the data register read | 294 295#### BMC Status Update Sequence 296 297The BMC sends status updates to the host to signal loss of function, loss of 298channel state, or the presence of a command in the KCS data register. 299 300| Step | Description | 301|------|----------------------------------------------------------------| 302| 1 | The BMC writes the status value to the status register | 303| 2 | The BMC writes the dummy command to ODR | 304| 3 | The hardware sets OBF, which triggers a host interrupt | 305| 4 | If OBF is set, the host reads the BMC command from ODR | 306| 5 | The interrupt is acknowledged by the data register read | 307| 6 | The host observes the command is the dummy command | 308| 7 | The host reads the status register to capture the state change | 309 310#### LPC Window Ownership and Synchronisation 311 312Because the LPC FW window is shared between the host and the BMC we need 313strict rules on which entity is allowed to access it at specific times. 314 315Firstly, we have rules for modification: 316 317* The control data is only written during initialisation. The control area 318 is never modified once the channel is active. 319* Only the BMC may write to the Rx buffer described in the control area 320* Only the host may write to the Tx buffer described in the control area 321 322During packet transmission, the follow sequence occurs: 323 3241. The Tx side writes the packet to its Tx buffer 3252. The Tx side sends a `Tx Begin` message, indicating that the buffer ownership 326 is transferred 3273. The Rx side now owns the buffer, and reads the message from its Rx area 3284. The Rx side sends a `Rx Complete` once done, indicating that the buffer 329 ownership is transferred back to the Tx side. 330 331### LPC Binding Operation 332 333The binding operation is not symmetric as the BMC is the only side that can 334drive the status register. Each side's initialisation sequence is outlined 335below. 336 337The sequences below contain steps where the BMC updates the channel status and 338where commands are sent between the BMC and the host. The act of updating 339status or sending a command invokes the behaviour outlined in [KCS 340Control](#kcs-control). 341 342The packet transmission sequences assume that `BMC Active` and `Channel Active` 343are set. 344 345#### BMC Initialisation Sequence 346 347| Step | Description | 348|------|------------------------------------------| 349| 1 | The BMC initialises the control area: magic value, BMC versions and buffer parameters | 350| 2 | The BMC sets the status to `BMC Active` | 351 352#### Host Initialisation Sequence 353 354| Step | v1 | v2 | Description | 355|------|----|----|------------------------------------------------| 356| 1 | ✓ | ✓ | The host waits for the `BMC Active` state | 357| 2 | ✓ | ✓ | The host populates the its version fields | 358| 3 | | ✓ | The host derives and writes to `rx_size` the packet size associated with its desired MTU | 359| 4 | ✓ | ✓ | The host sends the `Initialise` command | 360| 5 | ✓ | ✓ | The BMC observes the `Initialise` command | 361| 6 | ✓ | ✓ | The BMC calculates and writes `negotiated_ver` | 362| 7 | | ✓ | The BMC calculates the MTUs and updates neither, one or both of `rx_size` and `tx_size` | 363| 8 | ✓ | ✓ | The BMC sets the status to `Channel Active` | 364| 9 | ✓ | ✓ | The host observes that `Channel Active` is set | 365| 10 | ✓ | ✓ | The host reads the negotiated version | 366| 11 | | ✓ | The host reads both `rx_size` and `tx_size` to derive the negotiated MTUs | 367 368#### Host Packet Transmission Sequence 369 370| Step | Description | 371|------|--------------------------------------------------------------| 372| 1 | The host waits on any previous `Rx Complete` message | 373| 2 | The host writes the packet to its Tx area (BMC Rx area) | 374| 3 | The host sends the `Tx Begin` command, transferring ownership of its Tx buffer to the BMC | 375| 4 | The BMC observes the `Tx Begin` command | 376| 5 | The BMC reads the packet from the its Rx area (host Tx area) | 377| 6 | The BMC sends the `Rx Complete` command, transferring ownership of its Rx buffer to the host | 378| 7 | The host observes the `Rx Complete` command | 379 380#### BMC Packet Transmission Sequence 381 382| Step | Description | 383|------|---------------------------------------------------------------| 384| 1 | The BMC waits on any previous `Rx Complete` message | 385| 2 | The BMC writes the packet to its Tx area (host Rx area) | 386| 3 | The BMC sends the `Tx Begin` command, transferring ownership of its Tx buffer to the host | 387| 4 | The host observes the `Tx Begin` command | 388| 5 | The host reads the packet from the host Rx area (BMC Tx area) | 389| 6 | The host sends the `Rx Complete` command, transferring ownership of its Rx buffer to the BMC | 390| 7 | The BMC observes the `Rx Complete` command | 391 392## Implementation Notes 393 394On the BMC the initial prototype implementation makes use of the following 395components: 396 397* An LPC KCS device exposed by a [binding-specific kernel driver][mctp-driver] 398* The reserved memory mapped by the LPC2AHB bridge via the [aspeed-lpc-ctrl 399 driver][aspeed-lpc-ctrl] 400* The astlpc binding found in [libmctp][libmctp] 401 402[mctp-driver]: https://github.com/openbmc/linux/commit/9a3b539a175cf4fe1f8fc2997e8a91abec25c37f 403[aspeed-lpc-ctrl]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/soc/aspeed/aspeed-lpc-ctrl.c?h=v5.7 404[libmctp]: https://github.com/openbmc/libmctp 405 406From the host side, the LPC Firmware and KCS IO cycles are driven by 407free-standing firmware. Some firmwares exploit [libmctp][libmctp] by 408implementing the driver hooks for direct access to the LPC devices. 409 410## Alternatives Considered 411 412### The KCS MCTP Binding (DSP0254) 413 414The KCS hardware (used as the full transfer channel) can be used to transfer 415arbitrarily-sized MCTP messages. However, there are much larger overheads in 416synchronisation between host and BMC for every byte transferred. 417 418### The MCTP Serial Binding (DSP0253) 419 420We could use the VUART hardware to transfer the MCTP packets according to the 421existing MCTP Serial Binding. However, the VUART device is already used for 422console data. Multiplexing both MCTP and console would be an alternative, but 423the complexity introduced would make low-level debugging both more difficult 424and less reliable. 425 426### The BT interface 427 428The BT interface allows for block-at-time transfers. However, the BT buffer 429size is only 64 bytes on the AST2500 hardware, which does not allow us to 430comply with the MCTP Base Specification (DSP0236) that requires a 64-byte 431payload size as the minimum. The 64-byte BT buffer does not allow for MCTP and 432transport headers. 433 434Additionally, we would like to develop the MCTP channel alongside the existing 435IPMI interfaces, to allow a gradual transition from IPMI to MCTP. As the BT 436channel is already used on OpenPOWER systems for IPMI transfers, we would not 437be able to support both in parallel. 438 439### Using the AST2500 LPC Mailbox 440 441This would require enabling the SuperIO interface, which allows the host to 442access the entire BMC address space, and so introduces security 443vulnerabilities. 444