1# Management Component Transport Protocol (MCTP) LPC Transport Binding Specification for ASPEED BMC Systems
2
3## Scope
4
5This design provides an efficient method to transfer MCTP packets between the
6host and BMC over the LPC bus on ASPEED BMC platforms.
7
8## References
9
10The following referenced documents are indispensable for the application of
11this document.
12
131. DMTF DSP0236, Management Component Transport Protocol (MCTP) Base
14Specification 1.0,
15http://www.dmtf.org/standards/published_documents/DSP0236_1.0.pdf
16
172. Intel (R) Low Pin Count (LPC) Interface Specification 1.1,
18https://www.intel.com/content/dam/www/program/design/us/en/documents/low-pin-count-interface-specification.pdf
19
203. IPMI Consortium, Intelligent Platform Management Interface Specification,
21v1.5 Revision 1.1 February 20, 2002,
22http://download.intel.com/design/servers/ipmi/IPMIv1_5rev1_1.pdf
23
24## Definitions
25
26**BTU: Baseline Transmission Unit**
27
28Defined by the MCTP base specification as the smallest maximum packet size all
29MCTP-compliant endpoints must accept.
30
31**IBF: Input Buffer Full**
32
33A hardware-defined flag bit in a KCS device's Status Register (STR). The IBF
34flag indicates that a value has been written by the host to the corresponding
35Input Data Register (IDR).
36
37**IDR: Input Data Register**
38
39One of the three register interfaces exposed by a KCS device. The IDR is a one
40byte buffer which is written by the host and read by the BMC.
41
42**KCS: Keyboard-Controller-Style**
43
44A set of bit definitions and operation of the registers typically used in
45keyboard microcontrollers and embedded controllers. The term "Keyboard
46Controller Style" reflects that the register definition was originally used as
47the legacy "8742" keyboard controller interface in PC architecture computer
48systems.  This interface is available built-in to several commercially
49available microcontrollers. Data is transferred across the KCS interface using
50a per-byte handshake.
51
52**LPC Bus: Low Pin Count Bus**
53
54A bus specification that implements ISA bus in a reduced physical form while
55extending ISA's capabilities.
56
57**LPC FW: LPC Firmware Cycles**
58
59LPC firmware cycles allow separate boot BIOS firmware memory cycles and
60application memory cycles with respect to the LPC bus. The ASPEED BMCs allow
61remapping of the LPC firmware cycles onto arbitrary regions of the BMC's
62physical address space, including RAM.
63
64**MTU: Maximum Transmission Unit**
65
66The largest payload the link will accept for a packet. The Maximum Transmission
67Unit represents a value that is at least as large as the BTU. Negotiation of
68MTU values larger than the BTU may improve throughput for data-intensive
69transfers.
70
71**OBF: Output Buffer Full**
72
73A hardware-defined flag bit in a KCS device's Status Register (STR). The OBF
74flag indicates that a value has been written by the BMC to the corresponding
75Output Data Register (ODR).
76
77**ODR: Output Data Register**
78
79One of the three register interfaces exposed by a KCS device. The ODR is a
80one byte buffer which is written by the BMC and read by the host.
81
82**STR: Status Register**
83
84One of the three register interfaces exposed by a KCS device. STR is a
85BMC-controlled, eight-bit register exposed to both the BMC and the host for
86indication of IBF and OBF events on the input (IDR) and output (ODR) buffers.
87Bits that are not defined by hardware can be software-controlled in a manner
88defined by a platform-specific ABI.
89
90## Conventions
91
92Where unspecified, state, command and sequence descriptions apply to all
93versions of the protocol unless marked otherwise.
94
95## MCTP over LPC Transport
96
97### Concepts
98
99The basic components used for the transfer are:
100
101* An interrupt mechanism using the IPMI KCS interface
102* A window of the LPC FW address space, where reads and writes are forwarded to
103  BMC memory, using the LPC2AHB hardware
104
105In order to transfer a packet, either side of the channel (BMC or host) will:
106
1071. Write the packet to the LPC FW window
108   * The BMC will perform writes by writing to the memory backing the LPC
109     window
110   * The host will perform writes by writing to the LPC bus, at predefined
111     addresses
1122. Trigger an interrupt on the remote side, by writing to the KCS data buffer
113
114On this indication, the remote side will:
115
1161. Read from the KCS status register, which shows that the single-byte KCS data
117   buffer is full
1182. Read the provided command from the KCS data buffer, acknowledging the
119   interrupt
1203. Read the MCTP packet from the LPC FW window
121
122### Scope
123
124The document limits itself to describing the operation of the binding protocol.
125The following issues of protocol ABI are considered out of scope:
126
1271. The LPC IO address and Serial IRQ parameters of the KCS device
1282. The concrete location of the control region in the LPC FW address space
129
130### KCS Interface
131
132The KCS hardware on the ASPEED BMCs is used as a method of indicating, to the
133remote side, that a packet is ready to be transferred through the LPC FW
134mapping.
135
136The KCS hardware consists of two single-byte buffers: the Output Data Register
137(ODR) and the Input Data Register (IDR). The ODR is written by the BMC and read
138by the host. The IDR is the obverse.
139
140The KCS unit also contains a status register (STR), allowing both host and BMC
141to determine if there is data in the ODR or IDR. These are single-bit flags,
142designated Input/Output Buffer Full (IBF/OBF), and are automatically set by
143hardware when data has been written to the corresponding ODR/IDR buffer (and
144cleared when data has been read).
145
146While the IBF and OBF flags are managed in hardware, the remaining
147software-defined bits in the status register are used to carry other required
148protocol state. A problematic feature of the KCS status register is described
149in the IPMI specification, which states that an interrupt may be triggered on
150writes to the KCS status register but hardware implementations are not required
151to do so. Comparatively, writes to the data registers must set the
152corresponding buffer-full flag and invoke an interrupt.
153
154To ensure interrupts are generated for status updates, we exploit the OBF
155interrupt to signal a status update by writing a dummy command to ODR after
156updating the status register, as outlined below.
157
158### LPC FW Window
159
160The window of BMC-memory-backed LPC FW address space has a predefined format,
161consisting of:
162
163* A control descriptor, describing static data about the rest of the window
164* A receive area for BMC-to-host packets
165* A transmit area, for host-to-BMC packets
166
167The control descriptor contains a version, and offset and size data for the
168transmit and receive areas. These offsets are relative to the start of the LPC
169FW window.
170
171Full definition of the control area is defined below, and it will be the base
172for all future versions.
173
174```c
175struct mctp_lpcmap_hdr {
176   uint32_t magic;
177
178   uint16_t bmc_ver_min;
179   uint16_t bmc_ver_cur;
180   uint16_t host_ver_min;
181   uint16_t host_ver_cur;
182   uint16_t negotiated_ver;
183   uint16_t pad0;
184
185   uint32_t rx_offset;
186   uint32_t rx_size;
187   uint32_t tx_offset;
188   uint32_t tx_size;
189} __attribute__((packed));
190```
191
192The magic value marking the beginning of the control area is the ASCII encoding
193of "MCTP":
194
195```c
196#define LPC_MAGIC 0x4d435450
197```
198
199All medium-specific metadata is in big-endian format. This includes:
200
2011. Control area data
2022. Medium-specific packet header fields
2033. Medium-specific packet trailer fields
204
205MCTP packet data is transferred exactly as is presented, and no data escaping is
206performed.
207
208In all versions of the protocol, the transmit and receive areas contain a
209medium-specific header comprising a 32-bit payload length field, followed
210immediately by the MCTP packet data to be transferred. The full MCTP packet,
211including MCTP header, is considered to be the payload for the purpose of the
212header's length field.
213
214```c
215struct mctp_lpcbuf_hdr {
216   uint32_t length;
217} __attribute__((packed));
218```
219
220A medium-specific packet trailer must immediately follow the payload. The length
221of the trailer is not accounted for in the length field of the medium-specific
222packet header: The length of the trailer is implied by the negotiated protocol
223version.
224
225For protocol versions 1 and 2, the medium-specific trailer length is zero.
226
227For protocol version 3, the medium-specific trailer comprises a CRC-32 checksum
228of the payload.
229
230```c
231struct mctp_lpcbuf_tlr {
232   uint32_t crc32;
233} __attribute__((packed));
234```
235
236Where the CRC-32 implementation is defined by the following characteristics (or
237equivalent):
238
2391. The polynomial `x^32 + x^26 + x^23 + x^22 + x^16 + x^12 + x^11 + x^10 + x^8 + x^7 + x^5 + x^4 + x^2 + x + 1`
2402. Initialising the remainder state to `2^32 - 1`
2413. Incrementally shifting and `XOR`ing data bytes through the reversed polynomial representation `0xEDB88320`
2424. `XOR`ing the calculated remainder with `2^32 - 1`
243
244For all defined versions, only a single MCTP packet is present in the Rx and Tx
245areas. This may change for future versions of the protocol.
246
247#### Negotiation of the Maximum Transmission Unit
248
249Version 1 of the protocol offers no mechanism for negotiation of the maximum
250transmission unit. The Rx and Tx buffers must be sized to accommodate packets
251up to the Baseline Transmission Unit, and the implementation assumes that the
252MTU is set to the BTU regardless of the values of `rx_size` and `tx_size`.
253
254Version 2 of the protocol exploits the `rx_size` and `tx_size` fields in the
255control region to negotiate the link MTU. Note that at the time that the MTU is
256under negotiation the protocol version has not been finalised, so the process
257is necessarily backwards-compatible.
258
259The relevant property that each endpoint must control is the MTU of packets it
260will receive, as this governs how the remote endpoint's packetisation impacts
261memory pressure at the local endpoint. As such while the BMC MUST populate
262`rx_size` for backwards compatibility with version 1, the host MAY write
263`rx_size` without regard for its current value if the host supports version 2.
264The BMC controls the value of `tx_size`, and MAY choose to adjust it in
265response to the host's proposed `rx_size` value. As such, when `Channel Active`
266is set by the BMC, the host MUST read both `rx_size` and `tx_size` in response
267to ensure both the BMC and the host have a consistent understanding of the MTU
268in each direction. It is convention for `rx_size` and `tx_size` to be set to
269the same value by the BMC as part of finalising the channel, though it is not
270invalid to have asymmetric MTUs.
271
272For all protocol versions, the following properties must be upheld for the Rx
273and Tx buffers to be considered valid:
274
275* Intersect neither eachother nor the control region
276* Not extend beyond the window allocated to MCTP in the LPC FW address space
277* Must accommodate at least BTU-sized payloads
278
279The BMC MAY choose to fail channel initialisation if these properties are
280violated in the negotiation process.
281
282### KCS Status and Control Sequences
283
284The KCS status flags and command set govern the state of the protocol, defining
285the ability to send and receive packets on the LPC bus.
286
287#### KCS Status Register Layout
288
289| Bit      | Managed By | Description |
290|----------|------------|-------------|
291|  7 (MSB) |  Software  | BMC Active  |
292|  6       |  Software  | Channel active, version negotiated |
293|  5       |  Software  | Unused      |
294|  4       |  Software  | Unused      |
295|  3       |  Hardware  | Command / Data |
296|  2       |  Software  | Unused      |
297|  1       |  Hardware  | Input Buffer Full |
298|  0 (LSB) |  Hardware  | Output Buffer Full |
299
300#### KCS Data Register Commands
301
302| Command | Description |
303|---------|-------------|
304|  0x00   | Initialise  |
305|  0x01   | Tx Begin    |
306|  0x02   | Rx Complete |
307|  0xff   | Dummy Value |
308
309#### Host Command to BMC Sequence
310
311The host sends commands to the BMC to signal channel initialisation, begin
312transmission of a packet, or to complete reception of a packet.
313
314| Step | Description                                             |
315|------|---------------------------------------------------------|
316|  1   | The host writes a command value to IDR                  |
317|  2   | The hardware sets IBF, which triggers a BMC interrupt   |
318|  3   | The BMC reads the status register for IBF               |
319|  4   | If IBF is set, the BMC reads the host command from IDR  |
320|  5   | The interrupt is acknowledged by the data register read |
321
322#### BMC Command to Host Sequence
323
324The BMC sends commands to the host to begin transmission of a packet or to
325complete reception of a packet.
326
327| Step | Description                                             |
328|------|---------------------------------------------------------|
329|  1   | The BMC writes a command value to ODR                   |
330|  2   | The hardware sets OBF, which triggers a host interrupt  |
331|  3   | The host reads the status register for OBF              |
332|  4   | If OBF is set, the host reads the BMC command from ODR  |
333|  5   | The interrupt is acknowledged by the data register read |
334
335#### BMC Status Update Sequence
336
337The BMC sends status updates to the host to signal loss of function, loss of
338channel state, or the presence of a command in the KCS data register.
339
340| Step | Description                                                    |
341|------|----------------------------------------------------------------|
342|  1   | The BMC writes the status value to the status register         |
343|  2   | The BMC writes the dummy command to ODR                        |
344|  3   | The hardware sets OBF, which triggers a host interrupt         |
345|  4   | If OBF is set, the host reads the BMC command from ODR         |
346|  5   | The interrupt is acknowledged by the data register read        |
347|  6   | The host observes the command is the dummy command             |
348|  7   | The host reads the status register to capture the state change |
349
350#### LPC Window Ownership and Synchronisation
351
352Because the LPC FW window is shared between the host and the BMC we need
353strict rules on which entity is allowed to access it at specific times.
354
355Firstly, we have rules for modification:
356
357* The control data is only written during initialisation. The control area
358  is never modified once the channel is active.
359* Only the BMC may write to the Rx buffer described in the control area
360* Only the host may write to the Tx buffer described in the control area
361
362During packet transmission, the follow sequence occurs:
363
3641. The Tx side writes the packet to its Tx buffer
3652. The Tx side sends a `Tx Begin` message, indicating that the buffer ownership
366   is transferred
3673. The Rx side now owns the buffer, and reads the message from its Rx area
3684. The Rx side sends a `Rx Complete` once done, indicating that the buffer
369   ownership is transferred back to the Tx side.
370
371### LPC Binding Operation
372
373The binding operation is not symmetric as the BMC is the only side that can
374drive the status register. Each side's initialisation sequence is outlined
375below.
376
377The sequences below contain steps where the BMC updates the channel status and
378where commands are sent between the BMC and the host. The act of updating
379status or sending a command invokes the behaviour outlined in [KCS
380Control](#kcs-control).
381
382The packet transmission sequences assume that `BMC Active` and `Channel Active`
383are set.
384
385#### BMC Initialisation Sequence
386
387| Step | Description                              |
388|------|------------------------------------------|
389|  1   | The BMC initialises the control area: magic value, BMC versions and buffer parameters |
390|  2   | The BMC sets the status to `BMC Active`  |
391
392#### Host Initialisation Sequence
393
394| Step | v1 | v2 | v3 | Description                                    |
395|------|----|----|----|------------------------------------------------|
396|  1   | ✓  | ✓  | ✓  | The host waits for the `BMC Active` state      |
397|  2   | ✓  | ✓  | ✓  | The host populates the its version fields      |
398|  3   |    | ✓  | ✓  | The host derives and writes to `rx_size` the packet size associated with its desired MTU |
399|  4   | ✓  | ✓  | ✓  | The host sends the `Initialise` command        |
400|  5   | ✓  | ✓  | ✓  | The BMC observes the `Initialise` command      |
401|  6   | ✓  | ✓  | ✓  | The BMC calculates and writes `negotiated_ver` |
402|  7   |    | ✓  | ✓  | The BMC calculates the MTUs and updates neither, one or both of `rx_size` and `tx_size` |
403|  8   | ✓  | ✓  | ✓  | The BMC sets the status to `Channel Active`    |
404|  9   | ✓  | ✓  | ✓  | The host observes that `Channel Active` is set |
405|  10  | ✓  | ✓  | ✓  | The host reads the negotiated version          |
406|  11  |    | ✓  | ✓  | The host reads both `rx_size` and `tx_size` to derive the negotiated MTUs |
407
408#### Host Packet Transmission Sequence
409
410| Step | v1 | v2 | v3 | Description                                                  |
411|------|----|----|----|--------------------------------------------------------------|
412|  1   |    |    | ✓  | The host calculates the CRC-32 over the packet data          |
413|  2   | ✓  | ✓  | ✓  | The host waits on any previous `Rx Complete` message         |
414|  3   | ✓  | ✓  | ✓  | The host writes the packet data and medium-specific metadata to its Tx area (BMC Rx area) |
415|  4   | ✓  | ✓  | ✓  | The host sends the `Tx Begin` command, transferring ownership of its Tx buffer to the BMC |
416|  5   | ✓  | ✓  | ✓  | The BMC observes the `Tx Begin` command                      |
417|  6   | ✓  | ✓  | ✓  | The BMC reads the packet data and medium-specific metadata from the its Rx area (host Tx area) |
418|  7   | ✓  | ✓  | ✓  | The BMC sends the `Rx Complete` command, transferring ownership of its Rx buffer to the host |
419|  8   | ✓  | ✓  | ✓  | The host observes the `Rx Complete` command                  |
420|  9   |    |    | ✓  | The BMC validates the provided CRC-32 over the packet data   |
421
422#### BMC Packet Transmission Sequence
423
424| Step | v1 | v2 | v3 | Description                                                   |
425|------|----|----|----|---------------------------------------------------------------|
426|  1   |    |    | ✓  | The BMC calculates the CRC-32 over the packet data            |
427|  2   | ✓  | ✓  | ✓  | The BMC waits on any previous `Rx Complete` message           |
428|  3   | ✓  | ✓  | ✓  | The BMC writes the packet data and medium-specific metadata to its Tx area (host Rx area) |
429|  4   | ✓  | ✓  | ✓  | The BMC sends the `Tx Begin` command, transferring ownership of its Tx buffer to the host |
430|  5   | ✓  | ✓  | ✓  | The host observes the `Tx Begin` command                      |
431|  6   | ✓  | ✓  | ✓  | The host reads the packet data and medium-specific metadata from the host Rx area (BMC Tx area) |
432|  7   | ✓  | ✓  | ✓  | The host sends the `Rx Complete` command, transferring ownership of its Rx buffer to the BMC |
433|  8   | ✓  | ✓  | ✓  | The BMC observes the `Rx Complete` command                    |
434|  9   |    |    | ✓  | The host validates the provided CRC-32 over the packet data   |
435
436## Implementation Notes
437
438On the BMC the initial prototype implementation makes use of the following
439components:
440
441* An LPC KCS device exposed by a [binding-specific kernel driver][mctp-driver]
442* The reserved memory mapped by the LPC2AHB bridge via the [aspeed-lpc-ctrl
443  driver][aspeed-lpc-ctrl]
444* The astlpc binding found in [libmctp][libmctp]
445
446[mctp-driver]: https://github.com/openbmc/linux/commit/9a3b539a175cf4fe1f8fc2997e8a91abec25c37f
447[aspeed-lpc-ctrl]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/soc/aspeed/aspeed-lpc-ctrl.c?h=v5.7
448[libmctp]: https://github.com/openbmc/libmctp
449
450From the host side, the LPC Firmware and KCS IO cycles are driven by
451free-standing firmware. Some firmwares exploit [libmctp][libmctp] by
452implementing the driver hooks for direct access to the LPC devices.
453
454## Alternatives Considered
455
456### The KCS MCTP Binding (DSP0254)
457
458The KCS hardware (used as the full transfer channel) can be used to transfer
459arbitrarily-sized MCTP messages. However, there are much larger overheads in
460synchronisation between host and BMC for every byte transferred.
461
462### The MCTP Serial Binding (DSP0253)
463
464We could use the VUART hardware to transfer the MCTP packets according to the
465existing MCTP Serial Binding. However, the VUART device is already used for
466console data. Multiplexing both MCTP and console would be an alternative, but
467the complexity introduced would make low-level debugging both more difficult
468and less reliable.
469
470### The BT interface
471
472The BT interface allows for block-at-time transfers. However, the BT buffer
473size is only 64 bytes on the AST2500 hardware, which does not allow us to
474comply with the MCTP Base Specification (DSP0236) that requires a 64-byte
475payload size as the minimum. The 64-byte BT buffer does not allow for MCTP and
476transport headers.
477
478Additionally, we would like to develop the MCTP channel alongside the existing
479IPMI interfaces, to allow a gradual transition from IPMI to MCTP. As the BT
480channel is already used on OpenPOWER systems for IPMI transfers, we would not
481be able to support both in parallel.
482
483### Using the AST2500 LPC Mailbox
484
485This would require enabling the SuperIO interface, which allows the host to
486access the entire BMC address space, and so introduces security
487vulnerabilities.
488