1### NVMe-MI over SMBus 2 3Author: Tony Lee <tony.lee@quantatw.com> 4 5Created: 3-8-2019 6 7#### Problem Description 8 9Currently, OpenBMC does not support NVMe drive information. NVMe-MI 10specification defines a command that can read the NVMe drive information via 11SMBus directly. The NVMe drive can provide its information or status, like 12vendor ID, temperature, etc. The aim of this proposal is to allow users to 13monitor NVMe drives so appropriate action can be taken. 14 15#### Background and References 16 17NVMe-MI specification defines a command called 18`NVM Express Basic Management Command` that can read the NVMe drives information 19via SMBus directly. [1]. This command uses SMBus Block Read protocol specified 20by the SMBus specification. [2]. 21 22For our purpose is retrieve NVMe drives information, therefore, using NVM 23Express Basic Management Command where describe in NVMe-MI specification to 24communicate with NVMe drives. According to different platforms, temperature 25sensor, present status, LED and power sequence will be customized. 26 27[1] NVM Express Management Interface Revision 1.0a April 8, 2017 in Appendix A. 28(https://nvmexpress.org/wp-content/uploads/NVM_Express_Management_Interface_1_0a_2017.04.08_-_gold.pdf) 29[2] System Management Bus (SMBus) Specification Version 3.0 20 Dec 2014 30(http://smbus.org/specs/SMBus_3_0_20141220.pdf) 31 32#### Requirements 33 34The implementation should: 35 36- Provide a daemon to monitor NVMe drives. Parameters to be monitored are Status 37 Flags, SMART Warnings, Temperature, Percentage Drive Life Used, Vendor ID, and 38 Serial Number. 39- Provide a D-bus interface to allow other services to access data. 40- Capability of communication over hardware channel I2C to NVMe drives. 41- Ability to turn the fault LED on/off for each drive by SmartWarnings if the 42 object path of fault LED is defined in the configuration file. 43 44#### Proposed Design 45 46Create a D-bus service "xyz.openbmc_project.nvme.manager" with object paths for 47each NVMe sensor: "/xyz/openbmc_project/sensors/temperature/nvme0", 48"/xyz/openbmc_project/sensors/temperature/nvme1", etc. There is a JSON 49configuration file for drive index, bus ID, and the fault LED object path for 50each drive. For example, 51 52```json 53{ 54 "NvmeDriveIndex": 0, 55 "NVMeDriveBusID": 16, 56 "NVMeDriveFaultLEDGroupPath": "/xyz/openbmc_project/led/groups/led_u2_0_fault", 57 "NVMeDrivePresentPin": 148, 58 "NVMeDrivePwrGoodPin": 161 59}, 60{ 61 "NvmeDriveIndex": 1, 62 "NVMeDriveBusID": 17, 63 "NVMeDriveFaultLEDGroupPath": "/xyz/openbmc_project/led/groups/led_u2_0_fault", 64 "NVMeDrivePresentPin": 149, 65 "NVMeDrivePwrGoodPin": 162 66} 67``` 68 69Structure like: 70 71Under the D-bus named "xyz.openbmc_project.nvme.manager": 72 73``` 74 /xyz/openbmc_project 75 └─/xyz/openbmc_project/sensors 76 └─/xyz/openbmc_project/sensors/temperature/nvme0 77``` 78 79/xyz/openbmc_project/sensors/temperature/nvme0 Which implements: 80 81- xyz.openbmc_project.Sensor.Value 82- xyz.openbmc_project.Sensor.Threshold.Warning 83- xyz.openbmc_project.Sensor.Threshold.Critical 84 85Under the D-bus named "xyz.openbmc_project.Inventory.Manager": 86 87``` 88/xyz/openbmc_project 89 └─/xyz/openbmc_project/inventory 90 └─/xyz/openbmc_project/inventory/system 91 └─/xyz/openbmc_project/inventory/system/chassis 92 └─/xyz/openbmc_project/inventory/system/chassis/motherboard 93 └─/xyz/openbmc_project/inventory/system/chassis/motherboard/nvme0 94``` 95 96/xyz/openbmc_project/inventory/system/chassis/motherboard/nvme0 Which 97implements: 98 99- xyz.openbmc_project.Inventory.Item 100- xyz.openbmc_project.Inventory.Decorator.Asset 101- xyz.openbmc_project.Nvme.Status 102 103Interface `xyz.openbmc_project.Sensor.Value`, it's for hwmon to monitor 104temperature and with the following properties: 105 106| Property | Type | Description | 107| -------- | ------ | -------------------- | 108| MaxValue | int64 | Sensor maximum value | 109| MinValue | int64 | Sensor minimum value | 110| Scale | int64 | Sensor value scale | 111| Unit | string | Sensor unit | 112| Value | int64 | Sensor value | 113 114Interface `xyz.openbmc_project.Nvme.Status` with the following properties: 115 116| Property | Type | Description | 117| ----------------- | ------ | -------------------------------------------- | 118| SmartWarnings | string | Indicates smart warnings for the state | 119| StatusFlags | string | Indicates the status of the drives | 120| DriveLifeUsed | string | A vendor specific estimate of the percentage | 121| TemperatureFault | bool | If warning type about temperature happened | 122| BackupdrivesFault | bool | If warning type about backup drives happened | 123| CapacityFault | bool | If warning type about capacity happened | 124| DegradesFault | bool | If warning type about degrades happened | 125| MediaFault | bool | If warning type about media happened | 126 127Interface `xyz.openbmc_project.Inventory.Item` with the following properties: 128 129| Property | Type | Description | 130| ---------- | ------ | ----------------------------------- | 131| PrettyName | string | The human readable name of the item | 132| Present | bool | Whether or not the item is present | 133 134Interface `xyz.openbmc_project.Inventory.Decorator.Asset` with the following 135properties: 136 137| Property | Type | Description | 138| ------------ | ------ | ------------------------------------------------- | 139| PartNumber | string | The item part number, typically a stocking number | 140| SerialNumber | string | The item serial number | 141| Manufacturer | string | The item manufacturer | 142| BuildDate | bool | The date of item manufacture in YYYYMMDD format | 143| Model | bool | The model of the item | 144 145##### xyz.openbmc_project.nvme.manager.service 146 147This service has several steps: 148 1491. It will register a D-bus called `xyz.openbmc_project.nvme.manager` 150 description above. 1512. Obtain the drive index, bus ID, GPIO present pin, power good pin and fault 152 LED object path from the json file mentioned above. 1533. Each cycle will do following steps: 154 1. Check if the present pin of target drive is true, if true, means drive 155 exists and go to next step. If not, means drive does not exists and remove 156 object path from D-bus by drive index. 157 2. Check if the power good pin of target drive is true, if true means drive 158 is ready then create object path by drive index and go to next step. If 159 not, means drive power abnormal, turn on fault LED and log in journal. 160 3. Send a NVMe-MI command via SMBus Block Read protocol by bus ID of target 161 drive to get data. Data get from NVMe drives are "Status Flags", "SMART 162 Warnings", "Temperature", "Percentage Drive Life Used", "Vendor ID", and 163 "Serial Number". 164 4. The data will be set to the properties in D-bus. 165 166This service will run automatically and look up NVMe drives every second. 167 168##### Fault LED 169 170When the value obtained from the command corresponds to one of the warning 171types, it will trigger the fault LED of corresponding device and issue events. 172 173##### Add SEL related to NVMe 174 175The events `TemperatureFault`, `BackupdrivesFault`, `CapacityFault`, 176`DegradesFault` and `MediaFault` will be generated for the NVMe errors. 177 178- Temperature Fault log : when the property `TemperatureFault` set to true 179- Backupdrives Fault log : when the property `BackupdrivesFault` set to true 180- Capacity Fault log : when the property `CapacityFault` set to true 181- Degrades Fault log : when the property `DegradesFault` set to true 182- Media Fault log: when the property `MediaFault` set to true 183 184#### Alternatives Considered 185 186NVMe-MI specification defines multiple commands that can communicate with NVMe 187drives over MCTP protocol. The NVMe-MI over MCTP has the following key 188capabilities: 189 190- Discover drives that are present and learn capabilities of each drives. 191- Store data about the host environment enabling a Management Controller to 192 query the data later. 193- A standard format for VPD and defined mechanisms to read/write VPD contents. 194- Inventorying, configuring and monitoring. 195 196For monitoring NVMe drives, using NVM Express Basic Management Command over 197SMBus directly is much simpler than NVMe-MI over MCTP protocol. 198 199#### Impacts 200 201This application is monitoring NVMe drives via SMbus and set values to D-bus. 202The impacts should be small in the system. 203 204#### Testing 205 206This implementation is to use NVMe-MI-Basic command over SMBus and then set the 207response data to D-bus. Testing will send SMBus command to the drives to get the 208information and compare with the properties in D-bus to make sure they are the 209same. The testing can be performed on different NVMe drives by different 210manufacturers. For example: Intel P4500/P4600 and Micron 9200 Max/Pro. 211 212Unit tests will test by function: 213 214- It tests the length of responded data is as same as design in the function of 215 getting NVMe information. 216- It tests the function of setting values to D-bus is as same as design. 217- It tests the function of turn the corresponding LED ON/OFF by different 218 Smartwarnings values. 219