1# Virtual Media (a.k.a. Remote Media) 2 3Author: 4Pawel Rapkiewicz <pawel.rapkiewicz@intel.com> <pawelr> 5 6Other contributors: 7Przemyslaw Czarnowski <przemyslaw.hawrylewicz.czarnowski@intel.com> 8Anna Platash <anna.platash@intel.com> 9 10Created: 116/4/2019 12 13## Problem Description 14 15Virtual Media allows users to remotely mount given ISO/IMG drive images through 16BMC to Server HOST. The Remote drive is visible in HOST as USB storage device, 17and operates in RO mode, or RW mode (keeping in mind container limitations, 18and write protection switches). This can even be used to install OS on bare 19metal system. This document focuses on few redirection options, like in-browser 20ISO/IMG image mounting, and remote CIFS/HTTPS image mounting. 21 22## References 23 24* Virtual Media is going to use Network Block Device as primary disk image 25forwarder. 26* NBDkit is being used, to serve images from remote storages over HTTPS/CIFS. 27USBGadget as way to expose Media Storage Device to HOST. 28 29## Requirements 30None 31 32## Proposed Design 33 34Virtual Media splits into two modes of operation, lets call it Proxy, and 35Legacy. 36 37* Proxy mode - works directly from browser and uses JavaScript/HTML5 to 38communicate over Secure WebSockets directly to HTTPS endpoint hosted by bmcweb 39on BMC. 40* Legacy mode - is initiated from browser using Redfish defined VirtualMedia 41schemas, then BMC process connects to external CIFS/HTTPS image pointed during 42initialization. 43 44Both methods inherit from default Redfish/BMCWeb authentication and privileges 45mechanisms. 46 47The component diagram below shows Virtual Media high-level overview 48 49```ascii 50+------------------+ +----------------------------------+ +-----------------------+ 51|Remote Device| | |BMC| +------------+ | |HOST| | 52+-------------/ | +---/ +--Dbus----->+VirtualMedia| | +----/ | 53| | | v +------------+ | | | 54| +------------+ | | +-+--------+ | | | 55| |WebBrowser +<----HTTPS------->+BMCWeb | +---------+ | | +----------+ | 56| +------------+ | | +-+--------+ |USBGadget+<--------->+USB Device| | 57+------------------+ | ^ +----+----+ | | +----------+ | 58 | | ^ | | | 59 | | +------+ v | | | 60 | +--->+UNIX | +----+----+ | | | 61 | |SOCKET+<->+NBDClient| | | | 62+------------------+ | +--->+ | +---------+ | | | 63|Remote Storage| | | | +------+ | | | 64+--------------/ | | | | | | 65| | | v | | | 66| +-----------+ | | +-+-------+ | | | 67| |ISO/IMG +<---CIFS/HTTPS+-->+NBDkit | | | | 68| +-----------+ | | +---------+ | +-----------------------+ 69| | | | 70+------------------+ +----------------------------------+ 71``` 72 73Virtual Media feature supports multiple, simultaneous connections in both 74modes. 75 76### Asynchronicity 77 78Mounting and unmounting of remote device could be time consuming. Virtual 79Media shall support asynchronicity at the level of DBus and optionally Redfish 80API. 81 82Default timeouts for DBus (25 seconds) and for Redfish (60 seconds) may be 83insufficient to perform mounting and unmounting in some cases. 84 85Asynchronous responses will be described later on in appropriate sections. 86 87### Network Block Device (NBD) 88 89Reader can notice that most connections on diagram are based on Network Block 90Device. After Sourceforge project description: 91 92> With this compiled into your kernel, Linux can use a remote server as one of 93> its block devices. Every time the client computer wants to read /dev/nbd0, 94> it will send a request to the server via TCP, which will reply with the data 95> requested. This can be used for stations with low disk space (or even 96> diskless - if you use an initrd) to borrow disk space from other computers. 97> Unlike NFS, it is possible to put any file system on it. But (also unlike 98> NFS), if someone has mounted NBD read/write, you must assure that no one else 99> will have it mounted. 100> 101> -- [https://nbd.sourceforge.io/](https://nbd.sourceforge.io/) 102 103In Virtual Media use case, it's being used to pull data from remote client, and 104present it into non BMC mounted `/dev/nbdXX` device. Then the block device is 105being provided to Host through USB Gadget. 106 107### USB Gadget 108 109Part of Linux kernel that makes *emulation* of certain USB device classes 110possible through USB "On-The-Go", if connect appropriately to Host. In Virtual 111Media case it emulates USB mass storage connected to HOST. The source or 112redirection is block device created by nbd-client `/dev/nbdXX` 113 114### Proxy Mode 115 116Proxy Mode uses browser JavaScript and WebSockets support, to create JS NBD 117Server. Browser is responsible for create HTTPS session, authenticate user, and 118receive given privileges, then upgrade HTTPS session to WSS, through mechanisms 119described by [RFC6455](HTTPS://tools.ietf.org/html/rfc6455). Since WSS upgrade, 120JS application is responsible for handling all required by specification NBD 121Server commands. 122 123Multiple, simultaneous connections are supported per opening connections on 124different URIs in HTTPS server. Number of available simultaneous connections is 125being defined in configuration file described in next chapter. 126 127Encryption for proxy is supported through HTTPS/WSS channel and inherits 128encryption mechanisms directly from HTTPS server, all data transactions go 129through bmcweb. 130 131The initialization of connection will look as on diagram: 132 133```ascii 134┌───────┐ ┌──────┐ ┌────────────┐ ┌─────────┐ ┌────┐ ┌─────────┐ 135│Browser│ │bmcweb│ │VirtualMedia│ │NBDClient│ │uDEV│ │USBGadget│ 136└───┬───┘ └──┬───┘ └─────┬──────┘ └────┬────┘ └─┬──┘ └────┬────┘ 137 │ establish HTTPS session │ │ │ │ │ 138 │─────────────────────────> │ │ │ │ 139 │ │ │ │ │ │ 140 │ upgrade to WSS on │ │ │ │ │ 141 │ /nbd/X endpoint │ ╔══════════════╧════╗ │ │ │ 142 │─────────────────────────> ║* bmcweb creates ░║ │ │ │ 143 │ │ ║ /tmp/nbd.X.sock ║ │ │ │ 144 │ │ ║* bmcweb locks new ║ │ │ │ 145 │ │ ║ connections on ║ │ │ │ 146 │ │ ║ endpoint /nbd/X ║ │ │ │ 147 │ │ ╚══════════════╤════╝ │ │ │ 148 │ │ Mount from: │ │ │ │ 149 │ │ /tmp/nbd.X.sock │ │ │ │ 150 │ │ ────────────────> │ │ │ 151 │ │ │ │ │ │ 152 │ │ │ Spawn NBDClient from │ │ │ 153 │ │ │ /tmp/nbd.x.sock │ │ │ 154 │ │ │ to /dev/nbdX ┌┴┐ │ │ 155 │ │ │ ──────────────────> │ │ │ │ 156 │ │ │ │ │ │ │ 157 │ │ │ Block Device │ │ │ │ 158 │ │ │ properties changed │ │ │ │ 159 │ │ │ <───────────────────────────────│ │ 160 │ │ │ │ │ │ │ 161 │ │ │ Configure USB mass │ │ 162 │ │ │ storage from /dev/nbd/X │ │ 163 │ │ │ ─────────────────────────────────────────>│ 164 │ │ │ │ │ │ │ 165 │ │ Completion │ | | │ │ 166 │ │ signal │ | | │ │ 167 │ │ <───────────────│ │ │ │ │ 168 │ │ │ │ │ │ │ 169 │ │ │ Data │ │ │ │ 170 │<─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─>│ 171 │ │ │ │ │ │ │ 172``` 173 174### Legacy Mode 175 176Legacy Mode uses VirtualMedia schema, defined by DMTF, for mounting external 177CIFS/HTTPS images. The current implementation supports only stream mounting 178as for now. In this case Redfish is only used as mechanism for Virtual Media 179initialization, and is not responsible for data transmission. For data there 180is a separate component responsible for handling CIFS/HTTPS traffic called 181NBDkit. 182 183Multiple, simultaneous connections are supported through spawning additional 184nbkit instances, the number of available instances for CIFS/HTTPS is configured 185and described in details in next chapter. 186 187Encryption is based on remote storage connection, and follows Intel's Best 188security practices, as remote server support such (i.e. HTTPS requires SSL, and 189pure HTTP is not supported, for CIFS protocol version 3.0 allows enabling 190encryption and that will be provided). 191 192The flow looks like below: 193 194```ascii 195┌───────┐ ┌──────────┐ ┌──────┐ ┌────────────┐ ┌──────┐┌─────────┐┌────┐ ┌─────────┐ 196│Browser│ │CIFS/HTTPS│ │bmcweb│ │VirtualMedia│ │NBDkit││NBDClient││uDEV│ │USBGadget│ 197└───┬───┘ └────┬─────┘ └──┬───┘ └─────┬──────┘ └──┬───┘└────┬────┘└─┬──┘ └────┬────┘ 198 │establish HTTPS session│ │ │ │ │ │ 199 │───────────────────────> │ │ │ │ │ 200 │ │ │ │ │ │ │ │ 201 │Create new VirtualMedia│ │ │ │ │ │ 202 │ mountpoint via POST │ │ │ │ │ │ 203 │───────────────────────> │ │ │ │ │ 204 │ │ │ │ │ │ │ │ 205 │ │ │ Mount from │ │ │ │ │ 206 │ │ │ CIFS/HTTPS │ │ │ │ │ 207 │ │ │ location │ │ │ │ │ 208 │ │ │────────────> │ │ │ │ 209 │ │ │ │ │ │ │ │ 210 │ │ │ │Spawn NBDKit mounting│ │ │ │ 211 │ │ │ │ given location │ │ │ │ 212 │ │ │ │ appropriate │ │ │ │ 213 │ │ │ │ /tmp/nbd.X.sock ┌┴┐ │ │ │ 214 │ │ │ │ ──────────────────>│ │ │ │ │ 215 │ │ │ │ │ │ │ │ │ 216 │ │ │ │ Spawn NBDClient from │ │ │ 217 │ │ │ │ /tmp/nbd.X.sock to /dev/nbdX ┌┴┐ │ │ 218 │ │ │ │ ────────────────────────────>│ │ │ │ 219 │ │ │ │ │ │ │ │ │ │ 220 │ │ │ │ Block Device properties changed │ │ 221 │ │ │ │ <────────────────────────────────────── │ 222 │ │ │ │ │ │ │ │ │ │ 223 │ │ │ │ Configure USB mass storage from /dev/nbd/X │ 224 │ │ │ │ ───────────────────────────────────────────────>│ 225 │ │ │ │ │ │ │ │ │ │ 226 │ │ │ Completion │ │ │ │ │ │ │ 227 │ │ │ signal │ │ │ │ │ │ │ 228 │ │ |<───────────│ │ │ │ │ │ │ 229 │ │ │ │ │ │ │ │ │ │ 230 │ │ │ │ Data │ │ │ │ │ │ 231 │ │ <─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─>│ 232``` 233 234### Redfish support 235 236Virtual Media Service will be exposed as Redfish VirtualMedia endpoint as 237defined by DMTF. Here are some examples. 238 239#### Virtual Media Collection schema 240Members in collection will be defined based on configuration file described in 241next sections. And will be visible despite media is inserted or not. 242 243```json 244{ 245 "@odata.type": "#VirtualMediaCollection.VirtualMediaCollection", 246 "Name": "Virtual Media Services", 247 "Description": "Redfish-BMC Virtual Media Service Settings", 248 "Members@odata.count": 2, 249 "Members": [ 250 { 251 "@odata.id": "/redfish/v1/Managers/BMC/VirtualMedia/ISO0" 252 } , 253 { 254 "@odata.id": "/redfish/v1/Managers/BMC/VirtualMedia/1" 255 } 256 ], 257 "@odata.context": "/redfish/v1/$metadata#VirtualMediaCollection.VirtualMediaCollection", 258 "@odata.id": "/redfish/v1/Managers/BMC/VirtualMedia" 259} 260``` 261 262#### Virtual Media schema 263 264```json 265{ 266 "@odata.type": "#VirtualMedia.v1_1_0.VirtualMedia", 267 "Id": "ISO0", 268 "Name": "Virtual Removable Media", 269 "Actions": { 270 "#VirtualMedia.InsertMedia": { 271 "target": "/redfish/v1/Managers/bmc/VirtualMedia/ISO0/Actions/VirtualMedia.InsertMedia" 272 }, 273 "#VirtualMedia.EjectMedia": { 274 "target": "/redfish/v1/Managers/bmc/VirtualMedia/ISO0/Actions/VirtualMedia.EjectMedia" 275 } 276 }, 277 "MediaTypes": [ 278 "CD", 279 "USBStick" 280 ], 281 "Image": "https://192.168.0.1/Images/os.iso", 282 "ImageName": "Os", 283 "ConnectedVia": "URI", 284 "Inserted": true, 285 "WriteProtected": false, 286 "@odata.context": "/redfish/v1/$metadata#VirtualMedia.VirtualMedia", 287 "@odata.id": "/redfish/v1/Managers/BMC/VirtualMedia/ISO0", 288 "Oem": { 289 "OpenBMC": { 290 "@odata.type": "#OemVirtualMedia.v1_0_0.VirtualMedia", 291 "WebSocketEndpoint": "/nbd/0" 292 } 293 }, 294} 295``` 296 297Schema will look similar for both Proxy and Legacy Mode. Some key 298differences as follows: 299 300| Field Name | Proxy Mode | Legacy Mode | Comment | 301| -------------------- | ---------- | -------------------------------- | ----------------------------------- | 302| InsertMedia | N/A | action as described by DMTF spec | Action can return Task object if<br> process is time consumming | 303| Image | N/A | image location | | 304| ImageName | N/A | image name | | 305| ConnectedVia | "Applet" | as described by DMTF spec | applies only for connected media | 306| TransferMethod | "Stream" | "Stream" | "upload" is not supported by design | 307| TransferProtocolType | "OEM" | as described by DMTF spec | | 308 309#### Virtual Media OEM Extension 310 311Virtual Media schema is adapted to Legacy Mode where image is given by user 312directly via Redfish action and whole connection is processed between service 313and web server. 314 315For [Proxy Mode](#Proxy-Mode) nbd data is 316served by client web browser. Having multiple connections, in order to setup 317connection, client needs the information about the location of websocket 318created by web server. This value is exposed as OEM ``WebSocketEndpoint`` 319property for each item. 320 321### Inactivity timeout 322 323Virtual Media supports inactivity timeout, which will break Virtual Media 324connection after certain number of seconds of inactivity. Because nbdclient has 325mechanism for caching image, also kernel has home buffer mechanisms for block 326device, the idea is to prepare a patch on USBGadget driver, that will write USB 327gadget statistics under /proc/USBGadget/lun.X file. Those statistics will be 328observed by Virtual Media application. 329 330### Virtual Media Service 331 332Virtual Media Service is separate application that will coexist on DBus. It 333will be initialized from configuration json file, and expose all available 334for provisioning VirtualMedia objects. Virtual Media is responsible for: 335 336* Exposing current Virtual Media configuration to DBus user. 337* Spawning nbdclient, and monitor its lifetime, for Proxy connections. 338* Spawning nbdkit, and monitor its lifetime, for CIFS/HTTPS connections. 339* Monitoring uDEV for all NBD related block device changes, and 340 configure/de-configure USB Gadget accordingly. 341* Monitor NBD device inactivity period to support inactivity timeout. 342 343### Configuration 344 345Upon process startup, Virtual Media reads its config file, with the following 346structure: 347 348```json 349"InactivityTimeout": 1800, # Timeout of inactivity on device in seconds, that will lead to automatic disconnection 350"MountPoints": { 351 "ISO0": { 352 "EndpointId": "/nbd/0", # bmcweb endpoint (URL) configured for this type of connection 353 "Mode": 0, # 0 - Proxy Mode, 1 - Legacy Mode 354 "NBDDevice": "/dev/nbd0", # nbd endpoint on device usually matches numeric value with EndpointId 355 "UnixSocket": "/tmp/nbd.sock", # defines which Unix socket will be occupied by connection 356 "Timeout": 30, # timeout in seconds passed to nbdclient 357 "BlockSize": 512, # Block size passed to nbdclient 358 } 359}, 360``` 361 362### DBus Interface 363 364Virtual Media will expose the following object structure. All object paths 365are representation of configuration file described above 366 367```ascii 368/xyz/openbmc_project/VirtualMedia/Proxy/ISO0 369/xyz/openbmc_project/VirtualMedia/Proxy/1 370/xyz/openbmc_project/VirtualMedia/Legacy/0 371/xyz/openbmc_project/VirtualMedia/Legacy/1 372``` 373 374Each of object will implement ``xyz.openbmc_project.VirtualMedia.Process`` 375interface, which will be defined as follow: 376 377| Name | type | input | return | description | 378| -------- | -------- | ----- | ------- | ------------------------------------------------------------------- | 379| Active | Property | - | BOOLEAN | `True`, if object is occupied by active process, `False` otherwise | 380| ExitCode | Property | - | INT32 | If process terminates this property will contain returned exit code | 381 382Each object will also expose configuration of its own under 383``xyz.openbmc_project.VirtualMedia.MountPoint`` (all properties are RO) 384 385| Name | type | input | return | description | 386| -------------------------- | -------- | ----- | ------- | ------------------------------------------------------------------------- | 387| EndPointId | Property | - | STRING | As per configuration | 388| Mode | Property | - | BYTE | As per configuration | 389| Device | Property | - | STRING | As per configuration | 390| Socket | Property | - | STRING | As per configuration | 391| Timeout | Property | - | UINT16 | As per configuration | 392| BlockSize | Property | - | UINT16 | As per configuration | 393| RemainingInactivityTimeout | Property | - | UINT16 | Seconds to drop connection by server, for activated endpoint, 0 otherwise | 394| ImageURL | Property | - | STRING | URL to mounted image | 395| WriteProtected | Property | - | BOOLEAN | 'True', if the image is mounted as read only, 'False' otherwise | 396 397Another interface exposed by each object are stats under 398```xyz.openbmc_project.VirtualMedia.Stats``` (all properties are RO): 399 400| Name | type | input | return | description | 401| ------- | -------- | ----- | ------ | ---------------------------------------- | 402| ReadIO | Property | - | UINT64 | Number of read IOs since image mounting | 403| WriteIO | Property | - | UINT64 | Number of write IOs since image mounting | 404 405Depends on object path, object will expose different interface for mounting image. 406 407Mounting can be a time consuming task, so event driven mechanism has to be 408introduced. Mount and Unmount calls will trigger asynchronous action and will 409end immediately, giving appropriate signal containing status on task 410completion. 411 412For Proxy its ```xyz.openbmc_project.VirtualMedia.Proxy```, defined as follow: 413 414| Name | type | input | return | description | 415| ---------- | ------ | ----- | ------- | ------------------------------------------------- | 416| Mount | Method | - | BOOLEAN | Perform a mount to HOST operation on given object | 417| Unmount | Method | - | BOOLEAN | Perform an unmount from HOST on given object | 418| Completion | Signal | - | INT32 | Returns 0 for success or errno on failure | 419 420For Legacy its ```xyz.openbmc_project.VirtualMedia.Legacy```, defined as follow: 421 422| Name | type | input | return | description | 423| ---------- | ------ | ------ | ------- | ------------------------------------------------------------------------------------------ | 424| Mount | Method | STRING | BOOLEAN | Perform a mount to HOST operation on given object, with location given as STRING parameter | 425| Mount | Method | STRING<br>BOOLEAN<br>VARIANT<UNIX_FD,INT> | BOOLEAN | Perform a mount to HOST operation on given object, with parameters:<br><br>`STRING` : url to image. It should start with either `smb://` or `https://` prefix<br>`BOOLEAN` : RW flag for mounted gadget (should be consistent with remote image capabilities)<br>`VARIANT<UNIX_FD,INT>` : file descriptor of named pipe used for passing null-delimited secret data (username and password). When there is no data to pass `-1` should be passed as `INT` | 426| Unmount | Method | - | BOOLEAN | Perform an unmount from HOST on given object | 427| Completion | Signal | - | INT32 | Returns 0 for success or errno on failure | 428 429## Alternatives Considered 430Existing implementation in OpenBMC 431 432## Impact 433Shall not affect usability of current Virtual Media implementation 434 435## Testing 436TBD 437