1aa4d16e4SJack WangWhat: /sys/class/rnbd-client 2aa4d16e4SJack WangDate: Feb 2020 3aa4d16e4SJack WangKernelVersion: 5.7 4aa4d16e4SJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 5aa4d16e4SJack WangDescription: Provide information about RNBD-client. 6aa4d16e4SJack Wang All sysfs files that are not read-only provide the usage information on read: 7aa4d16e4SJack Wang 8002a9c2fSMauro Carvalho Chehab Example:: 9002a9c2fSMauro Carvalho Chehab 10aa4d16e4SJack Wang # cat /sys/class/rnbd-client/ctl/map_device 11aa4d16e4SJack Wang 12aa4d16e4SJack Wang > Usage: echo "sessname=<name of the rtrs session> path=<[srcaddr,]dstaddr> 13aa4d16e4SJack Wang > [path=<[srcaddr,]dstaddr>] device_path=<full path on remote side> 14aa4d16e4SJack Wang > [access_mode=<ro|rw|migration>] > map_device 15aa4d16e4SJack Wang > 16aa4d16e4SJack Wang > addr ::= [ ip:<ipv4> | ip:<ipv6> | gid:<gid> ] 17aa4d16e4SJack Wang 18aa4d16e4SJack WangWhat: /sys/class/rnbd-client/ctl/map_device 19aa4d16e4SJack WangDate: Feb 2020 20aa4d16e4SJack WangKernelVersion: 5.7 21aa4d16e4SJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 22002a9c2fSMauro Carvalho ChehabDescription: Expected format is the following:: 23aa4d16e4SJack Wang 24aa4d16e4SJack Wang sessname=<name of the rtrs session> 25aa4d16e4SJack Wang path=<[srcaddr,]dstaddr> [path=<[srcaddr,]dstaddr> ...] 26aa4d16e4SJack Wang device_path=<full path on remote side> 27aa4d16e4SJack Wang [access_mode=<ro|rw|migration>] 28aa4d16e4SJack Wang 29aa4d16e4SJack Wang Where: 30aa4d16e4SJack Wang 31002a9c2fSMauro Carvalho Chehab sessname: 32002a9c2fSMauro Carvalho Chehab accepts a string not bigger than 256 chars, which identifies 33aa4d16e4SJack Wang a given session on the client and on the server. 34aa4d16e4SJack Wang I.e. "clt_hostname-srv_hostname" could be a natural choice. 35aa4d16e4SJack Wang 36002a9c2fSMauro Carvalho Chehab path: 37002a9c2fSMauro Carvalho Chehab describes a connection between the client and the server by 38aa4d16e4SJack Wang specifying destination and, when required, the source address. 39002a9c2fSMauro Carvalho Chehab The addresses are to be provided in the following format:: 40aa4d16e4SJack Wang 41aa4d16e4SJack Wang ip:<IPv6> 42aa4d16e4SJack Wang ip:<IPv4> 43aa4d16e4SJack Wang gid:<GID> 44aa4d16e4SJack Wang 45002a9c2fSMauro Carvalho Chehab for example:: 46aa4d16e4SJack Wang 47aa4d16e4SJack Wang path=ip:10.0.0.66 48002a9c2fSMauro Carvalho Chehab 49aa4d16e4SJack Wang The single addr is treated as the destination. 50aa4d16e4SJack Wang The connection will be established to this server from any client IP address. 51aa4d16e4SJack Wang 52002a9c2fSMauro Carvalho Chehab :: 53002a9c2fSMauro Carvalho Chehab 54aa4d16e4SJack Wang path=ip:10.0.0.66,ip:10.0.1.66 55002a9c2fSMauro Carvalho Chehab 56aa4d16e4SJack Wang First addr is the source address and the second is the destination. 57aa4d16e4SJack Wang 58aa4d16e4SJack Wang If multiple "path=" options are specified multiple connection 59aa4d16e4SJack Wang will be established and data will be sent according to 60aa4d16e4SJack Wang the selected multipath policy (see RTRS mp_policy sysfs entry description). 61aa4d16e4SJack Wang 62002a9c2fSMauro Carvalho Chehab device_path: 63002a9c2fSMauro Carvalho Chehab Path to the block device on the server side. Path is specified 64aa4d16e4SJack Wang relative to the directory on server side configured in the 65aa4d16e4SJack Wang 'dev_search_path' module parameter of the rnbd_server. 66aa4d16e4SJack Wang The rnbd_server prepends the <device_path> received from client 67aa4d16e4SJack Wang with <dev_search_path> and tries to open the 68aa4d16e4SJack Wang <dev_search_path>/<device_path> block device. On success, 6947479b79SGioh Kim a /dev/rnbd<N> device file, a /sys/block/rnbd<N>/ 70aa4d16e4SJack Wang directory and an entry in /sys/class/rnbd-client/ctl/devices 71aa4d16e4SJack Wang will be created. 72aa4d16e4SJack Wang 73aa4d16e4SJack Wang If 'dev_search_path' contains '%SESSNAME%', then each session can 74aa4d16e4SJack Wang have different devices namespace, e.g. server was configured with 75aa4d16e4SJack Wang the following parameter "dev_search_path=/run/rnbd-devs/%SESSNAME%", 76aa4d16e4SJack Wang client has this string "sessname=blya device_path=sda", then server 77aa4d16e4SJack Wang will try to open: /run/rnbd-devs/blya/sda. 78aa4d16e4SJack Wang 79002a9c2fSMauro Carvalho Chehab access_mode: 80002a9c2fSMauro Carvalho Chehab the access_mode parameter specifies if the device is to be 81aa4d16e4SJack Wang mapped as "ro" read-only or "rw" read-write. The server allows 82aa4d16e4SJack Wang a device to be exported in rw mode only once. The "migration" 83aa4d16e4SJack Wang access mode has to be specified if a second mapping in read-write 84aa4d16e4SJack Wang mode is desired. 85aa4d16e4SJack Wang 86aa4d16e4SJack Wang By default "rw" is used. 87aa4d16e4SJack Wang 88*015fcf13SGioh Kim nr_poll_queues 89*015fcf13SGioh Kim specifies the number of poll-mode queues. If the IO has HIPRI flag, 90*015fcf13SGioh Kim the block-layer will send the IO via the poll-mode queue. 91*015fcf13SGioh Kim For fast network and device the polling is faster than interrupt-base 92*015fcf13SGioh Kim IO handling because it saves time for context switching, switching to 93*015fcf13SGioh Kim another process, handling the interrupt and switching back to the 94*015fcf13SGioh Kim issuing process. 95*015fcf13SGioh Kim 96*015fcf13SGioh Kim Set -1 if you want to set it as the number of CPUs 97*015fcf13SGioh Kim By default rnbd client creates only irq-mode queues. 98*015fcf13SGioh Kim 99*015fcf13SGioh Kim NOTICE: MUST make a unique session for a device using the poll-mode queues. 100*015fcf13SGioh Kim 101aa4d16e4SJack Wang Exit Codes: 102aa4d16e4SJack Wang 103aa4d16e4SJack Wang If the device is already mapped it will fail with EEXIST. If the input 104aa4d16e4SJack Wang has an invalid format it will return EINVAL. If the device path cannot 105aa4d16e4SJack Wang be found on the server, it will fail with ENOENT. 106aa4d16e4SJack Wang 107aa4d16e4SJack Wang Finding device file after mapping 108aa4d16e4SJack Wang --------------------------------- 109aa4d16e4SJack Wang 110aa4d16e4SJack Wang After mapping, the device file can be found by: 1117578d5cdSGioh Kim o The symlink /sys/class/rnbd-client/ctl/devices/<device_id>@<session_name> 112aa4d16e4SJack Wang points to /sys/block/<dev-name>. The last part of the symlink destination 113aa4d16e4SJack Wang is the same as the device name. By extracting the last part of the 114aa4d16e4SJack Wang path the path to the device /dev/<dev-name> can be build. 115aa4d16e4SJack Wang 1167578d5cdSGioh Kim * /dev/block/$(cat /sys/class/rnbd-client/ctl/devices/<device_id>@<session_name>/dev) 117aa4d16e4SJack Wang 118aa4d16e4SJack Wang How to find the <device_id> of the device is described on the next 119aa4d16e4SJack Wang section. 120aa4d16e4SJack Wang 121aa4d16e4SJack WangWhat: /sys/class/rnbd-client/ctl/devices/ 122aa4d16e4SJack WangDate: Feb 2020 123aa4d16e4SJack WangKernelVersion: 5.7 124aa4d16e4SJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 125aa4d16e4SJack WangDescription: For each device mapped on the client a new symbolic link is created as 1267578d5cdSGioh Kim /sys/class/rnbd-client/ctl/devices/<device_id>@<session_name>, which points 127aa4d16e4SJack Wang to the block device created by rnbd (/sys/block/rnbd<N>/). 128aa4d16e4SJack Wang The <device_id> of each device is created as follows: 129aa4d16e4SJack Wang 130aa4d16e4SJack Wang - If the 'device_path' provided during mapping contains slashes ("/"), 131aa4d16e4SJack Wang they are replaced by exclamation mark ("!") and used as as the 132aa4d16e4SJack Wang <device_id>. Otherwise, the <device_id> will be the same as the 133aa4d16e4SJack Wang "device_path" provided. 134