1745b6a3dSJack WangWhat: /sys/class/rtrs-client 2745b6a3dSJack WangDate: Feb 2020 3745b6a3dSJack WangKernelVersion: 5.7 4745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 5745b6a3dSJack WangDescription: When a user of RTRS API creates a new session, a directory entry with 6745b6a3dSJack Wang the name of that session is created under /sys/class/rtrs-client/<session-name>/ 7745b6a3dSJack Wang 8745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/add_path 9745b6a3dSJack WangDate: Feb 2020 10745b6a3dSJack WangKernelVersion: 5.7 11745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 12745b6a3dSJack WangDescription: RW, adds a new path (connection) to an existing session. Expected format is the 13745b6a3dSJack Wang following: 14745b6a3dSJack Wang 15745b6a3dSJack Wang <[source addr,]destination addr> 16745b6a3dSJack Wang *addr ::= [ ip:<ipv4|ipv6> | gid:<gid> ] 17745b6a3dSJack Wang 18745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/max_reconnect_attempts 19745b6a3dSJack WangDate: Feb 2020 20745b6a3dSJack WangKernelVersion: 5.7 21745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 22745b6a3dSJack WangDescription: Maximum number reconnect attempts the client should make before giving up 23745b6a3dSJack Wang after connection breaks unexpectedly. 24745b6a3dSJack Wang 25745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/mp_policy 26745b6a3dSJack WangDate: Feb 2020 27745b6a3dSJack WangKernelVersion: 5.7 28745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 29745b6a3dSJack WangDescription: Multipath policy specifies which path should be selected on each IO: 30745b6a3dSJack Wang 31745b6a3dSJack Wang round-robin (0): 32745b6a3dSJack Wang select path in per CPU round-robin manner. 33745b6a3dSJack Wang 34745b6a3dSJack Wang min-inflight (1): 35745b6a3dSJack Wang select path with minimum inflights. 36745b6a3dSJack Wang 37745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/ 38745b6a3dSJack WangDate: Feb 2020 39745b6a3dSJack WangKernelVersion: 5.7 40745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 41745b6a3dSJack WangDescription: Each path belonging to a given session is listed here by its source and 42745b6a3dSJack Wang destination address. When a new path is added to a session by writing to 43745b6a3dSJack Wang the "add_path" entry, a directory <src@dst> is created. 44745b6a3dSJack Wang 45745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/state 46745b6a3dSJack WangDate: Feb 2020 47745b6a3dSJack WangKernelVersion: 5.7 48745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 49745b6a3dSJack WangDescription: RO, Contains "connected" if the session is connected to the peer and fully 50745b6a3dSJack Wang functional. Otherwise the file contains "disconnected" 51745b6a3dSJack Wang 52745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/reconnect 53745b6a3dSJack WangDate: Feb 2020 54745b6a3dSJack WangKernelVersion: 5.7 55745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 56745b6a3dSJack WangDescription: Write "1" to the file in order to reconnect the path. 57745b6a3dSJack Wang Operation is blocking and returns 0 if reconnect was successful. 58745b6a3dSJack Wang 59745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/disconnect 60745b6a3dSJack WangDate: Feb 2020 61745b6a3dSJack WangKernelVersion: 5.7 62745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 63745b6a3dSJack WangDescription: Write "1" to the file in order to disconnect the path. 64745b6a3dSJack Wang Operation blocks until RTRS path is disconnected. 65745b6a3dSJack Wang 66745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/remove_path 67745b6a3dSJack WangDate: Feb 2020 68745b6a3dSJack WangKernelVersion: 5.7 69745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 70745b6a3dSJack WangDescription: Write "1" to the file in order to disconnected and remove the path 71745b6a3dSJack Wang from the session. Operation blocks until the path is disconnected 72745b6a3dSJack Wang and removed from the session. 73745b6a3dSJack Wang 74745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_name 75745b6a3dSJack WangDate: Feb 2020 76745b6a3dSJack WangKernelVersion: 5.7 77745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 78745b6a3dSJack WangDescription: RO, Contains the the name of HCA the connection established on. 79745b6a3dSJack Wang 80745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_port 81745b6a3dSJack WangDate: Feb 2020 82745b6a3dSJack WangKernelVersion: 5.7 83745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 84745b6a3dSJack WangDescription: RO, Contains the port number of active port traffic is going through. 85745b6a3dSJack Wang 86745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/src_addr 87745b6a3dSJack WangDate: Feb 2020 88745b6a3dSJack WangKernelVersion: 5.7 89745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 90745b6a3dSJack WangDescription: RO, Contains the source address of the path 91745b6a3dSJack Wang 92745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/dst_addr 93745b6a3dSJack WangDate: Feb 2020 94745b6a3dSJack WangKernelVersion: 5.7 95745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 96745b6a3dSJack WangDescription: RO, Contains the destination address of the path 97745b6a3dSJack Wang 98745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reset_all 99745b6a3dSJack WangDate: Feb 2020 100745b6a3dSJack WangKernelVersion: 5.7 101745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 102745b6a3dSJack WangDescription: RW, Read will return usage help, write 0 will clear all the statistics. 103745b6a3dSJack Wang 104745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/cpu_migration 105745b6a3dSJack WangDate: Feb 2020 106745b6a3dSJack WangKernelVersion: 5.7 107745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 108745b6a3dSJack WangDescription: RTRS expects that each HCA IRQ is pinned to a separate CPU. If it's 109745b6a3dSJack Wang not the case, the processing of an I/O response could be processed on a 110745b6a3dSJack Wang different CPU than where it was originally submitted. This file shows 111745b6a3dSJack Wang how many interrupts where generated on a non expected CPU. 112745b6a3dSJack Wang "from:" is the CPU on which the IRQ was expected, but not generated. 113745b6a3dSJack Wang "to:" is the CPU on which the IRQ was generated, but not expected. 114745b6a3dSJack Wang 115745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reconnects 116745b6a3dSJack WangDate: Feb 2020 117745b6a3dSJack WangKernelVersion: 5.7 118745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 119745b6a3dSJack WangDescription: Contains 2 unsigned int values, the first one records number of successful 120745b6a3dSJack Wang reconnects in the path lifetime, the second one records number of failed 121745b6a3dSJack Wang reconnects in the path lifetime. 122745b6a3dSJack Wang 123745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/rdma 124745b6a3dSJack WangDate: Feb 2020 125745b6a3dSJack WangKernelVersion: 5.7 126745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 127745b6a3dSJack WangDescription: Contains statistics regarding rdma operations and inflight operations. 128745b6a3dSJack Wang The output consists of 6 values: 129745b6a3dSJack Wang 130745b6a3dSJack Wang <read-count> <read-total-size> <write-count> <write-total-size> \ 131745b6a3dSJack Wang <inflights> <failovered> 132