1What: /sys/class/rtrs-client 2Date: Feb 2020 3KernelVersion: 5.7 4Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 5Description: When a user of RTRS API creates a new session, a directory entry with 6 the name of that session is created under /sys/class/rtrs-client/<session-name>/ 7 8What: /sys/class/rtrs-client/<session-name>/add_path 9Date: Feb 2020 10KernelVersion: 5.7 11Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 12Description: RW, adds a new path (connection) to an existing session. Expected format is the 13 following:: 14 15 <[source addr,]destination addr> 16 *addr ::= [ ip:<ipv4|ipv6> | gid:<gid> ] 17 18What: /sys/class/rtrs-client/<session-name>/max_reconnect_attempts 19Date: Feb 2020 20KernelVersion: 5.7 21Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 22Description: Maximum number reconnect attempts the client should make before giving up 23 after connection breaks unexpectedly. 24 25What: /sys/class/rtrs-client/<session-name>/mp_policy 26Date: Feb 2020 27KernelVersion: 5.7 28Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 29Description: Multipath policy specifies which path should be selected on each IO: 30 31 round-robin (0): 32 select path in per CPU round-robin manner. 33 34 min-inflight (1): 35 select path with minimum inflights. 36 37 min-latency (2): 38 select path with minimum latency. 39 40What: /sys/class/rtrs-client/<session-name>/paths/ 41Date: Feb 2020 42KernelVersion: 5.7 43Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 44Description: Each path belonging to a given session is listed here by its source and 45 destination address. When a new path is added to a session by writing to 46 the "add_path" entry, a directory <src@dst> is created. 47 48What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/state 49Date: Feb 2020 50KernelVersion: 5.7 51Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 52Description: RO, Contains "connected" if the session is connected to the peer and fully 53 functional. Otherwise the file contains "disconnected" 54 55What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/reconnect 56Date: Feb 2020 57KernelVersion: 5.7 58Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 59Description: Write "1" to the file in order to reconnect the path. 60 Operation is blocking and returns 0 if reconnect was successful. 61 62What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/disconnect 63Date: Feb 2020 64KernelVersion: 5.7 65Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 66Description: Write "1" to the file in order to disconnect the path. 67 Operation blocks until RTRS path is disconnected. 68 69What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/remove_path 70Date: Feb 2020 71KernelVersion: 5.7 72Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 73Description: Write "1" to the file in order to disconnected and remove the path 74 from the session. Operation blocks until the path is disconnected 75 and removed from the session. 76 77What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_name 78Date: Feb 2020 79KernelVersion: 5.7 80Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 81Description: RO, Contains the the name of HCA the connection established on. 82 83What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_port 84Date: Feb 2020 85KernelVersion: 5.7 86Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 87Description: RO, Contains the port number of active port traffic is going through. 88 89What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/src_addr 90Date: Feb 2020 91KernelVersion: 5.7 92Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 93Description: RO, Contains the source address of the path 94 95What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/dst_addr 96Date: Feb 2020 97KernelVersion: 5.7 98Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 99Description: RO, Contains the destination address of the path 100 101What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/cur_latency 102Date: Feb 2020 103KernelVersion: 5.7 104Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 105Description: RO, Contains the latency time calculated by the heart-beat messages. 106 Whenever the client sends heart-beat message, it checks the time gap 107 between sending the heart-beat message and receiving the ACK. 108 This value can be changed regularly. 109 110What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reset_all 111Date: Feb 2020 112KernelVersion: 5.7 113Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 114Description: RW, Read will return usage help, write 0 will clear all the statistics. 115 116What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/cpu_migration 117Date: Feb 2020 118KernelVersion: 5.7 119Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 120Description: RTRS expects that each HCA IRQ is pinned to a separate CPU. If it's 121 not the case, the processing of an I/O response could be processed on a 122 different CPU than where it was originally submitted. This file shows 123 how many interrupts where generated on a non expected CPU. 124 125 "from:" 126 is the CPU on which the IRQ was expected, but not generated. 127 "to:" 128 is the CPU on which the IRQ was generated, but not expected. 129 130What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reconnects 131Date: Feb 2020 132KernelVersion: 5.7 133Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 134Description: Contains 2 unsigned int values, the first one records number of successful 135 reconnects in the path lifetime, the second one records number of failed 136 reconnects in the path lifetime. 137 138What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/rdma 139Date: Feb 2020 140KernelVersion: 5.7 141Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 142Description: Contains statistics regarding rdma operations and inflight operations. 143 The output consists of 6 values:: 144 145 <read-count> <read-total-size> <write-count> \ 146 <write-total-size> <inflights> <failovered> 147