1745b6a3dSJack WangWhat:		/sys/class/rtrs-client
2745b6a3dSJack WangDate:		Feb 2020
3745b6a3dSJack WangKernelVersion:	5.7
4745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
5745b6a3dSJack WangDescription:	When a user of RTRS API creates a new session, a directory entry with
6745b6a3dSJack Wang		the name of that session is created under /sys/class/rtrs-client/<session-name>/
7745b6a3dSJack Wang
8745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/add_path
9745b6a3dSJack WangDate:		Feb 2020
10745b6a3dSJack WangKernelVersion:	5.7
11745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
12745b6a3dSJack WangDescription:	RW, adds a new path (connection) to an existing session. Expected format is the
13745b6a3dSJack Wang		following:
14745b6a3dSJack Wang
15745b6a3dSJack Wang		<[source addr,]destination addr>
16745b6a3dSJack Wang		*addr ::= [ ip:<ipv4|ipv6> | gid:<gid> ]
17745b6a3dSJack Wang
18745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/max_reconnect_attempts
19745b6a3dSJack WangDate:		Feb 2020
20745b6a3dSJack WangKernelVersion:	5.7
21745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
22745b6a3dSJack WangDescription:	Maximum number reconnect attempts the client should make before giving up
23745b6a3dSJack Wang		after connection breaks unexpectedly.
24745b6a3dSJack Wang
25745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/mp_policy
26745b6a3dSJack WangDate:		Feb 2020
27745b6a3dSJack WangKernelVersion:	5.7
28745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
29745b6a3dSJack WangDescription:	Multipath policy specifies which path should be selected on each IO:
30745b6a3dSJack Wang
31745b6a3dSJack Wang		round-robin (0):
32745b6a3dSJack Wang		select path in per CPU round-robin manner.
33745b6a3dSJack Wang
34745b6a3dSJack Wang		min-inflight (1):
35745b6a3dSJack Wang		select path with minimum inflights.
36745b6a3dSJack Wang
37745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/paths/
38745b6a3dSJack WangDate:		Feb 2020
39745b6a3dSJack WangKernelVersion:	5.7
40745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
41745b6a3dSJack WangDescription:	Each path belonging to a given session is listed here by its source and
42745b6a3dSJack Wang		destination address. When a new path is added to a session by writing to
43745b6a3dSJack Wang		the "add_path" entry, a directory <src@dst> is created.
44745b6a3dSJack Wang
45745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/state
46745b6a3dSJack WangDate:		Feb 2020
47745b6a3dSJack WangKernelVersion:	5.7
48745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
49745b6a3dSJack WangDescription:	RO, Contains "connected" if the session is connected to the peer and fully
50745b6a3dSJack Wang		functional.  Otherwise the file contains "disconnected"
51745b6a3dSJack Wang
52745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/reconnect
53745b6a3dSJack WangDate:		Feb 2020
54745b6a3dSJack WangKernelVersion:	5.7
55745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
56745b6a3dSJack WangDescription:	Write "1" to the file in order to reconnect the path.
57745b6a3dSJack Wang		Operation is blocking and returns 0 if reconnect was successful.
58745b6a3dSJack Wang
59745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/disconnect
60745b6a3dSJack WangDate:		Feb 2020
61745b6a3dSJack WangKernelVersion:	5.7
62745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
63745b6a3dSJack WangDescription:	Write "1" to the file in order to disconnect the path.
64745b6a3dSJack Wang		Operation blocks until RTRS path is disconnected.
65745b6a3dSJack Wang
66745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/remove_path
67745b6a3dSJack WangDate:		Feb 2020
68745b6a3dSJack WangKernelVersion:	5.7
69745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
70745b6a3dSJack WangDescription:	Write "1" to the file in order to disconnected and remove the path
71745b6a3dSJack Wang		from the session.  Operation blocks until the path is disconnected
72745b6a3dSJack Wang		and removed from the session.
73745b6a3dSJack Wang
74745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_name
75745b6a3dSJack WangDate:		Feb 2020
76745b6a3dSJack WangKernelVersion:	5.7
77745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
78745b6a3dSJack WangDescription:	RO, Contains the the name of HCA the connection established on.
79745b6a3dSJack Wang
80745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_port
81745b6a3dSJack WangDate:		Feb 2020
82745b6a3dSJack WangKernelVersion:	5.7
83745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
84745b6a3dSJack WangDescription:	RO, Contains the port number of active port traffic is going through.
85745b6a3dSJack Wang
86745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/src_addr
87745b6a3dSJack WangDate:		Feb 2020
88745b6a3dSJack WangKernelVersion:	5.7
89745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
90745b6a3dSJack WangDescription:	RO, Contains the source address of the path
91745b6a3dSJack Wang
92745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/dst_addr
93745b6a3dSJack WangDate:		Feb 2020
94745b6a3dSJack WangKernelVersion:	5.7
95745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
96745b6a3dSJack WangDescription:	RO, Contains the destination address of the path
97745b6a3dSJack Wang
98745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reset_all
99745b6a3dSJack WangDate:		Feb 2020
100745b6a3dSJack WangKernelVersion:	5.7
101745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
102745b6a3dSJack WangDescription:	RW, Read will return usage help, write 0 will clear all the statistics.
103745b6a3dSJack Wang
104745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/cpu_migration
105745b6a3dSJack WangDate:		Feb 2020
106745b6a3dSJack WangKernelVersion:	5.7
107745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
108745b6a3dSJack WangDescription:	RTRS expects that each HCA IRQ is pinned to a separate CPU. If it's
109745b6a3dSJack Wang		not the case, the processing of an I/O response could be processed on a
110745b6a3dSJack Wang		different CPU than where it was originally submitted.  This file shows
111745b6a3dSJack Wang		how many interrupts where generated on a non expected CPU.
112745b6a3dSJack Wang		"from:" is the CPU on which the IRQ was expected, but not generated.
113745b6a3dSJack Wang		"to:" is the CPU on which the IRQ was generated, but not expected.
114745b6a3dSJack Wang
115745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reconnects
116745b6a3dSJack WangDate:		Feb 2020
117745b6a3dSJack WangKernelVersion:	5.7
118745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
119745b6a3dSJack WangDescription:	Contains 2 unsigned int values, the first one records number of successful
120745b6a3dSJack Wang		reconnects in the path lifetime, the second one records number of failed
121745b6a3dSJack Wang		reconnects in the path lifetime.
122745b6a3dSJack Wang
123745b6a3dSJack WangWhat:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/rdma
124745b6a3dSJack WangDate:		Feb 2020
125745b6a3dSJack WangKernelVersion:	5.7
126745b6a3dSJack WangContact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
127745b6a3dSJack WangDescription:	Contains statistics regarding rdma operations and inflight operations.
128745b6a3dSJack Wang		The output consists of 6 values:
129745b6a3dSJack Wang
130745b6a3dSJack Wang		<read-count> <read-total-size> <write-count> <write-total-size> \
131745b6a3dSJack Wang		<inflights> <failovered>
132