1====================
2Userspace MAD access
3====================
4
5Device files
6============
7
8  Each port of each InfiniBand device has a "umad" device and an
9  "issm" device attached.  For example, a two-port HCA will have two
10  umad devices and two issm devices, while a switch will have one
11  device of each type (for switch port 0).
12
13Creating MAD agents
14===================
15
16  A MAD agent can be created by filling in a struct ib_user_mad_reg_req
17  and then calling the IB_USER_MAD_REGISTER_AGENT ioctl on a file
18  descriptor for the appropriate device file.  If the registration
19  request succeeds, a 32-bit id will be returned in the structure.
20  For example::
21
22	struct ib_user_mad_reg_req req = { /* ... */ };
23	ret = ioctl(fd, IB_USER_MAD_REGISTER_AGENT, (char *) &req);
24        if (!ret)
25		my_agent = req.id;
26	else
27		perror("agent register");
28
29  Agents can be unregistered with the IB_USER_MAD_UNREGISTER_AGENT
30  ioctl.  Also, all agents registered through a file descriptor will
31  be unregistered when the descriptor is closed.
32
33  2014
34       a new registration ioctl is now provided which allows additional
35       fields to be provided during registration.
36       Users of this registration call are implicitly setting the use of
37       pkey_index (see below).
38
39Receiving MADs
40==============
41
42  MADs are received using read().  The receive side now supports
43  RMPP. The buffer passed to read() must be at least one
44  struct ib_user_mad + 256 bytes. For example:
45
46  If the buffer passed is not large enough to hold the received
47  MAD (RMPP), the errno is set to ENOSPC and the length of the
48  buffer needed is set in mad.length.
49
50  Example for normal MAD (non RMPP) reads::
51
52	struct ib_user_mad *mad;
53	mad = malloc(sizeof *mad + 256);
54	ret = read(fd, mad, sizeof *mad + 256);
55	if (ret != sizeof mad + 256) {
56		perror("read");
57		free(mad);
58	}
59
60  Example for RMPP reads::
61
62	struct ib_user_mad *mad;
63	mad = malloc(sizeof *mad + 256);
64	ret = read(fd, mad, sizeof *mad + 256);
65	if (ret == -ENOSPC)) {
66		length = mad.length;
67		free(mad);
68		mad = malloc(sizeof *mad + length);
69		ret = read(fd, mad, sizeof *mad + length);
70	}
71	if (ret < 0) {
72		perror("read");
73		free(mad);
74	}
75
76  In addition to the actual MAD contents, the other struct ib_user_mad
77  fields will be filled in with information on the received MAD.  For
78  example, the remote LID will be in mad.lid.
79
80  If a send times out, a receive will be generated with mad.status set
81  to ETIMEDOUT.  Otherwise when a MAD has been successfully received,
82  mad.status will be 0.
83
84  poll()/select() may be used to wait until a MAD can be read.
85
86Sending MADs
87============
88
89  MADs are sent using write().  The agent ID for sending should be
90  filled into the id field of the MAD, the destination LID should be
91  filled into the lid field, and so on.  The send side does support
92  RMPP so arbitrary length MAD can be sent. For example::
93
94	struct ib_user_mad *mad;
95
96	mad = malloc(sizeof *mad + mad_length);
97
98	/* fill in mad->data */
99
100	mad->hdr.id  = my_agent;	/* req.id from agent registration */
101	mad->hdr.lid = my_dest;		/* in network byte order... */
102	/* etc. */
103
104	ret = write(fd, &mad, sizeof *mad + mad_length);
105	if (ret != sizeof *mad + mad_length)
106		perror("write");
107
108Transaction IDs
109===============
110
111  Users of the umad devices can use the lower 32 bits of the
112  transaction ID field (that is, the least significant half of the
113  field in network byte order) in MADs being sent to match
114  request/response pairs.  The upper 32 bits are reserved for use by
115  the kernel and will be overwritten before a MAD is sent.
116
117P_Key Index Handling
118====================
119
120  The old ib_umad interface did not allow setting the P_Key index for
121  MADs that are sent and did not provide a way for obtaining the P_Key
122  index of received MADs.  A new layout for struct ib_user_mad_hdr
123  with a pkey_index member has been defined; however, to preserve binary
124  compatibility with older applications, this new layout will not be used
125  unless one of IB_USER_MAD_ENABLE_PKEY or IB_USER_MAD_REGISTER_AGENT2 ioctl's
126  are called before a file descriptor is used for anything else.
127
128  In September 2008, the IB_USER_MAD_ABI_VERSION will be incremented
129  to 6, the new layout of struct ib_user_mad_hdr will be used by
130  default, and the IB_USER_MAD_ENABLE_PKEY ioctl will be removed.
131
132Setting IsSM Capability Bit
133===========================
134
135  To set the IsSM capability bit for a port, simply open the
136  corresponding issm device file.  If the IsSM bit is already set,
137  then the open call will block until the bit is cleared (or return
138  immediately with errno set to EAGAIN if the O_NONBLOCK flag is
139  passed to open()).  The IsSM bit will be cleared when the issm file
140  is closed.  No read, write or other operations can be performed on
141  the issm file.
142
143/dev files
144==========
145
146  To create the appropriate character device files automatically with
147  udev, a rule like::
148
149    KERNEL=="umad*", NAME="infiniband/%k"
150    KERNEL=="issm*", NAME="infiniband/%k"
151
152  can be used.  This will create device nodes named::
153
154    /dev/infiniband/umad0
155    /dev/infiniband/issm0
156
157  for the first port, and so on.  The InfiniBand device and port
158  associated with these devices can be determined from the files::
159
160    /sys/class/infiniband_mad/umad0/ibdev
161    /sys/class/infiniband_mad/umad0/port
162
163  and::
164
165    /sys/class/infiniband_mad/issm0/ibdev
166    /sys/class/infiniband_mad/issm0/port
167