1========================================= 2user_events: User-based Event Tracing 3========================================= 4 5:Author: Beau Belgrave 6 7Overview 8-------- 9User based trace events allow user processes to create events and trace data 10that can be viewed via existing tools, such as ftrace, perf and eBPF. 11To enable this feature, build your kernel with CONFIG_USER_EVENTS=y. 12 13Programs can view status of the events via 14/sys/kernel/debug/tracing/user_events_status and can both register and write 15data out via /sys/kernel/debug/tracing/user_events_data. 16 17Programs can also use /sys/kernel/debug/tracing/dynamic_events to register and 18delete user based events via the u: prefix. The format of the command to 19dynamic_events is the same as the ioctl with the u: prefix applied. 20 21Typically programs will register a set of events that they wish to expose to 22tools that can read trace_events (such as ftrace and perf). The registration 23process gives back two ints to the program for each event. The first int is the 24status index. This index describes which byte in the 25/sys/kernel/debug/tracing/user_events_status file represents this event. The 26second int is the write index. This index describes the data when a write() or 27writev() is called on the /sys/kernel/debug/tracing/user_events_data file. 28 29The structures referenced in this document are contained with the 30/include/uap/linux/user_events.h file in the source tree. 31 32**NOTE:** *Both user_events_status and user_events_data are under the tracefs 33filesystem and may be mounted at different paths than above.* 34 35Registering 36----------- 37Registering within a user process is done via ioctl() out to the 38/sys/kernel/debug/tracing/user_events_data file. The command to issue is 39DIAG_IOCSREG. 40 41This command takes a struct user_reg as an argument:: 42 43 struct user_reg { 44 u32 size; 45 u64 name_args; 46 u32 status_index; 47 u32 write_index; 48 }; 49 50The struct user_reg requires two inputs, the first is the size of the structure 51to ensure forward and backward compatibility. The second is the command string 52to issue for registering. Upon success two outputs are set, the status index 53and the write index. 54 55User based events show up under tracefs like any other event under the 56subsystem named "user_events". This means tools that wish to attach to the 57events need to use /sys/kernel/debug/tracing/events/user_events/[name]/enable 58or perf record -e user_events:[name] when attaching/recording. 59 60**NOTE:** *The write_index returned is only valid for the FD that was used* 61 62Command Format 63^^^^^^^^^^^^^^ 64The command string format is as follows:: 65 66 name[:FLAG1[,FLAG2...]] [Field1[;Field2...]] 67 68Supported Flags 69^^^^^^^^^^^^^^^ 70**BPF_ITER** - EBPF programs attached to this event will get the raw iovec 71struct instead of any data copies for max performance. 72 73Field Format 74^^^^^^^^^^^^ 75:: 76 77 type name [size] 78 79Basic types are supported (__data_loc, u32, u64, int, char, char[20], etc). 80User programs are encouraged to use clearly sized types like u32. 81 82**NOTE:** *Long is not supported since size can vary between user and kernel.* 83 84The size is only valid for types that start with a struct prefix. 85This allows user programs to describe custom structs out to tools, if required. 86 87For example, a struct in C that looks like this:: 88 89 struct mytype { 90 char data[20]; 91 }; 92 93Would be represented by the following field:: 94 95 struct mytype myname 20 96 97Deleting 98----------- 99Deleting an event from within a user process is done via ioctl() out to the 100/sys/kernel/debug/tracing/user_events_data file. The command to issue is 101DIAG_IOCSDEL. 102 103This command only requires a single string specifying the event to delete by 104its name. Delete will only succeed if there are no references left to the 105event (in both user and kernel space). User programs should use a separate file 106to request deletes than the one used for registration due to this. 107 108Status 109------ 110When tools attach/record user based events the status of the event is updated 111in realtime. This allows user programs to only incur the cost of the write() or 112writev() calls when something is actively attached to the event. 113 114User programs call mmap() on /sys/kernel/debug/tracing/user_events_status to 115check the status for each event that is registered. The byte to check in the 116file is given back after the register ioctl() via user_reg.status_index. 117Currently the size of user_events_status is a single page, however, custom 118kernel configurations can change this size to allow more user based events. In 119all cases the size of the file is a multiple of a page size. 120 121For example, if the register ioctl() gives back a status_index of 3 you would 122check byte 3 of the returned mmap data to see if anything is attached to that 123event. 124 125Administrators can easily check the status of all registered events by reading 126the user_events_status file directly via a terminal. The output is as follows:: 127 128 Byte:Name [# Comments] 129 ... 130 131 Active: ActiveCount 132 Busy: BusyCount 133 Max: MaxCount 134 135For example, on a system that has a single event the output looks like this:: 136 137 1:test 138 139 Active: 1 140 Busy: 0 141 Max: 4096 142 143If a user enables the user event via ftrace, the output would change to this:: 144 145 1:test # Used by ftrace 146 147 Active: 1 148 Busy: 1 149 Max: 4096 150 151**NOTE:** *A status index of 0 will never be returned. This allows user 152programs to have an index that can be used on error cases.* 153 154Status Bits 155^^^^^^^^^^^ 156The byte being checked will be non-zero if anything is attached. Programs can 157check specific bits in the byte to see what mechanism has been attached. 158 159The following values are defined to aid in checking what has been attached: 160 161**EVENT_STATUS_FTRACE** - Bit set if ftrace has been attached (Bit 0). 162 163**EVENT_STATUS_PERF** - Bit set if perf/eBPF has been attached (Bit 1). 164 165Writing Data 166------------ 167After registering an event the same fd that was used to register can be used 168to write an entry for that event. The write_index returned must be at the start 169of the data, then the remaining data is treated as the payload of the event. 170 171For example, if write_index returned was 1 and I wanted to write out an int 172payload of the event. Then the data would have to be 8 bytes (2 ints) in size, 173with the first 4 bytes being equal to 1 and the last 4 bytes being equal to the 174value I want as the payload. 175 176In memory this would look like this:: 177 178 int index; 179 int payload; 180 181User programs might have well known structs that they wish to use to emit out 182as payloads. In those cases writev() can be used, with the first vector being 183the index and the following vector(s) being the actual event payload. 184 185For example, if I have a struct like this:: 186 187 struct payload { 188 int src; 189 int dst; 190 int flags; 191 }; 192 193It's advised for user programs to do the following:: 194 195 struct iovec io[2]; 196 struct payload e; 197 198 io[0].iov_base = &write_index; 199 io[0].iov_len = sizeof(write_index); 200 io[1].iov_base = &e; 201 io[1].iov_len = sizeof(e); 202 203 writev(fd, (const struct iovec*)io, 2); 204 205**NOTE:** *The write_index is not emitted out into the trace being recorded.* 206 207EBPF 208---- 209EBPF programs that attach to a user-based event tracepoint are given a pointer 210to a struct user_bpf_context. The bpf context contains the data type (which can 211be a user or kernel buffer, or can be a pointer to the iovec) and the data 212length that was emitted (minus the write_index). 213 214Example Code 215------------ 216See sample code in samples/user_events. 217