perf auxtrace: Uninline functions that touch perf_session

So that we don't carry the session.h include directive in auxtrace.h,
which in turn opens a can of worms of files that were gett

perf auxtrace: Uninline functions that touch perf_session

So that we don't carry the session.h include directive in auxtrace.h,
which in turn opens a can of worms of files that were getting all sorts
of things via that include, fix them all.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: https://lkml.kernel.org/n/tip-d2d83aovpgri2z75wlitquni@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf test: Auto bump rlimit(MEMLOCK) for BPF test sake

I noticed that the 'perf test bpf' was failing:

# perf test bpf
41: BPF filter

perf test: Auto bump rlimit(MEMLOCK) for BPF test sake

I noticed that the 'perf test bpf' was failing:

# perf test bpf
41: BPF filter :
41.1: Basic BPF filtering : Skip
41.2: BPF pinning : Skip
41.3: BPF prologue generation : Skip
41.4: BPF relocation checker : Skip
# ulimit -l
64
#

Using verbose mode we get just a line bout -EPERF being returned from
libbpf's bpf_load_program_xattr(), that ends up being used in 'perf
test bpf' initial program loading capability query:

Missing basic BPF support, skip this test: Operation not permitted

Not that informative, but on a separate problem when creating BPF maps
bumping rlimit(MEMLOCK) helped, so I tried it here as well, works:

# ulimit -l 128
# perf test bpf
41: BPF filter :
41.1: Basic BPF filtering : Ok
41.2: BPF pinning : Ok
41.3: BPF prologue generation : Ok
41.4: BPF relocation checker : Ok
#

So use the recently added rlimit__bump_memlock() helper:

# ulimit -l 64
# perf test bpf
41: BPF filter :
41.1: Basic BPF filtering : Ok
41.2: BPF pinning : Ok
41.3: BPF prologue generation : Ok
41.4: BPF relocation checker : Ok
# ulimit -l
64
#

I.e. the bumping of memlock is restricted to the 'perf test' instance,
not changing the global value.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: https://lkml.kernel.org/n/tip-b9fubkhr4jm192lu7y8hgjvo@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf tools: Remove trim() implementation, use tools/lib's strim()

Moving more stuff out of tools/perf/util/ and using the kernel idiom.

Cc: Adrian Hunter <adrian.hunter@intel.com>

perf tools: Remove trim() implementation, use tools/lib's strim()

Moving more stuff out of tools/perf/util/ and using the kernel idiom.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: https://lkml.kernel.org/n/tip-wpj8rktj62yse5dq6ckny6de@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf tests: Add a test for time-utils

Test time ranges work as expected.

Committer testing:

$ perf test "time utils"
59: time utils

perf tests: Add a test for time-utils

Test time ranges work as expected.

Committer testing:

$ perf test "time utils"
59: time utils : Ok
$ perf test -v "time utils"
59: time utils :
--- start ---
test child forked, pid 31711

parse_nsec_time("0")
0

parse_nsec_time("1")
1000000000

parse_nsec_time("0.000000001")
1

parse_nsec_time("1.000000001")
1000000001

parse_nsec_time("123456.123456")
123456123456000

parse_nsec_time("1234567.123456789")
1234567123456789

parse_nsec_time("18446744073.709551615")
18446744073709551615

perf_time__parse_str("1234567.123456789,1234567.123456789")
start time 1234567123456789, end time 1234567123456789

perf_time__parse_str("1234567.123456789,1234567.123456790")
start time 1234567123456789, end time 1234567123456790

perf_time__parse_str("1234567.123456789,")
start time 1234567123456789, end time 0

perf_time__parse_str(",1234567.123456789")
start time 0, end time 1234567123456789

perf_time__parse_str("0,1234567.123456789")
start time 0, end time 1234567123456789

perf_time__parse_for_ranges("1234567.123456789,1234567.123456790")
start time 1234567123456789, end time 1234567123456790

perf_time__parse_for_ranges("10%/1")
first_sample_time 7654321000000000 last_sample_time 7654321000000100
start time 0: 7654321000000000, end time 0: 7654321000000009

perf_time__parse_for_ranges("10%/2")
first_sample_time 7654321000000000 last_sample_time 7654321000000100
start time 0: 7654321000000010, end time 0: 7654321000000019

perf_time__parse_for_ranges("10%/1,10%/2")
first_sample_time 11223344000000000 last_sample_time 11223344000000100
start time 0: 11223344000000000, end time 0: 11223344000000009
start time 1: 11223344000000010, end time 1: 11223344000000019

perf_time__parse_for_ranges("10%/1,10%/3,10%/10")
first_sample_time 11223344000000000 last_sample_time 11223344000000100
start time 0: 11223344000000000, end time 0: 11223344000000009
start time 1: 11223344000000020, end time 1: 11223344000000029
start time 2: 11223344000000090, end time 2: 11223344000000100

test child finished with 0
---- end ----
time utils: Ok
$

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jin Yao <yao.jin@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20190604130017.31207-19-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf tests: Add map_groups__merge_in test

Add map_groups__merge_in test to test the map_groups__merge_in function
usage - merging kcore maps into existing eBPF maps.

Committer t

perf tests: Add map_groups__merge_in test

Add map_groups__merge_in test to test the map_groups__merge_in function
usage - merging kcore maps into existing eBPF maps.

Committer testing:

# perf test merge
59: map_groups__merge_in : Ok
# perf test -v merge
59: map_groups__merge_in :
--- start ---
test child forked, pid 8349
test child finished with 0
---- end ----
map_groups__merge_in: Ok
#

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Song Liu <songliubraving@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stanislav Fomichev <sdf@google.com>
Link: http://lkml.kernel.org/r/20190508132010.14512-10-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf tests: Use shebangs in the shell scripts

Since the first line was used as a test identification, it needs to be
skipped by shell_test__description() function now.

Further n

perf tests: Use shebangs in the shell scripts

Since the first line was used as a test identification, it needs to be
skipped by shell_test__description() function now.

Further notes from Hendrik:

It might be worth to note that adding the shebang is necessary to spot
them as scripts.

Using /bin/sh looks fine to. Just briefly checked whether the scripts
contains some bash-specifics, which is not the case.

Signed-off-by: Michael Petlan <mpetlan@redhat.com>
Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com>
Cc: Jiri Olsa <jolsa@redhat.com>
LPU-Reference: 2127419430.57657104.1542836358464.JavaMail.zimbra@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf test: S390 does not support watchpoints in test 22

S390 does not support the perf_event_open system call for
attribute type PERF_TYPE_BREAKPOINT. This results in test
failure fo

perf test: S390 does not support watchpoints in test 22

S390 does not support the perf_event_open system call for
attribute type PERF_TYPE_BREAKPOINT. This results in test
failure for test 22:

[root@s8360046 perf]# ./perf test 22
22: Watchpoint :
22.1: Read Only Watchpoint : FAILED!
22.2: Write Only Watchpoint : FAILED!
22.3: Read / Write Watchpoint : FAILED!
22.4: Modify Watchpoint : FAILED!
[root@s8360046 perf]#

Add s390 support to avoid these tests being executed on
s390 platform:

[root@s8360046 perf]# ./perf test 22
[root@s8360046 perf]# ./perf test -v 22
22: Watchpoint : Disabled
[root@s8360046 perf]#

Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Ravi Bangoria <ravi.bangoria@linux.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Hendrik Brueckner <brueckner@linux.vnet.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Link: http://lkml.kernel.org/r/20180928105335.67179-1-tmricht@linux.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf test: Add watchpoint test

We don't have a 'perf test' entry available to test the watchpoint
functionality.

Add a simple set of tests:

- Read only watchpoint

perf test: Add watchpoint test

We don't have a 'perf test' entry available to test the watchpoint
functionality.

Add a simple set of tests:

- Read only watchpoint
- Write only watchpoint
- Read / Write watchpoint
- Runtime watchpoint modification

Ex.: on powerpc:

$ sudo perf test 22
22: Watchpoint :
22.1: Read Only Watchpoint : Ok
22.2: Write Only Watchpoint : Ok
22.3: Read / Write Watchpoint : Ok
22.4: Modify Watchpoint : Ok

Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.ibm.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Hendrik Brueckner <brueckner@linux.ibm.com>
Cc: Kate Stewart <kstewart@linuxfoundation.org>
Cc: Kim Phillips <kim.phillips@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Cc: Sandipan Das <sandipan@linux.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Thomas Richter <tmricht@linux.ibm.com>
Link: http://lkml.kernel.org/r/20180912061229.22832-1-ravi.bangoria@linux.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf tests: Fix indexing when invoking subtests

Recently, the subtest numbering was changed to start from 1. While it
is fine for displaying results, this should not be the case when th

perf tests: Fix indexing when invoking subtests

Recently, the subtest numbering was changed to start from 1. While it
is fine for displaying results, this should not be the case when the
subtests are actually invoked.

Typically, the subtests are stored in zero-indexed arrays and invoked
based on the index passed to the main test function. Since the index
now starts from 1, the second subtest in the array (index 1) gets
invoked instead of the first (index 0). This applies to all of the
following subtests but for the last one, the subtest always fails
because it does not meet the boundary condition of the subtest index
being lesser than the number of subtests.

This can be observed on powerpc64 and x86_64 systems running Fedora 28
as shown below.

Before:

# perf test "builtin clang support"
55: builtin clang support :
55.1: builtin clang compile C source to IR : Ok
55.2: builtin clang compile C source to ELF object : FAILED!

# perf test "LLVM search and compile"
38: LLVM search and compile :
38.1: Basic BPF llvm compile : Ok
38.2: kbuild searching : Ok
38.3: Compile source for BPF prologue generation : Ok
38.4: Compile source for BPF relocation : FAILED!

# perf test "BPF filter"
40: BPF filter :
40.1: Basic BPF filtering : Ok
40.2: BPF pinning : Ok
40.3: BPF prologue generation : Ok
40.4: BPF relocation checker : FAILED!

After:

# perf test "builtin clang support"
55: builtin clang support :
55.1: builtin clang compile C source to IR : Ok
55.2: builtin clang compile C source to ELF object : Ok

# perf test "LLVM search and compile"
38: LLVM search and compile :
38.1: Basic BPF llvm compile : Ok
38.2: kbuild searching : Ok
38.3: Compile source for BPF prologue generation : Ok
38.4: Compile source for BPF relocation : Ok

# perf test "BPF filter"
40: BPF filter :
40.1: Basic BPF filtering : Ok
40.2: BPF pinning : Ok
40.3: BPF prologue generation : Ok
40.4: BPF relocation checker : Ok

Signed-off-by: Sandipan Das <sandipan@linux.ibm.com>
Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Hendrik Brueckner <brueckner@linux.ibm.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Cc: Ravi Bangoria <ravi.bangoria@linux.ibm.com>
Cc: Thomas Richter <tmricht@linux.ibm.com>
Fixes: 9ef0112442bd ("perf test: Fix subtest number when showing results")
Link: http://lkml.kernel.org/r/20180726171733.33208-1-sandipan@linux.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf test: Fix subtest number when showing results

Perf test 40 for example has several subtests numbered 1-4 when
displaying the start of the subtest. When the subtest results
are d

perf test: Fix subtest number when showing results

Perf test 40 for example has several subtests numbered 1-4 when
displaying the start of the subtest. When the subtest results
are displayed the subtests are numbered 0-3.

Use this command to generate trace output:

[root@s35lp76 perf]# ./perf test -Fv 40 2>/tmp/bpf1

Fix this by adjusting the subtest number when show the
subtest result.

Output before:

[root@s35lp76 perf]# egrep '(^40\.[0-4]| subtest [0-4]:)' /tmp/bpf1
40.1: Basic BPF filtering :
BPF filter subtest 0: Ok
40.2: BPF pinning :
BPF filter subtest 1: Ok
40.3: BPF prologue generation :
BPF filter subtest 2: Ok
40.4: BPF relocation checker :
BPF filter subtest 3: Ok
[root@s35lp76 perf]#

Output after:

root@s35lp76 ~]# egrep '(^40\.[0-4]| subtest [0-4]:)' /tmp/bpf1
40.1: Basic BPF filtering :
BPF filter subtest 1: Ok
40.2: BPF pinning :
BPF filter subtest 2: Ok
40.3: BPF prologue generation :
BPF filter subtest 3: Ok
40.4: BPF relocation checker :
BPF filter subtest 4: Ok
[root@s35lp76 ~]#

Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Link: http://lkml.kernel.org/r/20180724134858.100644-1-tmricht@linux.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf test shell: Prevent temporary editor files from being considered test scripts

Allows a perf shell test developer to concurrently edit and run their
test scripts, avoiding perf test

perf test shell: Prevent temporary editor files from being considered test scripts

Allows a perf shell test developer to concurrently edit and run their
test scripts, avoiding perf test attempts to execute their editor
temporary files, such as seen here:

$ sudo taskset -c 0 ./perf test -vvvvvvvv -F 63
63: 0VIM 8.0 :
--- start ---
sh: 1: ./tests/shell/.record+probe_libc_inet_pton.sh.swp: Permission denied
---- end ----
0VIM 8.0: FAILED!

Signed-off-by: Kim Phillips <kim.phillips@arm.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Hendrik Brueckner <brueckner@linux.vnet.ibm.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Michael Petlan <mpetlan@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sandipan Das <sandipan@linux.vnet.ibm.com>
Cc: Thomas Richter <tmricht@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/20180629124658.15a506b41fc4539c08eb9426@arm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf tests: Let 'perf test list' display subtests

The output of perf test and perf test list differ because perf test list
does not display subtests. Correct this behavior and also let

perf tests: Let 'perf test list' display subtests

The output of perf test and perf test list differ because perf test list
does not display subtests. Correct this behavior and also let perf test
list report subtests.

For example:

$ ./perf test 2>&1 |wc -l
65

Without this commit:
$ ./perf test list 2>&1 |wc -l
57

With this commit:
$ ./perf test list 2>&1 |wc -l
65

Signed-off-by: Hendrik Brueckner <brueckner@linux.ibm.com>
Reviewed-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Michael Petlan <mpetlan@redhat.com>
Cc: linux-s390@vger.kernel.org
LPU-Reference: 1523605343-11970-1-git-send-email-brueckner@linux.ibm.com
Link: https://lkml.kernel.org/n/tip-efb74jw7x2xs2bucp5hf4ilu@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf tests: Disable breakpoint accounting test for powerpc

We disable this test as instruction breakpoints (HW_BREAKPOINT_X) are
not available for powerpc.

Before applying patch

perf tests: Disable breakpoint accounting test for powerpc

We disable this test as instruction breakpoints (HW_BREAKPOINT_X) are
not available for powerpc.

Before applying patch:

21: Breakpoint accounting :
--- start ---
test child forked, pid 3635
failed opening event 0
failed opening event 0
watchpoints count 1, breakpoints count 0, has_ioctl 1, share 0
test child finished with -2
---- end ----
Breakpoint accounting: Skip

After applying patch:

21: Breakpoint accounting : Disabled

Signed-off-by: Sandipan Das <sandipan@linux.vnet.ibm.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Cc: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/20180412162140.2992-1-sandipan@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf tests: Add mem2node object test

Adding mem2node object automated test.

The test prepares few artificial nodes - memory maps and verifies the
mem2node object returns proper

perf tests: Add mem2node object test

Adding mem2node object automated test.

The test prepares few artificial nodes - memory maps and verifies the
mem2node object returns proper node values to given addresses.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20180309101442.9224-4-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf tests: Add breakpoint accounting/modify test

Adding test that:

- detects the number of watch/break-points,
skip test if any is missing
- detects PERF_EVENT_IOC_

perf tests: Add breakpoint accounting/modify test

Adding test that:

- detects the number of watch/break-points,
skip test if any is missing
- detects PERF_EVENT_IOC_MODIFY_ATTRIBUTES ioctl,
skip test if it's missing
- detects if watchpoints and breakpoints share
same slots
- create all possible watchpoints on cpu 0
- change one of it to breakpoint
- in case wp and bp do not share slots,
we create another watchpoint to ensure
the slot accounting is correct

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Hari Bathini <hbathini@linux.vnet.ibm.com>
Cc: Jin Yao <yao.jin@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Kan Liang <kan.liang@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Milind Chabbi <chabbi.milind@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Oleg Nesterov <onestero@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Will Deacon <will.deacon@arm.com>
Link: http://lkml.kernel.org/r/20180312134548.31532-9-jolsa@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>

show more ...

perf test: Handle properly readdir DT_UNKNOWN

Some system can return DT_UNKNOWN in readdir's struct dirent::d_type and
we must handle it properly. In this case we can directly check if t

perf test: Handle properly readdir DT_UNKNOWN

Some system can return DT_UNKNOWN in readdir's struct dirent::d_type and
we must handle it properly. In this case we can directly check if the
entity we found is directory and skip it.

Reported-by: Michael Petlan <mpetlan@redhat.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20171206174535.25380-1-jolsa@kernel.org
[ Split from a larger patch ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

Merge branch 'linus' into perf/core, to fix conflicts

Conflicts:
tools/perf/arch/arm/annotate/instructions.c
tools/perf/arch/arm64/annotate/instructions.c

Merge branch 'linus' into perf/core, to fix conflicts

Conflicts:
tools/perf/arch/arm/annotate/instructions.c
tools/perf/arch/arm64/annotate/instructions.c
tools/perf/arch/powerpc/annotate/instructions.c
tools/perf/arch/s390/annotate/instructions.c
tools/perf/arch/x86/tests/intel-cqm.c
tools/perf/ui/tui/progress.c
tools/perf/util/zlib.c

Signed-off-by: Ingo Molnar <mingo@kernel.org>

show more ...

License cleanup: add SPDX GPL-2.0 license identifier to files with no license

Many source files in the tree are missing licensing information, which
makes it harder for compliance tools

License cleanup: add SPDX GPL-2.0 license identifier to files with no license

Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.

By default all files without license information are under the default
license of the kernel, which is GPL version 2.

Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.

This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.

How this work was done:

Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,

Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.

The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.

The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.

Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).

All documentation files were explicitly excluded.

The following heuristics were used to determine which SPDX license
identifiers to apply.

- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.

For non */uapi/* files that summary was:

SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139

and resulted in the first patch in this series.

If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:

SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930

and resulted in the second patch in this series.

- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:

SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1

and that resulted in the third patch in this series.

- when the two scanners agreed on the detected license(s), that became
the concluded license(s).

- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.

- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).

- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.

- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.

In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.

Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.

Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.

In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.

Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct

This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.

These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.

Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

perf tools: Make copyfile_offset() static

There are no usage outside util.c and this is the only remaining reason
for fcntl.h to be included in util.h, to get the loff_t definition in

perf tools: Make copyfile_offset() static

There are no usage outside util.c and this is the only remaining reason
for fcntl.h to be included in util.h, to get the loff_t definition in
Alpine Linux, so make it static.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-2dzlsao7k6ihozs5karw6kpx@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf test: Add test cases for new data source encoding

Add some simple tests to perf test to test data source printing.

v2: Make the tests actually checked for the correct name of F

perf test: Add test cases for new data source encoding

Add some simple tests to perf test to test data source printing.

v2: Make the tests actually checked for the correct name of Forward
v3: Adjust to new encoding

Committer notes:

Avoid the in place declaration to make this build with older compilers,
for instance, in Debian 7 we get:

tests/mem.c: In function 'test__mem':
tests/mem.c:30:5: error: missing initializer [-Werror=missing-field-initializers]
tests/mem.c:30:5: error: (near initialization for '(anonymous).<anonymous>.mem_snoop') [-Werror=missing-field-initializers]

So just zero a struct, then go on building the unions as needed,
reusing settings from the previous test, i.e. local -> remote, etc.

Signed-off-by: Andi Kleen <ak@linux.intel.com>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Link: http://lkml.kernel.org/r/20170816222156.19953-5-andi@firstfloor.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf test: Make 'list' use same filtering code as main 'perf test'

Before:

# perf test Synth
39: Synthesize thread map : Ok
41: Synthesize cpu map : Ok
42:

perf test: Make 'list' use same filtering code as main 'perf test'

Before:

# perf test Synth
39: Synthesize thread map : Ok
41: Synthesize cpu map : Ok
42: Synthesize stat config : Ok
43: Synthesize stat : Ok
44: Synthesize stat round : Ok
45: Synthesize attr update : Ok
# perf test list Synth
#

After:

# perf test Synth
39: Synthesize thread map : Ok
41: Synthesize cpu map : Ok
42: Synthesize stat config : Ok
43: Synthesize stat : Ok
44: Synthesize stat round : Ok
45: Synthesize attr update : Ok
# perf test list Synth
39: Synthesize thread map
41: Synthesize cpu map
42: Synthesize stat config
43: Synthesize stat
44: Synthesize stat round
45: Synthesize attr update
#

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Michael Petlan <mpetlan@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Thomas Richter <tmricht@linux.vnet.ibm.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-v95tqqzuwawsmds3zn2mosje@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf test: Add infrastructure to run shell based tests

To allow testing by directly using perf tools in scripts, checking that
the effects on the system are the ones expected and that th

perf test: Add infrastructure to run shell based tests

To allow testing by directly using perf tools in scripts, checking that
the effects on the system are the ones expected and that the output
produced is as well the desired one.

For instance, adding a probe at a well known location with 'perf probe',
then checking that the results from using that probe to record are the
desired ones, etc.

The next csets will introduce tests using this new testing
infrastructure.

The scripts should return 0 for Ok, 1 for FAIL and 2 for SKIP.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Michael Petlan <mpetlan@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Thomas Richter <tmricht@linux.vnet.ibm.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-swbpn7amrjqffh83lsr39s9p@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf test: Add 'struct test *' to the test functions

This way we'll be able to pass more test specific parameters without
having to change this function signature.

Will be used

perf test: Add 'struct test *' to the test functions

This way we'll be able to pass more test specific parameters without
having to change this function signature.

Will be used by the upcoming 'shell tests', shell scripts that will
call perf tools and check if they work as expected, comparing its
effects on the system (think 'perf probe foo') the output produced, etc.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Michael Petlan <mpetlan@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Thomas Richter <tmricht@linux.vnet.ibm.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-wq250w7j1opbzyiynozuajbl@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf test: Make 'list' subcommand match main 'perf test' numbering/matching

Before:

# perf test Synth
39: Synthesize thread map : Ok
41: Synthesize cpu map : Ok

perf test: Make 'list' subcommand match main 'perf test' numbering/matching

Before:

# perf test Synth
39: Synthesize thread map : Ok
41: Synthesize cpu map : Ok
42: Synthesize stat config : Ok
43: Synthesize stat : Ok
44: Synthesize stat round : Ok
45: Synthesize attr update : Ok
#
# perf test list Synth
1: Synthesize thread map
2: Synthesize cpu map
3: Synthesize stat config
4: Synthesize stat
5: Synthesize stat round
6: Synthesize attr update
#

After:

# perf test Synth
39: Synthesize thread map : Ok
41: Synthesize cpu map : Ok
42: Synthesize stat config : Ok
43: Synthesize stat : Ok
44: Synthesize stat round : Ok
45: Synthesize attr update : Ok
#
# perf test list Synth
39: Synthesize thread map
41: Synthesize cpu map
42: Synthesize stat config
43: Synthesize stat
44: Synthesize stat round
45: Synthesize attr update
#

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Matt Fleming <matt.fleming@intel.com>
Cc: Michael Petlan <mpetlan@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Thomas Richter <tmricht@linux.vnet.ibm.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-pjhuhkphs7o3tkbqrukfv6bz@git.kernel.org
Fixes: e8210cefb7e1 ("perf tests: Introduce iterator function for tests")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

perf test: Disable breakpoint signal tests for powerpc

The following tests are failing on powerpc:

# perf test break
18: Breakpoint overflow signal handler : FAILED!

perf test: Disable breakpoint signal tests for powerpc

The following tests are failing on powerpc:

# perf test break
18: Breakpoint overflow signal handler : FAILED!
19: Breakpoint overflow sampling : FAILED!

The powerpc kenel so far does not have support to even create
instruction breakpoints using the perf event interface, so those tests
fail early in the config phase.

I added a '->is_supported()' callback to test struct to be able to
disable specific tests. It seems better than putting ifdefs directly to
the test array.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/20170601205450.GA398@krava
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...