xref: /openbmc/linux/tools/testing/ktest/sample.conf (revision 9900b5dc)
1a75fececSSteven Rostedt#
2a57419b3SSteven Rostedt# Config file for ktest.pl
3a75fececSSteven Rostedt#
4a75fececSSteven Rostedt# Note, all paths must be absolute
5a75fececSSteven Rostedt#
6a75fececSSteven Rostedt
7a57419b3SSteven Rostedt# Options set in the beginning of the file are considered to be
8a57419b3SSteven Rostedt# default options. These options can be overriden by test specific
9a57419b3SSteven Rostedt# options, with the following exceptions:
10a75fececSSteven Rostedt#
11a75fececSSteven Rostedt#  LOG_FILE
12a75fececSSteven Rostedt#  CLEAR_LOG
13a75fececSSteven Rostedt#  POWEROFF_ON_SUCCESS
14a75fececSSteven Rostedt#  REBOOT_ON_SUCCESS
15a75fececSSteven Rostedt#
16a57419b3SSteven Rostedt# Test specific options are set after the label:
17a57419b3SSteven Rostedt#
18a57419b3SSteven Rostedt# TEST_START
19a57419b3SSteven Rostedt#
20a57419b3SSteven Rostedt# The options after a TEST_START label are specific to that test.
21a57419b3SSteven Rostedt# Each TEST_START label will set up a new test. If you want to
22a57419b3SSteven Rostedt# perform a test more than once, you can add the ITERATE label
23a57419b3SSteven Rostedt# to it followed by the number of times you want that test
24a57419b3SSteven Rostedt# to iterate. If the ITERATE is left off, the test will only
25a57419b3SSteven Rostedt# be performed once.
26a57419b3SSteven Rostedt#
27a57419b3SSteven Rostedt# TEST_START ITERATE 10
28a57419b3SSteven Rostedt#
29a57419b3SSteven Rostedt# You can skip a test by adding SKIP (before or after the ITERATE
30a57419b3SSteven Rostedt# and number)
31a57419b3SSteven Rostedt#
32a57419b3SSteven Rostedt# TEST_START SKIP
33a57419b3SSteven Rostedt#
34a57419b3SSteven Rostedt# TEST_START SKIP ITERATE 10
35a57419b3SSteven Rostedt#
36a57419b3SSteven Rostedt# TEST_START ITERATE 10 SKIP
37a57419b3SSteven Rostedt#
38a57419b3SSteven Rostedt# The SKIP label causes the options and the test itself to be ignored.
39a57419b3SSteven Rostedt# This is useful to set up several different tests in one config file, and
40a57419b3SSteven Rostedt# only enabling the ones you want to use for a current test run.
41a57419b3SSteven Rostedt#
42a57419b3SSteven Rostedt# You can add default options anywhere in the file as well
43a57419b3SSteven Rostedt# with the DEFAULTS tag. This allows you to have default options
44a57419b3SSteven Rostedt# after the test options to keep the test options at the top
45a57419b3SSteven Rostedt# of the file. You can even place the DEFAULTS tag between
46a57419b3SSteven Rostedt# test cases (but not in the middle of a single test case)
47a57419b3SSteven Rostedt#
48a57419b3SSteven Rostedt# TEST_START
49a57419b3SSteven Rostedt# MIN_CONFIG = /home/test/config-test1
50a57419b3SSteven Rostedt#
51a57419b3SSteven Rostedt# DEFAULTS
52a57419b3SSteven Rostedt# MIN_CONFIG = /home/test/config-default
53a57419b3SSteven Rostedt#
54a57419b3SSteven Rostedt# TEST_START ITERATE 10
55a57419b3SSteven Rostedt#
56a57419b3SSteven Rostedt# The above will run the first test with MIN_CONFIG set to
57a57419b3SSteven Rostedt# /home/test/config-test-1. Then 10 tests will be executed
58a57419b3SSteven Rostedt# with MIN_CONFIG with /home/test/config-default.
59a57419b3SSteven Rostedt#
60a57419b3SSteven Rostedt# You can also disable defaults with the SKIP option
61a57419b3SSteven Rostedt#
62a57419b3SSteven Rostedt# DEFAULTS SKIP
63a57419b3SSteven Rostedt# MIN_CONFIG = /home/test/config-use-sometimes
64a57419b3SSteven Rostedt#
65a57419b3SSteven Rostedt# DEFAULTS
66a57419b3SSteven Rostedt# MIN_CONFIG = /home/test/config-most-times
67a57419b3SSteven Rostedt#
68a57419b3SSteven Rostedt# The above will ignore the first MIN_CONFIG. If you want to
69a57419b3SSteven Rostedt# use the first MIN_CONFIG, remove the SKIP from the first
70a57419b3SSteven Rostedt# DEFAULTS tag and add it to the second. Be careful, options
71a57419b3SSteven Rostedt# may only be declared once per test or default. If you have
72a57419b3SSteven Rostedt# the same option name under the same test or as default
73a57419b3SSteven Rostedt# ktest will fail to execute, and no tests will run.
74a57419b3SSteven Rostedt#
753d1cc414SSteven Rostedt# DEFAULTS OVERRIDE
763d1cc414SSteven Rostedt#
773d1cc414SSteven Rostedt# Options defined in the DEFAULTS section can not be duplicated
783d1cc414SSteven Rostedt# even if they are defined in two different DEFAULT sections.
793d1cc414SSteven Rostedt# This is done to catch mistakes where an option is added but
803d1cc414SSteven Rostedt# the previous option was forgotten about and not commented.
813d1cc414SSteven Rostedt#
823d1cc414SSteven Rostedt# The OVERRIDE keyword can be added to a section to allow this
833d1cc414SSteven Rostedt# section to override other DEFAULT sections values that have
843d1cc414SSteven Rostedt# been defined previously. It will only override options that
853d1cc414SSteven Rostedt# have been defined before its use. Options defined later
863d1cc414SSteven Rostedt# in a non override section will still error. The same option
873d1cc414SSteven Rostedt# can not be defined in the same section even if that section
883d1cc414SSteven Rostedt# is marked OVERRIDE.
893d1cc414SSteven Rostedt#
90ab7a3f52SSteven Rostedt#
91ab7a3f52SSteven Rostedt#
9245d73a5dSSteven Rostedt# Both TEST_START and DEFAULTS sections can also have the IF keyword
9345d73a5dSSteven Rostedt# The value after the IF must evaluate into a 0 or non 0 positive
9445d73a5dSSteven Rostedt# integer, and can use the config variables (explained below).
9545d73a5dSSteven Rostedt#
9645d73a5dSSteven Rostedt# DEFAULTS IF ${IS_X86_32}
9745d73a5dSSteven Rostedt#
9845d73a5dSSteven Rostedt# The above will process the DEFAULTS section if the config
9945d73a5dSSteven Rostedt# variable IS_X86_32 evaluates to a non zero positive integer
10045d73a5dSSteven Rostedt# otherwise if it evaluates to zero, it will act the same
10145d73a5dSSteven Rostedt# as if the SKIP keyword was used.
10245d73a5dSSteven Rostedt#
10345d73a5dSSteven Rostedt# The ELSE keyword can be used directly after a section with
10445d73a5dSSteven Rostedt# a IF statement.
10545d73a5dSSteven Rostedt#
10645d73a5dSSteven Rostedt# TEST_START IF ${RUN_NET_TESTS}
10745d73a5dSSteven Rostedt# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-network
10845d73a5dSSteven Rostedt#
10945d73a5dSSteven Rostedt# ELSE
11045d73a5dSSteven Rostedt#
11145d73a5dSSteven Rostedt# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-normal
11245d73a5dSSteven Rostedt#
11345d73a5dSSteven Rostedt#
11445d73a5dSSteven Rostedt# The ELSE keyword can also contain an IF statement to allow multiple
11545d73a5dSSteven Rostedt# if then else sections. But all the sections must be either
11645d73a5dSSteven Rostedt# DEFAULT or TEST_START, they can not be a mixture.
11745d73a5dSSteven Rostedt#
11845d73a5dSSteven Rostedt# TEST_START IF ${RUN_NET_TESTS}
11945d73a5dSSteven Rostedt# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-network
12045d73a5dSSteven Rostedt#
12145d73a5dSSteven Rostedt# ELSE IF ${RUN_DISK_TESTS}
12245d73a5dSSteven Rostedt# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-tests
12345d73a5dSSteven Rostedt#
12445d73a5dSSteven Rostedt# ELSE IF ${RUN_CPU_TESTS}
12545d73a5dSSteven Rostedt# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-cpu
12645d73a5dSSteven Rostedt#
12745d73a5dSSteven Rostedt# ELSE
12845d73a5dSSteven Rostedt# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-network
12945d73a5dSSteven Rostedt#
130ab7a3f52SSteven Rostedt# The if statement may also have comparisons that will and for
131ab7a3f52SSteven Rostedt# == and !=, strings may be used for both sides.
132ab7a3f52SSteven Rostedt#
133ab7a3f52SSteven Rostedt# BOX_TYPE := x86_32
134ab7a3f52SSteven Rostedt#
135ab7a3f52SSteven Rostedt# DEFAULTS IF ${BOX_TYPE} == x86_32
136ab7a3f52SSteven Rostedt# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-32
137ab7a3f52SSteven Rostedt# ELSE
138ab7a3f52SSteven Rostedt# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-64
139ab7a3f52SSteven Rostedt#
1409900b5dcSSteven Rostedt# The DEFINED keyword can be used by the IF statements too.
1419900b5dcSSteven Rostedt# It returns true if the given config variable or option has been defined
1429900b5dcSSteven Rostedt# or false otherwise.
1439900b5dcSSteven Rostedt#
1449900b5dcSSteven Rostedt#
1459900b5dcSSteven Rostedt# DEFAULTS IF DEFINED USE_CC
1469900b5dcSSteven Rostedt# CC := ${USE_CC}
1479900b5dcSSteven Rostedt# ELSE
1489900b5dcSSteven Rostedt# CC := gcc
1499900b5dcSSteven Rostedt#
1509900b5dcSSteven Rostedt#
1519900b5dcSSteven Rostedt# As well as NOT DEFINED.
1529900b5dcSSteven Rostedt#
1539900b5dcSSteven Rostedt# DEFAULTS IF NOT DEFINED MAKE_CMD
1549900b5dcSSteven Rostedt# MAKE_CMD := make ARCH=x86
1559900b5dcSSteven Rostedt#
1569900b5dcSSteven Rostedt#
1572ed3b161SSteven Rostedt#
1582ed3b161SSteven Rostedt# INCLUDE file
1592ed3b161SSteven Rostedt#
1602ed3b161SSteven Rostedt# The INCLUDE keyword may be used in DEFAULT sections. This will
1612ed3b161SSteven Rostedt# read another config file and process that file as well. The included
1622ed3b161SSteven Rostedt# file can include other files, add new test cases or default
1632ed3b161SSteven Rostedt# statements. Config variables will be passed to these files and changes
1642ed3b161SSteven Rostedt# to config variables will be seen by top level config files. Including
1652ed3b161SSteven Rostedt# a file is processed just like the contents of the file was cut and pasted
1662ed3b161SSteven Rostedt# into the top level file, except, that include files that end with
1672ed3b161SSteven Rostedt# TEST_START sections will have that section ended at the end of
1682ed3b161SSteven Rostedt# the include file. That is, an included file is included followed
1692ed3b161SSteven Rostedt# by another DEFAULT keyword.
1702ed3b161SSteven Rostedt#
1712ed3b161SSteven Rostedt# Unlike other files referenced in this config, the file path does not need
1722ed3b161SSteven Rostedt# to be absolute. If the file does not start with '/', then the directory
1732ed3b161SSteven Rostedt# that the current config file was located in is used. If no config by the
1742ed3b161SSteven Rostedt# given name is found there, then the current directory is searched.
1752ed3b161SSteven Rostedt#
1762ed3b161SSteven Rostedt# INCLUDE myfile
1772ed3b161SSteven Rostedt# DEFAULT
1782ed3b161SSteven Rostedt#
1792ed3b161SSteven Rostedt# is the same as:
1802ed3b161SSteven Rostedt#
1812ed3b161SSteven Rostedt# INCLUDE myfile
1822ed3b161SSteven Rostedt#
1832ed3b161SSteven Rostedt# Note, if the include file does not contain a full path, the file is
1842ed3b161SSteven Rostedt# searched first by the location of the original include file, and then
1852ed3b161SSteven Rostedt# by the location that ktest.pl was executed in.
1862ed3b161SSteven Rostedt#
187a75fececSSteven Rostedt
18877d942ceSSteven Rostedt#### Config variables ####
18977d942ceSSteven Rostedt#
19077d942ceSSteven Rostedt# This config file can also contain "config variables".
19177d942ceSSteven Rostedt# These are assigned with ":=" instead of the ktest option
19277d942ceSSteven Rostedt# assigment "=".
19377d942ceSSteven Rostedt#
19477d942ceSSteven Rostedt# The difference between ktest options and config variables
19577d942ceSSteven Rostedt# is that config variables can be used multiple times,
19677d942ceSSteven Rostedt# where each instance will override the previous instance.
19777d942ceSSteven Rostedt# And that they only live at time of processing this config.
19877d942ceSSteven Rostedt#
19977d942ceSSteven Rostedt# The advantage to config variables are that they can be used
20077d942ceSSteven Rostedt# by any option or any other config variables to define thing
20177d942ceSSteven Rostedt# that you may use over and over again in the options.
20277d942ceSSteven Rostedt#
20377d942ceSSteven Rostedt# For example:
20477d942ceSSteven Rostedt#
20577d942ceSSteven Rostedt# USER      := root
20677d942ceSSteven Rostedt# TARGET    := mybox
20777d942ceSSteven Rostedt# TEST_CASE := ssh ${USER}@${TARGET} /path/to/my/test
20877d942ceSSteven Rostedt#
20977d942ceSSteven Rostedt# TEST_START
21077d942ceSSteven Rostedt# MIN_CONFIG = config1
21177d942ceSSteven Rostedt# TEST = ${TEST_CASE}
21277d942ceSSteven Rostedt#
21377d942ceSSteven Rostedt# TEST_START
21477d942ceSSteven Rostedt# MIN_CONFIG = config2
21577d942ceSSteven Rostedt# TEST = ${TEST_CASE}
21677d942ceSSteven Rostedt#
21777d942ceSSteven Rostedt# TEST_CASE := ssh ${USER}@${TARGET} /path/to/my/test2
21877d942ceSSteven Rostedt#
21977d942ceSSteven Rostedt# TEST_START
22077d942ceSSteven Rostedt# MIN_CONFIG = config1
22177d942ceSSteven Rostedt# TEST = ${TEST_CASE}
22277d942ceSSteven Rostedt#
22377d942ceSSteven Rostedt# TEST_START
22477d942ceSSteven Rostedt# MIN_CONFIG = config2
22577d942ceSSteven Rostedt# TEST = ${TEST_CASE}
22677d942ceSSteven Rostedt#
22777d942ceSSteven Rostedt# TEST_DIR := /home/me/test
22877d942ceSSteven Rostedt#
22977d942ceSSteven Rostedt# BUILD_DIR = ${TEST_DIR}/linux.git
23077d942ceSSteven Rostedt# OUTPUT_DIR = ${TEST_DIR}/test
23177d942ceSSteven Rostedt#
23277d942ceSSteven Rostedt# Note, the config variables are evaluated immediately, thus
23377d942ceSSteven Rostedt# updating TARGET after TEST_CASE has been assigned does nothing
23477d942ceSSteven Rostedt# to TEST_CASE.
23577d942ceSSteven Rostedt#
23677d942ceSSteven Rostedt# As shown in the example, to evaluate a config variable, you
23777d942ceSSteven Rostedt# use the ${X} convention. Simple $X will not work.
23877d942ceSSteven Rostedt#
23977d942ceSSteven Rostedt# If the config variable does not exist, the ${X} will not
24077d942ceSSteven Rostedt# be evaluated. Thus:
24177d942ceSSteven Rostedt#
24277d942ceSSteven Rostedt# MAKE_CMD = PATH=/mypath:${PATH} make
24377d942ceSSteven Rostedt#
24477d942ceSSteven Rostedt# If PATH is not a config variable, then the ${PATH} in
24577d942ceSSteven Rostedt# the MAKE_CMD option will be evaluated by the shell when
24677d942ceSSteven Rostedt# the MAKE_CMD option is passed into shell processing.
247a57419b3SSteven Rostedt
2482a62512bSSteven Rostedt#### Using options in other options ####
2492a62512bSSteven Rostedt#
2502a62512bSSteven Rostedt# Options that are defined in the config file may also be used
2512a62512bSSteven Rostedt# by other options. All options are evaulated at time of
2522a62512bSSteven Rostedt# use (except that config variables are evaluated at config
2532a62512bSSteven Rostedt# processing time).
2542a62512bSSteven Rostedt#
2552a62512bSSteven Rostedt# If an ktest option is used within another option, instead of
2562a62512bSSteven Rostedt# typing it again in that option you can simply use the option
2572a62512bSSteven Rostedt# just like you can config variables.
2582a62512bSSteven Rostedt#
2592a62512bSSteven Rostedt# MACHINE = mybox
2602a62512bSSteven Rostedt#
2612a62512bSSteven Rostedt# TEST = ssh root@${MACHINE} /path/to/test
2622a62512bSSteven Rostedt#
2632a62512bSSteven Rostedt# The option will be used per test case. Thus:
2642a62512bSSteven Rostedt#
2652a62512bSSteven Rostedt# TEST_TYPE = test
2662a62512bSSteven Rostedt# TEST = ssh root@{MACHINE}
2672a62512bSSteven Rostedt#
2682a62512bSSteven Rostedt# TEST_START
2692a62512bSSteven Rostedt# MACHINE = box1
2702a62512bSSteven Rostedt#
2712a62512bSSteven Rostedt# TEST_START
2722a62512bSSteven Rostedt# MACHINE = box2
2732a62512bSSteven Rostedt#
2742a62512bSSteven Rostedt# For both test cases, MACHINE will be evaluated at the time
2752a62512bSSteven Rostedt# of the test case. The first test will run ssh root@box1
2762a62512bSSteven Rostedt# and the second will run ssh root@box2.
2772a62512bSSteven Rostedt
278a57419b3SSteven Rostedt#### Mandatory Default Options ####
279a57419b3SSteven Rostedt
280a57419b3SSteven Rostedt# These options must be in the default section, although most
281a57419b3SSteven Rostedt# may be overridden by test options.
282a75fececSSteven Rostedt
283a75fececSSteven Rostedt# The machine hostname that you will test
284a75fececSSteven Rostedt#MACHINE = target
285a75fececSSteven Rostedt
286a75fececSSteven Rostedt# The box is expected to have ssh on normal bootup, provide the user
287a75fececSSteven Rostedt#  (most likely root, since you need privileged operations)
288a75fececSSteven Rostedt#SSH_USER = root
289a75fececSSteven Rostedt
290a75fececSSteven Rostedt# The directory that contains the Linux source code
291a75fececSSteven Rostedt#BUILD_DIR = /home/test/linux.git
292a75fececSSteven Rostedt
293a75fececSSteven Rostedt# The directory that the objects will be built
294a75fececSSteven Rostedt# (can not be same as BUILD_DIR)
295a75fececSSteven Rostedt#OUTPUT_DIR = /home/test/build/target
296a75fececSSteven Rostedt
297a75fececSSteven Rostedt# The location of the compiled file to copy to the target
298a75fececSSteven Rostedt# (relative to OUTPUT_DIR)
299a75fececSSteven Rostedt#BUILD_TARGET = arch/x86/boot/bzImage
300a75fececSSteven Rostedt
301a75fececSSteven Rostedt# The place to put your image on the test machine
302a75fececSSteven Rostedt#TARGET_IMAGE = /boot/vmlinuz-test
303a75fececSSteven Rostedt
304a75fececSSteven Rostedt# A script or command to reboot the box
305a57419b3SSteven Rostedt#
306a75fececSSteven Rostedt# Here is a digital loggers power switch example
307a75fececSSteven Rostedt#POWER_CYCLE = wget --no-proxy -O /dev/null -q  --auth-no-challenge 'http://admin:admin@power/outlet?5=CCL'
308a57419b3SSteven Rostedt#
309a75fececSSteven Rostedt# Here is an example to reboot a virtual box on the current host
310a75fececSSteven Rostedt# with the name "Guest".
311a57419b3SSteven Rostedt#POWER_CYCLE = virsh destroy Guest; sleep 5; virsh start Guest
312a75fececSSteven Rostedt
313a75fececSSteven Rostedt# The script or command that reads the console
314a57419b3SSteven Rostedt#
315a75fececSSteven Rostedt#  If you use ttywatch server, something like the following would work.
316a75fececSSteven Rostedt#CONSOLE = nc -d localhost 3001
317a57419b3SSteven Rostedt#
318a75fececSSteven Rostedt# For a virtual machine with guest name "Guest".
319a57419b3SSteven Rostedt#CONSOLE =  virsh console Guest
320a75fececSSteven Rostedt
321a75fececSSteven Rostedt# Required version ending to differentiate the test
322a75fececSSteven Rostedt# from other linux builds on the system.
323a75fececSSteven Rostedt#LOCALVERSION = -test
324a75fececSSteven Rostedt
325a75fececSSteven Rostedt# The grub title name for the test kernel to boot
326a75fececSSteven Rostedt# (Only mandatory if REBOOT_TYPE = grub)
327a75fececSSteven Rostedt#
328a57419b3SSteven Rostedt# Note, ktest.pl will not update the grub menu.lst, you need to
329a57419b3SSteven Rostedt# manually add an option for the test. ktest.pl will search
330a57419b3SSteven Rostedt# the grub menu.lst for this option to find what kernel to
331a57419b3SSteven Rostedt# reboot into.
332a57419b3SSteven Rostedt#
333a75fececSSteven Rostedt# For example, if in the /boot/grub/menu.lst the test kernel title has:
334a75fececSSteven Rostedt# title Test Kernel
335a57419b3SSteven Rostedt# kernel vmlinuz-test
336a75fececSSteven Rostedt#GRUB_MENU = Test Kernel
337a75fececSSteven Rostedt
338a75fececSSteven Rostedt# A script to reboot the target into the test kernel
339a75fececSSteven Rostedt# (Only mandatory if REBOOT_TYPE = script)
340a75fececSSteven Rostedt#REBOOT_SCRIPT =
341a75fececSSteven Rostedt
342a75fececSSteven Rostedt#### Optional Config Options (all have defaults) ####
343a75fececSSteven Rostedt
344a57419b3SSteven Rostedt# Start a test setup. If you leave this off, all options
345a57419b3SSteven Rostedt# will be default and the test will run once.
346a57419b3SSteven Rostedt# This is a label and not really an option (it takes no value).
347a57419b3SSteven Rostedt# You can append ITERATE and a number after it to iterate the
348a57419b3SSteven Rostedt# test a number of times, or SKIP to ignore this test.
349a57419b3SSteven Rostedt#
350a57419b3SSteven Rostedt#TEST_START
351a57419b3SSteven Rostedt#TEST_START ITERATE 5
352a57419b3SSteven Rostedt#TEST_START SKIP
353a75fececSSteven Rostedt
354dc895688SSteven Rostedt# Have the following options as default again. Used after tests
355dc895688SSteven Rostedt# have already been defined by TEST_START. Optionally, you can
356dc895688SSteven Rostedt# just define all default options before the first TEST_START
357dc895688SSteven Rostedt# and you do not need this option.
358dc895688SSteven Rostedt#
359dc895688SSteven Rostedt# This is a label and not really an option (it takes no value).
360dc895688SSteven Rostedt# You can append SKIP to this label and the options within this
361dc895688SSteven Rostedt# section will be ignored.
362dc895688SSteven Rostedt#
363dc895688SSteven Rostedt# DEFAULTS
364dc895688SSteven Rostedt# DEFAULTS SKIP
365dc895688SSteven Rostedt
366a75fececSSteven Rostedt# The default test type (default test)
367a75fececSSteven Rostedt# The test types may be:
368a75fececSSteven Rostedt#   build   - only build the kernel, do nothing else
369cd8e368fSSteven Rostedt#   install - build and install, but do nothing else (does not reboot)
370cd8e368fSSteven Rostedt#   boot    - build, install, and boot the kernel
371a75fececSSteven Rostedt#   test    - build, boot and if TEST is set, run the test script
372a57419b3SSteven Rostedt#          (If TEST is not set, it defaults back to boot)
373a75fececSSteven Rostedt#   bisect - Perform a bisect on the kernel (see BISECT_TYPE below)
374a75fececSSteven Rostedt#   patchcheck - Do a test on a series of commits in git (see PATCHCHECK below)
375a75fececSSteven Rostedt#TEST_TYPE = test
376a75fececSSteven Rostedt
377a57419b3SSteven Rostedt# Test to run if there is a successful boot and TEST_TYPE is test.
378a57419b3SSteven Rostedt# Must exit with 0 on success and non zero on error
379a57419b3SSteven Rostedt# default (undefined)
380a57419b3SSteven Rostedt#TEST = ssh user@machine /root/run_test
381a57419b3SSteven Rostedt
382a57419b3SSteven Rostedt# The build type is any make config type or special command
383a75fececSSteven Rostedt#  (default randconfig)
384a75fececSSteven Rostedt#   nobuild - skip the clean and build step
385a57419b3SSteven Rostedt#   useconfig:/path/to/config - use the given config and run
386a57419b3SSteven Rostedt#              oldconfig on it.
387a57419b3SSteven Rostedt# This option is ignored if TEST_TYPE is patchcheck or bisect
388a75fececSSteven Rostedt#BUILD_TYPE = randconfig
389a75fececSSteven Rostedt
390a75fececSSteven Rostedt# The make command (default make)
391a75fececSSteven Rostedt# If you are building a 32bit x86 on a 64 bit host
392a75fececSSteven Rostedt#MAKE_CMD = CC=i386-gcc AS=i386-as make ARCH=i386
393a75fececSSteven Rostedt
394dc895688SSteven Rostedt# Any build options for the make of the kernel (not for other makes, like configs)
395dc895688SSteven Rostedt# (default "")
396dc895688SSteven Rostedt#BUILD_OPTIONS = -j20
397dc895688SSteven Rostedt
3988b37ca8cSSteven Rostedt# If you need an initrd, you can add a script or code here to install
3998b37ca8cSSteven Rostedt# it. The environment variable KERNEL_VERSION will be set to the
400a57419b3SSteven Rostedt# kernel version that is used. Remember to add the initrd line
401a57419b3SSteven Rostedt# to your grub menu.lst file.
402a57419b3SSteven Rostedt#
403a57419b3SSteven Rostedt# Here's a couple of examples to use:
4048b37ca8cSSteven Rostedt#POST_INSTALL = ssh user@target /sbin/mkinitrd --allow-missing -f /boot/initramfs-test.img $KERNEL_VERSION
405a57419b3SSteven Rostedt#
406a57419b3SSteven Rostedt# or on some systems:
407a57419b3SSteven Rostedt#POST_INSTALL = ssh user@target /sbin/dracut -f /boot/initramfs-test.img $KERNEL_VERSION
4088b37ca8cSSteven Rostedt
409e0a8742eSSteven Rostedt# If for some reason you just want to boot the kernel and you do not
410e0a8742eSSteven Rostedt# want the test to install anything new. For example, you may just want
411e0a8742eSSteven Rostedt# to boot test the same kernel over and over and do not want to go through
412e0a8742eSSteven Rostedt# the hassle of installing anything, you can set this option to 1
413e0a8742eSSteven Rostedt# (default 0)
414e0a8742eSSteven Rostedt#NO_INSTALL = 1
415e0a8742eSSteven Rostedt
4160bd6c1a3SSteven Rostedt# If there is a script that you require to run before the build is done
4170bd6c1a3SSteven Rostedt# you can specify it with PRE_BUILD.
4180bd6c1a3SSteven Rostedt#
4190bd6c1a3SSteven Rostedt# One example may be if you must add a temporary patch to the build to
4200bd6c1a3SSteven Rostedt# fix a unrelated bug to perform a patchcheck test. This will apply the
4210bd6c1a3SSteven Rostedt# patch before each build that is made. Use the POST_BUILD to do a git reset --hard
4220bd6c1a3SSteven Rostedt# to remove the patch.
4230bd6c1a3SSteven Rostedt#
4240bd6c1a3SSteven Rostedt# (default undef)
4250bd6c1a3SSteven Rostedt#PRE_BUILD = cd ${BUILD_DIR} && patch -p1 < /tmp/temp.patch
4260bd6c1a3SSteven Rostedt
4270bd6c1a3SSteven Rostedt# To specify if the test should fail if the PRE_BUILD fails,
4280bd6c1a3SSteven Rostedt# PRE_BUILD_DIE needs to be set to 1. Otherwise the PRE_BUILD
4290bd6c1a3SSteven Rostedt# result is ignored.
4300bd6c1a3SSteven Rostedt# (default 0)
4310bd6c1a3SSteven Rostedt# PRE_BUILD_DIE = 1
4320bd6c1a3SSteven Rostedt
4330bd6c1a3SSteven Rostedt# If there is a script that should run after the build is done
4340bd6c1a3SSteven Rostedt# you can specify it with POST_BUILD.
4350bd6c1a3SSteven Rostedt#
4360bd6c1a3SSteven Rostedt# As the example in PRE_BUILD, POST_BUILD can be used to reset modifications
4370bd6c1a3SSteven Rostedt# made by the PRE_BUILD.
4380bd6c1a3SSteven Rostedt#
4390bd6c1a3SSteven Rostedt# (default undef)
4400bd6c1a3SSteven Rostedt#POST_BUILD = cd ${BUILD_DIR} && git reset --hard
4410bd6c1a3SSteven Rostedt
4420bd6c1a3SSteven Rostedt# To specify if the test should fail if the POST_BUILD fails,
4430bd6c1a3SSteven Rostedt# POST_BUILD_DIE needs to be set to 1. Otherwise the POST_BUILD
4440bd6c1a3SSteven Rostedt# result is ignored.
4450bd6c1a3SSteven Rostedt# (default 0)
4460bd6c1a3SSteven Rostedt#POST_BUILD_DIE = 1
4470bd6c1a3SSteven Rostedt
448a75fececSSteven Rostedt# Way to reboot the box to the test kernel.
449a75fececSSteven Rostedt# Only valid options so far are "grub" and "script"
450a75fececSSteven Rostedt# (default grub)
451a75fececSSteven Rostedt# If you specify grub, it will assume grub version 1
452a75fececSSteven Rostedt# and will search in /boot/grub/menu.lst for the title $GRUB_MENU
453a75fececSSteven Rostedt# and select that target to reboot to the kernel. If this is not
454a75fececSSteven Rostedt# your setup, then specify "script" and have a command or script
455a75fececSSteven Rostedt# specified in REBOOT_SCRIPT to boot to the target.
456a57419b3SSteven Rostedt#
457a57419b3SSteven Rostedt# The entry in /boot/grub/menu.lst must be entered in manually.
458a57419b3SSteven Rostedt# The test will not modify that file.
459a75fececSSteven Rostedt#REBOOT_TYPE = grub
460a75fececSSteven Rostedt
461a75fececSSteven Rostedt# The min config that is needed to build for the machine
462a57419b3SSteven Rostedt# A nice way to create this is with the following:
463a75fececSSteven Rostedt#
464a57419b3SSteven Rostedt#   $ ssh target
465a57419b3SSteven Rostedt#   $ lsmod > mymods
466a57419b3SSteven Rostedt#   $ scp mymods host:/tmp
467a57419b3SSteven Rostedt#   $ exit
468a57419b3SSteven Rostedt#   $ cd linux.git
469a57419b3SSteven Rostedt#   $ rm .config
470a57419b3SSteven Rostedt#   $ make LSMOD=mymods localyesconfig
471a57419b3SSteven Rostedt#   $ grep '^CONFIG' .config > /home/test/config-min
472a57419b3SSteven Rostedt#
473a57419b3SSteven Rostedt# If you want even less configs:
474a57419b3SSteven Rostedt#
475a57419b3SSteven Rostedt#   log in directly to target (do not ssh)
476a57419b3SSteven Rostedt#
477a57419b3SSteven Rostedt#   $ su
478a57419b3SSteven Rostedt#   # lsmod | cut -d' ' -f1 | xargs rmmod
479a57419b3SSteven Rostedt#
480a57419b3SSteven Rostedt#   repeat the above several times
481a57419b3SSteven Rostedt#
482a57419b3SSteven Rostedt#   # lsmod > mymods
483a57419b3SSteven Rostedt#   # reboot
484a57419b3SSteven Rostedt#
485a57419b3SSteven Rostedt# May need to reboot to get your network back to copy the mymods
486a57419b3SSteven Rostedt# to the host, and then remove the previous .config and run the
487a57419b3SSteven Rostedt# localyesconfig again. The CONFIG_MIN generated like this will
488a57419b3SSteven Rostedt# not guarantee network activity to the box so the TEST_TYPE of
489a57419b3SSteven Rostedt# test may fail.
490a57419b3SSteven Rostedt#
491a57419b3SSteven Rostedt# You might also want to set:
492a75fececSSteven Rostedt#   CONFIG_CMDLINE="<your options here>"
493a75fececSSteven Rostedt#  randconfig may set the above and override your real command
494a75fececSSteven Rostedt#  line options.
495a57419b3SSteven Rostedt# (default undefined)
496a75fececSSteven Rostedt#MIN_CONFIG = /home/test/config-min
497a75fececSSteven Rostedt
498a75fececSSteven Rostedt# Sometimes there's options that just break the boot and
499a75fececSSteven Rostedt# you do not care about. Here are a few:
500a75fececSSteven Rostedt#   # CONFIG_STAGING is not set
501a75fececSSteven Rostedt#  Staging drivers are horrible, and can break the build.
502a75fececSSteven Rostedt#   # CONFIG_SCSI_DEBUG is not set
503a75fececSSteven Rostedt#  SCSI_DEBUG may change your root partition
504a75fececSSteven Rostedt#   # CONFIG_KGDB_SERIAL_CONSOLE is not set
505a75fececSSteven Rostedt#  KGDB may cause oops waiting for a connection that's not there.
506a75fececSSteven Rostedt# This option points to the file containing config options that will be prepended
507a75fececSSteven Rostedt# to the MIN_CONFIG (or be the MIN_CONFIG if it is not set)
508a57419b3SSteven Rostedt#
509a57419b3SSteven Rostedt# Note, config options in MIN_CONFIG will override these options.
510a57419b3SSteven Rostedt#
511a57419b3SSteven Rostedt# (default undefined)
512a75fececSSteven Rostedt#ADD_CONFIG = /home/test/config-broken
513a75fececSSteven Rostedt
514dc895688SSteven Rostedt# The location on the host where to write temp files
51548920630SSteven Rostedt# (default /tmp/ktest/${MACHINE})
51648920630SSteven Rostedt#TMP_DIR = /tmp/ktest/${MACHINE}
517dc895688SSteven Rostedt
518dc895688SSteven Rostedt# Optional log file to write the status (recommended)
519dc895688SSteven Rostedt#  Note, this is a DEFAULT section only option.
520dc895688SSteven Rostedt# (default undefined)
521dc895688SSteven Rostedt#LOG_FILE = /home/test/logfiles/target.log
522dc895688SSteven Rostedt
523dc895688SSteven Rostedt# Remove old logfile if it exists before starting all tests.
524dc895688SSteven Rostedt#  Note, this is a DEFAULT section only option.
525dc895688SSteven Rostedt# (default 0)
526dc895688SSteven Rostedt#CLEAR_LOG = 0
527dc895688SSteven Rostedt
528dc895688SSteven Rostedt# Line to define a successful boot up in console output.
529dc895688SSteven Rostedt# This is what the line contains, not the entire line. If you need
530dc895688SSteven Rostedt# the entire line to match, then use regural expression syntax like:
531dc895688SSteven Rostedt#  (do not add any quotes around it)
532dc895688SSteven Rostedt#
533dc895688SSteven Rostedt#  SUCCESS_LINE = ^MyBox Login:$
534dc895688SSteven Rostedt#
535dc895688SSteven Rostedt# (default "login:")
536dc895688SSteven Rostedt#SUCCESS_LINE = login:
537dc895688SSteven Rostedt
5382b803365SSteven Rostedt# To speed up between reboots, defining a line that the
5392b803365SSteven Rostedt# default kernel produces that represents that the default
5402b803365SSteven Rostedt# kernel has successfully booted and can be used to pass
5412b803365SSteven Rostedt# a new test kernel to it. Otherwise ktest.pl will wait till
5422b803365SSteven Rostedt# SLEEP_TIME to continue.
5432b803365SSteven Rostedt# (default undefined)
5442b803365SSteven Rostedt#REBOOT_SUCCESS_LINE = login:
5452b803365SSteven Rostedt
5461c8a617aSSteven Rostedt# In case the console constantly fills the screen, having
5471c8a617aSSteven Rostedt# a specified time to stop the test after success is recommended.
5481c8a617aSSteven Rostedt# (in seconds)
5491c8a617aSSteven Rostedt# (default 10)
5501c8a617aSSteven Rostedt#STOP_AFTER_SUCCESS = 10
5511c8a617aSSteven Rostedt
5521c8a617aSSteven Rostedt# In case the console constantly fills the screen, having
5531c8a617aSSteven Rostedt# a specified time to stop the test after failure is recommended.
5541c8a617aSSteven Rostedt# (in seconds)
5551c8a617aSSteven Rostedt# (default 60)
5561c8a617aSSteven Rostedt#STOP_AFTER_FAILURE = 60
5571c8a617aSSteven Rostedt
5582d01b26aSSteven Rostedt# In case the console constantly fills the screen, having
5592d01b26aSSteven Rostedt# a specified time to stop the test if it never succeeds nor fails
5602d01b26aSSteven Rostedt# is recommended.
5612d01b26aSSteven Rostedt# Note: this is ignored if a success or failure is detected.
5622d01b26aSSteven Rostedt# (in seconds)
5632d01b26aSSteven Rostedt# (default 600, -1 is to never stop)
5642d01b26aSSteven Rostedt#STOP_TEST_AFTER = 600
5652d01b26aSSteven Rostedt
566dc895688SSteven Rostedt# Stop testing if a build fails. If set, the script will end if
567dc895688SSteven Rostedt# a failure is detected, otherwise it will save off the .config,
568dc895688SSteven Rostedt# dmesg and bootlog in a directory called
569dc895688SSteven Rostedt# MACHINE-TEST_TYPE_BUILD_TYPE-fail-yyyymmddhhmmss
570dc895688SSteven Rostedt# if the STORE_FAILURES directory is set.
571dc895688SSteven Rostedt# (default 1)
572dc895688SSteven Rostedt# Note, even if this is set to zero, there are some errors that still
573dc895688SSteven Rostedt# stop the tests.
574dc895688SSteven Rostedt#DIE_ON_FAILURE = 1
575dc895688SSteven Rostedt
576dc895688SSteven Rostedt# Directory to store failure directories on failure. If this is not
577dc895688SSteven Rostedt# set, DIE_ON_FAILURE=0 will not save off the .config, dmesg and
578dc895688SSteven Rostedt# bootlog. This option is ignored if DIE_ON_FAILURE is not set.
579dc895688SSteven Rostedt# (default undefined)
580dc895688SSteven Rostedt#STORE_FAILURES = /home/test/failures
581dc895688SSteven Rostedt
582dc895688SSteven Rostedt# Build without doing a make mrproper, or removing .config
583dc895688SSteven Rostedt# (default 0)
584dc895688SSteven Rostedt#BUILD_NOCLEAN = 0
585dc895688SSteven Rostedt
586dc895688SSteven Rostedt# As the test reads the console, after it hits the SUCCESS_LINE
587dc895688SSteven Rostedt# the time it waits for the monitor to settle down between reads
588dc895688SSteven Rostedt# can usually be lowered.
589dc895688SSteven Rostedt# (in seconds) (default 1)
590dc895688SSteven Rostedt#BOOTED_TIMEOUT = 1
591dc895688SSteven Rostedt
592dc895688SSteven Rostedt# The timeout in seconds when we consider the box hung after
593dc895688SSteven Rostedt# the console stop producing output. Be sure to leave enough
594dc895688SSteven Rostedt# time here to get pass a reboot. Some machines may not produce
595dc895688SSteven Rostedt# any console output for a long time during a reboot. You do
596dc895688SSteven Rostedt# not want the test to fail just because the system was in
597dc895688SSteven Rostedt# the process of rebooting to the test kernel.
598dc895688SSteven Rostedt# (default 120)
599dc895688SSteven Rostedt#TIMEOUT = 120
600dc895688SSteven Rostedt
601dc895688SSteven Rostedt# In between tests, a reboot of the box may occur, and this
602dc895688SSteven Rostedt# is the time to wait for the console after it stops producing
603dc895688SSteven Rostedt# output. Some machines may not produce a large lag on reboot
604dc895688SSteven Rostedt# so this should accommodate it.
605dc895688SSteven Rostedt# The difference between this and TIMEOUT, is that TIMEOUT happens
606dc895688SSteven Rostedt# when rebooting to the test kernel. This sleep time happens
607dc895688SSteven Rostedt# after a test has completed and we are about to start running
608dc895688SSteven Rostedt# another test. If a reboot to the reliable kernel happens,
609dc895688SSteven Rostedt# we wait SLEEP_TIME for the console to stop producing output
610dc895688SSteven Rostedt# before starting the next test.
6112b803365SSteven Rostedt#
6122b803365SSteven Rostedt# You can speed up reboot times even more by setting REBOOT_SUCCESS_LINE.
613dc895688SSteven Rostedt# (default 60)
614dc895688SSteven Rostedt#SLEEP_TIME = 60
615dc895688SSteven Rostedt
616dc895688SSteven Rostedt# The time in between bisects to sleep (in seconds)
617dc895688SSteven Rostedt# (default 60)
618dc895688SSteven Rostedt#BISECT_SLEEP_TIME = 60
619dc895688SSteven Rostedt
62027d934b2SSteven Rostedt# The time in between patch checks to sleep (in seconds)
62127d934b2SSteven Rostedt# (default 60)
62227d934b2SSteven Rostedt#PATCHCHECK_SLEEP_TIME = 60
62327d934b2SSteven Rostedt
624dc895688SSteven Rostedt# Reboot the target box on error (default 0)
625dc895688SSteven Rostedt#REBOOT_ON_ERROR = 0
626dc895688SSteven Rostedt
627dc895688SSteven Rostedt# Power off the target on error (ignored if REBOOT_ON_ERROR is set)
628dc895688SSteven Rostedt#  Note, this is a DEFAULT section only option.
629dc895688SSteven Rostedt# (default 0)
630dc895688SSteven Rostedt#POWEROFF_ON_ERROR = 0
631dc895688SSteven Rostedt
632dc895688SSteven Rostedt# Power off the target after all tests have completed successfully
633dc895688SSteven Rostedt#  Note, this is a DEFAULT section only option.
634dc895688SSteven Rostedt# (default 0)
635dc895688SSteven Rostedt#POWEROFF_ON_SUCCESS = 0
636dc895688SSteven Rostedt
637dc895688SSteven Rostedt# Reboot the target after all test completed successfully (default 1)
638dc895688SSteven Rostedt# (ignored if POWEROFF_ON_SUCCESS is set)
639dc895688SSteven Rostedt#REBOOT_ON_SUCCESS = 1
640dc895688SSteven Rostedt
641dc895688SSteven Rostedt# In case there are isses with rebooting, you can specify this
642dc895688SSteven Rostedt# to always powercycle after this amount of time after calling
643dc895688SSteven Rostedt# reboot.
644dc895688SSteven Rostedt# Note, POWERCYCLE_AFTER_REBOOT = 0 does NOT disable it. It just
645dc895688SSteven Rostedt# makes it powercycle immediately after rebooting. Do not define
646dc895688SSteven Rostedt# it if you do not want it.
647dc895688SSteven Rostedt# (default undefined)
648dc895688SSteven Rostedt#POWERCYCLE_AFTER_REBOOT = 5
649dc895688SSteven Rostedt
650dc895688SSteven Rostedt# In case there's isses with halting, you can specify this
651dc895688SSteven Rostedt# to always poweroff after this amount of time after calling
652dc895688SSteven Rostedt# halt.
653dc895688SSteven Rostedt# Note, POWEROFF_AFTER_HALT = 0 does NOT disable it. It just
654dc895688SSteven Rostedt# makes it poweroff immediately after halting. Do not define
655dc895688SSteven Rostedt# it if you do not want it.
656dc895688SSteven Rostedt# (default undefined)
657dc895688SSteven Rostedt#POWEROFF_AFTER_HALT = 20
658dc895688SSteven Rostedt
659dc895688SSteven Rostedt# A script or command to power off the box (default undefined)
660dc895688SSteven Rostedt# Needed for POWEROFF_ON_ERROR and SUCCESS
661dc895688SSteven Rostedt#
662dc895688SSteven Rostedt# Example for digital loggers power switch:
663dc895688SSteven Rostedt#POWER_OFF = wget --no-proxy -O /dev/null -q  --auth-no-challenge 'http://admin:admin@power/outlet?5=OFF'
664dc895688SSteven Rostedt#
665dc895688SSteven Rostedt# Example for a virtual guest call "Guest".
666dc895688SSteven Rostedt#POWER_OFF = virsh destroy Guest
667dc895688SSteven Rostedt
668d1fbd7e6SSteven Rostedt# The way to execute a command on the target
669d1fbd7e6SSteven Rostedt# (default ssh $SSH_USER@$MACHINE $SSH_COMMAND";)
670d1fbd7e6SSteven Rostedt# The variables SSH_USER, MACHINE and SSH_COMMAND are defined
671d1fbd7e6SSteven Rostedt#SSH_EXEC = ssh $SSH_USER@$MACHINE $SSH_COMMAND";
672d1fbd7e6SSteven Rostedt
673d1fbd7e6SSteven Rostedt# The way to copy a file to the target
674d1fbd7e6SSteven Rostedt# (default scp $SRC_FILE $SSH_USER@$MACHINE:$DST_FILE)
675d1fbd7e6SSteven Rostedt# The variables SSH_USER, MACHINE, SRC_FILE and DST_FILE are defined.
676d1fbd7e6SSteven Rostedt#SCP_TO_TARGET = scp $SRC_FILE $SSH_USER@$MACHINE:$DST_FILE
677d1fbd7e6SSteven Rostedt
678d1fbd7e6SSteven Rostedt# The nice way to reboot the target
679d1fbd7e6SSteven Rostedt# (default ssh $SSH_USER@$MACHINE reboot)
680d1fbd7e6SSteven Rostedt# The variables SSH_USER and MACHINE are defined.
681d1fbd7e6SSteven Rostedt#REBOOT = ssh $SSH_USER@$MACHINE reboot
682d1fbd7e6SSteven Rostedt
683f1a5b962SSteven Rostedt# The way triple faults are detected is by testing the kernel
684f1a5b962SSteven Rostedt# banner. If the kernel banner for the kernel we are testing is
685f1a5b962SSteven Rostedt# found, and then later a kernel banner for another kernel version
686f1a5b962SSteven Rostedt# is found, it is considered that we encountered a triple fault,
687f1a5b962SSteven Rostedt# and there is no panic or callback, but simply a reboot.
688f1a5b962SSteven Rostedt# To disable this (because it did a false positive) set the following
689f1a5b962SSteven Rostedt# to 0.
690f1a5b962SSteven Rostedt# (default 1)
691f1a5b962SSteven Rostedt#DETECT_TRIPLE_FAULT = 0
692f1a5b962SSteven Rostedt
693a75fececSSteven Rostedt#### Per test run options ####
694a57419b3SSteven Rostedt# The following options are only allowed in TEST_START sections.
695a57419b3SSteven Rostedt# They are ignored in the DEFAULTS sections.
696a75fececSSteven Rostedt#
697a57419b3SSteven Rostedt# All of these are optional and undefined by default, although
698a57419b3SSteven Rostedt#  some of these options are required for TEST_TYPE of patchcheck
699a57419b3SSteven Rostedt#  and bisect.
700a75fececSSteven Rostedt#
701a57419b3SSteven Rostedt#
702a57419b3SSteven Rostedt# CHECKOUT = branch
703a75fececSSteven Rostedt#
704a75fececSSteven Rostedt#  If the BUILD_DIR is a git repository, then you can set this option
705a75fececSSteven Rostedt#  to checkout the given branch before running the TEST. If you
706a75fececSSteven Rostedt#  specify this for the first run, that branch will be used for
707a57419b3SSteven Rostedt#  all preceding tests until a new CHECKOUT is set.
708a75fececSSteven Rostedt#
709a57419b3SSteven Rostedt#
7109064af52SSteven Rostedt# TEST_NAME = name
7119064af52SSteven Rostedt#
7129064af52SSteven Rostedt#  If you want the test to have a name that is displayed in
7139064af52SSteven Rostedt#  the test result banner at the end of the test, then use this
7149064af52SSteven Rostedt#  option. This is useful to search for the RESULT keyword and
7159064af52SSteven Rostedt#  not have to translate a test number to a test in the config.
716a57419b3SSteven Rostedt#
717a57419b3SSteven Rostedt# For TEST_TYPE = patchcheck
718a75fececSSteven Rostedt#
719a75fececSSteven Rostedt#  This expects the BUILD_DIR to be a git repository, and
720a57419b3SSteven Rostedt#  will checkout the PATCHCHECK_START commit.
721a75fececSSteven Rostedt#
722a57419b3SSteven Rostedt#  The option BUILD_TYPE will be ignored.
723a75fececSSteven Rostedt#
724a57419b3SSteven Rostedt#  The MIN_CONFIG will be used for all builds of the patchcheck. The build type
725a57419b3SSteven Rostedt#  used for patchcheck is oldconfig.
726a75fececSSteven Rostedt#
727a57419b3SSteven Rostedt#  PATCHCHECK_START is required and is the first patch to
728a57419b3SSteven Rostedt#   test (the SHA1 of the commit). You may also specify anything
729a57419b3SSteven Rostedt#   that git checkout allows (branch name, tage, HEAD~3).
730a57419b3SSteven Rostedt#
731a57419b3SSteven Rostedt#  PATCHCHECK_END is the last patch to check (default HEAD)
732a57419b3SSteven Rostedt#
733a57419b3SSteven Rostedt#  PATCHCHECK_TYPE is required and is the type of test to run:
734a75fececSSteven Rostedt#      build, boot, test.
735a75fececSSteven Rostedt#
736a75fececSSteven Rostedt#   Note, the build test will look for warnings, if a warning occurred
7371990207dSSteven Rostedt#     in a file that a commit touches, the build will fail, unless
7381990207dSSteven Rostedt#     IGNORE_WARNINGS is set for the given commit's sha1
7391990207dSSteven Rostedt#
7401990207dSSteven Rostedt#   IGNORE_WARNINGS can be used to disable the failure of patchcheck
7411990207dSSteven Rostedt#     on a particuler commit (SHA1). You can add more than one commit
7421990207dSSteven Rostedt#     by adding a list of SHA1s that are space delimited.
743a75fececSSteven Rostedt#
744a75fececSSteven Rostedt#   If BUILD_NOCLEAN is set, then make mrproper will not be run on
745a75fececSSteven Rostedt#   any of the builds, just like all other TEST_TYPE tests. But
746a75fececSSteven Rostedt#   what makes patchcheck different from the other tests, is if
747a75fececSSteven Rostedt#   BUILD_NOCLEAN is not set, only the first and last patch run
748a75fececSSteven Rostedt#   make mrproper. This helps speed up the test.
749a75fececSSteven Rostedt#
750a75fececSSteven Rostedt# Example:
751a57419b3SSteven Rostedt#   TEST_START
752a57419b3SSteven Rostedt#   TEST_TYPE = patchcheck
753a57419b3SSteven Rostedt#   CHECKOUT = mybranch
754a57419b3SSteven Rostedt#   PATCHCHECK_TYPE = boot
755a57419b3SSteven Rostedt#   PATCHCHECK_START = 747e94ae3d1b4c9bf5380e569f614eb9040b79e7
756d1fbd7e6SSteven Rostedt#   PATCHCHECK_END = HEAD~2
7571990207dSSteven Rostedt#   IGNORE_WARNINGS = 42f9c6b69b54946ffc0515f57d01dc7f5c0e4712 0c17ca2c7187f431d8ffc79e81addc730f33d128
758a75fececSSteven Rostedt#
759a75fececSSteven Rostedt#
760a57419b3SSteven Rostedt#
761a57419b3SSteven Rostedt# For TEST_TYPE = bisect
762a75fececSSteven Rostedt#
763a75fececSSteven Rostedt#  You can specify a git bisect if the BUILD_DIR is a git repository.
764a75fececSSteven Rostedt#  The MIN_CONFIG will be used for all builds of the bisect. The build type
765a75fececSSteven Rostedt#  used for bisecting is oldconfig.
766a75fececSSteven Rostedt#
767a57419b3SSteven Rostedt#  The option BUILD_TYPE will be ignored.
768a57419b3SSteven Rostedt#
769a57419b3SSteven Rostedt#  BISECT_TYPE is the type of test to perform:
770a75fececSSteven Rostedt#	build	- bad fails to build
771a75fececSSteven Rostedt#	boot	- bad builds but fails to boot
772a75fececSSteven Rostedt#	test	- bad boots but fails a test
773a75fececSSteven Rostedt#
774a57419b3SSteven Rostedt# BISECT_GOOD is the commit (SHA1) to label as good (accepts all git good commit types)
775a57419b3SSteven Rostedt# BISECT_BAD is the commit to label as bad (accepts all git bad commit types)
776a75fececSSteven Rostedt#
777a75fececSSteven Rostedt# The above three options are required for a bisect operation.
778a75fececSSteven Rostedt#
779a57419b3SSteven Rostedt# BISECT_REPLAY = /path/to/replay/file (optional, default undefined)
780a75fececSSteven Rostedt#
781a75fececSSteven Rostedt#   If an operation failed in the bisect that was not expected to
782a75fececSSteven Rostedt#   fail. Then the test ends. The state of the BUILD_DIR will be
783a57419b3SSteven Rostedt#   left off at where the failure occurred. You can examine the
784a75fececSSteven Rostedt#   reason for the failure, and perhaps even find a git commit
785a75fececSSteven Rostedt#   that would work to continue with. You can run:
786a75fececSSteven Rostedt#
787a75fececSSteven Rostedt#   git bisect log > /path/to/replay/file
788a75fececSSteven Rostedt#
789a57419b3SSteven Rostedt#   The adding:
790a75fececSSteven Rostedt#
791a57419b3SSteven Rostedt#    BISECT_REPLAY= /path/to/replay/file
792a75fececSSteven Rostedt#
793a57419b3SSteven Rostedt#   And running the test again. The test will perform the initial
794a57419b3SSteven Rostedt#    git bisect start, git bisect good, and git bisect bad, and
795a57419b3SSteven Rostedt#    then it will run git bisect replay on this file, before
796a57419b3SSteven Rostedt#    continuing with the bisect.
797a57419b3SSteven Rostedt#
798a57419b3SSteven Rostedt# BISECT_START = commit (optional, default undefined)
799a57419b3SSteven Rostedt#
800a57419b3SSteven Rostedt#   As with BISECT_REPLAY, if the test failed on a commit that
801a75fececSSteven Rostedt#   just happen to have a bad commit in the middle of the bisect,
802a57419b3SSteven Rostedt#   and you need to skip it. If BISECT_START is defined, it
803a57419b3SSteven Rostedt#   will checkout that commit after doing the initial git bisect start,
804a57419b3SSteven Rostedt#   git bisect good, git bisect bad, and running the git bisect replay
805a57419b3SSteven Rostedt#   if the BISECT_REPLAY is set.
806a75fececSSteven Rostedt#
807c23dca7cSSteven Rostedt# BISECT_SKIP = 1 (optional, default 0)
808c23dca7cSSteven Rostedt#
809c23dca7cSSteven Rostedt#   If BISECT_TYPE is set to test but the build fails, ktest will
810c23dca7cSSteven Rostedt#   simply fail the test and end their. You could use BISECT_REPLAY
811c23dca7cSSteven Rostedt#   and BISECT_START to resume after you found a new starting point,
812c23dca7cSSteven Rostedt#   or you could set BISECT_SKIP to 1. If BISECT_SKIP is set to 1,
813c23dca7cSSteven Rostedt#   when something other than the BISECT_TYPE fails, ktest.pl will
814c23dca7cSSteven Rostedt#   run "git bisect skip" and try again.
815c23dca7cSSteven Rostedt#
8163410f6fdSSteven Rostedt# BISECT_FILES = <path> (optional, default undefined)
8173410f6fdSSteven Rostedt#
8183410f6fdSSteven Rostedt#   To just run the git bisect on a specific path, set BISECT_FILES.
8193410f6fdSSteven Rostedt#   For example:
8203410f6fdSSteven Rostedt#
8213410f6fdSSteven Rostedt#     BISECT_FILES = arch/x86 kernel/time
8223410f6fdSSteven Rostedt#
8233410f6fdSSteven Rostedt#   Will run the bisect with "git bisect start -- arch/x86 kernel/time"
8243410f6fdSSteven Rostedt#
825a57419b3SSteven Rostedt# BISECT_REVERSE = 1 (optional, default 0)
826a75fececSSteven Rostedt#
827a75fececSSteven Rostedt#   In those strange instances where it was broken forever
828a75fececSSteven Rostedt#   and you are trying to find where it started to work!
829a57419b3SSteven Rostedt#   Set BISECT_GOOD to the commit that was last known to fail
830a57419b3SSteven Rostedt#   Set BISECT_BAD to the commit that is known to start working.
831a57419b3SSteven Rostedt#   With BISECT_REVERSE = 1, The test will consider failures as
832a57419b3SSteven Rostedt#   good, and success as bad.
833a75fececSSteven Rostedt#
834c960bb9fSSteven Rostedt# BISECT_MANUAL = 1 (optional, default 0)
835c960bb9fSSteven Rostedt#
836c960bb9fSSteven Rostedt#   In case there's a problem with automating the bisect for
837c960bb9fSSteven Rostedt#   whatever reason. (Can't reboot, want to inspect each iteration)
838c960bb9fSSteven Rostedt#   Doing a BISECT_MANUAL will have the test wait for you to
839c960bb9fSSteven Rostedt#   tell it if the test passed or failed after each iteration.
840c960bb9fSSteven Rostedt#   This is basicall the same as running git bisect yourself
841c960bb9fSSteven Rostedt#   but ktest will rebuild and install the kernel for you.
842c960bb9fSSteven Rostedt#
843a57419b3SSteven Rostedt# BISECT_CHECK = 1 (optional, default 0)
844a75fececSSteven Rostedt#
845a75fececSSteven Rostedt#   Just to be sure the good is good and bad is bad, setting
846a57419b3SSteven Rostedt#   BISECT_CHECK to 1 will start the bisect by first checking
847a57419b3SSteven Rostedt#   out BISECT_BAD and makes sure it fails, then it will check
848a57419b3SSteven Rostedt#   out BISECT_GOOD and makes sure it succeeds before starting
849a57419b3SSteven Rostedt#   the bisect (it works for BISECT_REVERSE too).
850a75fececSSteven Rostedt#
851a57419b3SSteven Rostedt#   You can limit the test to just check BISECT_GOOD or
852a57419b3SSteven Rostedt#   BISECT_BAD with BISECT_CHECK = good or
853a57419b3SSteven Rostedt#   BISECT_CHECK = bad, respectively.
854a75fececSSteven Rostedt#
855a75fececSSteven Rostedt# Example:
856a57419b3SSteven Rostedt#   TEST_START
857a57419b3SSteven Rostedt#   TEST_TYPE = bisect
858a57419b3SSteven Rostedt#   BISECT_GOOD = v2.6.36
859a57419b3SSteven Rostedt#   BISECT_BAD = b5153163ed580e00c67bdfecb02b2e3843817b3e
860a57419b3SSteven Rostedt#   BISECT_TYPE = build
861a57419b3SSteven Rostedt#   MIN_CONFIG = /home/test/config-bisect
862d1fbd7e6SSteven Rostedt#
863d1fbd7e6SSteven Rostedt#
864d1fbd7e6SSteven Rostedt#
865d1fbd7e6SSteven Rostedt# For TEST_TYPE = config_bisect
866d1fbd7e6SSteven Rostedt#
867d1fbd7e6SSteven Rostedt#  In those cases that you have two different configs. One of them
868d1fbd7e6SSteven Rostedt#  work, the other does not, and you do not know what config causes
869d1fbd7e6SSteven Rostedt#  the problem.
870d1fbd7e6SSteven Rostedt#  The TEST_TYPE config_bisect will bisect the bad config looking for
871d1fbd7e6SSteven Rostedt#  what config causes the failure.
872d1fbd7e6SSteven Rostedt#
873d1fbd7e6SSteven Rostedt#  The way it works is this:
874d1fbd7e6SSteven Rostedt#
875d1fbd7e6SSteven Rostedt#   First it finds a config to work with. Since a different version, or
876d1fbd7e6SSteven Rostedt#   MIN_CONFIG may cause different dependecies, it must run through this
877d1fbd7e6SSteven Rostedt#   preparation.
878d1fbd7e6SSteven Rostedt#
879d1fbd7e6SSteven Rostedt#   Overwrites any config set in the bad config with a config set in
880d1fbd7e6SSteven Rostedt#   either the MIN_CONFIG or ADD_CONFIG. Thus, make sure these configs
881d1fbd7e6SSteven Rostedt#   are minimal and do not disable configs you want to test:
882d1fbd7e6SSteven Rostedt#   (ie.  # CONFIG_FOO is not set).
883d1fbd7e6SSteven Rostedt#
884d1fbd7e6SSteven Rostedt#   An oldconfig is run on the bad config and any new config that
885d1fbd7e6SSteven Rostedt#   appears will be added to the configs to test.
886d1fbd7e6SSteven Rostedt#
887d1fbd7e6SSteven Rostedt#   Finally, it generates a config with the above result and runs it
888d1fbd7e6SSteven Rostedt#   again through make oldconfig to produce a config that should be
889d1fbd7e6SSteven Rostedt#   satisfied by kconfig.
890d1fbd7e6SSteven Rostedt#
891d1fbd7e6SSteven Rostedt#   Then it starts the bisect.
892d1fbd7e6SSteven Rostedt#
893d1fbd7e6SSteven Rostedt#   The configs to test are cut in half. If all the configs in this
894d1fbd7e6SSteven Rostedt#   half depend on a config in the other half, then the other half
895d1fbd7e6SSteven Rostedt#   is tested instead. If no configs are enabled by either half, then
896d1fbd7e6SSteven Rostedt#   this means a circular dependency exists and the test fails.
897d1fbd7e6SSteven Rostedt#
898d1fbd7e6SSteven Rostedt#   A config is created with the test half, and the bisect test is run.
899d1fbd7e6SSteven Rostedt#
900d1fbd7e6SSteven Rostedt#   If the bisect succeeds, then all configs in the generated config
901d1fbd7e6SSteven Rostedt#   are removed from the configs to test and added to the configs that
902d1fbd7e6SSteven Rostedt#   will be enabled for all builds (they will be enabled, but not be part
903d1fbd7e6SSteven Rostedt#   of the configs to examine).
904d1fbd7e6SSteven Rostedt#
905d1fbd7e6SSteven Rostedt#   If the bisect fails, then all test configs that were not enabled by
906d1fbd7e6SSteven Rostedt#   the config file are removed from the test. These configs will not
907d1fbd7e6SSteven Rostedt#   be enabled in future tests. Since current config failed, we consider
908d1fbd7e6SSteven Rostedt#   this to be a subset of the config that we started with.
909d1fbd7e6SSteven Rostedt#
910d1fbd7e6SSteven Rostedt#   When we are down to one config, it is considered the bad config.
911d1fbd7e6SSteven Rostedt#
912d1fbd7e6SSteven Rostedt#   Note, the config chosen may not be the true bad config. Due to
913d1fbd7e6SSteven Rostedt#   dependencies and selections of the kbuild system, mulitple
914d1fbd7e6SSteven Rostedt#   configs may be needed to cause a failure. If you disable the
915d1fbd7e6SSteven Rostedt#   config that was found and restart the test, if the test fails
916d1fbd7e6SSteven Rostedt#   again, it is recommended to rerun the config_bisect with a new
917d1fbd7e6SSteven Rostedt#   bad config without the found config enabled.
918d1fbd7e6SSteven Rostedt#
919d1fbd7e6SSteven Rostedt#  The option BUILD_TYPE will be ignored.
920d1fbd7e6SSteven Rostedt#
921d1fbd7e6SSteven Rostedt#  CONFIG_BISECT_TYPE is the type of test to perform:
922d1fbd7e6SSteven Rostedt#	build	- bad fails to build
923d1fbd7e6SSteven Rostedt#	boot	- bad builds but fails to boot
924d1fbd7e6SSteven Rostedt#	test	- bad boots but fails a test
925d1fbd7e6SSteven Rostedt#
926d1fbd7e6SSteven Rostedt#  CONFIG_BISECT is the config that failed to boot
927d1fbd7e6SSteven Rostedt#
928c960bb9fSSteven Rostedt#  If BISECT_MANUAL is set, it will pause between iterations.
929c960bb9fSSteven Rostedt#  This is useful to use just ktest.pl just for the config bisect.
930c960bb9fSSteven Rostedt#  If you set it to build, it will run the bisect and you can
931c960bb9fSSteven Rostedt#  control what happens in between iterations. It will ask you if
932c960bb9fSSteven Rostedt#  the test succeeded or not and continue the config bisect.
93330f75da5SSteven Rostedt#
93430f75da5SSteven Rostedt# CONFIG_BISECT_GOOD (optional)
93530f75da5SSteven Rostedt#  If you have a good config to start with, then you
93630f75da5SSteven Rostedt#  can specify it with CONFIG_BISECT_GOOD. Otherwise
93730f75da5SSteven Rostedt#  the MIN_CONFIG is the base.
938c960bb9fSSteven Rostedt#
939d1fbd7e6SSteven Rostedt# Example:
940d1fbd7e6SSteven Rostedt#   TEST_START
941d1fbd7e6SSteven Rostedt#   TEST_TYPE = config_bisect
942d1fbd7e6SSteven Rostedt#   CONFIG_BISECT_TYPE = build
943d1fbd7e6SSteven Rostedt#   CONFIG_BISECT = /home/test/�onfig-bad
944d1fbd7e6SSteven Rostedt#   MIN_CONFIG = /home/test/config-min
945c960bb9fSSteven Rostedt#   BISECT_MANUAL = 1
946d1fbd7e6SSteven Rostedt#
9474c4ab120SSteven Rostedt#
9484c4ab120SSteven Rostedt#
9494c4ab120SSteven Rostedt# For TEST_TYPE = make_min_config
9504c4ab120SSteven Rostedt#
9514c4ab120SSteven Rostedt#  After doing a make localyesconfig, your kernel configuration may
9524c4ab120SSteven Rostedt#  not be the most useful minimum configuration. Having a true minimum
9534c4ab120SSteven Rostedt#  config that you can use against other configs is very useful if
9544c4ab120SSteven Rostedt#  someone else has a config that breaks on your code. By only forcing
9554c4ab120SSteven Rostedt#  those configurations that are truly required to boot your machine
9564c4ab120SSteven Rostedt#  will give you less of a chance that one of your set configurations
9574c4ab120SSteven Rostedt#  will make the bug go away. This will give you a better chance to
9584c4ab120SSteven Rostedt#  be able to reproduce the reported bug matching the broken config.
9594c4ab120SSteven Rostedt#
9604c4ab120SSteven Rostedt#  Note, this does take some time, and may require you to run the
9614c4ab120SSteven Rostedt#  test over night, or perhaps over the weekend. But it also allows
9624c4ab120SSteven Rostedt#  you to interrupt it, and gives you the current minimum config
9634c4ab120SSteven Rostedt#  that was found till that time.
9644c4ab120SSteven Rostedt#
9654c4ab120SSteven Rostedt#  Note, this test automatically assumes a BUILD_TYPE of oldconfig
9664c4ab120SSteven Rostedt#  and its test type acts like boot.
9674c4ab120SSteven Rostedt#  TODO: add a test version that makes the config do more than just
9684c4ab120SSteven Rostedt#   boot, like having network access.
9694c4ab120SSteven Rostedt#
970b9066f6cSSteven Rostedt#  To save time, the test does not just grab any option and test
971b9066f6cSSteven Rostedt#  it. The Kconfig files are examined to determine the dependencies
972b9066f6cSSteven Rostedt#  of the configs. If a config is chosen that depends on another
973b9066f6cSSteven Rostedt#  config, that config will be checked first. By checking the
974b9066f6cSSteven Rostedt#  parents first, we can eliminate whole groups of configs that
975b9066f6cSSteven Rostedt#  may have been enabled.
976b9066f6cSSteven Rostedt#
977b9066f6cSSteven Rostedt#  For example, if a USB device config is chosen and depends on CONFIG_USB,
978b9066f6cSSteven Rostedt#  the CONFIG_USB will be tested before the device. If CONFIG_USB is
979b9066f6cSSteven Rostedt#  found not to be needed, it, as well as all configs that depend on
980b9066f6cSSteven Rostedt#  it, will be disabled and removed from the current min_config.
981b9066f6cSSteven Rostedt#
9824c4ab120SSteven Rostedt#  OUTPUT_MIN_CONFIG is the path and filename of the file that will
9834c4ab120SSteven Rostedt#   be created from the MIN_CONFIG. If you interrupt the test, set
9844c4ab120SSteven Rostedt#   this file as your new min config, and use it to continue the test.
9854c4ab120SSteven Rostedt#   This file does not need to exist on start of test.
9864c4ab120SSteven Rostedt#   This file is not created until a config is found that can be removed.
98735ce5952SSteven Rostedt#   If this file exists, you will be prompted if you want to use it
98835ce5952SSteven Rostedt#   as the min_config (overriding MIN_CONFIG) if START_MIN_CONFIG
98935ce5952SSteven Rostedt#   is not defined.
9904c4ab120SSteven Rostedt#   (required field)
9914c4ab120SSteven Rostedt#
9924c4ab120SSteven Rostedt#  START_MIN_CONFIG is the config to use to start the test with.
9934c4ab120SSteven Rostedt#   you can set this as the same OUTPUT_MIN_CONFIG, but if you do
9944c4ab120SSteven Rostedt#   the OUTPUT_MIN_CONFIG file must exist.
9954c4ab120SSteven Rostedt#   (default MIN_CONFIG)
9964c4ab120SSteven Rostedt#
9974c4ab120SSteven Rostedt#  IGNORE_CONFIG is used to specify a config file that has configs that
9984c4ab120SSteven Rostedt#   you already know must be set. Configs are written here that have
9994c4ab120SSteven Rostedt#   been tested and proved to be required. It is best to define this
10004c4ab120SSteven Rostedt#   file if you intend on interrupting the test and running it where
10014c4ab120SSteven Rostedt#   it left off. New configs that it finds will be written to this file
10024c4ab120SSteven Rostedt#   and will not be tested again in later runs.
10034c4ab120SSteven Rostedt#   (optional)
10044c4ab120SSteven Rostedt#
10054c4ab120SSteven Rostedt# Example:
10064c4ab120SSteven Rostedt#
10074c4ab120SSteven Rostedt#  TEST_TYPE = make_min_config
10084c4ab120SSteven Rostedt#  OUTPUT_MIN_CONFIG = /path/to/config-new-min
10094c4ab120SSteven Rostedt#  START_MIN_CONFIG = /path/to/config-min
10104c4ab120SSteven Rostedt#  IGNORE_CONFIG = /path/to/config-tested
10114c4ab120SSteven Rostedt#
1012