/openbmc/openbmc-test-automation/lib/ |
H A D | gen_plug_in.py | 7423c01a Tue Oct 04 10:27:21 CDT 2016 Michael Walsh <micwalsh@us.ibm.com> Providing plug-in support: Typically, a test program is written to perform certain basic tests on a test machine. For example, one might write an "obmc_boot" program that performs various boot tests on the Open BMC machine. Experience has shown that over time, additional testing needs often arise. Examples of such additional testing needs might include: - Data base logging of results - Performance measurements - Memory leak analysis - Hardware verification - Error log (sels) analysis - SOL_console The developer could add additional parms to obmc_boot and likewise add supporting code in obmc_boot each time a need arises. Users would employ these new functions as follows: obmc_boot --perf=1 --mem_leak=1 --db_logging=1 --db_userid=xxxx However, another option would be to add general-purpose plug-in support to obmc_boot. This would allow the user to indicate to obmc_boot which plug-in packages it ought to run. Such plug-in packages could be written in any langauge whatsoever: Robot, python, bash, perl, C++. An example call to obmc_boot would then look something like this: obmc_boot --plug_in_dir_paths="Perf:Mem_leak:DB_logging" Now all the obmc_boot developer needs to do is call the plug-in processing module (process_plug_in_packages.py) at various call points which are agreed upon by the obmc_boot developer and the plug-in developers. Example call points which can be implemented are: setup - Called at the start of obmc_boot pre_boot - Called before each boot test initiated by obmc_boot post_boot - Called after each boot test initiated by obmc_boot cleanup - Called at the end of obmc_boot This allows the choice of options to be passed as data to obmc_boot. The advantages of this approach are: - Much less maintenance of the original test program (obmc_boot). - Since plug-ins are separate from the main test program, users are free to have plug-ins that suit their environments. One user may wish to log results to a database that is of no interest to the rest of the world. Such a plug-in can be written and need never be pushed to gerrit/github. - One can even write temporary plug-ins designed just to collect data or stop when a particular defect occurs. In our current environment, the concept has proven exceedingly useful. We have over 40 permanent plug-ins and in our temp plug-in directory, we still have over 80 plug-ins. Change-Id: Iee0ea950cffaef202d56da4dae7c044b6366a59c Signed-off-by: Michael Walsh <micwalsh@us.ibm.com>
|
H A D | gen_arg.py | 7423c01a Tue Oct 04 10:27:21 CDT 2016 Michael Walsh <micwalsh@us.ibm.com> Providing plug-in support: Typically, a test program is written to perform certain basic tests on a test machine. For example, one might write an "obmc_boot" program that performs various boot tests on the Open BMC machine. Experience has shown that over time, additional testing needs often arise. Examples of such additional testing needs might include: - Data base logging of results - Performance measurements - Memory leak analysis - Hardware verification - Error log (sels) analysis - SOL_console The developer could add additional parms to obmc_boot and likewise add supporting code in obmc_boot each time a need arises. Users would employ these new functions as follows: obmc_boot --perf=1 --mem_leak=1 --db_logging=1 --db_userid=xxxx However, another option would be to add general-purpose plug-in support to obmc_boot. This would allow the user to indicate to obmc_boot which plug-in packages it ought to run. Such plug-in packages could be written in any langauge whatsoever: Robot, python, bash, perl, C++. An example call to obmc_boot would then look something like this: obmc_boot --plug_in_dir_paths="Perf:Mem_leak:DB_logging" Now all the obmc_boot developer needs to do is call the plug-in processing module (process_plug_in_packages.py) at various call points which are agreed upon by the obmc_boot developer and the plug-in developers. Example call points which can be implemented are: setup - Called at the start of obmc_boot pre_boot - Called before each boot test initiated by obmc_boot post_boot - Called after each boot test initiated by obmc_boot cleanup - Called at the end of obmc_boot This allows the choice of options to be passed as data to obmc_boot. The advantages of this approach are: - Much less maintenance of the original test program (obmc_boot). - Since plug-ins are separate from the main test program, users are free to have plug-ins that suit their environments. One user may wish to log results to a database that is of no interest to the rest of the world. Such a plug-in can be written and need never be pushed to gerrit/github. - One can even write temporary plug-ins designed just to collect data or stop when a particular defect occurs. In our current environment, the concept has proven exceedingly useful. We have over 40 permanent plug-ins and in our temp plug-in directory, we still have over 80 plug-ins. Change-Id: Iee0ea950cffaef202d56da4dae7c044b6366a59c Signed-off-by: Michael Walsh <micwalsh@us.ibm.com>
|
H A D | gen_robot_print.py | 7423c01a Tue Oct 04 10:27:21 CDT 2016 Michael Walsh <micwalsh@us.ibm.com> Providing plug-in support: Typically, a test program is written to perform certain basic tests on a test machine. For example, one might write an "obmc_boot" program that performs various boot tests on the Open BMC machine. Experience has shown that over time, additional testing needs often arise. Examples of such additional testing needs might include: - Data base logging of results - Performance measurements - Memory leak analysis - Hardware verification - Error log (sels) analysis - SOL_console The developer could add additional parms to obmc_boot and likewise add supporting code in obmc_boot each time a need arises. Users would employ these new functions as follows: obmc_boot --perf=1 --mem_leak=1 --db_logging=1 --db_userid=xxxx However, another option would be to add general-purpose plug-in support to obmc_boot. This would allow the user to indicate to obmc_boot which plug-in packages it ought to run. Such plug-in packages could be written in any langauge whatsoever: Robot, python, bash, perl, C++. An example call to obmc_boot would then look something like this: obmc_boot --plug_in_dir_paths="Perf:Mem_leak:DB_logging" Now all the obmc_boot developer needs to do is call the plug-in processing module (process_plug_in_packages.py) at various call points which are agreed upon by the obmc_boot developer and the plug-in developers. Example call points which can be implemented are: setup - Called at the start of obmc_boot pre_boot - Called before each boot test initiated by obmc_boot post_boot - Called after each boot test initiated by obmc_boot cleanup - Called at the end of obmc_boot This allows the choice of options to be passed as data to obmc_boot. The advantages of this approach are: - Much less maintenance of the original test program (obmc_boot). - Since plug-ins are separate from the main test program, users are free to have plug-ins that suit their environments. One user may wish to log results to a database that is of no interest to the rest of the world. Such a plug-in can be written and need never be pushed to gerrit/github. - One can even write temporary plug-ins designed just to collect data or stop when a particular defect occurs. In our current environment, the concept has proven exceedingly useful. We have over 40 permanent plug-ins and in our temp plug-in directory, we still have over 80 plug-ins. Change-Id: Iee0ea950cffaef202d56da4dae7c044b6366a59c Signed-off-by: Michael Walsh <micwalsh@us.ibm.com>
|
H A D | gen_valid.py | 7423c01a Tue Oct 04 10:27:21 CDT 2016 Michael Walsh <micwalsh@us.ibm.com> Providing plug-in support: Typically, a test program is written to perform certain basic tests on a test machine. For example, one might write an "obmc_boot" program that performs various boot tests on the Open BMC machine. Experience has shown that over time, additional testing needs often arise. Examples of such additional testing needs might include: - Data base logging of results - Performance measurements - Memory leak analysis - Hardware verification - Error log (sels) analysis - SOL_console The developer could add additional parms to obmc_boot and likewise add supporting code in obmc_boot each time a need arises. Users would employ these new functions as follows: obmc_boot --perf=1 --mem_leak=1 --db_logging=1 --db_userid=xxxx However, another option would be to add general-purpose plug-in support to obmc_boot. This would allow the user to indicate to obmc_boot which plug-in packages it ought to run. Such plug-in packages could be written in any langauge whatsoever: Robot, python, bash, perl, C++. An example call to obmc_boot would then look something like this: obmc_boot --plug_in_dir_paths="Perf:Mem_leak:DB_logging" Now all the obmc_boot developer needs to do is call the plug-in processing module (process_plug_in_packages.py) at various call points which are agreed upon by the obmc_boot developer and the plug-in developers. Example call points which can be implemented are: setup - Called at the start of obmc_boot pre_boot - Called before each boot test initiated by obmc_boot post_boot - Called after each boot test initiated by obmc_boot cleanup - Called at the end of obmc_boot This allows the choice of options to be passed as data to obmc_boot. The advantages of this approach are: - Much less maintenance of the original test program (obmc_boot). - Since plug-ins are separate from the main test program, users are free to have plug-ins that suit their environments. One user may wish to log results to a database that is of no interest to the rest of the world. Such a plug-in can be written and need never be pushed to gerrit/github. - One can even write temporary plug-ins designed just to collect data or stop when a particular defect occurs. In our current environment, the concept has proven exceedingly useful. We have over 40 permanent plug-ins and in our temp plug-in directory, we still have over 80 plug-ins. Change-Id: Iee0ea950cffaef202d56da4dae7c044b6366a59c Signed-off-by: Michael Walsh <micwalsh@us.ibm.com>
|
H A D | gen_misc.py | 7423c01a Tue Oct 04 10:27:21 CDT 2016 Michael Walsh <micwalsh@us.ibm.com> Providing plug-in support: Typically, a test program is written to perform certain basic tests on a test machine. For example, one might write an "obmc_boot" program that performs various boot tests on the Open BMC machine. Experience has shown that over time, additional testing needs often arise. Examples of such additional testing needs might include: - Data base logging of results - Performance measurements - Memory leak analysis - Hardware verification - Error log (sels) analysis - SOL_console The developer could add additional parms to obmc_boot and likewise add supporting code in obmc_boot each time a need arises. Users would employ these new functions as follows: obmc_boot --perf=1 --mem_leak=1 --db_logging=1 --db_userid=xxxx However, another option would be to add general-purpose plug-in support to obmc_boot. This would allow the user to indicate to obmc_boot which plug-in packages it ought to run. Such plug-in packages could be written in any langauge whatsoever: Robot, python, bash, perl, C++. An example call to obmc_boot would then look something like this: obmc_boot --plug_in_dir_paths="Perf:Mem_leak:DB_logging" Now all the obmc_boot developer needs to do is call the plug-in processing module (process_plug_in_packages.py) at various call points which are agreed upon by the obmc_boot developer and the plug-in developers. Example call points which can be implemented are: setup - Called at the start of obmc_boot pre_boot - Called before each boot test initiated by obmc_boot post_boot - Called after each boot test initiated by obmc_boot cleanup - Called at the end of obmc_boot This allows the choice of options to be passed as data to obmc_boot. The advantages of this approach are: - Much less maintenance of the original test program (obmc_boot). - Since plug-ins are separate from the main test program, users are free to have plug-ins that suit their environments. One user may wish to log results to a database that is of no interest to the rest of the world. Such a plug-in can be written and need never be pushed to gerrit/github. - One can even write temporary plug-ins designed just to collect data or stop when a particular defect occurs. In our current environment, the concept has proven exceedingly useful. We have over 40 permanent plug-ins and in our temp plug-in directory, we still have over 80 plug-ins. Change-Id: Iee0ea950cffaef202d56da4dae7c044b6366a59c Signed-off-by: Michael Walsh <micwalsh@us.ibm.com>
|
H A D | gen_print.py | 7423c01a Tue Oct 04 10:27:21 CDT 2016 Michael Walsh <micwalsh@us.ibm.com> Providing plug-in support: Typically, a test program is written to perform certain basic tests on a test machine. For example, one might write an "obmc_boot" program that performs various boot tests on the Open BMC machine. Experience has shown that over time, additional testing needs often arise. Examples of such additional testing needs might include: - Data base logging of results - Performance measurements - Memory leak analysis - Hardware verification - Error log (sels) analysis - SOL_console The developer could add additional parms to obmc_boot and likewise add supporting code in obmc_boot each time a need arises. Users would employ these new functions as follows: obmc_boot --perf=1 --mem_leak=1 --db_logging=1 --db_userid=xxxx However, another option would be to add general-purpose plug-in support to obmc_boot. This would allow the user to indicate to obmc_boot which plug-in packages it ought to run. Such plug-in packages could be written in any langauge whatsoever: Robot, python, bash, perl, C++. An example call to obmc_boot would then look something like this: obmc_boot --plug_in_dir_paths="Perf:Mem_leak:DB_logging" Now all the obmc_boot developer needs to do is call the plug-in processing module (process_plug_in_packages.py) at various call points which are agreed upon by the obmc_boot developer and the plug-in developers. Example call points which can be implemented are: setup - Called at the start of obmc_boot pre_boot - Called before each boot test initiated by obmc_boot post_boot - Called after each boot test initiated by obmc_boot cleanup - Called at the end of obmc_boot This allows the choice of options to be passed as data to obmc_boot. The advantages of this approach are: - Much less maintenance of the original test program (obmc_boot). - Since plug-ins are separate from the main test program, users are free to have plug-ins that suit their environments. One user may wish to log results to a database that is of no interest to the rest of the world. Such a plug-in can be written and need never be pushed to gerrit/github. - One can even write temporary plug-ins designed just to collect data or stop when a particular defect occurs. In our current environment, the concept has proven exceedingly useful. We have over 40 permanent plug-ins and in our temp plug-in directory, we still have over 80 plug-ins. Change-Id: Iee0ea950cffaef202d56da4dae7c044b6366a59c Signed-off-by: Michael Walsh <micwalsh@us.ibm.com>
|
/openbmc/openbmc-test-automation/bin/ |
H A D | validate_plug_ins.py | 7423c01a Tue Oct 04 10:27:21 CDT 2016 Michael Walsh <micwalsh@us.ibm.com> Providing plug-in support: Typically, a test program is written to perform certain basic tests on a test machine. For example, one might write an "obmc_boot" program that performs various boot tests on the Open BMC machine. Experience has shown that over time, additional testing needs often arise. Examples of such additional testing needs might include: - Data base logging of results - Performance measurements - Memory leak analysis - Hardware verification - Error log (sels) analysis - SOL_console The developer could add additional parms to obmc_boot and likewise add supporting code in obmc_boot each time a need arises. Users would employ these new functions as follows: obmc_boot --perf=1 --mem_leak=1 --db_logging=1 --db_userid=xxxx However, another option would be to add general-purpose plug-in support to obmc_boot. This would allow the user to indicate to obmc_boot which plug-in packages it ought to run. Such plug-in packages could be written in any langauge whatsoever: Robot, python, bash, perl, C++. An example call to obmc_boot would then look something like this: obmc_boot --plug_in_dir_paths="Perf:Mem_leak:DB_logging" Now all the obmc_boot developer needs to do is call the plug-in processing module (process_plug_in_packages.py) at various call points which are agreed upon by the obmc_boot developer and the plug-in developers. Example call points which can be implemented are: setup - Called at the start of obmc_boot pre_boot - Called before each boot test initiated by obmc_boot post_boot - Called after each boot test initiated by obmc_boot cleanup - Called at the end of obmc_boot This allows the choice of options to be passed as data to obmc_boot. The advantages of this approach are: - Much less maintenance of the original test program (obmc_boot). - Since plug-ins are separate from the main test program, users are free to have plug-ins that suit their environments. One user may wish to log results to a database that is of no interest to the rest of the world. Such a plug-in can be written and need never be pushed to gerrit/github. - One can even write temporary plug-ins designed just to collect data or stop when a particular defect occurs. In our current environment, the concept has proven exceedingly useful. We have over 40 permanent plug-ins and in our temp plug-in directory, we still have over 80 plug-ins. Change-Id: Iee0ea950cffaef202d56da4dae7c044b6366a59c Signed-off-by: Michael Walsh <micwalsh@us.ibm.com>
|
H A D | process_plug_in_packages.py | 7423c01a Tue Oct 04 10:27:21 CDT 2016 Michael Walsh <micwalsh@us.ibm.com> Providing plug-in support: Typically, a test program is written to perform certain basic tests on a test machine. For example, one might write an "obmc_boot" program that performs various boot tests on the Open BMC machine. Experience has shown that over time, additional testing needs often arise. Examples of such additional testing needs might include: - Data base logging of results - Performance measurements - Memory leak analysis - Hardware verification - Error log (sels) analysis - SOL_console The developer could add additional parms to obmc_boot and likewise add supporting code in obmc_boot each time a need arises. Users would employ these new functions as follows: obmc_boot --perf=1 --mem_leak=1 --db_logging=1 --db_userid=xxxx However, another option would be to add general-purpose plug-in support to obmc_boot. This would allow the user to indicate to obmc_boot which plug-in packages it ought to run. Such plug-in packages could be written in any langauge whatsoever: Robot, python, bash, perl, C++. An example call to obmc_boot would then look something like this: obmc_boot --plug_in_dir_paths="Perf:Mem_leak:DB_logging" Now all the obmc_boot developer needs to do is call the plug-in processing module (process_plug_in_packages.py) at various call points which are agreed upon by the obmc_boot developer and the plug-in developers. Example call points which can be implemented are: setup - Called at the start of obmc_boot pre_boot - Called before each boot test initiated by obmc_boot post_boot - Called after each boot test initiated by obmc_boot cleanup - Called at the end of obmc_boot This allows the choice of options to be passed as data to obmc_boot. The advantages of this approach are: - Much less maintenance of the original test program (obmc_boot). - Since plug-ins are separate from the main test program, users are free to have plug-ins that suit their environments. One user may wish to log results to a database that is of no interest to the rest of the world. Such a plug-in can be written and need never be pushed to gerrit/github. - One can even write temporary plug-ins designed just to collect data or stop when a particular defect occurs. In our current environment, the concept has proven exceedingly useful. We have over 40 permanent plug-ins and in our temp plug-in directory, we still have over 80 plug-ins. Change-Id: Iee0ea950cffaef202d56da4dae7c044b6366a59c Signed-off-by: Michael Walsh <micwalsh@us.ibm.com>
|