1Contributing to OpenBMC Test Automation 2======================================= 3Guide to working on OpenBMC test automation. This document will always be a 4work-in-progress, feel free to propose changes. 5 6Submitting changes via Gerrit server 7------------------------------------ 8- Reference [OpenBMC CLA signers](https://github.com/openbmc/openbmc-tools/blob/master/emilyshaffer/cla-signers/cla-signers) 9- Reference [OpenBMC docs](https://github.com/openbmc/docs/blob/master/contributing.md#submitting-changes-via-gerrit-server) 10 11Robot Coding Guidelines 12----------------------- 13- For this project, we will write Robot keyword definitions in either Robot 14 or Python. Robot code should be quite simple. Therefore, if the algorithm 15 in question is the least bit complex, please write it in Python. 16- Observe a maximum line length of 79 characters. 17- Avoid trailing space at the end of any line of Robot code. 18- Avoid the use of tabs. 19- Robot supports delimiting cells with either two or more spaces or with a 20 pipe symbol (e.g. "\|"). Our team has chosen to use spaces rather than the 21 pipe character. Make sure all space delimiters in Robot code are the 22 **minimum** of two spaces. There may be some exceptions to this rule. 23 24 Exceptions to two-space delimiter rule: 25 - When you wish to line up resource, library or variable values: 26 ``` 27 Library Lib1 28 Resource Resource1 29 *** Variables *** 30 ${var1} ${EMPTY} 31 ``` 32 - When you wish to line up fields for test templates: 33 ``` 34 [Template] Set System LED State 35 # LED Name LED State 36 power On 37 power Off 38 ``` 39 - When you wish to indent if/else or loop bodies for visual effect: 40 ``` 41 Run Keyword If '${this}' == '${that}' 42 ... Log Bla, bla... 43 ... ELSE 44 ... Run Keywords Key1 parms 45 ... AND Key2 parms 46 ``` 47- Use spaces to make conditions more readable: 48 49 Correct example: 50 ``` 51 Run Keyword If '${var1}' == '${0}' My Keyword 52 ``` 53 Incorrect example: 54 ``` 55 Run Keyword If '${var1}'=='${0}' My Keyword 56 ``` 57- When you define or call a Robot keyword, Robot pays no attention to spaces, 58 underscores or case. However, our team will observe the following 59 conventions in both our definitions and our calls: 60 - Separate words with single spaces. 61 - Capitalize the first character of each word. 62 - Capitalize all characters in any word that is an acronym (e.g. JSON, BMC, 63 etc). 64 65 Examples: 66 ``` 67 *** Keywords *** 68 69 This Is Correct 70 71 # This keyword name is correct. 72 73 this_is_incorrect 74 75 # This keyword name is incorrect because of 1) the 76 # underscores instead of spaces and 2) the failure to 77 # capitalize each word in the keyword. 78 79 soisthis 80 81 # This keyword name is incorrect because of 1) a failure to 82 # separate words with spaces and 2) a failure to capitalize 83 # each word in the keyword. 84 85 BMC Is An Acronym 86 87 # This keyword name is correct. Note that "BMC" is an 88 # acronym and as such is entirely uppercase. 89 ``` 90- Documentation strings: 91 - Each documentation string should be phrased as an **English command**. 92 Punctuate it correctly with the first word capitalized and a period at 93 the end. 94 95 Correct example: 96 ``` 97 Boot BMC 98 [Documentation] Boot the BMC. 99 ``` 100 Incorrect example: 101 ``` 102 Boot BMC 103 [Documentation] This keyword boots the BMC. 104 105 # The doc string above is not phrased as a command. 106 ``` 107 - Doc strings should be just one terse, descriptive sentence. 108 Remember that this doc string shows up in the HTML log file. Put 109 additional commentary below in standard comment lines. 110 111 Correct example: 112 ``` 113 Stop SOL Console Logging 114 115 [Documentation] Stop system console logging and return log output. 116 ``` 117 Incorrect example: 118 ``` 119 Stop SOL Console Logging 120 121 [Documentation] Stop system console logging. If there are multiple 122 ... system console processes, they will all be 123 ... stopped. If there is no existing log file this 124 ... keyword will return an error message to that 125 ... effect (and write that message to targ_file_path, 126 ... if specified). NOTE: This keyword will not fail 127 ... if there is no running system console process. 128 129 # This doc string is way too long. 130 ``` 131- Tags: 132 - Create a tag for every test case with a tag name that mirrors the test case 133 name as follows: 134 ``` 135 Create Intermediate File 136 137 [Tags] Create_Intermediate_File 138 ``` 139- Description of argument(s): 140 - As shown in the following example, if your keyword has any arguments, include 141 a "**Description of argument(s)**" section. This effectively serves as the 142 help text for anyone wanting to use or understand your keyword. Include 143 real data examples wherever possible and applicable. Leave at least 2 spaces 144 between the argument name and the description. Align all description text as 145 shown in the example below. 146 147 Example: 148 ``` 149 Get URL List 150 [Documentation] Return list of URLs under given URL. 151 [Arguments] ${openbmc_url} ${policy} 152 153 # Description of argument(s): 154 # openbmc_url URL for list operation (e.g. 155 # "/xyz/openbmc_project/inventory"). 156 # policy Power restore policy (e.g "RESTORE_LAST_STATE", 157 # ${RESTORE_LAST_STATE}). 158 ``` 159- Variable assignments: 160 161 When assigning a variable as output from a keyword, do not precede the 162 equal sign with a space. 163 164 Correct examples: 165 ``` 166 ${var1}= Set Variable ${1} 167 ${var1}= My Keyword 168 ``` 169 Incorrect examples: 170 171 ``` 172 ${var1} = Set Variable ${1} 173 ${var1} = My Keyword 174 ``` 175- General variable naming conventions: 176 - Variable names should be lower case with few exceptions: 177 - Environment variables should be all upper case. 178 - Variables intended to be set by Robot -v parameters may be all 179 upper case. 180 - Words within a variable name should be separated by underscores: 181 182 Correct examples: 183 ``` 184 ${host_name} 185 ${program_pid} 186 ``` 187 Incorrect examples: 188 ``` 189 ${HostName} 190 ${ProgramPid} 191 ``` 192- Special variable naming conventions. 193 194 For certain very commonly used kinds of variables, please observe these 195 conventions in order to achieve consistency throughout the code. 196 197 - hosts 198 199 When a variable is intended to contain **either** an IP address **or** 200 a host name (either long or short), please give it a suffix of "_host". 201 202 Examples: 203 ``` 204 openbmc_host 205 os_host 206 pdu_host 207 openbmc_serial_host 208 ``` 209 - host names 210 211 For host names (long or short, e.g. "bmc1" or "bmc1.example.com"), use 212 a suffix of _host_name. 213 214 Examples: 215 ``` 216 openbmc_host_name 217 os_host_name 218 pdu_host_name 219 openbmc_serial_host_name 220 ``` 221 - Short host names 222 223 For short host names (e.g. "bmc1"), use a suffix of _host_short_name. 224 225 Examples: 226 ``` 227 openbmc_host_short_name 228 os_host_short_name 229 pdu_host_short_name 230 openbmc_serial_host_short_name 231 ``` 232 - IP addresses 233 234 For IP addresses, use a suffix of _ip. 235 236 Example: 237 ``` 238 openbmc_ip 239 os_ip 240 pdu_ip 241 openbmc_serial_ip 242 ``` 243 - Files and directories: 244 - Files: 245 - If your variable is to contain only the file's name, use a suffix 246 of _file_name. 247 248 Examples: 249 ``` 250 ffdc_file_name = "bmc1.170428.120200.ffdc" 251 ``` 252 - If your variable is to contain the path to a file, use a suffix of 253 _file_path. Bear in mind that a file path can be relative or 254 absolute so that should not be a consideration in whether to use 255 the "_file_path" suffix. 256 257 Examples: 258 ``` 259 status_file_path = "bmc1.170428.120200.status" 260 status_file_path = "subdir/bmc1.170428.120200.status" 261 status_file_path = "./bmc1.170428.120200.status" 262 status_file_path = "../bmc1.170428.120200.status" 263 status_file_path = "/home/user1/status/bmc1.170428.120200.status" 264 ``` 265 To re-iterate, it doesn't matter whether the contents of the 266 variable are a relative or absolute path (as shown in the 267 examples above). A file path is simply a value with enough 268 information in it for the program to find the file. 269 270 - If the variable **must** contain an absolute path (which should be 271 the rare case), use a suffix _abs_file_path. 272 273 - Directories: 274 - Directory variables should follow the same conventions as file 275 variables. 276 277 - If your variable is to contain only the directory's name, use a 278 suffix of _dir_name. 279 280 Example: 281 ``` 282 ffdc_dir_name = "ffdc" 283 ``` 284 - If your variable is to contain the path to a directory, use a 285 suffix of _dir_path. Bear in mind that a dir path can be 286 relative or absolute so that should not be a consideration in 287 whether to use _dir_path. 288 289 Examples: 290 ``` 291 status_dir_path = "status/" 292 status_dir_path = "subdir/status" 293 status_dir_path = "./status/" 294 status_dir_path = "../status/" 295 status_dir_path = "/home/user1/status/" 296 ``` 297 To re-iterate, it doesn't matter whether the contents of 298 the variable are a relative or absolute path (as shown in 299 the examples above). A dir path is simply a value with 300 enough information in it for the program to find the 301 directory. 302 303 - If the variable **must** contain an absolute path (which 304 should be the rare case), use a suffix _abs_dir_path. 305 - IMPORTANT: As a programming convention, do pre- 306 processing on all dir_path variables to ensure that they 307 contain a trailing slash. If we follow that convention 308 religiously, that when changes are made in other parts of 309 the program, the programmer can count on the value having 310 a trailing slash. Therefore they can safely do this kind 311 of thing: 312 ``` 313 my_file_path = my_dir_path + my_file_name 314 ``` 315 - Setup/Teardown keywords 316 317 Use standardized names for setup and teardown keywords: 318 - Suite Setup Execution 319 - Suite Teardown Execution 320 - Test Setup Execution 321 - Test Teardown Execution 322- Traditional comments (i.e. using the hashtag style comments) 323 - Please leave one space following the hashtag. 324 ``` 325 #wrong 326 327 # Right 328 ``` 329 - Please use proper English punction: 330 - Capitalize the first word in the sentence or phrase. 331 - End sentences (or stand-alone phrases) with a period. 332 333 - Do not keep commented out code in your program. Instead, remove it 334 entirely. 335 336Python Coding Guidelines 337----------------------- 338- The minimum required Python version is 2.7.x. 339- Run pep8 on all Python files and correct errors. 340 341 Example as run from a Linux command line: 342 ``` 343 pep8 my_pgm.py 344 345 my_pgm.py:41:1: E302 expected 2 blank lines, found 1 346 my_pgm.py:58:52: W291 trailing whitespace 347 ``` 348- Include doc strings in every function and follow the guidelines in 349 https://www.python.org/dev/peps/pep-0257/. 350 351 Example: 352 ``` 353 r""" 354 Return the function name associated with the indicated stack frame. 355 356 Description of argument(s): 357 stack_frame_ix The index of the stack frame whose 358 function name should be returned. If 359 the caller does not specify a value, 360 this function will set the value to 1 361 which is the index of the caller's 362 stack frame. If the caller is the 363 wrapper function "print_func_name", 364 this function will bump it up by 1. 365 """ 366 ``` 367- As shown in the prior example, if your function has any arguments, include 368 a "Description of argument(s)" section. This effectively serves as the 369 help text for anyone wanting to use or understand your function. Include 370 real data examples wherever possible and applicable. 371- Function definitions: 372 - Put each function parameter on its own line: 373 ``` 374 def func1(parm1, 375 376 parm2): 377 ``` 378- Do not keep commented out code in your program. Instead, remove it 379 entirely. 380- When you define or call a Robot keyword, Robot pays no attention to spaces, 381 underscores or case. However, our team will observe the following 382 conventions in both our definitions and our calls: 383 - Separate words with single spaces. 384 - Capitalize the first character of each word. 385 - Capitalize all characters in any word that is an acronym (e.g. JSON, BMC, 386 etc). 387 388 Examples: 389 ``` 390 *** Keywords *** 391 392 This Is Correct 393 394 # This keyword name is correct. 395 396 this_is_incorrect 397 398 # This keyword name is incorrect because of 1) the 399 # underscores instead of spaces and 2) the failure to 400 # capitalize each word in the keyword. 401 402 soisthis 403 404 # This keyword name is incorrect because of 1) a failure to 405 # separate words with spaces and 2) a failure to capitalize 406 # each word in the keyword. 407 408 BMC Is An Acronym 409 410 # This keyword name is correct. Note that "BMC" is an 411 # acronym and as such is entirely uppercase. 412 ``` 413- Documentation strings: 414 - Each documentation string should be phrased as an **English command**. 415 Punctuate it correctly with the first word capitalized and a period at 416 the end. 417 418 Correct example: 419 ``` 420 Boot BMC 421 [Documentation] Boot the BMC. 422 ``` 423 Incorrect example: 424 ``` 425 Boot BMC 426 [Documentation] This keyword boots the BMC. 427 428 # The doc string above is not phrased as a command. 429 ``` 430 - Doc strings should be just one terse, descriptive sentence. 431 Remember that this doc string shows up in the HTML log file. Put 432 additional commentary below in standard comment lines. 433 434 Correct example: 435 ``` 436 Stop SOL Console Logging 437 438 [Documentation] Stop system console logging and return log output. 439 ``` 440 Incorrect example: 441 ``` 442 Stop SOL Console Logging 443 444 [Documentation] Stop system console logging. If there are multiple 445 ... system console processes, they will all be 446 ... stopped. If there is no existing log file this 447 ... keyword will return an error message to that 448 ... effect (and write that message to targ_file_path, 449 ... if specified). NOTE: This keyword will not fail 450 ... if there is no running system console process. 451 452 # This doc string is way too long. 453 ``` 454- Tags: 455 - Create a tag for every test case with a tag name that mirrors the test case 456 name as follows: 457 ``` 458 Create Intermediate File 459 460 [Tags] Create_Intermediate_File 461 ``` 462- General variable naming conventions: 463 - Variable names should be lower case with few exceptions: 464 - Environment variables should be all upper case. 465 - Variables intended to be set by Robot -v parameters may be all 466 upper case. 467 - Words within a variable name should be separated by underscores: 468 469 Correct examples: 470 ``` 471 ${host_name} 472 ${program_pid} 473 ``` 474 Incorrect examples: 475 ``` 476 ${HostName} 477 ${ProgramPid} 478 ``` 479- Special variable naming conventions. 480 481 For certain very commonly used kinds of variables, please observe these 482 conventions in order to achieve consistency throughout the code. 483 484 - hosts 485 486 When a variable is intended to contain **either** an IP address **or** 487 a host name (either long or short), please give it a suffix of "_host". 488 489 Examples: 490 ``` 491 openbmc_host 492 os_host 493 pdu_host 494 openbmc_serial_host 495 ``` 496 - host names 497 498 For host names (long or short, e.g. "bmc1" or "bmc1.example.com"), use 499 a suffix of _host_name. 500 501 Examples: 502 ``` 503 openbmc_host_name 504 os_host_name 505 pdu_host_name 506 openbmc_serial_host_name 507 ``` 508 - Short host names 509 510 For short host names (e.g. "bmc1"), use a suffix of _host_short_name. 511 512 Examples: 513 ``` 514 openbmc_host_short_name 515 os_host_short_name 516 pdu_host_short_name 517 openbmc_serial_host_short_name 518 ``` 519 - IP addresses 520 521 For IP addresses, use a suffix of _ip. 522 523 Example: 524 ``` 525 openbmc_ip 526 os_ip 527 pdu_ip 528 openbmc_serial_ip 529 ``` 530- Files and directories: 531 - Files: 532 - If your variable is to contain only the file's name, use a suffix 533 of _file_name. 534 535 Examples: 536 ``` 537 ffdc_file_name = "bmc1.170428.120200.ffdc" 538 ``` 539 - If your variable is to contain the path to a file, use a suffix of 540 _file_path. Bear in mind that a file path can be relative or 541 absolute so that should not be a consideration in whether to use 542 the "_file_path" suffix. 543 544 Examples: 545 ``` 546 status_file_path = "bmc1.170428.120200.status" 547 status_file_path = "subdir/bmc1.170428.120200.status" 548 status_file_path = "./bmc1.170428.120200.status" 549 status_file_path = "../bmc1.170428.120200.status" 550 status_file_path = "/home/user1/status/bmc1.170428.120200.status" 551 ``` 552 To re-iterate, it doesn't matter whether the contents of the 553 variable are a relative or absolute path (as shown in the 554 examples above). A file path is simply a value with enough 555 information in it for the program to find the file. 556 557 - If the variable **must** contain an absolute path (which should be 558 the rare case), use a suffix _abs_file_path. 559 560 - Directories: 561 - Directory variables should follow the same conventions as file 562 variables. 563 564 - If your variable is to contain only the directory's name, use a 565 suffix of _dir_name. 566 567 Example: 568 ``` 569 ffdc_dir_name = "ffdc" 570 ``` 571 - If your variable is to contain the path to a directory, use a 572 suffix of _dir_path. Bear in mind that a dir path can be 573 relative or absolute so that should not be a consideration in 574 whether to use _dir_path. 575 576 Examples: 577 ``` 578 status_dir_path = "status/" 579 status_dir_path = "subdir/status" 580 status_dir_path = "./status/" 581 status_dir_path = "../status/" 582 status_dir_path = "/home/user1/status/" 583 ``` 584 To re-iterate, it doesn't matter whether the contents of 585 the variable are a relative or absolute path (as shown in 586 the examples above). A dir path is simply a value with 587 enough information in it for the program to find the 588 directory. 589 590 - If the variable **must** contain an absolute path (which 591 should be the rare case), use a suffix _abs_dir_path. 592 - IMPORTANT: As a programming convention, do pre- 593 processing on all dir_path variables to ensure that they 594 contain a trailing slash. If we follow that convention 595 religiously, that when changes are made in other parts of 596 the program, the programmer can count on the value having 597 a trailing slash. Therefore they can safely do this kind 598 of thing: 599 ``` 600 my_file_path = my_dir_path + my_file_name 601 ``` 602- Traditional comments (i.e. using the hashtag style comments) 603 - Please leave one space following the hashtag. 604 ``` 605 #wrong 606 607 # Right 608 ``` 609 - Please use proper English punction: 610 - Capitalize the first word in the sentence or phrase. 611 - End sentences (or stand-alone phrases) with a period. 612 613 - Do not keep commented out code in your program. Instead, remove it 614 entirely. 615 616Python Coding Guidelines 617----------------------- 618- Run pep8 on all Python files and correct errors. 619 620 Example as run from a Linux command line: 621 ``` 622 pep8 my_pgm.py 623 624 my_pgm.py:41:1: E302 expected 2 blank lines, found 1 625 my_pgm.py:58:52: W291 trailing whitespace 626 ``` 627- Include doc strings in every function and follow the guidelines in 628 https://www.python.org/dev/peps/pep-0257/. 629 630 Example: 631 ``` 632 r""" 633 Return the function name associated with the indicated stack frame. 634 635 Description of argument(s): 636 stack_frame_ix The index of the stack frame whose 637 function name should be returned. If 638 the caller does not specify a value, 639 this function will set the value to 1 640 which is the index of the caller's 641 stack frame. If the caller is the 642 wrapper function "print_func_name", 643 this function will bump it up by 1. 644 """ 645 ``` 646- As shown in the prior example, if your function has any arguments, include 647 a "Description of argument(s)" section. This effectively serves as the 648 help text for anyone wanting to use or understand your function. Include 649 real data examples wherever possible and applicable. 650- Function definitions: 651 - Put each function parameter on its own line: 652 ``` 653 def func1(parm1, 654 655 parm2): 656 ``` 657- Do not keep commented out code in your program. Instead, remove it 658 entirely. 659 660Template Usage Guidelines 661------------------------- 662We have several templates in the templates/ sub-directory. If there is a 663template that applies to your programming situation (Python, bash, etc.), 664it should be used to create new programs as in the following example 665 666- Example: 667 668 ``` 669 $ cd templates 670 $ cp python_pgm_template ../bin/my_new_program 671 ``` 672 673These templates have much of your preliminary work done for you and will help 674us all follow a similar structure. 675 676- Features: 677 - Help text and arg parsing started for you. 678 - Support for "stock" parameters like "quiet", "debug", "test_mode". 679 - "exit_function" and "signal_handler" defined. 680 - "validate_parms" function pre-created. 681 - "main" function follows conventional startup sequence: 682 683 ``` 684 if not gen_get_options(parser, stock_list): 685 return False 686 687 if not validate_parms(): 688 return False 689 690 qprint_pgm_header() 691 692 # Your code here. 693 ``` 694