1Contributing to OpenBMC Test Automation 2======================================= 3Guide to working on OpenBMC test automation. This document will always be a 4work-in-progress, feel free to propose changes. 5 6Submitting changes via Gerrit server 7------------------------------------ 8- Reference [OpenBMC docs](https://github.com/openbmc/docs/blob/master/contributing.md#submitting-changes-via-gerrit-server) 9 10Robot Coding Guidelines 11----------------------- 12- For this project, we will write Robot keyword definitions in either Robot 13 or Python. Robot code should be quite simple. Therefore, if the algorithm 14 in question is the least bit complex, please write it in Python. 15- Observe a maximum line length of 79 characters. 16- Avoid trailing space at the end of any line of Robot code. 17- Avoid the use of tabs. 18- Robot supports delimiting cells with either two or more spaces or with a 19 pipe symbol (e.g. "\|"). Our team has chosen to use spaces rather than the 20 pipe character. Make sure all space delimiters in Robot code are the 21 **minimum** of two spaces. There may be some exceptions to this rule. 22 23 Exceptions to two-space delimiter rule: 24 - When you wish to line up resource, library or variable values: 25 ``` 26 Library Lib1 27 Resource Resource1 28 *** Variables *** 29 ${var1} ${EMPTY} 30 ``` 31 - When you wish to line up fields for test templates: 32 ``` 33 [Template] Set System LED State 34 # LED Name LED State 35 power On 36 power Off 37 ``` 38 - When you wish to indent if/else or loop bodies for visual effect: 39 ``` 40 Run Keyword If '${this}' == '${that}' 41 ... Log Bla, bla... 42 ... ELSE 43 ... Run Keywords Key1 parms 44 ... AND Key2 parms 45 ``` 46- Use spaces to make conditions more readable: 47 48 Correct example: 49 ``` 50 Run Keyword If '${var1}' == '${0}' My Keyword 51 ``` 52 Incorrect example: 53 ``` 54 Run Keyword If '${var1}'=='${0}' My Keyword 55 ``` 56- When you define or call a Robot keyword, Robot pays no attention to spaces, 57 underscores or case. However, our team will observe the following 58 conventions in both our definitions and our calls: 59 - Separate words with single spaces. 60 - Capitalize the first character of each word. 61 - Capitalize all characters in any word that is an acronym (e.g. JSON, BMC, 62 etc). 63 64 Examples: 65 ``` 66 *** Keywords *** 67 68 This Is Correct 69 70 # This keyword name is correct. 71 72 this_is_incorrect 73 74 # This keyword name is incorrect because of 1) the 75 # underscores instead of spaces and 2) the failure to 76 # capitalize each word in the keyword. 77 78 soisthis 79 80 # This keyword name is incorrect because of 1) a failure to 81 # separate words with spaces and 2) a failure to capitalize 82 # each word in the keyword. 83 84 BMC Is An Acronym 85 86 # This keyword name is correct. Note that "BMC" is an 87 # acronym and as such is entirely uppercase. 88 ``` 89- Documentation strings: 90 - Each documentation string should be phrased as an **English command**. 91 Punctuate it correctly with the first word capitalized and a period at 92 the end. 93 94 Correct example: 95 ``` 96 Boot BMC 97 [Documentation] Boot the BMC. 98 ``` 99 Incorrect example: 100 ``` 101 Boot BMC 102 [Documentation] This keyword boots the BMC. 103 104 # The doc string above is not phrased as a command. 105 ``` 106 - Doc strings should be just one terse, descriptive sentence. 107 Remember that this doc string shows up in the HTML log file. Put 108 additional commentary below in standard comment lines. 109 110 Correct example: 111 ``` 112 Stop SOL Console Logging 113 114 [Documentation] Stop system console logging and return log output. 115 ``` 116 Incorrect example: 117 ``` 118 Stop SOL Console Logging 119 120 [Documentation] Stop system console logging. If there are multiple 121 ... system console processes, they will all be 122 ... stopped. If there is no existing log file this 123 ... keyword will return an error message to that 124 ... effect (and write that message to targ_file_path, 125 ... if specified). NOTE: This keyword will not fail 126 ... if there is no running system console process. 127 128 # This doc string is way too long. 129 ``` 130- Tags: 131 - Create a tag for every test case with a tag name that mirrors the test case 132 name as follows: 133 ``` 134 Create Intermediate File 135 136 [Tags] Create_Intermediate_File 137 ``` 138- Description of argument(s): 139 - As shown in the following example, if your keyword has any arguments, include 140 a "**Description of argument(s)**" section. This effectively serves as the 141 help text for anyone wanting to use or understand your keyword. Include 142 real data examples wherever possible and applicable. Leave at least 2 spaces 143 between the argument name and the description. Align all description text as 144 shown in the example below. 145 146 Example: 147 ``` 148 Get URL List 149 [Documentation] Return list of URLs under given URL. 150 [Arguments] ${openbmc_url} ${policy} 151 152 # Description of argument(s): 153 # openbmc_url URL for list operation (e.g. 154 # "/xyz/openbmc_project/inventory"). 155 # policy Power restore policy (e.g "RESTORE_LAST_STATE", 156 # ${RESTORE_LAST_STATE}). 157 ``` 158- Variable assignments: 159 160 When assigning a variable as output from a keyword, do not precede the 161 equal sign with a space. 162 163 Correct examples: 164 ``` 165 ${var1}= Set Variable ${1} 166 ${var1}= My Keyword 167 ``` 168 Incorrect examples: 169 170 ``` 171 ${var1} = Set Variable ${1} 172 ${var1} = My Keyword 173 ``` 174- General variable naming conventions: 175 - Variable names should be lower case with few exceptions: 176 - Environment variables should be all upper case. 177 - Variables intended to be set by Robot -v parameters may be all 178 upper case. 179 - Words within a variable name should be separated by underscores: 180 181 Correct examples: 182 ``` 183 ${host_name} 184 ${program_pid} 185 ``` 186 Incorrect examples: 187 ``` 188 ${HostName} 189 ${ProgramPid} 190 ``` 191- Special variable naming conventions. 192 193 For certain very commonly used kinds of variables, please observe these 194 conventions in order to achieve consistency throughout the code. 195 196 - hosts 197 198 When a variable is intended to contain **either** an IP address **or** 199 a host name (either long or short), please give it a suffix of "_host". 200 201 Examples: 202 ``` 203 openbmc_host 204 os_host 205 pdu_host 206 openbmc_serial_host 207 ``` 208 - host names 209 210 For host names (long or short, e.g. "bmc1" or "bmc1.ibm.com"), use a 211 suffix of _host_name. 212 213 Examples: 214 ``` 215 openbmc_host_name 216 os_host_name 217 pdu_host_name 218 openbmc_serial_host_name 219 ``` 220 - Short host names 221 222 For short host names (e.g. "bmc1"), use a suffix of _host_short_name. 223 224 Examples: 225 ``` 226 openbmc_host_short_name 227 os_host_short_name 228 pdu_host_short_name 229 openbmc_serial_host_short_name 230 ``` 231 - IP addresses 232 233 For IP addresses, use a suffix of _ip. 234 235 Example: 236 ``` 237 openbmc_ip 238 os_ip 239 pdu_ip 240 openbmc_serial_ip 241 ``` 242 - Files and directories: 243 - Files: 244 - If your variable is to contain only the file's name, use a suffix 245 of _file_name. 246 247 Examples: 248 ``` 249 ffdc_file_name = "bmc1.170428.120200.ffdc" 250 ``` 251 - If your variable is to contain the path to a file, use a suffix of 252 _file_path. Bear in mind that a file path can be relative or 253 absolute so that should not be a consideration in whether to use 254 the "_file_path" suffix. 255 256 Examples: 257 ``` 258 status_file_path = "bmc1.170428.120200.status" 259 status_file_path = "subdir/bmc1.170428.120200.status" 260 status_file_path = "./bmc1.170428.120200.status" 261 status_file_path = "../bmc1.170428.120200.status" 262 status_file_path = "/home/user1/status/bmc1.170428.120200.status" 263 ``` 264 To re-iterate, it doesn't matter whether the contents of the 265 variable are a relative or absolute path (as shown in the 266 examples above). A file path is simply a value with enough 267 information in it for the program to find the file. 268 269 - If the variable **must** contain an absolute path (which should be 270 the rare case), use a suffix _abs_file_path. 271 272 - Directories: 273 - Directory variables should follow the same conventions as file 274 variables. 275 276 - If your variable is to contain only the directory's name, use a 277 suffix of _dir_name. 278 279 Example: 280 ``` 281 ffdc_dir_name = "ffdc" 282 ``` 283 - If your variable is to contain the path to a directory, use a 284 suffix of _dir_path. Bear in mind that a dir path can be 285 relative or absolute so that should not be a consideration in 286 whether to use _dir_path. 287 288 Examples: 289 ``` 290 status_dir_path = "status/" 291 status_dir_path = "subdir/status" 292 status_dir_path = "./status/" 293 status_dir_path = "../status/" 294 status_dir_path = "/home/user1/status/" 295 ``` 296 To re-iterate, it doesn't matter whether the contents of 297 the variable are a relative or absolute path (as shown in 298 the examples above). A dir path is simply a value with 299 enough information in it for the program to find the 300 directory. 301 302 - If the variable **must** contain an absolute path (which 303 should be the rare case), use a suffix _abs_dir_path. 304 - IMPORTANT: As a programming convention, do pre- 305 processing on all dir_path variables to ensure that they 306 contain a trailing slash. If we follow that convention 307 religiously, that when changes are made in other parts of 308 the program, the programmer can count on the value having 309 a trailing slash. Therefore they can safely do this kind 310 of thing: 311 ``` 312 my_file_path = my_dir_path + my_file_name 313 ``` 314 - Setup/Teardown keywords 315 316 Use standardized names for setup and teardown keywords: 317 - Suite Setup Execution 318 - Suite Teardown Execution 319 - Test Setup Execution 320 - Test Teardown Execution 321- Traditional comments (i.e. using the hashtag style comments) 322 - Please leave one space following the hashtag. 323 ``` 324 #wrong 325 326 # Right 327 ``` 328 - Please use proper English punction: 329 - Capitalize the first word in the sentence or phrase. 330 - End sentences (or stand-alone phrases) with a period. 331 332 - Do not keep commented out code in your program. Instead, remove it 333 entirely. 334 335Python Coding Guidelines 336----------------------- 337- The minimum required Python version is 2.7.x. 338- Run pep8 on all Python files and correct errors. 339 340 Example as run from a Linux command line: 341 ``` 342 pep8 my_pgm.py 343 344 my_pgm.py:41:1: E302 expected 2 blank lines, found 1 345 my_pgm.py:58:52: W291 trailing whitespace 346 ``` 347- Include doc strings in every function and follow the guidelines in 348 https://www.python.org/dev/peps/pep-0257/. 349 350 Example: 351 ``` 352 r""" 353 Return the function name associated with the indicated stack frame. 354 355 Description of argument(s): 356 stack_frame_ix The index of the stack frame whose 357 function name should be returned. If 358 the caller does not specify a value, 359 this function will set the value to 1 360 which is the index of the caller's 361 stack frame. If the caller is the 362 wrapper function "print_func_name", 363 this function will bump it up by 1. 364 """ 365 ``` 366- As shown in the prior example, if your function has any arguments, include 367 a "Description of argument(s)" section. This effectively serves as the 368 help text for anyone wanting to use or understand your function. Include 369 real data examples wherever possible and applicable. 370- Function definitions: 371 - Put each function parameter on its own line: 372 ``` 373 def func1(parm1, 374 375 parm2): 376 ``` 377- Do not keep commented out code in your program. Instead, remove it 378 entirely. 379- When you define or call a Robot keyword, Robot pays no attention to spaces, 380 underscores or case. However, our team will observe the following 381 conventions in both our definitions and our calls: 382 - Separate words with single spaces. 383 - Capitalize the first character of each word. 384 - Capitalize all characters in any word that is an acronym (e.g. JSON, BMC, 385 etc). 386 387 Examples: 388 ``` 389 *** Keywords *** 390 391 This Is Correct 392 393 # This keyword name is correct. 394 395 this_is_incorrect 396 397 # This keyword name is incorrect because of 1) the 398 # underscores instead of spaces and 2) the failure to 399 # capitalize each word in the keyword. 400 401 soisthis 402 403 # This keyword name is incorrect because of 1) a failure to 404 # separate words with spaces and 2) a failure to capitalize 405 # each word in the keyword. 406 407 BMC Is An Acronym 408 409 # This keyword name is correct. Note that "BMC" is an 410 # acronym and as such is entirely uppercase. 411 ``` 412- Documentation strings: 413 - Each documentation string should be phrased as an **English command**. 414 Punctuate it correctly with the first word capitalized and a period at 415 the end. 416 417 Correct example: 418 ``` 419 Boot BMC 420 [Documentation] Boot the BMC. 421 ``` 422 Incorrect example: 423 ``` 424 Boot BMC 425 [Documentation] This keyword boots the BMC. 426 427 # The doc string above is not phrased as a command. 428 ``` 429 - Doc strings should be just one terse, descriptive sentence. 430 Remember that this doc string shows up in the HTML log file. Put 431 additional commentary below in standard comment lines. 432 433 Correct example: 434 ``` 435 Stop SOL Console Logging 436 437 [Documentation] Stop system console logging and return log output. 438 ``` 439 Incorrect example: 440 ``` 441 Stop SOL Console Logging 442 443 [Documentation] Stop system console logging. If there are multiple 444 ... system console processes, they will all be 445 ... stopped. If there is no existing log file this 446 ... keyword will return an error message to that 447 ... effect (and write that message to targ_file_path, 448 ... if specified). NOTE: This keyword will not fail 449 ... if there is no running system console process. 450 451 # This doc string is way too long. 452 ``` 453- Tags: 454 - Create a tag for every test case with a tag name that mirrors the test case 455 name as follows: 456 ``` 457 Create Intermediate File 458 459 [Tags] Create_Intermediate_File 460 ``` 461- General variable naming conventions: 462 - Variable names should be lower case with few exceptions: 463 - Environment variables should be all upper case. 464 - Variables intended to be set by Robot -v parameters may be all 465 upper case. 466 - Words within a variable name should be separated by underscores: 467 468 Correct examples: 469 ``` 470 ${host_name} 471 ${program_pid} 472 ``` 473 Incorrect examples: 474 ``` 475 ${HostName} 476 ${ProgramPid} 477 ``` 478- Special variable naming conventions. 479 480 For certain very commonly used kinds of variables, please observe these 481 conventions in order to achieve consistency throughout the code. 482 483 - hosts 484 485 When a variable is intended to contain **either** an IP address **or** 486 a host name (either long or short), please give it a suffix of "_host". 487 488 Examples: 489 ``` 490 openbmc_host 491 os_host 492 pdu_host 493 openbmc_serial_host 494 ``` 495 - host names 496 497 For host names (long or short, e.g. "bmc1" or "bmc1.ibm.com"), use a 498 suffix of _host_name. 499 500 Examples: 501 ``` 502 openbmc_host_name 503 os_host_name 504 pdu_host_name 505 openbmc_serial_host_name 506 ``` 507 - Short host names 508 509 For short host names (e.g. "bmc1"), use a suffix of _host_short_name. 510 511 Examples: 512 ``` 513 openbmc_host_short_name 514 os_host_short_name 515 pdu_host_short_name 516 openbmc_serial_host_short_name 517 ``` 518 - IP addresses 519 520 For IP addresses, use a suffix of _ip. 521 522 Example: 523 ``` 524 openbmc_ip 525 os_ip 526 pdu_ip 527 openbmc_serial_ip 528 ``` 529- Files and directories: 530 - Files: 531 - If your variable is to contain only the file's name, use a suffix 532 of _file_name. 533 534 Examples: 535 ``` 536 ffdc_file_name = "bmc1.170428.120200.ffdc" 537 ``` 538 - If your variable is to contain the path to a file, use a suffix of 539 _file_path. Bear in mind that a file path can be relative or 540 absolute so that should not be a consideration in whether to use 541 the "_file_path" suffix. 542 543 Examples: 544 ``` 545 status_file_path = "bmc1.170428.120200.status" 546 status_file_path = "subdir/bmc1.170428.120200.status" 547 status_file_path = "./bmc1.170428.120200.status" 548 status_file_path = "../bmc1.170428.120200.status" 549 status_file_path = "/home/user1/status/bmc1.170428.120200.status" 550 ``` 551 To re-iterate, it doesn't matter whether the contents of the 552 variable are a relative or absolute path (as shown in the 553 examples above). A file path is simply a value with enough 554 information in it for the program to find the file. 555 556 - If the variable **must** contain an absolute path (which should be 557 the rare case), use a suffix _abs_file_path. 558 559 - Directories: 560 - Directory variables should follow the same conventions as file 561 variables. 562 563 - If your variable is to contain only the directory's name, use a 564 suffix of _dir_name. 565 566 Example: 567 ``` 568 ffdc_dir_name = "ffdc" 569 ``` 570 - If your variable is to contain the path to a directory, use a 571 suffix of _dir_path. Bear in mind that a dir path can be 572 relative or absolute so that should not be a consideration in 573 whether to use _dir_path. 574 575 Examples: 576 ``` 577 status_dir_path = "status/" 578 status_dir_path = "subdir/status" 579 status_dir_path = "./status/" 580 status_dir_path = "../status/" 581 status_dir_path = "/home/user1/status/" 582 ``` 583 To re-iterate, it doesn't matter whether the contents of 584 the variable are a relative or absolute path (as shown in 585 the examples above). A dir path is simply a value with 586 enough information in it for the program to find the 587 directory. 588 589 - If the variable **must** contain an absolute path (which 590 should be the rare case), use a suffix _abs_dir_path. 591 - IMPORTANT: As a programming convention, do pre- 592 processing on all dir_path variables to ensure that they 593 contain a trailing slash. If we follow that convention 594 religiously, that when changes are made in other parts of 595 the program, the programmer can count on the value having 596 a trailing slash. Therefore they can safely do this kind 597 of thing: 598 ``` 599 my_file_path = my_dir_path + my_file_name 600 ``` 601- Traditional comments (i.e. using the hashtag style comments) 602 - Please leave one space following the hashtag. 603 ``` 604 #wrong 605 606 # Right 607 ``` 608 - Please use proper English punction: 609 - Capitalize the first word in the sentence or phrase. 610 - End sentences (or stand-alone phrases) with a period. 611 612 - Do not keep commented out code in your program. Instead, remove it 613 entirely. 614 615Python Coding Guidelines 616----------------------- 617- Run pep8 on all Python files and correct errors. 618 619 Example as run from a Linux command line: 620 ``` 621 pep8 my_pgm.py 622 623 my_pgm.py:41:1: E302 expected 2 blank lines, found 1 624 my_pgm.py:58:52: W291 trailing whitespace 625 ``` 626- Include doc strings in every function and follow the guidelines in 627 https://www.python.org/dev/peps/pep-0257/. 628 629 Example: 630 ``` 631 r""" 632 Return the function name associated with the indicated stack frame. 633 634 Description of argument(s): 635 stack_frame_ix The index of the stack frame whose 636 function name should be returned. If 637 the caller does not specify a value, 638 this function will set the value to 1 639 which is the index of the caller's 640 stack frame. If the caller is the 641 wrapper function "print_func_name", 642 this function will bump it up by 1. 643 """ 644 ``` 645- As shown in the prior example, if your function has any arguments, include 646 a "Description of argument(s)" section. This effectively serves as the 647 help text for anyone wanting to use or understand your function. Include 648 real data examples wherever possible and applicable. 649- Function definitions: 650 - Put each function parameter on its own line: 651 ``` 652 def func1(parm1, 653 654 parm2): 655 ``` 656- Do not keep commented out code in your program. Instead, remove it 657 entirely. 658 659Template Usage Guidelines 660------------------------- 661We have several templates in the templates/ sub-directory. If there is a 662template that applies to your programming situation (Python, bash, etc.), 663it should be used to create new programs as in the following example 664 665- Example: 666 667 ``` 668 $ cd templates 669 $ cp python_pgm_template ../bin/my_new_program 670 ``` 671 672These templates have much of your preliminary work done for you and will help 673us all follow a similar structure. 674 675- Features: 676 - Help text and arg parsing started for you. 677 - Support for "stock" parameters like "quiet", "debug", "test_mode". 678 - "exit_function" and "signal_handler" defined. 679 - "validate_parms" function pre-created. 680 - "main" function follows conventional startup sequence: 681 682 ``` 683 if not gen_get_options(parser, stock_list): 684 return False 685 686 if not validate_parms(): 687 return False 688 689 qprint_pgm_header() 690 691 # Your code here. 692 ``` 693