xref: /openbmc/u-boot/tools/moveconfig.py (revision 6b44ae6b)
1#!/usr/bin/env python2
2#
3# Author: Masahiro Yamada <yamada.masahiro@socionext.com>
4#
5# SPDX-License-Identifier:	GPL-2.0+
6#
7
8"""
9Move config options from headers to defconfig files.
10
11Since Kconfig was introduced to U-Boot, we have worked on moving
12config options from headers to Kconfig (defconfig).
13
14This tool intends to help this tremendous work.
15
16
17Usage
18-----
19
20This tool takes one input file.  (let's say 'recipe' file here.)
21The recipe describes the list of config options you want to move.
22Each line takes the form:
23<config_name> <type> <default>
24(the fields must be separated with whitespaces.)
25
26<config_name> is the name of config option.
27
28<type> is the type of the option.  It must be one of bool, tristate,
29string, int, and hex.
30
31<default> is the default value of the option.  It must be appropriate
32value corresponding to the option type.  It must be either y or n for
33the bool type.  Tristate options can also take m (although U-Boot has
34not supported the module feature).
35
36You can add two or more lines in the recipe file, so you can move
37multiple options at once.
38
39Let's say, for example, you want to move CONFIG_CMD_USB and
40CONFIG_SYS_TEXT_BASE.
41
42The type should be bool, hex, respectively.  So, the recipe file
43should look like this:
44
45  $ cat recipe
46  CONFIG_CMD_USB bool n
47  CONFIG_SYS_TEXT_BASE hex 0x00000000
48
49Next you must edit the Kconfig to add the menu entries for the configs
50you are moving.
51
52And then run this tool giving the file name of the recipe
53
54  $ tools/moveconfig.py recipe
55
56The tool walks through all the defconfig files to move the config
57options specified by the recipe file.
58
59The log is also displayed on the terminal.
60
61Each line is printed in the format
62<defconfig_name>   :  <action>
63
64<defconfig_name> is the name of the defconfig
65(without the suffix _defconfig).
66
67<action> shows what the tool did for that defconfig.
68It looks like one of the followings:
69
70 - Move 'CONFIG_... '
71   This config option was moved to the defconfig
72
73 - Default value 'CONFIG_...'.  Do nothing.
74   The value of this option is the same as default.
75   We do not have to add it to the defconfig.
76
77 - 'CONFIG_...' already exists in Kconfig.  Do nothing.
78   This config option is already defined in Kconfig.
79   We do not need/want to touch it.
80
81 - Undefined.  Do nothing.
82   This config option was not found in the config header.
83   Nothing to do.
84
85 - Failed to process.  Skip.
86   An error occurred during processing this defconfig.  Skipped.
87   (If -e option is passed, the tool exits immediately on error.)
88
89Finally, you will be asked, Clean up headers? [y/n]:
90
91If you say 'y' here, the unnecessary config defines are removed
92from the config headers (include/configs/*.h).
93It just uses the regex method, so you should not rely on it.
94Just in case, please do 'git diff' to see what happened.
95
96
97How does it works?
98------------------
99
100This tool runs configuration and builds include/autoconf.mk for every
101defconfig.  The config options defined in Kconfig appear in the .config
102file (unless they are hidden because of unmet dependency.)
103On the other hand, the config options defined by board headers are seen
104in include/autoconf.mk.  The tool looks for the specified options in both
105of them to decide the appropriate action for the options.  If the option
106is found in the .config or the value is the same as the specified default,
107the option does not need to be touched.  If the option is found in
108include/autoconf.mk, but not in the .config, and the value is different
109from the default, the tools adds the option to the defconfig.
110
111For faster processing, this tool handles multi-threading.  It creates
112separate build directories where the out-of-tree build is run.  The
113temporary build directories are automatically created and deleted as
114needed.  The number of threads are chosen based on the number of the CPU
115cores of your system although you can change it via -j (--jobs) option.
116
117
118Toolchains
119----------
120
121Appropriate toolchain are necessary to generate include/autoconf.mk
122for all the architectures supported by U-Boot.  Most of them are available
123at the kernel.org site, some are not provided by kernel.org.
124
125The default per-arch CROSS_COMPILE used by this tool is specified by
126the list below, CROSS_COMPILE.  You may wish to update the list to
127use your own.  Instead of modifying the list directly, you can give
128them via environments.
129
130
131Available options
132-----------------
133
134 -c, --color
135   Surround each portion of the log with escape sequences to display it
136   in color on the terminal.
137
138 -d, --defconfigs
139  Specify a file containing a list of defconfigs to move
140
141 -n, --dry-run
142   Peform a trial run that does not make any changes.  It is useful to
143   see what is going to happen before one actually runs it.
144
145 -e, --exit-on-error
146   Exit immediately if Make exits with a non-zero status while processing
147   a defconfig file.
148
149 -H, --headers-only
150   Only cleanup the headers; skip the defconfig processing
151
152 -j, --jobs
153   Specify the number of threads to run simultaneously.  If not specified,
154   the number of threads is the same as the number of CPU cores.
155
156 -v, --verbose
157   Show any build errors as boards are built
158
159To see the complete list of supported options, run
160
161  $ tools/moveconfig.py -h
162
163"""
164
165import fnmatch
166import multiprocessing
167import optparse
168import os
169import re
170import shutil
171import subprocess
172import sys
173import tempfile
174import time
175
176SHOW_GNU_MAKE = 'scripts/show-gnu-make'
177SLEEP_TIME=0.03
178
179# Here is the list of cross-tools I use.
180# Most of them are available at kernel.org
181# (https://www.kernel.org/pub/tools/crosstool/files/bin/), except the followings:
182# arc: https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases
183# blackfin: http://sourceforge.net/projects/adi-toolchain/files/
184# nds32: http://osdk.andestech.com/packages/nds32le-linux-glibc-v1.tgz
185# nios2: https://sourcery.mentor.com/GNUToolchain/subscription42545
186# sh: http://sourcery.mentor.com/public/gnu_toolchain/sh-linux-gnu
187CROSS_COMPILE = {
188    'arc': 'arc-linux-',
189    'aarch64': 'aarch64-linux-',
190    'arm': 'arm-unknown-linux-gnueabi-',
191    'avr32': 'avr32-linux-',
192    'blackfin': 'bfin-elf-',
193    'm68k': 'm68k-linux-',
194    'microblaze': 'microblaze-linux-',
195    'mips': 'mips-linux-',
196    'nds32': 'nds32le-linux-',
197    'nios2': 'nios2-linux-gnu-',
198    'openrisc': 'or32-linux-',
199    'powerpc': 'powerpc-linux-',
200    'sh': 'sh-linux-gnu-',
201    'sparc': 'sparc-linux-',
202    'x86': 'i386-linux-'
203}
204
205STATE_IDLE = 0
206STATE_DEFCONFIG = 1
207STATE_AUTOCONF = 2
208STATE_SAVEDEFCONFIG = 3
209
210ACTION_MOVE = 0
211ACTION_DEFAULT_VALUE = 1
212ACTION_ALREADY_EXIST = 2
213ACTION_UNDEFINED = 3
214
215COLOR_BLACK        = '0;30'
216COLOR_RED          = '0;31'
217COLOR_GREEN        = '0;32'
218COLOR_BROWN        = '0;33'
219COLOR_BLUE         = '0;34'
220COLOR_PURPLE       = '0;35'
221COLOR_CYAN         = '0;36'
222COLOR_LIGHT_GRAY   = '0;37'
223COLOR_DARK_GRAY    = '1;30'
224COLOR_LIGHT_RED    = '1;31'
225COLOR_LIGHT_GREEN  = '1;32'
226COLOR_YELLOW       = '1;33'
227COLOR_LIGHT_BLUE   = '1;34'
228COLOR_LIGHT_PURPLE = '1;35'
229COLOR_LIGHT_CYAN   = '1;36'
230COLOR_WHITE        = '1;37'
231
232### helper functions ###
233def get_devnull():
234    """Get the file object of '/dev/null' device."""
235    try:
236        devnull = subprocess.DEVNULL # py3k
237    except AttributeError:
238        devnull = open(os.devnull, 'wb')
239    return devnull
240
241def check_top_directory():
242    """Exit if we are not at the top of source directory."""
243    for f in ('README', 'Licenses'):
244        if not os.path.exists(f):
245            sys.exit('Please run at the top of source directory.')
246
247def get_make_cmd():
248    """Get the command name of GNU Make.
249
250    U-Boot needs GNU Make for building, but the command name is not
251    necessarily "make". (for example, "gmake" on FreeBSD).
252    Returns the most appropriate command name on your system.
253    """
254    process = subprocess.Popen([SHOW_GNU_MAKE], stdout=subprocess.PIPE)
255    ret = process.communicate()
256    if process.returncode:
257        sys.exit('GNU Make not found')
258    return ret[0].rstrip()
259
260def color_text(color_enabled, color, string):
261    """Return colored string."""
262    if color_enabled:
263        return '\033[' + color + 'm' + string + '\033[0m'
264    else:
265        return string
266
267def log_msg(color_enabled, color, defconfig, msg):
268    """Return the formated line for the log."""
269    return defconfig[:-len('_defconfig')].ljust(37) + ': ' + \
270        color_text(color_enabled, color, msg) + '\n'
271
272def update_cross_compile():
273    """Update per-arch CROSS_COMPILE via enviroment variables
274
275    The default CROSS_COMPILE values are available
276    in the CROSS_COMPILE list above.
277
278    You can override them via enviroment variables
279    CROSS_COMPILE_{ARCH}.
280
281    For example, if you want to override toolchain prefixes
282    for ARM and PowerPC, you can do as follows in your shell:
283
284    export CROSS_COMPILE_ARM=...
285    export CROSS_COMPILE_POWERPC=...
286    """
287    archs = []
288
289    for arch in os.listdir('arch'):
290        if os.path.exists(os.path.join('arch', arch, 'Makefile')):
291            archs.append(arch)
292
293    # arm64 is a special case
294    archs.append('aarch64')
295
296    for arch in archs:
297        env = 'CROSS_COMPILE_' + arch.upper()
298        cross_compile = os.environ.get(env)
299        if cross_compile:
300            CROSS_COMPILE[arch] = cross_compile
301
302def cleanup_one_header(header_path, patterns, dry_run):
303    """Clean regex-matched lines away from a file.
304
305    Arguments:
306      header_path: path to the cleaned file.
307      patterns: list of regex patterns.  Any lines matching to these
308                patterns are deleted.
309      dry_run: make no changes, but still display log.
310    """
311    with open(header_path) as f:
312        lines = f.readlines()
313
314    matched = []
315    for i, line in enumerate(lines):
316        for pattern in patterns:
317            m = pattern.search(line)
318            if m:
319                print '%s: %s: %s' % (header_path, i + 1, line),
320                matched.append(i)
321                break
322
323    if dry_run or not matched:
324        return
325
326    with open(header_path, 'w') as f:
327        for i, line in enumerate(lines):
328            if not i in matched:
329                f.write(line)
330
331def cleanup_headers(config_attrs, dry_run):
332    """Delete config defines from board headers.
333
334    Arguments:
335      config_attrs: A list of dictionaris, each of them includes the name,
336                    the type, and the default value of the target config.
337      dry_run: make no changes, but still display log.
338    """
339    while True:
340        choice = raw_input('Clean up headers? [y/n]: ').lower()
341        print choice
342        if choice == 'y' or choice == 'n':
343            break
344
345    if choice == 'n':
346        return
347
348    patterns = []
349    for config_attr in config_attrs:
350        config = config_attr['config']
351        patterns.append(re.compile(r'#\s*define\s+%s\W' % config))
352        patterns.append(re.compile(r'#\s*undef\s+%s\W' % config))
353
354    for dir in 'include', 'arch', 'board':
355        for (dirpath, dirnames, filenames) in os.walk(dir):
356            for filename in filenames:
357                if not fnmatch.fnmatch(filename, '*~'):
358                    cleanup_one_header(os.path.join(dirpath, filename),
359                                       patterns, dry_run)
360
361### classes ###
362class KconfigParser:
363
364    """A parser of .config and include/autoconf.mk."""
365
366    re_arch = re.compile(r'CONFIG_SYS_ARCH="(.*)"')
367    re_cpu = re.compile(r'CONFIG_SYS_CPU="(.*)"')
368
369    def __init__(self, config_attrs, options, build_dir):
370        """Create a new parser.
371
372        Arguments:
373          config_attrs: A list of dictionaris, each of them includes the name,
374                        the type, and the default value of the target config.
375          options: option flags.
376          build_dir: Build directory.
377        """
378        self.config_attrs = config_attrs
379        self.options = options
380        self.build_dir = build_dir
381
382    def get_cross_compile(self):
383        """Parse .config file and return CROSS_COMPILE.
384
385        Returns:
386          A string storing the compiler prefix for the architecture.
387        """
388        arch = ''
389        cpu = ''
390        dotconfig = os.path.join(self.build_dir, '.config')
391        for line in open(dotconfig):
392            m = self.re_arch.match(line)
393            if m:
394                arch = m.group(1)
395                continue
396            m = self.re_cpu.match(line)
397            if m:
398                cpu = m.group(1)
399
400        assert arch, 'Error: arch is not defined in %s' % defconfig
401
402        # fix-up for aarch64
403        if arch == 'arm' and cpu == 'armv8':
404            arch = 'aarch64'
405
406        return CROSS_COMPILE.get(arch, '')
407
408    def parse_one_config(self, config_attr, defconfig_lines, autoconf_lines):
409        """Parse .config, defconfig, include/autoconf.mk for one config.
410
411        This function looks for the config options in the lines from
412        defconfig, .config, and include/autoconf.mk in order to decide
413        which action should be taken for this defconfig.
414
415        Arguments:
416          config_attr: A dictionary including the name, the type,
417                       and the default value of the target config.
418          defconfig_lines: lines from the original defconfig file.
419          autoconf_lines: lines from the include/autoconf.mk file.
420
421        Returns:
422          A tupple of the action for this defconfig and the line
423          matched for the config.
424        """
425        config = config_attr['config']
426        not_set = '# %s is not set' % config
427
428        if config_attr['type'] in ('bool', 'tristate') and \
429           config_attr['default'] == 'n':
430            default = not_set
431        else:
432            default = config + '=' + config_attr['default']
433
434        for line in defconfig_lines:
435            line = line.rstrip()
436            if line.startswith(config + '=') or line == not_set:
437                return (ACTION_ALREADY_EXIST, line)
438
439        if config_attr['type'] in ('bool', 'tristate'):
440            value = not_set
441        else:
442            value = '(undefined)'
443
444        for line in autoconf_lines:
445            line = line.rstrip()
446            if line.startswith(config + '='):
447                value = line
448                break
449
450        if value == default:
451            action = ACTION_DEFAULT_VALUE
452        elif value == '(undefined)':
453            action = ACTION_UNDEFINED
454        else:
455            action = ACTION_MOVE
456
457        return (action, value)
458
459    def update_defconfig(self, defconfig):
460        """Parse files for the config options and update the defconfig.
461
462        This function parses the given defconfig, the generated .config
463        and include/autoconf.mk searching the target options.
464        Move the config option(s) to the defconfig or do nothing if unneeded.
465        Also, display the log to show what happened to this defconfig.
466
467        Arguments:
468          defconfig: defconfig name.
469        """
470
471        defconfig_path = os.path.join('configs', defconfig)
472        dotconfig_path = os.path.join(self.build_dir, '.config')
473        autoconf_path = os.path.join(self.build_dir, 'include', 'autoconf.mk')
474        results = []
475
476        with open(defconfig_path) as f:
477            defconfig_lines = f.readlines()
478
479        with open(autoconf_path) as f:
480            autoconf_lines = f.readlines()
481
482        for config_attr in self.config_attrs:
483            result = self.parse_one_config(config_attr, defconfig_lines,
484                                           autoconf_lines)
485            results.append(result)
486
487        log = ''
488
489        for (action, value) in results:
490            if action == ACTION_MOVE:
491                actlog = "Move '%s'" % value
492                log_color = COLOR_LIGHT_GREEN
493            elif action == ACTION_DEFAULT_VALUE:
494                actlog = "Default value '%s'.  Do nothing." % value
495                log_color = COLOR_LIGHT_BLUE
496            elif action == ACTION_ALREADY_EXIST:
497                actlog = "'%s' already defined in Kconfig.  Do nothing." % value
498                log_color = COLOR_LIGHT_PURPLE
499            elif action == ACTION_UNDEFINED:
500                actlog = "Undefined.  Do nothing."
501                log_color = COLOR_DARK_GRAY
502            else:
503                sys.exit("Internal Error. This should not happen.")
504
505            log += log_msg(self.options.color, log_color, defconfig, actlog)
506
507        # Some threads are running in parallel.
508        # Print log in one shot to not mix up logs from different threads.
509        print log,
510
511        if not self.options.dry_run:
512            with open(dotconfig_path, 'a') as f:
513                for (action, value) in results:
514                    if action == ACTION_MOVE:
515                        f.write(value + '\n')
516
517        os.remove(os.path.join(self.build_dir, 'include', 'config', 'auto.conf'))
518        os.remove(autoconf_path)
519
520class Slot:
521
522    """A slot to store a subprocess.
523
524    Each instance of this class handles one subprocess.
525    This class is useful to control multiple threads
526    for faster processing.
527    """
528
529    def __init__(self, config_attrs, options, devnull, make_cmd):
530        """Create a new process slot.
531
532        Arguments:
533          config_attrs: A list of dictionaris, each of them includes the name,
534                        the type, and the default value of the target config.
535          options: option flags.
536          devnull: A file object of '/dev/null'.
537          make_cmd: command name of GNU Make.
538        """
539        self.options = options
540        self.build_dir = tempfile.mkdtemp()
541        self.devnull = devnull
542        self.make_cmd = (make_cmd, 'O=' + self.build_dir)
543        self.parser = KconfigParser(config_attrs, options, self.build_dir)
544        self.state = STATE_IDLE
545        self.failed_boards = []
546
547    def __del__(self):
548        """Delete the working directory
549
550        This function makes sure the temporary directory is cleaned away
551        even if Python suddenly dies due to error.  It should be done in here
552        because it is guranteed the destructor is always invoked when the
553        instance of the class gets unreferenced.
554
555        If the subprocess is still running, wait until it finishes.
556        """
557        if self.state != STATE_IDLE:
558            while self.ps.poll() == None:
559                pass
560        shutil.rmtree(self.build_dir)
561
562    def add(self, defconfig, num, total):
563        """Assign a new subprocess for defconfig and add it to the slot.
564
565        If the slot is vacant, create a new subprocess for processing the
566        given defconfig and add it to the slot.  Just returns False if
567        the slot is occupied (i.e. the current subprocess is still running).
568
569        Arguments:
570          defconfig: defconfig name.
571
572        Returns:
573          Return True on success or False on failure
574        """
575        if self.state != STATE_IDLE:
576            return False
577        cmd = list(self.make_cmd)
578        cmd.append(defconfig)
579        self.ps = subprocess.Popen(cmd, stdout=self.devnull,
580                                   stderr=subprocess.PIPE)
581        self.defconfig = defconfig
582        self.state = STATE_DEFCONFIG
583        self.num = num
584        self.total = total
585        return True
586
587    def poll(self):
588        """Check the status of the subprocess and handle it as needed.
589
590        Returns True if the slot is vacant (i.e. in idle state).
591        If the configuration is successfully finished, assign a new
592        subprocess to build include/autoconf.mk.
593        If include/autoconf.mk is generated, invoke the parser to
594        parse the .config and the include/autoconf.mk, and then set the
595        slot back to the idle state.
596
597        Returns:
598          Return True if the subprocess is terminated, False otherwise
599        """
600        if self.state == STATE_IDLE:
601            return True
602
603        if self.ps.poll() == None:
604            return False
605
606        if self.ps.poll() != 0:
607            errmsg = 'Failed to process.'
608            errout = self.ps.stderr.read()
609            if errout.find('gcc: command not found') != -1:
610                errmsg = 'Compiler not found ('
611                errmsg += color_text(self.options.color, COLOR_YELLOW,
612                                     self.cross_compile)
613                errmsg += color_text(self.options.color, COLOR_LIGHT_RED,
614                                     ')')
615            print >> sys.stderr, log_msg(self.options.color,
616                                         COLOR_LIGHT_RED,
617                                         self.defconfig,
618                                         errmsg),
619            if self.options.verbose:
620                print >> sys.stderr, color_text(self.options.color,
621                                                COLOR_LIGHT_CYAN, errout)
622            if self.options.exit_on_error:
623                sys.exit("Exit on error.")
624            else:
625                # If --exit-on-error flag is not set,
626                # skip this board and continue.
627                # Record the failed board.
628                self.failed_boards.append(self.defconfig)
629                self.state = STATE_IDLE
630                return True
631
632        if self.state == STATE_AUTOCONF:
633            self.parser.update_defconfig(self.defconfig)
634
635            print ' %d defconfigs out of %d\r' % (self.num + 1, self.total),
636            sys.stdout.flush()
637
638            """Save off the defconfig in a consistent way"""
639            cmd = list(self.make_cmd)
640            cmd.append('savedefconfig')
641            self.ps = subprocess.Popen(cmd, stdout=self.devnull,
642                                       stderr=subprocess.PIPE)
643            self.state = STATE_SAVEDEFCONFIG
644            return False
645
646        if self.state == STATE_SAVEDEFCONFIG:
647            defconfig_path = os.path.join(self.build_dir, 'defconfig')
648            shutil.move(defconfig_path,
649                        os.path.join('configs', self.defconfig))
650            self.state = STATE_IDLE
651            return True
652
653        self.cross_compile = self.parser.get_cross_compile()
654        cmd = list(self.make_cmd)
655        if self.cross_compile:
656            cmd.append('CROSS_COMPILE=%s' % self.cross_compile)
657        cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
658        cmd.append('include/config/auto.conf')
659        """This will be screen-scraped, so be sure the expected text will be
660        returned consistently on every machine by setting LANG=C"""
661        self.ps = subprocess.Popen(cmd, stdout=self.devnull,
662                                   env=dict(os.environ, LANG='C'),
663                                   stderr=subprocess.PIPE)
664        self.state = STATE_AUTOCONF
665        return False
666
667    def get_failed_boards(self):
668        """Returns a list of failed boards (defconfigs) in this slot.
669        """
670        return self.failed_boards
671
672class Slots:
673
674    """Controller of the array of subprocess slots."""
675
676    def __init__(self, config_attrs, options):
677        """Create a new slots controller.
678
679        Arguments:
680          config_attrs: A list of dictionaris containing the name, the type,
681                        and the default value of the target CONFIG.
682          options: option flags.
683        """
684        self.options = options
685        self.slots = []
686        devnull = get_devnull()
687        make_cmd = get_make_cmd()
688        for i in range(options.jobs):
689            self.slots.append(Slot(config_attrs, options, devnull, make_cmd))
690
691    def add(self, defconfig, num, total):
692        """Add a new subprocess if a vacant slot is found.
693
694        Arguments:
695          defconfig: defconfig name to be put into.
696
697        Returns:
698          Return True on success or False on failure
699        """
700        for slot in self.slots:
701            if slot.add(defconfig, num, total):
702                return True
703        return False
704
705    def available(self):
706        """Check if there is a vacant slot.
707
708        Returns:
709          Return True if at lease one vacant slot is found, False otherwise.
710        """
711        for slot in self.slots:
712            if slot.poll():
713                return True
714        return False
715
716    def empty(self):
717        """Check if all slots are vacant.
718
719        Returns:
720          Return True if all the slots are vacant, False otherwise.
721        """
722        ret = True
723        for slot in self.slots:
724            if not slot.poll():
725                ret = False
726        return ret
727
728    def show_failed_boards(self):
729        """Display all of the failed boards (defconfigs)."""
730        failed_boards = []
731
732        for slot in self.slots:
733            failed_boards += slot.get_failed_boards()
734
735        if len(failed_boards) > 0:
736            msg = [ "The following boards were not processed due to error:" ]
737            msg += failed_boards
738            for line in msg:
739                print >> sys.stderr, color_text(self.options.color,
740                                                COLOR_LIGHT_RED, line)
741
742            with open('moveconfig.failed', 'w') as f:
743                for board in failed_boards:
744                    f.write(board + '\n')
745
746def move_config(config_attrs, options):
747    """Move config options to defconfig files.
748
749    Arguments:
750      config_attrs: A list of dictionaris, each of them includes the name,
751                    the type, and the default value of the target config.
752      options: option flags
753    """
754    if len(config_attrs) == 0:
755        print 'Nothing to do. exit.'
756        sys.exit(0)
757
758    print 'Move the following CONFIG options (jobs: %d)' % options.jobs
759    for config_attr in config_attrs:
760        print '  %s (type: %s, default: %s)' % (config_attr['config'],
761                                                config_attr['type'],
762                                                config_attr['default'])
763
764    if options.defconfigs:
765        defconfigs = [line.strip() for line in open(options.defconfigs)]
766        for i, defconfig in enumerate(defconfigs):
767            if not defconfig.endswith('_defconfig'):
768                defconfigs[i] = defconfig + '_defconfig'
769            if not os.path.exists(os.path.join('configs', defconfigs[i])):
770                sys.exit('%s - defconfig does not exist. Stopping.' %
771                         defconfigs[i])
772    else:
773        # All the defconfig files to be processed
774        defconfigs = []
775        for (dirpath, dirnames, filenames) in os.walk('configs'):
776            dirpath = dirpath[len('configs') + 1:]
777            for filename in fnmatch.filter(filenames, '*_defconfig'):
778                defconfigs.append(os.path.join(dirpath, filename))
779
780    slots = Slots(config_attrs, options)
781
782    # Main loop to process defconfig files:
783    #  Add a new subprocess into a vacant slot.
784    #  Sleep if there is no available slot.
785    for i, defconfig in enumerate(defconfigs):
786        while not slots.add(defconfig, i, len(defconfigs)):
787            while not slots.available():
788                # No available slot: sleep for a while
789                time.sleep(SLEEP_TIME)
790
791    # wait until all the subprocesses finish
792    while not slots.empty():
793        time.sleep(SLEEP_TIME)
794
795    print ''
796    slots.show_failed_boards()
797
798def bad_recipe(filename, linenum, msg):
799    """Print error message with the file name and the line number and exit."""
800    sys.exit("%s: line %d: error : " % (filename, linenum) + msg)
801
802def parse_recipe(filename):
803    """Parse the recipe file and retrieve the config attributes.
804
805    This function parses the given recipe file and gets the name,
806    the type, and the default value of the target config options.
807
808    Arguments:
809      filename: path to file to be parsed.
810    Returns:
811      A list of dictionaris, each of them includes the name,
812      the type, and the default value of the target config.
813    """
814    config_attrs = []
815    linenum = 1
816
817    for line in open(filename):
818        tokens = line.split()
819        if len(tokens) != 3:
820            bad_recipe(filename, linenum,
821                       "%d fields in this line.  Each line must contain 3 fields"
822                       % len(tokens))
823
824        (config, type, default) = tokens
825
826        # prefix the option name with CONFIG_ if missing
827        if not config.startswith('CONFIG_'):
828            config = 'CONFIG_' + config
829
830        # sanity check of default values
831        if type == 'bool':
832            if not default in ('y', 'n'):
833                bad_recipe(filename, linenum,
834                           "default for bool type must be either y or n")
835        elif type == 'tristate':
836            if not default in ('y', 'm', 'n'):
837                bad_recipe(filename, linenum,
838                           "default for tristate type must be y, m, or n")
839        elif type == 'string':
840            if default[0] != '"' or default[-1] != '"':
841                bad_recipe(filename, linenum,
842                           "default for string type must be surrounded by double-quotations")
843        elif type == 'int':
844            try:
845                int(default)
846            except:
847                bad_recipe(filename, linenum,
848                           "type is int, but default value is not decimal")
849        elif type == 'hex':
850            if len(default) < 2 or default[:2] != '0x':
851                bad_recipe(filename, linenum,
852                           "default for hex type must be prefixed with 0x")
853            try:
854                int(default, 16)
855            except:
856                bad_recipe(filename, linenum,
857                           "type is hex, but default value is not hexadecimal")
858        else:
859            bad_recipe(filename, linenum,
860                       "unsupported type '%s'. type must be one of bool, tristate, string, int, hex"
861                       % type)
862
863        config_attrs.append({'config': config, 'type': type, 'default': default})
864        linenum += 1
865
866    return config_attrs
867
868def main():
869    try:
870        cpu_count = multiprocessing.cpu_count()
871    except NotImplementedError:
872        cpu_count = 1
873
874    parser = optparse.OptionParser()
875    # Add options here
876    parser.add_option('-c', '--color', action='store_true', default=False,
877                      help='display the log in color')
878    parser.add_option('-d', '--defconfigs', type='string',
879                      help='a file containing a list of defconfigs to move')
880    parser.add_option('-n', '--dry-run', action='store_true', default=False,
881                      help='perform a trial run (show log with no changes)')
882    parser.add_option('-e', '--exit-on-error', action='store_true',
883                      default=False,
884                      help='exit immediately on any error')
885    parser.add_option('-H', '--headers-only', dest='cleanup_headers_only',
886                      action='store_true', default=False,
887                      help='only cleanup the headers')
888    parser.add_option('-j', '--jobs', type='int', default=cpu_count,
889                      help='the number of jobs to run simultaneously')
890    parser.add_option('-v', '--verbose', action='store_true', default=False,
891                      help='show any build errors as boards are built')
892    parser.usage += ' recipe_file\n\n' + \
893                    'The recipe_file should describe config options you want to move.\n' + \
894                    'Each line should contain config_name, type, default_value\n\n' + \
895                    'Example:\n' + \
896                    'CONFIG_FOO bool n\n' + \
897                    'CONFIG_BAR int 100\n' + \
898                    'CONFIG_BAZ string "hello"\n'
899
900    (options, args) = parser.parse_args()
901
902    if len(args) != 1:
903        parser.print_usage()
904        sys.exit(1)
905
906    config_attrs = parse_recipe(args[0])
907
908    update_cross_compile()
909
910    check_top_directory()
911
912    if not options.cleanup_headers_only:
913        move_config(config_attrs, options)
914
915    cleanup_headers(config_attrs, options.dry_run)
916
917if __name__ == '__main__':
918    main()
919