Buildman is a builder. It is not make, although it runs make. It does not
produce any useful output on the terminal while building, except for
progress information (except with -v, see below). All the output (errors,
-warnings and binaries if you are ask for them) is stored in output
+warnings and binaries if you ask for them) is stored in output
directories, which you can look at while the build is progressing, or when
it is finished.
valid value, and all will be well. Otherwise buildman will perform random
actions. Use -n to check what the random actions might be.
-If you just want to build the current source tree, leave off the -b flag.
-This will display results and errors as they happen. You can still look
-at them later using -s. Note that buildman will assume that the source
-has changed, and will build all specified boards in this case.
+If you just want to build the current source tree, leave off the -b flag
+and add -e. This will display results and errors as they happen. You can
+still look at them later using -se. Note that buildman will assume that the
+source has changed, and will build all specified boards in this case.
Buildman is optimised for building many commits at once, for many boards.
On multi-core machines, Buildman is fast because it uses most of the
means to build all arm boards except nvidia, freescale and anything ending
with 'ball'.
-It is convenient to use the -n option to see whaat will be built based on
+It is convenient to use the -n option to see what will be built based on
the subset given.
Buildman does not store intermediate object files. It optionally copies
$ git checkout -b my-branch origin/master
$ # Add some commits to the branch, reading for testing
-2. Create ~/.buildman to tell buildman where to find tool chains. As an
-example:
+2. Create ~/.buildman to tell buildman where to find tool chains (see 'The
+.buildman file' later for details). As an example:
# Buildman settings file
to build x86 commits.
-2. Check the available toolchains
+3. Make sure you have the require Python pre-requisites
+
+Buildman uses multiprocessing, Queue, shutil, StringIO, ConfigParser and
+urllib2. These should normally be available, but if you get an error like
+this then you will need to obtain those modules:
+
+ ImportError: No module named multiprocessing
+
+
+4. Check the available toolchains
Run this check to make sure that you have a toolchain for every architecture.
be used (c88 and c99). This is a feature.
+5. Install new toolchains if needed
+
+You can download toolchains and update the [toolchain] section of the
+settings file to find them.
+
+To make this easier, buildman can automatically download and install
+toolchains from kernel.org. First list the available architectures:
+
+$ ./tools/buildman/buildman sandbox --fetch-arch list
+Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/
+Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/
+Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/
+Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.2.4/
+Available architectures: alpha am33_2.0 arm avr32 bfin cris crisv32 frv h8300
+hppa hppa64 i386 ia64 m32r m68k mips mips64 or32 powerpc powerpc64 s390x sh4
+sparc sparc64 tilegx x86_64 xtensa
+
+Then pick one and download it:
+
+$ ./tools/buildman/buildman sandbox --fetch-arch or32
+Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/
+Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/
+Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/
+Downloading: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1//x86_64-gcc-4.5.1-nolibc_or32-linux.tar.xz
+Unpacking to: /home/sjg/.buildman-toolchains
+Testing
+ - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/.'
+ - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin'
+ - found '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc'
+Tool chain test: OK
+
+Buildman should now be set up to use your new toolchain.
+
+At the time of writing, U-Boot has these architectures:
+
+ arc, arm, avr32, blackfin, m68k, microblaze, mips, nds32, nios2, openrisc
+ powerpc, sandbox, sh, sparc, x86
+
+Of these, only arc and nds32 are not available at kernel.org..
+
+
How to run it
=============
$ ./tools/buildman/buildman -b <branch> -n
If it can't detect the upstream branch, try checking out the branch, and
-doing something like 'git branch --set-upstream <branch> upstream/master'
-or something similar.
+doing something like 'git branch --set-upstream-to upstream/master'
+or something similar. Buildman will try to guess a suitable upstream branch
+if it can't find one (you will see a message like" Guessing upstream as ...).
As an example:
To find out how the build went, ask for a summary with -s. You can do this
-either before the build completes (presumably in another terminal) or or
+either before the build completes (presumably in another terminal) or
afterwards. Let's work through an example of how this is used:
$ ./tools/buildman/buildman -b lcd9b -s
At commit 16, the error moves - you can see that the old error at line 120
is fixed, but there is a new one at line 126. This is probably only because
-we added some code and moved the broken line father down the file.
+we added some code and moved the broken line further down the file.
If many boards have the same error, then -e will display the error only
once. This makes the output as concise as possible. To see which boards have
run_list_real 1996 2000 +4
do_nandboot 760 756 -4
do_mem_crc 168 68 -100
- colibri_t20_iris: all -9 rodata -29 text +20
+ colibri_t20 : all -9 rodata -29 text +20
u-boot: add: 1/0, grow: 2/-3 bytes: 140/-112 (28)
function old new delta
hash_command 80 160 +80
board was built) and by 96 bytes for powerpc. This increase was offset in both
cases by reductions in rodata and data/bss.
-Shown below the summary lines is the sizes for each board. Below each board
-is the sizes for each function. This information starts with:
+Shown below the summary lines are the sizes for each board. Below each board
+are the sizes for each function. This information starts with:
add - number of functions added / removed
grow - number of functions which grew / shrunk
increases, and vice versa.
-Providing 'make' flags
-======================
+The .buildman file
+==================
+
+The .buildman file provides information about the available toolchains and
+also allows build flags to be passed to 'make'. It consists of several
+sections, with the section name in square brackets. Within each section are
+a set of (tag, value) pairs.
+
+'[toolchain]' section
+
+ This lists the available toolchains. The tag here doesn't matter, but
+ make sure it is unique. The value is the path to the toolchain. Buildman
+ will look in that path for a file ending in 'gcc'. It will then execute
+ it to check that it is a C compiler, passing only the --version flag to
+ it. If the return code is 0, buildman assumes that it is a valid C
+ compiler. It uses the first part of the name as the architecture and
+ strips off the last part when setting the CROSS_COMPILE environment
+ variable (parts are delimited with a hyphen).
+
+ For example powerpc-linux-gcc will be noted as a toolchain for 'powerpc'
+ and CROSS_COMPILE will be set to powerpc-linux- when using it.
+
+'[toolchain-alias]' section
+
+ This converts toolchain architecture names to U-Boot names. For example,
+ if an x86 toolchains is called i386-linux-gcc it will not normally be
+ used for architecture 'x86'. Adding 'x86: i386 x86_64' to this section
+ will tell buildman that the i386 and x86_64 toolchains can be used for
+ the x86 architecture.
+
+'[make-flags]' section
-U-Boot's build system supports a few flags (such as BUILD_TAG) which affect
-the build product. These flags can be specified in the buildman settings
-file. They can also be useful when building U-Boot against other open source
-software.
+ U-Boot's build system supports a few flags (such as BUILD_TAG) which
+ affect the build product. These flags can be specified in the buildman
+ settings file. They can also be useful when building U-Boot against other
+ open source software.
-[make-flags]
-at91-boards=ENABLE_AT91_TEST=1
-snapper9260=${at91-boards} BUILD_TAG=442
-snapper9g45=${at91-boards} BUILD_TAG=443
+ [make-flags]
+ at91-boards=ENABLE_AT91_TEST=1
+ snapper9260=${at91-boards} BUILD_TAG=442
+ snapper9g45=${at91-boards} BUILD_TAG=443
-This will use 'make ENABLE_AT91_TEST=1 BUILD_TAG=442' for snapper9260
-and 'make ENABLE_AT91_TEST=1 BUILD_TAG=443' for snapper9g45. A special
-variable ${target} is available to access the target name (snapper9260 and
-snapper9g20 in this case). Variables are resolved recursively. Note that
-variables can only contain the characters A-Z, a-z, 0-9, hyphen (-) and
-underscore (_).
+ This will use 'make ENABLE_AT91_TEST=1 BUILD_TAG=442' for snapper9260
+ and 'make ENABLE_AT91_TEST=1 BUILD_TAG=443' for snapper9g45. A special
+ variable ${target} is available to access the target name (snapper9260
+ and snapper9g20 in this case). Variables are resolved recursively. Note
+ that variables can only contain the characters A-Z, a-z, 0-9, hyphen (-)
+ and underscore (_).
-It is expected that any variables added are dealt with in U-Boot's
-config.mk file and documented in the README.
+ It is expected that any variables added are dealt with in U-Boot's
+ config.mk file and documented in the README.
+
+ Note that you can pass ad-hoc options to the build using environment
+ variables, for example:
+
+ SOME_OPTION=1234 ./tools/buildman/buildman my_board
Quick Sanity Check
==================
If you have made changes and want to do a quick sanity check of the
-currently-checked-out source, run buildman without the -b flag. This will
-build the selected boards and display build status and errors as it runs
-(i.e. -v amd -e are enabled automatically).
+currently checked-out source, run buildman without the -b flag. This will
+build the selected boards and display build status as it runs (i.e. -v is
+enabled automatically). Use -e to see errors/warnings as well.
+
+
+Building Ranges
+===============
+
+You can build a range of commits by specifying a range instead of a branch
+when using the -b flag. For example:
+
+ upstream/master..us-buildman
+
+will build commits in us-buildman that are not in upstream/master.
Other options
flag to see the full errors and -l to see which boards caused which errors.
If you really want to see build results as they happen, use -v when doing a
-build (-e will be enabled automatically).
+build (and -e to see the errors/warnings too).
You don't need to stick around on that branch while buildman is running. It
checks out its own copy of the source code, so you can change branches,
This has mostly be written in my spare time as a response to my difficulties
in testing large series of patches. Apart from tidying up there is quite a
-bit of scope for improvement. Things like better error diffs, easier access
-to log files, error display while building. Also it would be nice it buildman
-could 'hunt' for problems, perhaps by building a few boards for each arch,
-or checking commits for changed files and building only boards which use
-those files.
+bit of scope for improvement. Things like better error diffs and easier
+access to log files. Also it would be nice if buildman could 'hunt' for
+problems, perhaps by building a few boards for each arch, or checking
+commits for changed files and building only boards which use those files.
Credits