Commit graph

126 commits

Author SHA1 Message Date
Simon Glass
c8d7393b73 buildman: Add a quick-start note
For those who just want to build a board, it is useful to see a quick hint
right at the start of the documentation. Add a few commands showing how to
download toolchains and build a board.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-31 19:37:08 -06:00
Simon Glass
c8785c5b49 buildman: Avoid overwriting existing toolchain entries
The current code for setting up the toolchain config always writes the new
paths to an item called 'toolchain'. This means that it will overwrite any
existing toolchain item with the same name. In practice, this means that:

   buildman --fetch-arch all

will fetch all toolchains, but only the path of the final one will be added
to the config. This normally works out OK, since most toolchains are the
same version (e.g. gcc 4.9) and will be found on the same path. But it is
not correct and toolchains for archs which don't use the same version will
not function as expected.

Adjust the code to use a complete glob of the toolchain path.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31 19:37:08 -06:00
Simon Glass
7e92e46e63 buildman: Drop the toolchain error when downloading toolchains
It doesn't make sense to complain about missing toolchains when the
--fetch-arch option is being used. The user is presumably aware that there
is a toolchain problem and is actively correcting it by running with this
option.

Refactor the code to avoid printing this confusing message.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-31 19:37:08 -06:00
Simon Glass
2289b2763c buildman: Fix a typo in TestSettingsHasPath()
The function comment should say 'buildman'. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31 19:37:08 -06:00
Simon Glass
713bea38dd buildman: Improve the toolchain progress/error output
Use colour to make it easier to see what is going on. Also print a message
before downloading a new toolchain. Mention --fetch-arch in the message that
is shown when there are no available toolchains, since this is the quickest
way to resolve the problem.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31 19:37:08 -06:00
Simon Glass
80e6a48750 buildman: Allow the toolchain error to be suppressed
When there are no toolchains a warning is printed. But in some cases this is
confusing, such as when the user is fetching new toolchains.

Adjust the function to supress the warning in this case.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-31 19:37:08 -06:00
Simon Glass
bd6f5d98de buildman: Fix the 'help' test to use the correct path
When buildman is run via a symlink, this test fails. Fix it to work the same
way as buildman itself.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-31 19:37:08 -06:00
Simon Glass
8e605a5e3e buildman: Automatically create a config file if needed
If there is no ~/.buildman file, buildman currently complains and exists. To
make things a little more friendly, create an empty one automatically. This
will not allow things to be built, but --fetch-arch can be used to handle
that.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31 19:37:08 -06:00
Simon Glass
8ea42101d2 buildman: Tidy up the README a little
Tidy up some problems found by a recent review.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31 19:37:08 -06:00
Stephen Warren
f79f1e0c0e buildman: allow more incremental building
One use-case for buildman is to continually run it interactively after
each small step in a large refactoring operation. This gives more
immediate feedback than making a number of commits and then going back and
testing them. For this to work well, buildman needs to be extremely fast.
At present, a couple issues prevent it being as fast as it could be:

1) Each time buildman runs "make %_defconfig", it runs "make mrproper"
first. This throws away all previous build results, requiring a
from-scratch build. Optionally avoiding this would speed up the build, at
the cost of potentially causing or missing some build issues.

2) A build tree is created per thread rather than per board. When a thread
switches between building different boards, this often causes many files
to be rebuilt due to changing config options. Using a separate build tree
for each board would avoid this. This does put more strain on the system's
disk cache, but it is worth it on my system at least.

This commit adds two command-line options to implement the changes
described above; -I ("--incremental") turns of "make mrproper" and -P
("--per-board-out-dir") creats a build directory per board rather than per
thread.

Tested:

    ./tools/buildman/buildman.py tegra
    ./tools/buildman/buildman.py -I -P tegra
    ./tools/buildman/buildman.py -b tegra_dev tegra
    ./tools/buildman/buildman.py -b tegra_dev -I -P tegra

... each once after deleting the buildman result/work directory, and once
"incrementally" after a previous identical invocation.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org> # v1
Tested-by: Simon Glass <sjg@chromium.org> # v1
Acked-by: Simon Glass <sjg@chromium.org>
2016-05-17 09:54:43 -06:00
Simon Glass
c81d0d215e buildman: Clarify the use of -V
This option outputs to the log file, not to the terminal. Clarify that in
the help, and add a mention of it in the README.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Stephen Warren <swarren@wwwdotorg.org>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2016-03-17 21:27:39 -06:00
Simon Glass
17bce66cbf buildman: Add a way to specific a full toolchain prefix
At present buildman allows you to specify the directory containing the
toolchain, but not the actual toolchain prefix. If there are multiple
toolchains in a single directory, this can be inconvenient.

Add a new 'toolchain-prefix' setting to the settings file, which allows
the full prefix (or path to the C compiler) to be specified.

Update the documentation to match.

Suggested-by: Stephen Warren <swarren@wwwdotorg.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
2016-03-17 21:27:39 -06:00
Simon Glass
d4c8572b71 buildman: Allow branch names which conflict with directories
At present if you try to use buildman with the branch 'test' it will
complain that it is unsure whether you mean the branch or the directory.
This is a feature of the 'git log' command that buildman uses. Fix it
by resolving the ambiguity.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-03-17 21:27:39 -06:00
Simon Glass
608e399fde buildman: Allow the toolchain architecture to be specified
At present the architecture is deduced from the toolchain filename. Allow it
to be specified by the caller.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com
2016-03-14 15:34:50 -06:00
Simon Glass
ff690df9fc buildman: Allow the toolchain priority to be specified
At present the priority of a toolchain is calculated from its filename based
on hard-coded rules. Allow it to be specified by the caller. We will use
this in a later patch. Also display the priority and provide a message when
it is overriden by another toolchain of higher priority.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-03-14 15:34:50 -06:00
Simon Glass
ad24ebace1 buildman: Fix up a few code inconsistencies in toolchain.py
Normally we use a single quote for strings unless there is a reason not to
(such as an embedded single quote). Fix a few counter-examples in this file.
Also add a missing function-argument comment.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-03-14 15:34:50 -06:00
Simon Glass
2bdeade0ea buildman: patman: Fix -H when installed as a symlink
It is convenient to install symlinks to buildman and patman in the search
patch, such as /usr/local/bin. But when this is done, the -H option fails to
work because it looks in the directory containing the symlink instead of its
target. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-03-14 15:34:50 -06:00
Bin Meng
e8aebc4744 tools: Update openrisc toolchain information
Since commit 87da2690ab
"openrisc: updating build tools naming convention", openrisc
kernel.org toolchain is out of date and cannot build U-Boot.
Update buildman and moveconfig tools to refer to the new one.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
2016-02-24 18:44:11 -05:00
Thomas Chou
8951523c6a buildman: README: add links for toolchains not available on kernel.org
Add links for toolchains not available on kernel.org.

The sh4 toolchains from kernel.org dose not work for some boards,
so use the sh from Sourcery.

Signed-off-by: Thomas Chou <thomas@wytron.com.tw>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2015-12-10 20:34:23 -07:00
Simon Glass
8270e3c12e buildman: Improve the config comparison feature
At present buildman can compare configurations between commits but the
feature is less useful than it could be. There is no summary by architecture
and changes are not reported on a per-board basis.

Correct these deficiencies so that it is possible to see exactly what is
changing for any number of boards.

Note that 'buildman -b <branch> -C' is recommended for any build where you
will be comparing configuration. Without -C the correct configuration will
not be reported since changes will often not be picked up.

Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2015-09-09 07:48:03 -06:00
Ulf Magnusson
90c36d8ab9 kconfiglib: update to the latest version
Corresponds to ba71a0e (Fix _parse_block() 'parent' documentation re.
ifs.) from upstream, just adding the SPDX tag.

Has performance improvements, code cleanup, Python 3 support, and various
small fixes, including the following:

  - Unset user values when loading a zero-byte .config. (5e54e2c)
  - Ignore indented .config assignments. (f8a7510)
  - Do not require $srctree to be set for non-kernel projects. (d56e9c1)
  - Report correct locations in the presence of continuation lines.
    (0cebc87)
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2015-08-28 12:33:13 -04:00
Bin Meng
9f244b27cf buildman: Correct '--fetch-arch' command documentation
The doc wrongly put sandbox in the '--fetch-arch' command. Remove it.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
2015-07-28 10:36:22 -06:00
Tom Rini
dd59211046 builderthread.py: Keep 'SPL'
On i.MX platforms the SPL binary is called "SPL" so make sure we keep
that.

Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org>
2015-07-14 18:03:15 -06:00
Masahiro Yamada
9d01b7872e kconfiglib: sync with the latest in Kconfiglib project
This commit imports some updates of kconfiglib.py from
https://github.com/ulfalizer/Kconfiglib

 - Warn about and ignore the "allnoconfig_y" Kconfig option
 - Statements in choices inherit menu/if deps
 - Add Symbol.is_allnoconfig_y()
 - Hint that modules are still supported despite warnings.
 - Add warning related to get_defconfig_filename().
 - Fix typo in docs.
 - Allow digits in $-references to symbols.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Philip Craig <philipjcraig@gmail.com>
Signed-off-by: Jakub Sitnicki <jsitnicki@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2015-06-12 12:43:06 -04:00
Michal Simek
1246231c48 buildman: Add gcc 4.9.0 with Microblaze toolchain
Also read gcc 4.9.0 at kernel.org which also have Microblaze toolchain.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Acked-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>

Fixed unit test failure by updating the test:
Signed-off-by: Simon Glass <sjg@chromium.org>
2015-04-23 12:31:54 -06:00
Tom Rini
f5e5ece0b7 buildman: Make -V (verbose_build) really be verbose
The help text for -V says we will pass V=1 but all it really did was not
pass in -s.  Change the logic to pass make V=1 with given to buildman -V or
-s to make otherwise.

Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org>
2015-04-18 16:24:25 -06:00
Tom Rini
0eb4c04503 buildman: Keep more outputs with the --keep-outputs flag
When told to keep outputs, be much more liberal in what files we keep.
In addition to adding 'MLO', keep anything that matches u-boot-spl.* (so
that we keep the map file as well) and anything we generate about
'u-boot itself.  A large number of bootable formats now match this and
thus it's easier to build many targets and then boot them afterwards
using buildman.

Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org>
2015-04-18 16:24:25 -06:00
Simon Glass
843312dcdd buildman: Allow comparison of build configuration
It is useful to be able to see CONFIG changes made by commits. Add this
feature to buildman using the -K flag so that all CONFIG changes are
reported.

The CONFIG options exist in a number of files. Each is reported
individually as well as a summary that covers all files. The output
shows three parts: green for additions, red for removals and yellow for
changes.

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-04-18 16:24:25 -06:00
Simon Glass
970f932a68 buildman: Store build config files
Store all config file output so that we can compare changes if requested.

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-04-18 16:24:24 -06:00
Simon Glass
88c8dcf949 buildman: Adjust the 'aborted' heuristic for writing output
At present buildman tries to detect an aborted build and doesn't record a
result in that case. This is to make sure that an abort (e.g. with Ctrl-C)
does not mark the build as done. Without this option, buildman would never
retry the build unless -f/-F are provided. The effect is that aborting the
build creates 'fake errors' on whatever builds buildman happens to be
working on at the time.

Unfortunately the current test is not reliable and this detection can
trigger if a required toolchain tool is missing. In this case the toolchain
problem is never reported.

Adjust the logic to continue processing the build result, mark the build as
done (and failed), but with a return code which indicates that it should be
retried.

The correct fix is to fully and correctly detect an aborted build, quit
buildman immediately and not write any partial build results in this case.
Unfortunately this is currently beyond my powers and is left as an exercise
for the reader (and patches are welcome).

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-04-18 16:24:24 -06:00
Simon Glass
40f11fce7c buildman: Show 'make' command line when -V is used
When a verbose build it selected, show the make command before the output of
that command.

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-04-18 16:24:24 -06:00
Marcel Ziswiler
e57c6e5b50 ARM: tegra: rename colibri_t20 board/configuration/device-tree
In accordance with our other modules supported by U-Boot and as agreed
upon for Apalis/Colibri T30 get rid of the carrier board in the board/
configuration/device-tree naming.

While at it also bring the prompt more in line with our other products.

Signed-off-by: Marcel Ziswiler <marcel@ziswiler.com>
Signed-off-by: Tom Warren <twarren@nvidia.com>
2015-03-30 10:04:43 -07:00
Simon Glass
63c619eefd buildman: Add a space before the list of boards
Tweak the output slightly so we don't get things like:

   - board1 board2+  board3 board4

There should be a space before the '+'.

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-03-05 16:14:32 -07:00
Simon Glass
2a76a64981 buildman: Correct toolchain download feature
Commit d908898 updated the ScanPath() function but not its documentation
and not all its callers.

This breaks the toolchain check after it is downloaded. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Heiko Schocher <hs@denx.de>
2015-03-05 16:14:01 -07:00
Albert ARIBAUD
d908898333 buildman: allow multiple toolchains in a single path
When buildman scans a toolchain path, it stops at the
first toolchain found. However, a single path can contains
several toolchains, each with its own prefix.

This patch lets buildman scan all toolchains in the path.

Signed-off-by: Albert ARIBAUD <albert.u.boot@aribaud.net>
Acked-by: Simon Glass <sjg@chromium.org>
2015-02-15 14:34:06 -07:00
Simon Glass
d2ce658de5 buildman: Add an option to write the full build output
Normally buildman runs with 'make -s' meaning that only errors and warnings
appear in the log file. Add a -V option to run make in verbose mode, and
with V=1, causing a full build log to be created.

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-01-14 21:16:54 -08:00
Simon Glass
827e37b558 buildman: Add the option to download toolchains from kernel.org
The site at https://www.kernel.org/pub/tools/crosstool/ is a convenient
repository of toolchains which can be used for U-Boot. Add a feature to
download and install a toolchain for a selected architecture automatically.

It isn't clear how long this site will stay in the current place and
format, but we should be able to rely on bug reports if it changes.

Suggested-by: Marek Vašut <marex@denx.de>
Suggested-by: Fabio Estevam <festevam@gmail.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2015-01-14 21:16:54 -08:00
Simon Glass
9b83bfdcb0 buildman: Allow architecture to alias to multiple toolchains
Some archs have need than one alias, so support a list of alises in the
..buildman file.

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-01-14 21:16:54 -08:00
Simon Glass
cc935295f2 buildman: Don't use the local settings when running tests
We should create a test setting file when running testes, not use whatever
happens to be on the local machine.

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-01-14 21:16:54 -08:00
Simon Glass
7024ab6316 buildman: Don't complain about missing sections in ~/.buildman
Silently ignore this since it is valid to have missing sections.

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-01-14 21:16:54 -08:00
Simon Glass
620053421c buildman: Add documentation about the .buildman file
This file is only partially documented. Add some more details.

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Wolfgang Denk <wd@denx.de>
2015-01-14 21:16:53 -08:00
Simon Glass
346996969d buildman: Add a note about Python pre-requisites
Since we need a few modules which might not be available in a bare-bones
distribution, add a note about that to the README.

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Wolfgang Denk <wd@denx.de>
2015-01-14 21:16:53 -08:00
Simon Glass
bb1501f2c2 buildman: Add an option to use the full tool chain path
In some cases there may be multiple toolchains with the same name in the
path. Provide an option to use the full path in the CROSS_COMPILE
environment variable.

Note: Wolfgang mentioned that this is dangerous since in some cases there
may be other tools on the path that are needed. So this is set up as an
option, not the default. I will need test confirmation (i.e. that this
commit fixes a real problem) before merging it.

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Steve Rae <srae@broadcom.com>
2015-01-14 21:16:53 -08:00
Simon Glass
f210b58734 buildman: Put the toolchain path first instead of last in PATH
If:

1. Toolchains A and B have the same filename
2. Toolchain A is in the PATH
3. Toolchain B is given in ~/.buildman and buildman uses it to build

then buildman will add toolchain B to the end of its path but will not
necessarily use it since U-Boot will find toolchain A first in the PATH.

Try to fix this by putting the toolchain first in the path instead of
last.

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-01-14 21:16:53 -08:00
Simon Glass
b53241230d buildman: Try to avoid hard-coded string parsing
The assumption that the compiler name will always end in gcc is incorrect
for clang and apparently on BSD.

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-01-14 21:16:53 -08:00
Simon Glass
5abab20dfb buildman: Allow specifying a range of commits to build
Adjust the -b flag to permit a range expression as well as a branch.

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Tested-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
2015-01-14 21:16:53 -08:00
Simon Glass
0740127f4d buildman: Don't remove entire output directory when testing
When running tests the output directory is often wiped. This is only safe if
a branch is being built. The output directory may contain other things
besides the buildman test output.

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-01-14 21:16:52 -08:00
Simon Glass
5971ab5c44 buildman: Add an option to flatten output directory trees
When building current source for a single board, buildman puts the output
in <output_dir>/current/current/<board>. Add an option to make it use
<output_dir>/<board> instead. This removes the unnecessary directories
in that case, controlled by the --no-subdirs/-N option.

Suggested-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2015-01-14 21:16:52 -08:00
Simon Glass
2a9e2c6a09 buildman: Try to guess the upstream commit
Buildman normally obtains the upstream commit by asking git. Provided that
the branch was created with 'git checkout -b <branch> <some_upstream>' then
this normally works.

When there is no upstream, we can try to guess one, by looking up through
the commits until we find a branch. Add a function to try this and print
a warning if buildman ends up relying on it.

Also update the documentation to match.

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Wolfgang Denk <wd@denx.de>
2015-01-14 21:16:52 -08:00
Simon Glass
1a91567510 buildman: Don't prune output space for 'current source' build
This is not needed since we always do a full (non-incremental) build. Also
it might be dangerous since it will try to delete everything below the
base directory.

Fix this potentially nasty bug.

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-01-14 21:16:52 -08:00