Commit Graph

43506 Commits

Author SHA1 Message Date
Jose Perez Carranza 333890953d postinst: Add a test case to verify postinst scripts behavior
Add test case that verify behavior of postinst scripts at
roofts time and when is delayed to the first boot directly
on the target.

(From OE-Core rev: 82b171f3b37e6733997fc1e7685b7cac5a3476e7)

Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Francisco Pedraza 23d1c4ffb7 selftest: Test needed to verify postinst order
It verifies the following:

1. Compile a minimal image.
2. The compiled image will add the layer with the recipe postinst,
previously created at:
"meta-selftest/recipes-test"
3. Run QEMU.
4. Validate the task execution order.
[YOCTO #5319]

(From OE-Core rev: a8ff789a3bfedcbc4358db7907a45270d8b1b76a)

Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Mariano Lopez 3f6e6f4e69 scripts/runqemu: Allow to use qemu from host.
This will add support to use qemu from the running host,
with this is possible to put qemu-native in ASSUME_PROVIDED
variable.

By default it will try to get qemu from the build sysroot,
and only if it fails will try to use the host's qemu.

(From OE-Core rev: fe7fd2cd3a9c4fb5b31bd3cab81c96a3b81cb540)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Leonardo Sandoval 2f0eb8e1b6 sysvinit-inittab: fix getty device removal
getty devices were not being removed in some cases because device name
was not at the end of the line, for example a ttyS1 device:

S1:12345:respawn:/bin/start_getty 115200 ttyS1 vt102

Removing this limitation allows sed to remove any line containing
the device.

(From OE-Core rev: 2e0b36981c1f91ed0d3d457c370df10a099407af)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Ross Burton 7db7950367 sqlite: build position-independent code
pseudo links against this and uses PIC, so some toolchain combinations will
refuse to link against sqlite unless it is also PIC.

(From OE-Core rev: 6a58e12d19c539deac9e90679a68438497a42fa4)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Lukasz Nowak b010c327c7 busybox: allow libiproute to handle table ids larger than 255
These changes are required for compatibility with ConnMan, which by default
uses table ids greater than 255.

(From OE-Core rev: e9114bdd8a83b88f59526780910c49e3092fdd57)

Signed-off-by: Lukasz Nowak <lnowak@tycoint.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Ross Burton d535aba974 oeqa: move lib/oe tests to oe-selftest
These tests don't get ran often (as demonstrated by the fact that some were not
ported to Python 3), so move them to oeqa/selftest so they get executed
frequently and can be extended easily.

[ YOCTO #7376 ]

(From OE-Core rev: 2001979ad41e6fdd5a37b0f90a96708f39c9df07)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Andreas Müller 94c29c1008 libsdl2: fix build on wayland(-dev)less hosts
* add sysroot prefix to wayland core protocols
* do not use pkg-config to find wayland-scanner

(From OE-Core rev: a8e7c5b415b99973c39a7ddd57cae45695fb0119)

Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Andreas Müller 00c691465c libsdl2: add wayland-protocols and -native to to depends of PACKAGECONFIG[wayland]
(From OE-Core rev: e49f139b4d5cfbf8b0391f4c434070ad30f37b10)

Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Andreas Müller ef8fd68b7f libsdl2: add EXTRA_OECONF[vardepsexclude] = "MACHINE"
fixes Martin's MACHINE checksum test [1]

[1] http://lists.openembedded.org/pipermail/openembedded-core/2016-November/129464.html

(From OE-Core rev: dcfbfb9b1fc2b5d54e3c04375cb8fc87df849f1f)

Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Jackie Huang 280c83335a extrausers.bbclass: Use PACKAGE_INSTALL instead of IMAGE_INSTALL
The initramfs image recipes changed to use PACKAGE_INSTALL
so they will not be affected by IMAGE_INSTALL, and will cause
error when inherit extrausers:

| ERROR: core-image-minimal-initramfs-1.0-r0 do_rootfs:
  core-image-minimal-initramfs: usermod command did not succeed.

So use PACKAGE_INSTALL as well in extrausers.bbclass to fix it.

(From OE-Core rev: fa541362e2d2cc0494a86a413b7b52dfe3eee908)

Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Leonardo Sandoval 030f62e9c8 selftest: buildoptions: skip read-only-image test depending on distro
Poky-tiny cannot build core-image-sato, so skip test (read-only-image)
in this case.

(From OE-Core rev: bcee8c614f28b38054f5d8c1c5251b3702cf113c)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Leonardo Sandoval e2ea2597c9 selftest: bblayers: remove linux kernel checks for show-recipes check
Preferred kernel recipes depends on the distro, so remove the kernel
checks to avoid failures on non-poky distros and make the test
distro agnostic.

(From OE-Core rev: ae92b72990b3ac804791b501d08126491fdddb7c)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Leonardo Sandoval 279c8ce310 selftest: bbtests: use minimal image so all distros can execute it
poky-tiny distro cannot build full-cmdline image, so use an image
(core-image-minimal) that can be built in all distros.

(From OE-Core rev: b293dd4200bbb3705c88af6113be7f43fbd0ed72)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Leonardo Sandoval 009adf73fa selftest: base: new object member to store the DISTRO value
Instead of quering it multiple times, query once and use it on
test method skip checks. Also, rename current distro sstate object
member to a more meaninful name.

(From OE-Core rev: bde9d99575a63ad2d7fd5974ce6ce19aad9a8984)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Mariano Lopez a92d27acd8 oeqa/utils/commands.py: Fix get_bb_vars() when called without arguments
Commit 9d55e9d489 broke calling get_bb_vars()
when called without arguments. This fix this issue.

(From OE-Core rev: 91f856426c7523e1ebdf6d6f93f5fa7e509d6e49)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 16:15:04 +00:00
Paul Eggleton 66adb6bdf7 oe-selftest: add basic tinfoil tests
Add some tests to verify that the new tinfoil API is operating
correctly.

(From OE-Core rev: 16afda66b861ba028c1152dcdcab2b7ebfbff965)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 0cba3cbcb4 oe-selftest: devtool: improve test_devtool_modify slightly
* Check that man .in file actually gets modified, since sed -i doesn't
  fail if it it doesn't
* Use a variable for man file path

(From OE-Core rev: 9ad36e945fa5b03726f78ba99e823eade8daa710)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 229091babb devtool: prevent BBHandledException from showing traceback
If we don't catch this then attempting to run devtool in non-memres mode
when bitbake is already running will produce a traceback instead of just
an error message.

(From OE-Core rev: e01b75dff599ffa2b66e6608b28bbb3564365eee)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 4448b2a122 devtool: extract: disable basehash mismatch errors
Using the setVariable commands here followed by buildFile will result in
"basehash mismatch" errors, and that's expected since we are deviating
*at runtime* from what was previously seen by changing these variable
values. Set BB_HASH_IGNORE_MISMATCH to turn off the errors.

(From OE-Core rev: b0169796f294bbec0397b7eae86454a46b68cdc5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 078ef4361d lib/oe/recipeutils: drop parse_recipe_simple()
This was intended to be used with tinfoil, but tinfoil now has its own
parse_recipe() method to do this which works properly in the memres
case.

(From OE-Core rev: cdfc6173cb06ca374b7d927442a0fdde8373ba48)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 85de43fccd devtool: fix extraction of source to work in memres mode
Extracting the source for a recipe (as used by devtool's extract, modify
and upgrade subcommands) requires us to run do_fetch, do_unpack,
do_patch and any tasks that the recipe has inserted inbetween, and do so
with a modified datastore primarily so that we can redirect WORKDIR and
STAMPS_DIR in order to have the files written out to a place of our
choosing and avoid stamping the tasks as having executed in a real build
context respectively. However, this all gets much more difficult when in
memres mode since we can't call internal functions such as
bb.build.exec_func() directly - instead we need to execute the tasks on
the server. To do this we use the buildFile command which already exists
for the purpose of supporting bitbake -b, and setVariable commands to
set up the appropriate datastore.

(I did look at passing the modified datastore to the buildFile command
instead of using setVar() on the main datastore, however its use of
databuilder makes that very difficult, and we'd also need a different
method of getting the changes in the datastore over to the worker as
well.)

(From OE-Core rev: eb63b5339014fc72ba4829714e0a96a98e135ee2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 86b55e6026 recipetool: add OE lib path
The autotools code imports oe.package; we weren't experiencing a problem
with this probably due to OE itself adding that path previously.

(From OE-Core rev: a61d7bf8447b2d2c65eb34315c86086ff35c8bc9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 275b3fe5f4 classes/patch: move several functions to oe.patch
Move patch_path(), src_patches() and should_apply() to oe.patch, making
them easier to call from elsewhere (particularly across the
UI/server boundary).

(From OE-Core rev: 2724511e18810cc8082c1b028e3b7c8a8b5def56)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 45adbe3709 classes/patch: move in logic to commit for additional tasks
If PATCHTOOL is "git", and PATCH_COMMIT_FUNCTIONS is set to "1", for
additional tasks between do_unpack and do_patch, make a git commit. This
logic was previously implemented in devtool itself, but it makes more
sense for it to be implemented in the patch class since that's where the
rest of the logic is for this (or in lib/oe/patch.py). It also makes
it possible for this to work with tinfoil2.

(From OE-Core rev: f24f59ea1d8bc335ea8576f6a346d0935f4a3548)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 5942a8dc7c classes/base: fix license file checksumming when source not under TMPDIR
With the changes to the code for extracting source for a recipe, we are
properly executing the tasks for a recipe, which means their stamps (and
therefore signatures) are important. When running devtool extract on
the lsof recipe I noticed that do_fetch and do_unpack were executing a
second time when we called for do_patch, and this turned out to be
because LIC_FILES_CHKSUM in that recipe contains an entry which
is an absolute path (has ${S} at the start). Normally this wouldn't be
an issue since S is under TMPDIR and thus the existing code would ignore
it, however devtool's extraction code extracts to a temporary directory
which is not under TMPDIR; the result was the path to this file was not
being ignored and the second time around when the license file had been
extracted it was incorporated into the signature. We don't want this, so
explicitly exclude S as well as B and WORKDIR for good measure.

(From OE-Core rev: 1c99d74a862f25e23ea6465fab7ddc9ce74d6974)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 70ce595237 devtool / recipetool: use tinfoil parsing API
Use Tinfoil.parse_recipe_file() and Tinfoil.parse_recipe() instead of
the recipeutils equivalents, and replace any local duplicate
implementations. This not only tidies up the code but also allows these
calls to work in memres mode.

(From OE-Core rev: f13b56266ee96dfab65a3a7db50e8051aa9f071a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 1f8cf63f8a oe-selftest: use tinfoil.parse_recipe()
Use tinfoil.parse_recipe() in order to allow oe-selftest to be used in
memres mode.

(From OE-Core rev: 499ee9006271112f22cfe08fa5ba5c21be95380b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 93d675304d oe-selftest: make tinfoil quiet when using to start QEMU
We don't need to see the parsing/cache loading message in the
oe-selftest output, so use the newly added quiet option to disable it.

(From OE-Core rev: c023bc55ff000d1de891d1a8e2a163e94bf63de6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 557382ab42 lib/oe/recipeutils: use cooker function instead of bb.providers
We now have a function in cooker itself that can do this lookup;
additionally, the rewritten tinfoil's cooker adapter has its own
implementation that can work remotely, so if we use it then this
function can work in that scenario as well.

(From OE-Core rev: 0a6a4be99c1e4ef3c0da53d63f18ad579545d6a8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 1e29b57e92 devtool: package: don't try to initialise tinfoil twice
setup_tinfoil() already calls prepare(), we don't need to call it again
ourselves and doing so with tinfoil2 results in "ERROR: Only one copy of
bitbake should be run against a build directory". Calling prepare()
twice should probably still be allowed, so that ought to be fixed
separately, but in the mean time this code is still wrong so fix it
here.

(From OE-Core rev: 38b8a7d4aff096ea0a62f2ddf3fe2de1df591bf5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton af867199a5 classes/image: suppress log_check mechanism for warnings/errors logged through BitBake
If you printed a warning through bb.warn() / bbwarn or an error through
bb.error() / bberror, this was also being picked up by our log_check
mechanism that was designed to pick up warnings and errors printed by
other programs used during do_rootfs. This meant you saw not only the
warning or error itself, you saw it a second time through log_check,
which is a bit ugly. Use the just-added BB_TASK_LOGGER to access the
logger and add a handler that we can use to find out if any warning or
error we find in the logs is one we should ignore as it has already been
printed.

Fixes [YOCTO #8223].

(From OE-Core rev: fb37304d27857df3c53c0867e81fbc8899b48089)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 13916a4fab lib/oe/rootfs: fix log_check warnings being printed twice with RPM packaging
We were calling _log_check() in the RPM-specific rootfs class as well as
in the base class; this is unnecessary and resulted in any errors/warnings
generated during the actual package installation time triggering two warnings
instead of one. Drop the call from RpmRootfs._create() to fix this.

(From OE-Core rev: 541c56d755ba0354297673e857628026ad9e4df2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Ola x Nilsson d65886bb69 oe-pkgdata-util: Make read-value handle override variables
Some variables in pkgdata files have a package-name override.  When
the bare variable can not be found, try with the override-variant.

PKGSIZE is one such variable, and already had special code to handle this.

Test included.

(From OE-Core rev: 6df99cda894033cba68bc6ab91e47f67e0d788a5)

Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton c61b525e91 bitbake: lib/bb/build: enable access to logger within tasks
In certain circumstances it can be useful to get access to BitBake's
logger within a task; the main example is in OpenEmbedded's image
construction code where we want to be able to check the log file for
errors and warnings, but we don't want to see any errors or warnings
that were emitted through the logger; so we need a way to exclude those.
In order to do this, pass the logger object into the task via a
BB_TASK_LOGGER variable, and add a logging handler class to bb.utils
that can be added to it in order to keep a list of warnings/errors that
have been emitted.

(Bitbake rev: f1cd6fab604f14d8686b1d783cbfe012d923ee42)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton 6688f6a115 bitbake: server/process: don't change UI process signal handler on terminate
On terminating the connection to the server, we were disabling SIGINT -
and this is executed on the UI side. I'm not sure whether the intention
here was to undo the SIGINT disabling we did in the server, and it was
just a mistake that it disabled rather than restored and it's run on the
wrong side, or whether we wanted to stop the user from breaking out of
the shutdown code - the commit message provides no clues either way.
Regardless, we do not want to permanently disable Ctrl+C here - it's
legitimate to terminate the connection to the server and then
re-establish it within the same process; at least currently, devtool
modify by virtue of using tinfoil in two separate parts of the code does
this, and the result of this disabling is that during the second tinfoil
usage we can potentially be parsing all recipes without the ability to
easily interrupt the process.

(Bitbake rev: 58c60a951229dcbd8253863fb24228d046c23f6e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton d856bce542 bitbake: siggen: add means of ignoring basehash mismatch
If you run the setVariable command to set variables then you end up
causing the basehash to not match the previously computed values, which
triggers error messages. These mismatches are expected, so add a means
of disabling them.

(Bitbake rev: 5a80c0e210f26526afbe8f266b7b1a9c03334967)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton 8d315820aa bitbake: runqueue: enable setVariable command to affect task execution
Allow the client to set variables with the setVariable command and have
those changes take effect when running tasks. This is accomplished by
collecting changes made by setVariable separately and pass these to the
worker so it can be applied on top of the datastore it creates.

(Bitbake rev: 69a3cd790da35c3898a8f50c284ad1a4677682a4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton 7229250411 bitbake: data_smart: support serialisation
The COW object used within VariableHistory can't be serialised itself,
but we can convert it to a dict when serializing and then back when
deserialising. This finally allows DataSmart objects to be serialized.
NOTE: "serialisation" here means pickling, not over XMLRPC or any other
transport.

(Bitbake rev: bbbb2a53d5decf3b613a92c4ff77c84bfc5d4903)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton 797a8ee040 bitbake: cooker: allow buildFile warning to be hidden programmatically
If we want to use this function/command internally, we don't want this
warning shown.

(Bitbake rev: 5cfbb60833e7b12d698c1c2970c17ccf2a4971bf)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton 8f8a9ef669 bitbake: tinfoil: pass datastore to server when expanding python references
If you're expanding a value that refers to the value of a variable in
python code, we need to ensure that the datastore that gets used to get
the value of that variable is the client-side datastore and not just the
part of it that's on the server side. For example, suppose you are in
client code doing the following:

d.setVar('HELLO', 'there')
result = d.expand('${@d.getVar("HELLO", True)}')

result should be "there" but if the client part wasn't taken into
account, it would be whatever value HELLO had in the server portion of
the datastore (if any).

(Bitbake rev: cbc22a0a9aadc8606b927dbac0f1407ec2736b35)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton f1f3a112a0 bitbake: tinfoil: implement server-side recipe parsing
It's not really practical for us to parse recipes on the client side, we
need to do it on the server because that's where we have the full python
environment (including any "pure" python functions defined in classes).
Thus, add some functions to tinfoil do this including a few shortcut
functions.

(Bitbake rev: 8f635815d191c9d848a92d51fdbf5e9fd3da1727)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton 8c33063a1d bitbake: remotedata: enable transporting datastore from the client to the server
For the purposes of server-side parsing and expansion allowing for
client-side use of the datastore, we need a means of sending a datastore
from the client back to the server, where the datastore probably
consists of a remote (server-side) original plus some client-side
modifications. To do this we need to take care of a couple of things:

1) xmlrpc can't handle nested dicts, so if you enable memres and simply
   try passing a serialised datastore then things break. Instead of
   serialising the entire datastore, just take the naive option of
   transferring the internal dict alone (as a list of tuples) for now.

2) Change the TinfoilDataStoreConnector object into simply the handle
   (number) when transmitting; it gets substituted with the real
   datastore when the server receives it.

(Bitbake rev: 784d2f1a024efe632fc9049ce5b78692d419d938)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton 7d5c9860de bitbake: tinfoil: rewrite as a wrapper around the UI
Rewrite tinfoil as a wrapper around the UI, instead of the earlier
approach of starting up just enough of cooker to do what we want. This
has several advantages:

* It now works when bitbake is memory-resident instead of failing with
  "ERROR: Only one copy of bitbake should be run against a build
  directory".

* We can now connect an actual UI, thus you get things like the recipe
  parsing / cache loading progress bar and parse error handling for free

* We can now handle events generated by the server if we wish to do so

* We can potentially extend this to do more stuff, e.g. actually running
  build operations - this needs to be made more practical before we can
  use it though (since you effectively have to become the UI yourself
  for this at the moment.)

The downside is that tinfoil no longer has direct access to cooker, the
global datastore, or the cache. To mitigate this I have extended
data_smart to provide remote access capability for the datastore, and
created "fake" cooker and cooker.recipecache / cooker.collection adapter
objects in order to avoid breaking too many tinfoil-using scripts that
might be out there (we've never officially documented tinfoil or
BitBake's internal code, but we can still make accommodations where
practical). I've at least gone far enough to support all of the
utilities that use tinfoil in OE-Core with some changes, but I know
there are scripts such as Chris Larson's "bb" out there that do make
other calls into BitBake code that I'm not currently providing access to
through the adapters.

Part of the fix for [YOCTO #5470].

(Bitbake rev: 3bbf8d611c859f74d563778115677a04f5c4ab43)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton e271d7dc60 bitbake: command: provide a means to shut down from the client in memres mode
In memory resident mode we don't really want to actually shut down since
it's only the client going away.

(Bitbake rev: 74db369c46043116359101cab70486afd82372c0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton d325d94f3f bitbake: data_smart: implement remote datastore functionality
This allows you to maintain a local reference to a remote datastore. The
actual implementation of the remote connection is delegated to a
connector object that the caller must define and supply. There is
support for getting variable values and expanding python references
(i.e. ${@...} remotely, however setting variables remotely is not
supported - any variable setting is done locally as if the datastore
were a copy (which it kind of is).

Loosely based on an earlier prototype implementation by Qing He.

(Bitbake rev: a3edc3eefa2d03c4ad5d12187b32fa4dc495082a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton 727f332829 bitbake: server/xmlrpc: send back 503 response with correct encoding
If you send back a string here you get "TypeError: 'str' does not
support the buffer interface" errors in bitbake-cookerdaemon.log and
"IncompleteRead(0 bytes read, 22 more expected)" errors on the client
side.

(Bitbake rev: 0d659a7dfe5fb096f8aa4380320f9e2a464b3cb5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton 7ebca83926 bitbake: knotty: fix --observe-only option
If we're in observe-only mode then we cannot run commands that would
affect the server's state, including getSetVariable, so prevent that
from being called in observe-only mode.

(Bitbake rev: 2c5a8661430edebff67ab4a108995033d182b5d6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:06 +00:00
Paul Eggleton 568409faa1 bitbake: knotty: make quiet option a level option
Allow you to specify -q / --quiet more than once to reduce the messages
even further. It will now operate as follows:

 Level  Option  Result
 -----  ------  ----------------------------------------
 0              Print usual output
 1      -q      Only show progress and warnings or above
 2      -qq     Only show warnings or above
 3+     -qqq    Only show errors

(Bitbake rev: 6cf2582e17c28ca04f5cfb59858c4a9778c700d4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:06 +00:00
Paul Eggleton 3fa98e19d5 bitbake: data_smart: fix resetting of reference on variablehistory
There is no "datasmart" member, only dataroot. This dates back to the
original implementation of variable history support - it's surprising we
haven't noticed the issue until now, but I guess it's rare to change a
copy of a datastore in a manner which using the old reference would
cause an issue.

(Bitbake rev: febd5534b07edfdef15cedb0578730c582c7373f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:06 +00:00