Commit Graph

923 Commits

Author SHA1 Message Date
Richard Purdie fb1654fee4 image: Fix wic environment issues
The wic environment function needs to run after the rootfs size is
setup. We move this code to a specific task, and depend on that task
from the wic images and other places its needed.
This fixes:

======================================================================
FAIL: test_image_env (oeqa.selftest.wic.Wic)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 218, in test_image_env
    self.assertTrue(var in content, "%s is not in .env file" % var)
AssertionError: False is not true : ROOTFS_SIZE is not in .env file

(From OE-Core rev: 606f9e2d7d8d389c8d4f5c3090139d3bb780e09c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:46 +00:00
Richard Purdie 1da8f52b67 insane: Start to clean up do_configure_qa code
This cleans up the do_configure_qa code so that the exit status from
package_qa_handle_error is handled correctly.

It also converts package_qa_check_license to use our standard QA check
configuration interface through package_qa_handle_error rather than
a mix of return values, bb.fatal and bb.build.FuncFailed.

Due to the merging of multiple lines into one message, we need to
tweak a QA test to account for this.

(From OE-Core rev: fbdf977a8094bb1e0baca85ffb3ed7f326986639)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:46 +00:00
Aníbal Limón b8ebac9794 DpkgRootfs: Fix logcheck_error false-positive when use multilib
Rootfs with dpkg was failing due to false-positive in logcheck_error
because current logic of DpkgPM handles missing dependencies failure
using apt-get -f install [1][2].

This support was broken due to addition of logcheck and don't take into
account dpkgpm cases, in order to fix add an attr for specify expected
errors regex'es by package manager.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/rootfs.py#n659
[2] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/package_manager.py#n2038

(From OE-Core rev: fa7a5ebef87883755491b847c2f4e1a7b021d585)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:31 +00:00
Matt Madison 4aeb69d35f package_manager.py: fixes for multilib deb packaging builds
* tmp/deploy/deb subdirectories do not get hyphens replaced
  with underscores, so don't do that translation when building
  the sources list.

* Fix MULTILIB_VARIANTS handling to be more general and
  work for all architectures

* Also include a fix for a warning generated by apt
  due to missing apt/preferences.d directory.

(From OE-Core rev: c3ee9b12ae704eadf02ff288d8035b5885d6218e)

Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:31 +00:00
Yi Zhao 235606f31b oeqa/runtime/logrotate: fix hardcoded root directory
Use $HOME instead of /home/root in case user changes the default root
directory by ROOT_HOME

(From OE-Core rev: 123ca1c3457e472bd3dab5563f7ba5312a9864bf)

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:30 +00:00
Yi Zhao cce6c3ed9a oeqa/runtime/smart: fix hardcoded root directory
Use $HOME instead of /home/root in case user changes the default root
directory by ROOT_HOME

(From OE-Core rev: c9adb8399687b60eb976ab32d960f4b32af71fea)

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:30 +00:00
Richard Purdie e306d5495a populate_sdk_ext: Change to include siginfo and non sstate task sigs
Right now, the locked task hashes list for the extensible SDK locks
down only the sstate tasks.

Whilst asthetically pleasing, this gives two problems:

* Half the task are left floating meaning checksum mismatches
  are a pain to debug
* The later code which copies relavent data files out the sstate
  cache can't use any of this data.

This patch modifies things so all the checksums are listed in the locked
file. An exclusion of tasks probably makes more sense for the library
function rather than an allowed list.

The only sstate task being deliberaly excluded here was do_package
so add in a function to explictly exclude those sstate object files.

The net result of this that siginfo files for all tasks are included in
the SDK, which means commands like "bitbake -S printdiff" now function.

(From OE-Core rev: 6b70479e47b8a8743d8b410d6bc08da1607a318e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie 0a4e1f968a image: Create separate tasks for rootfs construction
This patch splits the code in lib/oe/image into separate tasks, one per
image type. This removes the need for the simple task graph code and defers
to the bitbake task management code to handle this instead.

This is a good step forward in splitting up the monolithic code and starting
to make it more accessible to people.

It should also make it easier for people to hook in other tasks and processes
into the rootfs code.

Incidentally, the reason this code was all combined originally was due to
limitations of fakeroot where if you exited the session, you lost permissions
data. With pseudo this constraint was removed.

We did start to rework the rootfs/image code previously and got so far with
untangling it however we did prioritise some performance tweaks over splitting
into separate tasks and in hindsight, this was a mistake and should have been done
the other way around. That work was suspended due to changes in the people working
on the project but this split has always been intended, now is the time to finish
it IMO.

There were some side effects of doing this:

* The symlink for the manifest moves to the rootfs-postcommands class and into
  the manifest function.
* There is no seperate "symlink removal" and "symlink creation", they are merged
* The date/time stamps of the manifest and the built images can now be different since
  the tasks can be run separately and the datetime stamp will then be different
  between do_rootfs and the do_image_* tasks.

(From OE-Core rev: c2dab181c1cdabac3be6197f4b9ea4235cbbc140)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie fdced52387 image: Move pre/post process commands to bbclass
As the next step in splitting up do_image, move the pre and post processing
commands to separate tasks. This also creates the do_image_complete task
which acts as the end marker task for image generation.

(From OE-Core rev: 800528eaa421d451b596545125cb218e08989151)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie 1ee08426d7 sstatesig: Handle special case of gcc-source shared-workdir for printdiff
Often, bitbake -S printdiff would show that there was a checksum not found
which would turn out to be from gcc-source. This is due to it being a
shared-workdir recipe.

For now, hardcode the special case into the sstatesig code to stop people
(including me) puzzling over this.

If/as/when we add any other shared workdir recipes, we'll need to rethink
this.

(From OE-Core rev: f11342f0c838b520828927c9d69f7c17309c1b48)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:47 +00:00
Ioan-Adrian Ratiu 8bec5c55a2 x11vnc: remove all references to moved package
Together with the move to meta-oe, all references to x11vnc should be
removed from oe-core. There are three of these: a distro alias, a
packagegroup rdepends and a runtime test.

(From OE-Core rev: cfd1e4bcd66a9a542007115647cadb8480330fab)

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:20 +00:00
Ross Burton 5aa3b93009 oeqa/selftest/recipetool: update for libjpeg-turbo migration
The recipe creation test of libmatchbox now depends on libjpeg-turbo, not jpeg.

(From OE-Core rev: 58bb2d82f4f1b111641c480a27836be6119ee279)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:17 +00:00
Daniel Istrate c2bda6c000 scripts/oe-selftest: Allow to run tests on random/all MACHINEs
Add an option for random MACHINE into oe-selftest:
--machine [random/all]
1. random: will set a random MACHINE for each test
2. all: will run tests for all machines

Custom machine sets only weak default values (??=) for MACHINE in machine.inc.
This let test cases that require a specific MACHINE to be able to
override it, using (?= or =).

e.g.:
oe-selftest --run-tests signing --machine random -->
will run all tests switching MACHINE randomly for each test

oe-selftest --run-tests signing --machine all -->
for each machine will run all tests

oe-selftest --run-all-tests --machine random

Also update oeqa/selftest/base.py to accomodate this feature.

Fix for [YOCTO #5880].

(From OE-Core rev: 4a9c3653eecd9a3c1b45bab5e16c8d679c55f8bd)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:17 +00:00
Daniel Istrate 8e1435eb71 selftest: Added testcase decorators for 2 tests
1391 for test_devtool_add_fetch_simple from devtool
1392 for test_recipetool_create_simple from recipetool

(From OE-Core rev: d6ee070127a08be98958bbb8b11c80bb64165dc7)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:16 +00:00
Daniel Istrate 4ec2da71db selftest: moved tc test_buildhistory_does_not_change_signatures
Moved test_buildhistory_does_not_change_signatures from
buildhistory/BuildhistoryBase to buildoptions/BuildhistoryTests.

The test being in the base class was causing it to run
multiple times.

Fix for [YOCTO #8867]

(From OE-Core rev: 975e67e28ccba5dcb0fced43c1f9e7da183dc201)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:16 +00:00
Ross Burton 09705a4af0 oeqa/selftest: support sets in devtool comparisons
The devtool and recipetool tests do literal string comparisons, but for some
fields the ordering could be irrelevant and potentially non-deterministic.  For
example, the recipetool_create_simple test started failing with:

AssertionError: 'GPLv2 Unknown' != 'Unknown GPLv2' : values for LICENSE do not match

The ordering of the LICENSE field isn't relevant.  So, if the expected value is
a set(), split the string into a set too and compare those.

(From OE-Core rev: dab1b0ad42f08eb547732e854ba28a4cf0a23107)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:20 +00:00
Patrick Ohly c91078954e package_manager.py: add debugging support for rpm scriptlet execution
By default, smart is invoked with --log-level=warning, which hides all
output from pre/post install scriptlets. That makes it hard to debug
scriptlet failure or why they get postponed to first-boot via
/etc/rpm-postinst.

The new ROOTFS_RPM_DEBUG variabled is expected to be set to an integer in
local.conf an incrementally adds more output:
  0 = default, only warnings
  1 = --log-level=info (includes information about executing scriptlets and their output)
  2 = --log-level=debug
  3 = --log-level=debug plus dumps of scriplet content and command invocation

The default behavior is not changed yet, but it seems that level 1 would
be a better default.

(From OE-Core rev: 5cb597a19dbfe825e5b26d828e18644c9ee58f86)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:19 +00:00
Daniel Istrate ab79287688 selftest: Added MACHINE = "qemux86" to tests that use runqemu
It makes sense for tests that use runqemu to have MACHINE set as qemu.
This also avoid issues when running oe-selftest with --arch random/all
option.

(From OE-Core rev: 169e1eaa4fc5ed03e2307b68686a7f5b1db37a36)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:17 +00:00
Paul Eggleton 9da9308781 oe-selftest: devtool: add more explicit check for ls output
test_devtool_deploy_target is failing on the Yocto Project autobuilder
apparently when it attempts to cut out some fields from the list. It
doesn't fail here and I can't see what the problem lines are, so add a
check for lines with too few fields so we can get a look at them next
time it fails.

(From OE-Core rev: 2b19c74332c3899c84708946d749f88c4416cf73)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton c2435b1181 oe-selftest: add tests for simple devtool add / recipetool create URL case
Add an oe-selftest test case for the newly supported syntax with only
the remote URL specified (auto-detecting name and version).

(From OE-Core rev: 7c7df9f62fe15578af0420c63e320c317e058708)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton ebe5f0b872 recipetool: create: basic extraction of name/version from filename
Often the filename (e.g. source tarball) contains the name and version
of the software it contains.

(This isn't intended to be exhaustive, just to catch the common case.)

(From OE-Core rev: 944eacfb849ee69b41e12c9de4f264406281ac6a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton db5f9645ad recipetool: create: support extracting name and version from build scripts
Some build systems (notably autotools) support declaring the name and
version of the program being built; since we need those for the recipe
we can attempt to extract them. It's a little fuzzy as they are often
omitted or may not be appropriately formatted for our purposes, but it
does work on a reasonable number of software packages to be useful.

(From OE-Core rev: 3b3fd33190d89c09e62126eea0e45aa84fe5442e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Daniel Istrate d4b5a1f171 selftest/buildhistory.py: Test buildhistory does not change sigs
[YOCTO #5953] Add a test to ensure buildhistory does not
change signatures.

Also removed unused imports.

(From OE-Core rev: 9e8f5ff6b0bd6cffcbb991d75487ab6005974000)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22 16:08:49 +00:00
Daniel Istrate f80f8baa6f oeqa/selftest: Added testcase decorators for 2 testcases
bblayers: test_bitbakelayers_showrecipes    1384
wic:      test_directdisk_bootloader_config 1385

(From OE-Core rev: eeb3382f38f0cb9188cccb4d32deb38be0b50f67)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-16 12:12:15 +00:00
Mariano Lopez 9c0186fd57 rootfs.py: Change logic to unistall packages
In the current state some of the base utils (update-rc.d,
base-passwd, shadow, and update-alternatives) are unistalled
when there is no package manager in the image. Checking for
previous commits, the unistall of these utils were to be
done in a read-only filesystem.

It is a valid option to have these utils without a package
manager, and also make sense to remove them when building a
read-only filesystem.

This changes the check logic from having a package mananger
to if is a read-only filesystem to remove the utils.

Another change implemented with this patch is that delayed
post installs now doesn't depend if there is a package manager.
Also it is a valid option to have post install scripts without
package manger.

[YOCTO #8235]

(From OE-Core rev: 5aae19959a443c6ac4b0feef10715c8acf3c6376)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-12 23:42:51 +00:00
Benjamin Esquivel 23083e7e5e oeqa/systemd: get runtest target boot time and log
oeqa.runtime.systemd.SystemdJournalTests.test_systemd_boot_time
this test will query the target boot time from journactl and will
print it to the output, if the time is obtained, the test passes

it then compares the boot time against the default systemd's
timeout TimeoutStartSec and if the boot time is greater than the
it will print it too

this test prints the startup time in the test log like:
...
test_systemd_boot_time (oeqa.runtime.systemd.SystemdJournalTests) ...
Startup finished in 6.922s (kernel) + 52.089s (userspace) = 59.011s.

(From OE-Core rev: 59c6c13a8fa1bfc8e0615463e00ccdef04a87a47)

Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-12 23:42:51 +00:00
Benjamin Esquivel c6330a2783 oeqa/systemd: journalctl helper function
a function to request for the journalctl output to the current target
system with l_match_units support

(From OE-Core rev: b411fec74f39d1efa7276046a32d6048b35dd199)

Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-12 23:42:51 +00:00
Haris Okanovic 639cadd6f9 sdk.py / OpkgSdk: remove_packaging_data() after install
Run remove_packaging_data() on both host and target sysroots to wipe
opkg state after install, similar to what RpmSdk does.

Use case:

Opkg may download local package indexes (I.e. file:// URLs) by
sym-linking them into /var/lib/opkg/lists/ instead of copying [1].
This leaves behind broken symlinks under the lists directory when
using opkg to build SDK sysroots.

The -h option may be set via SDKTAROPTS in some configurations to create
symlink-less SDK archives for Windows file systems. Sysroots containing
broken symlinks will fail to archive under this configuration.

Testing:

Verified /var/lib/opkg/ is empty after running populate_sdk() in a Fido
based distribution.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/opkg/commit/?h=opkg-0.3.x&id=f9022a8520fcde8f1b71424d26a652c218fce685

(From OE-Core rev: c8e0ec2da9ad4ce1c103966906a85f68c15400dd)

Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Reviewed-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Natinst-ReviewBoard-ID: 119065
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:25 +00:00
Leonardo Sandoval 6650357f13 lib/oe/package_manager: Introducing PACKAGE_FEED_BASE_PATHS/PACKAGE_FEED_ARCHS
The recently introduced PACKAGE_FEED_PREFIX is not flexible enough for
constructing URIs, because the same PREFIX is used for all PACKAGE_FEED_URIS.
Also, the string 'PREFIX' is confusing because it is not at the beginning of
the URI. The variable PACKAGE_FEED_BASE_PATHS replaces PACKAGE_FEED_PREFIX,
allowing multiple base paths to be appended on each PACKAGE_FEED_URIS. In the
other hand, a new variable called PACKAGE_FEED_ARCHS, similar in concept to
PACKAGE_BASE_PATHS, defines package architectures defined by the user.

To demonstrate the usage of the PACKAGE_FEED_URIS, PACKAGE_FEED_BASE_PATHS and
PACKAGE_FEED_ARCHS, let's assume these variables are set on local.conf

    PACKAGE_FEED_URIS = "https://example.com/packagerepos/release \
                         https://example.com/packagerepos/updates"
    PACKAGE_FEED_BASE_PATHS = "rpm rpm-dev"
    PACKAGE_FEED_ARCHS = "all core2-64"

the resulting feeds would be

    https://example.com/packagerepos/release/rpm/all
    https://example.com/packagerepos/release/rpm/core2-64
    https://example.com/packagerepos/release/rpm-dev/all
    https://example.com/packagerepos/release/rpm-dev/core2-64
    https://example.com/packagerepos/updates/rpm/all
    https://example.com/packagerepos/updates/rpm/core2-64
    https://example.com/packagerepos/updates/rpm-dev/all
    https://example.com/packagerepos/updates/rpm-dev/core2-64

(From OE-Core rev: 229723a20095e80bde29e4b3398047f62f972170)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:25 +00:00
Mariano Lopez d7baeb5fad selftest/wic.py: Add test for custom bootloader config
This change just add anoher test to the wic module.
It will try to create a image with a custom bootloader
configuration. This test will use the example image
directdisk-bootloader-config to test the configfile
option for the bootloaders.

[YOCTO #8728]

(From OE-Core rev: 567cb51f15354c0398b986e32011420a5f3dd090)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:25 +00:00
Ross Burton 40cd22815c oeqa/sshcontrol: don't source profile
Instead of sourcing /etc/profile to get $PATH including /usr/sbin, just assign
to PATH in the ssh invocation.

The remote /etc/profile may not actually be manipulating PATH as we expect, and
there may be other commands which can interfere with the tests (such as resize
emitting a series of control characters on connection).

(From OE-Core rev: 0f3fb5bbf2fd7db82898fed3281af143387316ff)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:05 +00:00
Ross Burton d39192a848 oeqa/runtime/multilib: refactor ELF class extraction
Instead of duplicating the same code over and over, split it out to a separate
function.

(From OE-Core rev: 4f870f020bbf908ab87990803f3c278bf4e44843)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:05 +00:00
Paul Eggleton 1fbd76093d lib/oe/recipeutils: refactor patch_recipe_file() to use edit_metadata()
Use bb.utils.edit_metadata() to replace some of the logic in this
function; this avoids us effectively having two implementations of the
same thing. In the process fix the following issues:

* Insert values before any leading comments for the next variable
  instead of after them
* Insert overridden variables (e.g. RDEPENDS_${PN}) in the correct place
* Properly handle replacing varflag settings (e.g. SRC_URI[md5sum])

(From OE-Core rev: 0f81b83fc5fd908efa7f6b837137830ca65f6ed6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:04 +00:00
Paul Eggleton 21481bc17a lib/oe/recipeutils: check in validate_pn() for names instead of filenames
Ensure that the user specifies just the name portion instead of a file
name with extension. (We can't just look for . since there are recipe
names such as "glib-2.0" that legitimately contain .).

(From OE-Core rev: a4c6af737811adb2bca87e3896f8710738dd255e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:04 +00:00
Paul Eggleton 0fe742674e devtool: disable creating workspace for extract and search subcommands
For subcommands that don't actually involve the workspace, don't
auto-create the workspace.

(From OE-Core rev: 90cba7992bc1d227e242666cd486414bd4a45f7e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:03 +00:00
Paul Eggleton a360fa7d51 lib/oe/patch: improve extraction of patch header
For patches that we have to extract the header information by hand (i.e.
will not apply with "git am"), make the following improvements:

* If we can't extract author/date/subject, then try to do so from the
  commit that added the patch in git (assuming the metadata is tracked
  by git)
* Take only first Signed-off-by line instead of last
* Accept any case for "Signed-off-by" in case author has typed it by
  hand
* Improve conditional - we can skip the other cases if one matches

Implements [YOCTO #7624].

(From OE-Core rev: 13ec296b5c35aefa2c44f64f8bd1ef54c4a0a731)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:03 +00:00
Mike Crowe 804f5b83f3 image.py: avoid mkdir race when building multiple images
If multiple images are being built simultaneously against the same
sysroot then the call to os.makedirs in Image._write_wic_env can fail
with:

File: '.../meta/lib/oe/image.py', lineno: 341, function: _write_wic_env
     0337:        """
     0338:        stdir = self.d.getVar('STAGING_DIR_TARGET', True)
     0339:        outdir = os.path.join(stdir, 'imgdata')
     0340:        if not os.path.exists(outdir):
 *** 0341:            os.makedirs(outdir)
     0342:        basename = self.d.getVar('IMAGE_BASENAME', True)
     0343:        with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
     0344:            for var in self.d.getVar('WICVARS', True).split():
     0345:                value = self.d.getVar(var, True)
File: '/usr/lib/python2.7/os.py', lineno: 157, function: makedirs
     0153:            if e.errno != errno.EEXIST:
     0154:                raise
     0155:        if tail == curdir:           # xxx/newdir/. exists if xxx/newdir exists
     0156:            return
 *** 0157:    mkdir(name, mode)
     0158:
     0159:def removedirs(name):
     0160:    """removedirs(path)
     0161:
Exception: OSError: [Errno 17] File exists: '.../tmp-glibc/sysroots/cheetah/imgdata'

Using bb.utils.mkdirhier() protects against this.

There's also little point in checking to see if the directory already
exists - we might as well just try and create it regardless.

Once the directory has been created, there's no race on the actual file
since the filename contains IMAGE_BASENAME.

(From OE-Core rev: e3f216eef1b7a8ecfd00317e60704cff472238ba)

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:03 +00:00
Martin Jansa 312862f12a package_manager.py: define info_dir and status_file when OPKGLIBDIR isn't the default
* without this the do_rootfs task doesn't respect OPKGLIBDIR and
  info, status are created in different directory than opkg on
  target expects
* people who modify OPKGLIBDIR need to make sure that opkg.conf included
  in opkg package also sets info_dir and status_file options

(From OE-Core rev: 11f1956cf5d732cfed6d28c8c292b5053966895c)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:02 +00:00
Mike Crowe b00f73429a image.py: Avoid creating empty .env file in _write_wic_env
Creating a file for every image containing a few variables isn't
necessary if wic is not being used, so don't write the file if WICVARS
is empty.

(From OE-Core rev: d40779a7d3ce4f326c29ec7971731cab1f505a37)

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:02 +00:00
Dariusz Pelowski a88505b450 lib/oe/terminal: use C locale when determining version
(From OE-Core rev: 5e5bf4f7e242494ed9c2cbc13bec7f21494db17c)

Signed-off-by: Dariusz Pelowski <dariusz.pelowski@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:02 +00:00
Ross Burton b3641838d6 oeqa/qemurunner: pass nographic to runqemu if DISPLAY isn't set
Not everyone wants to run the tests with a qemu that has a graphical output, so
allow display to be None and pass nographic to runqemu in that case.

(From OE-Core rev: b1d85f13e79f8a493b9849551357a1a5cf0d0fec)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:01 +00:00
Paul Eggleton 2948169291 classes/populate_sdk_ext: tweak reporting of workspace exclusion
If you have a local workspace layer enabled when building the
extensible SDK, we explicitly exclude that from the SDK (mostly because
the SDK has its own for the user to use). Adjust the message we print
notifying the user of this so it's clear that we're excluding it from
the SDK, and scale it back from a warning to a note printed with
bb.plain().

(From OE-Core rev: 90f46f74a088a7b965d2205eceb9eff6f276dd38)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:00 +00:00
Leonardo Sandoval e864f71f4c terminal: Open a new window instead of split on older tmux versions (<1.9)
If an old version is detected (<1.9), create a new window instead of split:
the reason is that there is no easy way to get the active pane height if no
nested formats are supported.

(From OE-Core rev: 457bd6297ae99627c5f596c3c09086d787d5a5ab)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-25 08:08:08 +00:00
Mariano Lopez 49a465cd23 package_manager.py: Delete installed_pkgs.txt file
This change removes the file installed_pkgs.txt after it
was used for installation of complementary packages. This
file was causing confusion when left in the WORKDIR after
the build.

(From OE-Core rev: d0f3f3a294d509560bd12b93b26eeec65cfee314)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:42 +00:00
Mariano Lopez ace895d162 rootfs.py: Stop using installed_pkgs.txt
The method _uninstall_unneeded uses the file installed_pkgs.txt,
this file is left after the build and can cause confusion. This
changes allow to get the installed packages using functions of
rootfs instead of the installed_pkgs.txt file. With this change
now is possible to remove the file without breaking anything.

[YOCTO #8444]

(From OE-Core rev: bf935ac16f6175673417dda92a619946b52fac87)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:42 +00:00
Ross Burton ccb1616e54 lib/oe/distro_check: don't set empty proxy keys
If the proxies dictionary has a proxy set to None urllib will throw an exception
instead of not using a proxy (abridged stack):

File: '/home/ross/Yocto/poky/meta/lib/oe/distro_check.py', lineno: 43, function: get_links_from_url
 *** 0043:    with create_socket(url,d) as sock:
     0044:        webpage = sock.read()
File: '/home/ross/Yocto/poky/meta/lib/oe/distro_check.py', lineno: 5, function: create_socket
     0003:def create_socket(url, d):
     0004:    import urllib
 *** 0005:    socket = urllib.urlopen(url, proxies=get_proxies(d))
File: '/usr/lib/python2.7/urllib.py', lineno: 87, function: urlopen
     0086:    if data is None:
 *** 0087:        return opener.open(url)
File: '/usr/lib/python2.7/urllib.py', lineno: 203, function: open
     0201:        else:
     0202:            proxy = None
 *** 0203:        name = 'open_' + urltype
     0204:        self.type = urltype
     0205:        name = name.replace('-', '_')
Exception: TypeError: cannot concatenate 'str' and 'NoneType' objects

Filter out unset values so that the dictionary only has valid assignments in.

(From OE-Core rev: 2d91290ab5608dd1297d1c26ab807fc4574a8a6b)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:41 +00:00
Richard Purdie 8137a846fb lib/oe/copy_buildsystem: Don't expand BB_TASKDEPDATA
The value isn't a string so don't try and expand it.

(From OE-Core rev: ab87d3649c39326938d82d623efafb76905f770d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:41 +00:00
Ross Burton a6c68d85a9 oeqa/selftest/sstatetests: prettier output for allarch test
Instead of creating two lists of full paths and comparing them which in failure
produces a list of every stamp file (so all tasks, twice), reduce the filename
down to a recipe/task->hash dictionary and compare those, meaning unittest
shows the differences in the dictionaries.

In the future get_files() should be generalised so all tests in this class can
use it, and find a pair of hashes that don't match and run diffsigs on them.

(From OE-Core rev: b612628081b81b50965ae9454df4b2747c6997b2)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:41 +00:00
Daniel Istrate 92328b43bc oeqa/selftest/signing: Added new test for signing sstate.
[YOCTO #8182] Optional signing sstate archives and signature verification
[YOCTO #8559] Signing sstate archives with custom dir for gpg keys

(From OE-Core rev: 6a462fbb11db2085e4b6763a601c7fc4ac0025c8)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:41 +00:00
Daniel Istrate fbb03a8c90 oeqa/selftest/signing: New test for Signing packages in the package feeds.
[YOCTO # 8134] This test verifies features introduced in bug 8134.

It requires as resources the files from meta-selftest/files/signing:
For 'gpg --gen-key' the used input was:
key: RSA
key-size: 2048
key-valid: 0
realname: testuser
email: testuser@email.com
comment: nocomment
passphrase: test123

(From OE-Core rev: 6b9d22bfd5414b517a1f0468e1229dfa2294b5fd)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:41 +00:00