Commit Graph

4290 Commits

Author SHA1 Message Date
Robert Yang eb418c37ce insane.bbclass: fix package_qa_walk()
The parameter "path" would be redefined inside package_qa_walk() which
is useless,  so remove it.

(From OE-Core rev: b07d06d02cb675f5ed00ebe603441254f5671088)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:22 +00:00
Robert Yang e1850048a2 insane.bbclass: print all the QA messages
Fixed:
If more than one files have the same QA issue, it only prints the last
one, others are overrided, for example:
messages["host-user-contaminated"] = "foo1"
messages["host-user-contaminated"] = "foo2"
Only foo2 will be printed.

Add package_qa_add_message() to hanlde messages, so that all of them
will be printed. The package_qa_add_message() is from RP.

(From OE-Core rev: 1dcb38ca9d632887dd99ea9fa32dd955561dc18d)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:22 +00:00
Robert Yang 68c5e6d4d0 insane.bbclass:buildpaths: ignore ipkg/dpkg's CONTROL dir
Fixed race issue when:
WARN_QA_append = " buildpaths"
PACKAGE_CLASSES = "package_ipk"
$ bitbake e2fsprogs

[snip]
 *** 0595:    with open(path) as f:
     0596:        file_content = f.read()
     0597:        if tmpdir in file_content:
     0598:            messages["buildpaths"] = "File %s in package contained reference to tmpdir" % package_qa_clean_path(path,d)
     0599:
Exception: IOError: [Errno 2] No such file or directory: '/path/to/tmp/work/core2-64-poky-linux/e2fsprogs/1.42.9-r0/packages-split/e2fsprogs-ptest/CONTROL/control'

The similar to package_deb.

(From OE-Core rev: cd4313339853d2eddebf93a49d0f45a305985a31)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:21 +00:00
Ross Burton 258676bfa7 sstate: display the sysroot name when cleaning for clarity
When cleaning old builds from the sysroots, also print the sysroot architecture.

(From OE-Core rev: 98a98a4bf71f74b5c27291ee9a2ac14006377e49)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:21 +00:00
Ross Burton ba837f1055 autotools: don't output the full config.log on configure failure
The config.log written by autoconf includes many pages of useless output,
followed by an obfuscated error message, and then more pages of every variable
that's been set.  It's only understandable if you're well versed in how autoconf
behaves, and often in simple failure modes doesn't actually make it clear what
the problem was.

Instead of outputting the whole config.log to the console when do_configure()
fails, use bbfatal_log so the human-readable configure output (not the
config.log) is shown to the user, and tell the user where config.log can be
found if they need it.

[ YOCTO #8856 ]

(From OE-Core rev: 55e263c559098cce99d8c3ef57ccc39f1d4a6848)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:20 +00:00
Mariano Lopez 6ebda8e659 lib/oe/rootfs: Use list_pkgs() instead of list()
This patch changes the use list_pkgs() instead of list()
from class RpmPkgsList. The change is in two functions,
image_list_installed_packages from rootfs.py and
sdk_list_installed_packages from sdk.py.

With this change the functions calling the functions
listed above, must format the output as they required.
The formatting can be done using format_pkg_list() from
oe.utils.

The classes calling the afected functions are changed too
with this patch, to keep the same functionality using the
new data structure.

[YOCTO #7427]

(From OE-Core rev: 983ea373362514e5888bd1d7d9c4f136c94b00f2)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:15 +00:00
Ross Burton 18357688e9 sstate: replace verbose manifest removal with a single count
If there are lots of changes between the previous build and the build about to
start bitbake will potentially print pages of:

  DEBUG: Stamp $BUILD/stamps/corei7-64-poky-linux/libdrm/2.4.66-r0 is not reachable, removing related manifests

Instead of spamming the console with this list, write the list of manifests only
to the debug log and simply write a count to the console.  This way the user
doesn't get spammed but still knows what is happening if bitbake appears to
stall with heavy I/O.

(From OE-Core rev: e777b541c76dad293d1c214e46c00f8f78fe0539)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:37:35 +00:00
Markus Lehtonen 5dc38a349a sign_rpm.bbclass: fix task dependencies
do_rootfs task needs to depend on signing-keys:do_export_public_keys.
The rpm signing public key needs to be present in order to prevent a
crash because it is imported into the rootfs rpmdb before rootfs
creation starts.

(From OE-Core rev: f854f4549f0b01421464032406a5275494acd818)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:37:34 +00:00
Richard Purdie 9e867efdc8 sstate: Add packagedata to list of tasks not to recurse
If we "bitbake X -c packagedata" and the packagedata comes from sstate, we
don't need any of the tasks dependencies. This is similar to the
populate_lic case, we only care about the end result.

Therefore short circuit the dependencies so packagedata doesn't pull in
any other dependencies.

(From OE-Core rev: 76aa4f49db7d32bbd35703c64470fdfc63f403a4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:53 +00:00
Paul Eggleton 5e881c185c classes/populate_sdk_ext: fix task dependency regression
Fix a regression caused by OE-Core revision
eabeb26335b1a4eb1e68218160dbdbe8fdf36272 where we lost the task
dependency on packages in TOOLCHAIN_HOST_TASK (such as
meta-environment-extsdk-${MACHINE}) which led to errors about missing
packages when building the extensible SDK. This happened because I only
moved half of the dependencies coming from the standard SDK task to
do_sdk_depends.

While I'm at it, tidy up the do_populate_sdk_ext[depends] line.

(From OE-Core rev: 80309a2ba25de219b5339f1af6d5052d6229b676)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:53 +00:00
Richard Purdie 2e9f092c97 image: Handle image types containing '-' correctly
Shell function names can't contain '-' characters, which means our image
task names also can't. Add some mapping to use '_' instead of the '-' so
images like "rpi-sdimg" work again.

(From OE-Core rev: e609a4dea2f6d9744e7d2a6650bebf2c02398907)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:53 +00:00
Richard Purdie ef211a5ca1 bootimg/image-vm/image-live: Improve image dependencies
Now that image generation consists of separate tasks, we can improve
the dependencies in these classes. We can also incorporate the tasks
within do_image_complete instead of do_build so we have a specific task
for when the deploy directory is populated with images.

This fixes various expectations about do_image_completed and the way
I'd tried to use it to fix some sanity test problems.

(From OE-Core rev: 9543c4a9ce0ac7d9ced66fa14e48a1aa7401011e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:47 +00:00
Richard Purdie 0910bc65f5 image: Always run do_rootfs_wicenv
The wic command can be used externally but for this to work, the wic
environment file needs to be present. Therefore write this out
universally, it runs in parallel with other image construction so
any performance implications are negligible.

(From OE-Core rev: b2576f2eab10e4c5dd86449312b417a269cc578e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:47 +00:00
Richard Purdie 05716dd7ee bootimg/image: Enhance bootimg to respect RM_OLD_IMAGE
Currently, iso and hddimg links don't respect RM_OLD_IMAGE. This
updates them to use the common symlinks code so that they behave
like the rest of the system.

(From OE-Core rev: 6a05cb64dfafd531d50454ef7141ff0290d01ca9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:47 +00:00
Richard Purdie 1c869a96b2 rootfs-postcommands: Ensure license manifests respect RM_OLD_IMAGE
When RM_OLD_IMAGE is set, old manifest files should be removed
along with old image files and our QA tests expect this.

This patch ensures this happens.

(From OE-Core rev: fc951851a0e172641e2f24a4edce87ca2eb5cecd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:47 +00:00
Richard Purdie d27491b694 image: Ensure we don't expand TMPDIR in image commands
Similarly to DATETIME, don't expand TMPDIR in image commands.
This ensures some of the stamp comparisons we make in the
QA tests work correctly.

(From OE-Core rev: a8c377beadb85b0ff503ec8ddd1a2cd05e363c19)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:47 +00:00
Richard Purdie ce8a2068d4 image: Fix instability of do_image_* checksums
The do_image_* tasks contained the expanded version of DATETIME. Due
to the expansion, we couldn't exclude the value from the task checksum
which meant the task would rerun.

We fix this by deleting the DATETIME value during expansion so we don't
expand any references to at that time. This means the task's hash can be
stable rather than having hardcoded date/time values. It will get expanded
at execution time.

This also fixes errors shown by -S:

NOTE: Reparsing files to collect dependency data
Writing locked sigs to /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/locked-sigs.inc
ERROR: Bitbake's cached basehash does not match the one we just generated (/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/recipes-sato/images/core-image-sato.bb.do_image_tar)!
ERROR: The mismatched hashes were 77872792556367f1dde49a1425caf1a0 and 9bb0aca6286ab7dd22d3c69964beb665

(From OE-Core rev: ecbc1db7ed1f9848dee69507de8eb289b8ddeba0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:47 +00:00
Richard Purdie fb1654fee4 image: Fix wic environment issues
The wic environment function needs to run after the rootfs size is
setup. We move this code to a specific task, and depend on that task
from the wic images and other places its needed.
This fixes:

======================================================================
FAIL: test_image_env (oeqa.selftest.wic.Wic)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 218, in test_image_env
    self.assertTrue(var in content, "%s is not in .env file" % var)
AssertionError: False is not true : ROOTFS_SIZE is not in .env file

(From OE-Core rev: 606f9e2d7d8d389c8d4f5c3090139d3bb780e09c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:46 +00:00
Richard Purdie 1da8f52b67 insane: Start to clean up do_configure_qa code
This cleans up the do_configure_qa code so that the exit status from
package_qa_handle_error is handled correctly.

It also converts package_qa_check_license to use our standard QA check
configuration interface through package_qa_handle_error rather than
a mix of return values, bb.fatal and bb.build.FuncFailed.

Due to the merging of multiple lines into one message, we need to
tweak a QA test to account for this.

(From OE-Core rev: fbdf977a8094bb1e0baca85ffb3ed7f326986639)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:46 +00:00
Richard Purdie dd28695320 insane: Clean up horrible return value processing code
Whilst in theory return values are nice, handling the complex
aggregation of the return values is ugly. The code already uses QA_SANE
as a marker to signal problems. Whilst that isn't as idealistic, it
makes the code massively cleaner, so rely on this instead.

(From OE-Core rev: ebee3caeaa18daf4974ea90e264d64467681f57a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:46 +00:00
Jackie Huang 320a319220 license.bbclass: fix license manifest
The license manifest is wrong when there is no spaces before and after separates |&()
and we can get warning like:
WARNING: The license listed Artistic-1.0GPLv1+ was not in the licenses collected for recipe

(From OE-Core rev: 20d6a4ce4e395251604d92111ce5ecacde5ce172)

Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:08 +00:00
Andre McCurdy 76e35f1d76 kernel-yocto.bbclass: move do_kernel_link_vmlinux() into kernel.bbclass
Move do_kernel_link_vmlinux() from kernel-yocto.bbclass into
kernel.bbclass so that it's available to any kernel recipe.

Note that the task is not enabled by default in kernel-yocto.bbclass,
so don't enable by default in kernel.bbclass either. To enable, see
the example in linux-yocto.inc, ie:

  addtask kernel_link_vmlinux after do_compile before do_install

(From OE-Core rev: a29371848deda618a42f9a30f9856a44c2342fe6)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:08 +00:00
Andre McCurdy d453fa183c kernel-yocto.bbclass: remove do_kernel_link_vmlinux from SRCTREECOVEREDTASKS
The do_kernel_link_vmlinux() task modifies the build directory (not
the source tree) and should not be skipped when externalsrc is being
used.

(From OE-Core rev: 70b8a60d42831c701d10066eb57aaaad7a6fb7ae)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:08 +00:00
Elliot Smith e246905bfc toaster.bbclass: Separate artifact dump from image file dump
SDK artifacts were being dumped alongside the image file dump.
However, the dump was not being triggered correctly, as it
should be attached as a postfunc to do_populate_sdk, rather than
do_rootfs, as do_rootfs doesn't always fire when populating
the SDK directory.

By adding a postfunc to do_populate_sdk, the postfunc is fired
in the correct situations.

Also fixes the directory which is scanned for SDK artifacts, as
this is different from the image file directory.

[YOCTO #7603]

(From OE-Core rev: cd2eb45ce1f9be15e133d6fe940c1cff2199099b)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:07 +00:00
Peter Kjellerstedt 4b4dea75b5 useradd-staticids.bbclass: Remove unnecessary spaces
This removes unnecessary spaces inserted before semicolons in the
modified USERADD_PARAM_${PN} and GROUPADD_PARAM_${PN} variables. This
should not affect the handling of the variables as the only one that
actually sees the semicolons is the code in useradd.bbclass that uses
cut to split the variables at them, and any whitespace preceeding or
following the semicolons will be properly ignored.

(From OE-Core rev: acc17ef91a6f506e3cacdc0d4ebfa268b3f3affd)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:07 +00:00
Peter Kjellerstedt 4f2c352580 useradd-staticids.bbclass: Read passwd/group files before parsing
Read and merge the passwd/group files before parsing the user and
group definitions. This means they will only be read once per
recipe. This solves a problem where if a user was definied in multiple
files, it could generate group definitions for groups that should not
be created. E.g., if the first passwd file read defines a user as:

foobar:🔢:::

and the second passwd file defines it as:

foobar:::nogroup:The foobar user:/:/bin/sh

then a foobar group would be created even if the user will use the
nogroup as its primary group.

(From OE-Core rev: 3149319ad997379a01d87f2b1b5d14f66541997f)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:06 +00:00
Peter Kjellerstedt 4cbdb1577e useradd-staticids.bbclass: Simplify the logic for when to add groups
The original code was near impossible to follow, and missed a couple
of cases. For example, if one added the following line to the passwd
file specified in USERADD_UID_TABLES:

foobar12345:nogroup::/:/bin/sh

and then specified the user as:

USERADD_PARAM_${PN} = "--system foobar"

one would then assume that the foobar user would be created with the
primary group set to nogroup. However, it was not (the primary group
would be foobar), and the only way to get it correct was to explicitly
add --gid nogroup to the USERADD_PARAM_${PN}.

(From OE-Core rev: e98e98ce65cf9ffdef29462310a622ffddd1412b)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:06 +00:00
Peter Kjellerstedt b18e40ce2f useradd-staticids.bbclass: Simplify some logic
The [<on_true>, <on_false>][not <condition>] construct may solve the
problem of implementing a conditional operator, but it is not very
readable. At least I find this:

    uaargs.groupid = field[3] or uaargs.gid or uaargs.groupname

a lot more readable than this:

    uaargs.groupid = [uaargs.gid, uaargs.groupname][not uaargs.gid]
    uaargs.groupid = [field[3], uaargs.groupid][not field[3]]

Also, the official conditional operator since Python 2.5 (<on_true> if
<condition> else <on_false>) does not evaluate both <on_false> and
<on_true> as [<on_true>, <on_false>][not <condition>] does.

(From OE-Core rev: 274d38a6e2183ec88335a08b963f26c34b328558)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:06 +00:00
Peter Kjellerstedt b689aa0df3 useradd-staticids.bbclass: Make --no-user-group have effect
If --no-user-group is specified in USERADD_PARAM_${PN} for a user and
no --gid is specified, then we should not assume that the group name
for the user is the user name.

(From OE-Core rev: 0e5402c433739a416a76df532378533cb25365c7)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:06 +00:00
Peter Kjellerstedt c03ea8dddc useradd-staticids.bbclass: Treat mutually exclusive options as such
The useradd options --create-home/--no-create-home and
--user-group/--no-user-group are mutually exclusive and should be
treated as such.

(From OE-Core rev: 908eca7fb4af8a60026f53e2bb2cf1d5daf089ab)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:06 +00:00
Robert Yang 44d283a13d autotools.bbclass: use relative path to run configure script
The absolute path (/path/to/configure) caused VPATH in Makefile to be an
absolute path, and then it will be in elf files, use relative path to
run configure can fix the problem.

This will reduce a lot of buildpaths QA issues in a world buld.

[YOCTO #8894]

(From OE-Core rev: 2c6ad43af0a0c4db5d7fd342be8585bec66debfb)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:03 +00:00
Jianxun Zhang c0e9f2d2b6 kernel/kernel-arch: Explicitly mapping between i386/x86_64 and x86 for kernel ARCH
For a bare-bone kernel recipe which specifies 32 bit x86 target,
a 64 bit .config will be generated from do_configure task when
building 32-bit qemux86, once all of these conditions are true:

* arch of host is x86_64
* kernel source tree used in build has commit ffee0de41 which
  actually chooses i386 or x86_64 defconfig by asking host when
  ARCH is "x86" (arch/x86/Makefile)
* bare-bone kernel recipe inherits directly from kernel without
  other special treatments.

Build will fail because of the mismatched kernel architecture.

The patch sets ARCH i386 or x86_64 explicitly to configure
task to avoid this host contamination. Kernel artifact is also
changed so that it can map i386 and x64 back to arch/x86 when
needed.

(From OE-Core rev: 8d310b24927d0f348fb431895f0583733db2aad0)

Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 15:44:56 +00:00
Richard Purdie b567235c0c image/image-live: Add back IMAGE_TYPES_MASKED support
IMAGE_TYPES_MASKED support was accidentally removed. The original
idea behind it was to remove some of the hardcoding in the core
image code, so do that for image-live and ensure the dependency
and masked variables correctly reflect the needs of the class.
This means we can remove all the hardcoded special cases since
image-vm already has the needed markup.

(From OE-Core rev: 9a2d4a3b8d7bb1cf7f1fb7fe47d5c002d9941c89)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 11:59:42 +00:00
Otavio Salvador e914e2ac15 image.bbclass: Handle image base type dependency properly
When a base image type has an implicit dependency (from IMAGE_TYPEDEP)
this has to be taken into account. This is a regression introduced by
OE-Core:c2dab18 (image: Create separate tasks for rootfs construction).

The issue has been found when building meta-fsl-arm based images which
does not include the rootfs image type explicitly in IMAGE_FSTYPES but
instead is added, using IMAGE_TYPEDEP, for the 'sdcard.gz' image.

Reported-by: Fabio Berton <fabio.berton@ossystems.com.br>
(From OE-Core rev: 191c7be3a6cc52911f244323072433f6a1172bf1)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 11:59:42 +00:00
Paul Eggleton cf0aea78b0 classes/populate_sdk_ext: avoid unnecessary sstate being brought in
Create a separate task where we can just have the recursive dependencies
for the tasks corresponding to constructing the SDK content (i.e. from
the image contents). This avoids us recursing into dependencies from
buildtools and getting a bunch of nativesdk stuff, for example.

(This isn't an ideal way to have to implement it, but without
overcomplicating things on the BitBake side just for this use-case I
can't see a better way.)

(From OE-Core rev: eabeb26335b1a4eb1e68218160dbdbe8fdf36272)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 11:54:51 +00:00
Richard Purdie ea29bec6c7 insane/package: Fix cases where QA errors aren't fatal
When using package_qa_handle_error(), we need to check QA_SANE and
give a fatal error if issues were raised. The main insane checks do
this, the anonymous python in insane does not, nor does the code in
package.bbclass. This was likely missed when the function was
introduced to package.bbclass.

The impact is that errors can be shown but don't stop the build
(do_package succeeds), the initial build fails due to the errors
shown but subsequent builds succeed. This is clearly broken.

The solution is to check QA_SANE in places we use the handle_error
function.

[YOCTO #8907]

(From OE-Core rev: 7e36d13212763fe409747aba011c020c9aeba6d7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 11:54:51 +00:00
Paul Eggleton 2e620a4785 classes/populate_sdk_ext: check that extensible SDK prepared correctly
After the change to use --setscene-only when running bitbake to prepare
the SDK at the end of installation, add a check that the SDK got
prepared correctly by doing a dry-run and looking at the output for any
real tasks that we don't expect. In order to make this easier, the
preparation shell script was rewritten in python.

(From OE-Core rev: 2306683634435b990e63020fc5cf91753bbaf7b6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 11:54:51 +00:00
Paul Eggleton 4685c3378d classes/buildhistory: save auto.conf and bblayers.conf for extensible SDK
These form part of the configuration for the extensible SDK, we should
really be recording what goes into them.

(From OE-Core rev: bd1557bfd726e44c23ab6220867119c57ac6b596)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 11:54:51 +00:00
Paul Eggleton 39f6472605 classes/populate_sdk_ext: support auto.conf
If auto.conf exists in the user's configuration we need to also run it
through the same filter and write the result into the ext SDK, or we
risk missing configuration applied on an autobuilder.

Fixes [YOCTO #8904].

(From OE-Core rev: fb0a34a00f3eb10935a3fca4d0af74ae4de4f9a5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 11:54:51 +00:00
Paul Eggleton 91877aa60e classes/populate_sdk_ext.bbclass: handle if local.conf doesn't end with a newline
If there is no newline at the end of local.conf, appending
INHERIT = "uninative" won't work, it will corrupt the line and the
installed eSDK will build things, making the "Preparing build system..."
step take an age.

Fixes [YOCTO #8897].

(From OE-Core rev: 1e641f09e8cf0ba716c3d2aade0845abecc27827)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 11:54:51 +00:00
Mark Hatle ea05e05826 image.bbclass: Unconditional includes of populate_sdk_ext fails
populate_sdk_ext requires uninative support, which is only available on
glibc based SDKMACHINES.  For instance, when using mingw32 a dependency
error will occur:

NOTE: Runtime target 'nativesdk-glibc' is unbuildable, removing...
ERROR: Required build target 'core-image-minimal' has no buildable providers.
Missing or unbuildable dependency chain was: ['core-image-minimal', 'uninative-tarball', 'nativesdk-glibc']

This is dues to populate_sdk_ext.bbclass having:
do_populate_sdk_ext[depends] += "buildtools-tarball:do_populate_sdk uninative-tarball:do_populate_sdk"
addtask populate_sdk_ext

Since bitbake can't determine for dependency resolution if the task is going
to be run yet, it blows up and says it simply can't be resolved.

Workaround this problem by making the inherit conditional on the SDK_OS
containing 'linux'.

(From OE-Core rev: e471ce3464d5ae024315d4839cccd4c651f9ba83)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 11:54:48 +00:00
Matt Madison e265fbba3c package_deb.bbclass: add 'Multi-Arch: foreign' tag to allarch packages
This tells APT that it can use such packages to resolve dependencies
from packages of any architecture in a multilib build.

(From OE-Core rev: 7158c79a70e3d820c9701dacfa7206d13f95845a)

Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:31 +00:00
Matt Madison 9ea7428e67 package_deb.bbclass, cross-canadian.bbclass: DPKG_ARCH mapping function
Have DPKG_ARCH set by directly invoking a mapping function, rather
than using an anonymous Python function modify the variable under
the hood, so we can have proper handling of overrides.

Also bring in some additional mappings to Debian architecture names
that weren't being handled.

(From OE-Core rev: 8d042ea4e755cb0bb28b88333e10e04ec4e86a36)

Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:31 +00:00
Randy Witt 74c3667b2a populate_sdk_ext: Pass excluded_targets as a list to prune_lockedsigs
prune_lockedsigs expects excluded_targets to be a list, whereas
previously it was passed in as a string.

(From OE-Core rev: db3cd1e08b08b99342d269882f31ec7e1daba2c6)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie e306d5495a populate_sdk_ext: Change to include siginfo and non sstate task sigs
Right now, the locked task hashes list for the extensible SDK locks
down only the sstate tasks.

Whilst asthetically pleasing, this gives two problems:

* Half the task are left floating meaning checksum mismatches
  are a pain to debug
* The later code which copies relavent data files out the sstate
  cache can't use any of this data.

This patch modifies things so all the checksums are listed in the locked
file. An exclusion of tasks probably makes more sense for the library
function rather than an allowed list.

The only sstate task being deliberaly excluded here was do_package
so add in a function to explictly exclude those sstate object files.

The net result of this that siginfo files for all tasks are included in
the SDK, which means commands like "bitbake -S printdiff" now function.

(From OE-Core rev: 6b70479e47b8a8743d8b410d6bc08da1607a318e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie e1a558a434 populate_sdk: Switch from bzip2 to xz
xz gives *much* faster decompression times for the SDK which in itself
is a good reason to use it. It also gives better compression.

One downside is its slower but we care about the end user case
first, build performance secondary.

It also assumes the SDK user has a tar capable of understanding a xz
compressed file but that should be common enough now.

(From OE-Core rev: 193086137b899a0a4e774ffc337ed0da9947fd4f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie 3341f3fbee classes: Fix do_rootfs references
After the separation of do_rootfs, some rootfs references need changing
to image_complete.

(From OE-Core rev: 59a5f596ca29b1eb8283706e3c60fbb39f9c2c23)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie 0a4e1f968a image: Create separate tasks for rootfs construction
This patch splits the code in lib/oe/image into separate tasks, one per
image type. This removes the need for the simple task graph code and defers
to the bitbake task management code to handle this instead.

This is a good step forward in splitting up the monolithic code and starting
to make it more accessible to people.

It should also make it easier for people to hook in other tasks and processes
into the rootfs code.

Incidentally, the reason this code was all combined originally was due to
limitations of fakeroot where if you exited the session, you lost permissions
data. With pseudo this constraint was removed.

We did start to rework the rootfs/image code previously and got so far with
untangling it however we did prioritise some performance tweaks over splitting
into separate tasks and in hindsight, this was a mistake and should have been done
the other way around. That work was suspended due to changes in the people working
on the project but this split has always been intended, now is the time to finish
it IMO.

There were some side effects of doing this:

* The symlink for the manifest moves to the rootfs-postcommands class and into
  the manifest function.
* There is no seperate "symlink removal" and "symlink creation", they are merged
* The date/time stamps of the manifest and the built images can now be different since
  the tasks can be run separately and the datetime stamp will then be different
  between do_rootfs and the do_image_* tasks.

(From OE-Core rev: c2dab181c1cdabac3be6197f4b9ea4235cbbc140)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie fdced52387 image: Move pre/post process commands to bbclass
As the next step in splitting up do_image, move the pre and post processing
commands to separate tasks. This also creates the do_image_complete task
which acts as the end marker task for image generation.

(From OE-Core rev: 800528eaa421d451b596545125cb218e08989151)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie cdc0aeed9b image.bbclass: Separate out image generation into a new task, do_image
I've heard complaints from people trying to create more interesting image
types about how hard it is to understand the rootfs/image generation code
and that its a pain to develop/test/debug.

Having looked at it myself, the internal construction of shell functions which
then gets passed into a multiprocessing pool is rather convoluted and it places
rather odd constraints on when variables are expanded. Its therefore no wonder
people find it confusing/complex.

This patch starts the process of splitting this up by separating out image
generation from the do_rootfs task into a new do_image task.

(From OE-Core rev: 57578d0ca6c3aaf6edf0af2c4862d43c97415156)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Paul Eggleton 9348c91ce1 classes/populate_sdk_ext: disable signature warnings
The user of the extensible SDK doesn't need to see these.

(From OE-Core rev: 7045fabf73d4eef9c023edb9e0a8b8d1d3f04680)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:45 +00:00
Paul Eggleton d44dcd77ab classes/populate_sdk_ext: fix cascading from preparation failure
During extensible SDK installtion, if the build system preparation step
fails we try to put something at the end of the environment setup script
to show an error when it is sourced, in case the user doesn't realise
that the partially-installed SDK is broken. However, an apostrophe in
the message (actually a single quote) appears to terminate the string
and therefore breaks the command. Drop it to avoid that.

(From OE-Core rev: 21e591d182e24c399ae010a8eff9b89947061a46)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:45 +00:00
Richard Purdie 2c5e7e04f1 image: Really remove lockfiles flag
This was supposed to be removed by a previous patch but was readded.
Really remove it.

(From OE-Core rev: 5661d8cb7849df62358368743134c0aaf523965e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:42:12 +00:00
Mariano Lopez b1515064b5 image_types.bbclass: Rebuild when WICVARS change
The procces to do a wic image is to save a file with
variables required by wic and then call wic using this
file. Because this is external to bitbake if the vars
change, the image won't be rebuild; an example of such
is IMAGE_BOOT_FILES.

This patch adds these variables to vardeps of do_rootfs
when a wic image is build. This will rebuild the image
if a variable needed by wic changes.

[YOCTO #8693]

(From OE-Core rev: 12c54d50ed4c321dc272beb3c6cb770965c979f1)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:20 +00:00
Ross Burton 6d111c8290 testimage: remove VNC test, x11vnc isn't in oe-core anymore
Now that x11vnc won't be in Sato images we don't want to test it.

(From OE-Core rev: 90ff7219332d9651fcdfc895d61fb73a46ffde78)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:20 +00:00
Paul Eggleton ae1fc966f7 classes/buildhistory: actually use KiB in extensible SDK sizes files
I was using bytes here for the sizes and writing out KiB as the suffix.
Change it to actually write out size values in KiB.

(From OE-Core rev: e6b1840ee9aeb933f86fd2a92a90b94c1bc9db7c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:20 +00:00
Martin Jansa e2736f7db6 sanity.bbclass: add more information to error message about TUNE_PKGARCH missing in PACKAGE_ARCHS
* sometimes it's hard to see what's wrong here
* error message:
  Error, the PACKAGE_ARCHS variable does not contain TUNE_PKGARCH (cortexa7t2hfhf-vfp-vfpv4-neon).
  doesn't help much to understand what exactly went wrong and where

(From OE-Core rev: 4bedd149964b3b594d7b1a233a9cc436925288e9)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:18 +00:00
Alexander Kanavin 8b11ed8a75 qt4: remove recipes and classes
Qt4 is no longer supportd upstream, but it is still needed for LSB compliancy.
Qt4 recipes have been moved to a separate meta-qt4 layer to be consistent with
meta-qt3 and meta-qt5.

(From OE-Core rev: cb89d2b25b4edb1241bc5426a69a6bc44df9be2c)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:14 +00:00
Richard Purdie 401c632508 image: Condense do_rootfs function/flags
Various prefuncs and flags and addtask statements make sense to belong together
in one clearer function now, this patch cleans things up a bit.

(From OE-Core rev: eae0cf7875197f9520be54370bc670e27338aad3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:33 +00:00
Richard Purdie 0051510ae3 image/rootfs-postcommands: Separate out post rootfs commands to separate class
Reading image.bbclass is a little difficult as it has many post rootfs
helper functions and its hard to separate those from the core contents
of the rootfs/image code.

Moving it to a separate class would be one way of making it clearer
what these functions are. There are some comment layout improvements
but no code changes.

(From OE-Core rev: df4cb51c8e60fa46d4d15be8da3d84287ff08ae7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:33 +00:00
Richard Purdie 3428eddcd1 image: Remove pointless rootfs lock
It was added in aa3141e979 back in 2008
but I don't see why multiple images would need this now.

(From OE-Core rev: 2eaeac6d98c310cb2602ba194d934c5b7bed253d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:33 +00:00
Ming Liu 0f516f0efe image-vm.bbclass: uses IMAGE_LINK_NAME
Uses ${IMAGE_LINK_NAME} instead of ${IMAGE_BASENAME}-${MACHINE}.

(From OE-Core rev: 755627cdbe32ef1b46692669f8aba981d626a0e7)

Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:19 +00:00
Ming Liu c851096d97 image-live.bbclass: uses IMAGE_LINK_NAME
Uses ${IMAGE_LINK_NAME} instead of ${IMAGE_BASENAME}-${MACHINE}.

(From OE-Core rev: 5f9feffe4cc4c1c084ab266382460e0b8f37e023)

Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:19 +00:00
Alexander Kanavin dd5756b353 mirrors: add archive.apache.org to Apache mirrors
Historical releases may disappear from the main mirrors, so
then we need to fall back to the archive (which, in turn, may
not carry the current release, and so can't be the primary mirror).

(From OE-Core rev: 2eb171a1c9f1d2ce37965aad7b3f470a2c44ed83)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:18 +00:00
Paul Eggleton 35bd2544ec buildhistory: improve support for extensible SDK
* Ensure extensible SDK and standard SDK go into their own directories
* Record extra extensible SDK variables
* Write sstate-package-sizes.txt and sstate-task-sizes.txt files so you
  can analyse the size of the contents
* Add BUILDHISTORY_SDK_FILES (similar to BUILDHISTORY_IMAGE_FILES) and
  default it to pick up config files installed in the extensible SDK

(From OE-Core rev: 5f87ff116d9ccd5e95e638dfa84583169a3f99f3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:17 +00:00
Paul Eggleton ea0abcdd17 buildhistory: fix not recording SDK information
After OE-Core revision baa4e43a29e45df17eaa3456acc179b08d571db6 we lost
recording SDK the contents in buildhistory. This was due to the
SDK_POSTPROCESS_COMMAND variable being set with = in
populate_sdk_base.bbclass which overwrote any value set with += in
buildhistory.bbclass; to fix it, use _append in buildhistory.bbclass
instead.

Fixes [YOCTO #8839].

(From OE-Core rev: 11d1aa82ef4a00051e0a50a87a1efed1c50c73b5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:17 +00:00
Jianxun Zhang 4487e3a420 kernel-yocto: fix checkout bare-cloned kernel repositories
The existing code doesn't tell regular (with .git) and bare cases and
just move the unpacked repo to the place of kernel source. But later
steps will fail on a bare-cloned repo because we can not checkout
directly in a bare cloned repo.

This change performs another clone to fix the issue.

Note: This change doesn't cover the case that S and WORKDIR are same
and the repo is bare cloned.

(From OE-Core rev: ccfa2ee5c4f509de4c18a7054b2a66fc874d5d69)

Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:16 +00:00
Paul Eggleton 7540550048 classes/populate_sdk_ext: error out of install if buildtools install fails
If the installation of buildtools fails then we should fail the entire
installation instead of blindly continuing on.

(From OE-Core rev: 34bb63e6c72fb862e0ef0d2b26e1bfddaf7ddb99)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton ecce3d3774 classes/populate_sdk_ext: hide build configuration in devtool build* output
The configuration of the build system within the extensible SDK is
fixed, so there's not a lot of point in showing it; plus it just gets in
the way of the output that's interesting to the user in this context. So
let's hide it within the extensible SDK.

(From OE-Core rev: 6223b73c2dfc326382dbd3c0ae2d7a0cf2cc3b48)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton fd84d0fe35 classes/base: don't print header if BUILDCFG_HEADER not set
If we don't want a header printed at the start of task execution (which
we'd prefer not to within the extensible SDK)  we can accomplish that by
clearing BUILDCFG_VARS and BUILDCFG_HEADER, but that was still printing
a load of blank lines. To keep things simple, check if BUILDCFG_HEADER
is set to something before printing a header.

(From OE-Core rev: d896d77c1d0fc56ff7019bfee3cf06fbc3ede8f3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton a4f496ae43 classes/populate_sdk_ext: use uninative to set NATIVELSBSTRING
We inherit uninative in the extensible SDK configuration, and uninative
sets NATIVELSBSTRING to a fixed value, so we don't need to force the
value ourselves.

Fixes [YOCTO #8662].

(From OE-Core rev: 918814c05d670bccb05d61fa848fd0d93da3a7b0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Elliot Smith a6f8a3f12e toaster.bbclass: fix TypeError when parsing build stats
Reading IO stats fails because the IO read/write bytes are
being converted to strings, then added to a numeric running
total.

Fix this by converting IO stats to integers.

(From OE-Core rev: 8e2475eecafc0161d25684f5b8239273739de759)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22 16:15:36 +00:00
Jackie Huang 606c9e7f78 staging.bbclass: make already-stripped can be skipped
Add a check like what we does in package.bbclass
so that the already-stripped QA test can be skipped.

(From OE-Core rev: 2262fdb256954b22dadb2f7c6922e6046c269742)

Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22 16:08:50 +00:00
Richard Purdie 9f3681dca5 buildstats-summary/toaster: Cope with removal of get_bn()
The buildstats changes removed the no longer needed get_bn() function,
replace this with references to BUILDNAME.

(From OE-Core rev: e1a37899da56014693f08d1c39cb6ec0a4ed2bf4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 13:50:57 +00:00
Richard Purdie d7e766b70a toaster: Update for buildstats changes
This updates the toaster class to use the new data format from buildstats.
This does mean it will no longer read IO data from older builds, however
since that data is completely useless anyway, I don't consider that to be
an issue.

(From OE-Core rev: 5c3987226cd9669d5fc0993d0541048475af9fcb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:19 +00:00
Richard Purdie adfdca4df1 buildstats: Improve to add getrusage data and corrected IO stats
Add IO stats and getrusage() data to the task statistics. We
also drop the CPU percentage calculation since its pretty arbitrary
and not very accurate/useful.

In particular we can now see the user and sys times as well as the
wall clock times.

(From OE-Core rev: b849130f71d3ba32a6fa94c291ca6ce7c7c3b3d1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:19 +00:00
Richard Purdie 3187647d89 buildstats: Separate out the build and task data to allow improvements
The combined build and task data code makes changing things hard, separate
out the functions so that changes can be made to the task data whilst the
build data remains unchanged.

(From OE-Core rev: c79cfce4f820f20346d0565df8df626832976e28)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:19 +00:00
Richard Purdie 38a2553864 buildstats: Clean up e.data and bb.data references
Rather than bb.data and e.data, cleanup to use 'd' and match the
standard coding style.

(From OE-Core rev: dbce2c6122d492ba86873db4d02322f0df7a6752)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:18 +00:00
Richard Purdie 7b1e48f9c7 buildstats: Drop get_bn/set_pn and just use BUILDNAME
The current setting and getting of the "name" to use for buildstats is
convoluted and not particularly interesting. We only need this for the
e.getPkgs()[0] component of the path which is the first target listed
on the commandline. This is pretty arbitrary.

If we drop that piece, we can assume BUILDNAME is common for all events
and simply use this and query it. If BUILDNAME did change, that would
be a bug and it should be fixed elsewhere.

Also take the opportunity to share some common code since the function
now has the eventmask.

(From OE-Core rev: 918d83460639df273f38ae079ffeebd6a79b3373)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:18 +00:00
Richard Purdie 78371629b8 buildstats: Drop disk data from buildstats
The existing diskstats data from buildstats simply isn't useful. It
gives stats on the total IO counts on some random disk within the
system. This means that the count includes data from all other tasks
running at the same time and from any other process running on the
system.

I've been unable to find any use for the data so as a start at
reworking the class, remove the related code.

(From OE-Core rev: f9fb02909f332365cad329352956a29cff6eba77)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:18 +00:00
Randy Witt a5dd1dd890 uninative.bbclass: Choose the correct loader based on BUILD_ARCH
Previously UNINATIVE_LOADER was always ld-linux-x86-64.so.2. That is
incorrect when the host is 32-bit.

This change also changes to using ?= so the user can override
UNINATIVE_LOADER if so desired.

[YOCTO #8124]

(From OE-Core rev: b78fa0bcadd54bb29b6f1bb3a9308d4c454bf4e2)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-16 12:12:15 +00:00
Richard Purdie 388e5802df license: Fix BB_TASKDEPDATA references
We shouldn't try and expand what is a python dict object
and we don't want it to influence the task hashes, task
dependencies are already accounted for.

(From OE-Core rev: 697072130a3298aede7166f125aa982be23793ca)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-16 11:56:30 +00:00
Richard Purdie e0890b662e meta: Drop now pointless manual -dbg packaging
With the autodebug package generation logic, specifically setting FILES_${PN}-dbg
isn't needed in most cases, we can remove them.

(From OE-Core rev: 3ab59d49dd7c18e194b58d1248b4b87709b5a738)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-16 11:56:30 +00:00
Richard Purdie b7766e4bbe package: Add auto package splitting of .debug files
Creating FILES_${PN}-dbg is tedious and also pretty pointless. We might as well
assume ".debug" is a special directory name and split into -dbg automatically.

This change does so without changing the rest of the splitting logic too much.
It can be disabled for the cases where we really do want manual control of
the -dbg packages (e.g. qt4) with NOAUTOPACKAGEDEBUG = "1".

(From OE-Core rev: da5ec06814e105451cca11cce76b5c5231110524)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-16 11:56:30 +00:00
Richard Purdie 0da1d71809 nopackages: Add class for recipes which don't generate packages
It turns out writing the same list of packaging tasks multiple
times in multiple places is error prone. Move this to a new class
'nopackages", migrate existing users and add glibc-initial and
libgcc-initial since we don't want packages for those recipes.

This means the sstate for those recipes won't be installed, saving
small amounts of build time and bandwidth.

A reference to the old package_write task is also dropped.

(From OE-Core rev: cece583d58f82a50c3a4eac876eb326ac3b8f5e5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 22:30:57 +00:00
Richard Purdie 5003d14761 sstate: Ensure populate_lic dependencies are not followed
It was observed that do_fetch dependencies (e.g. subversion-native of tremor)
were being installed even when sstate was available and hence no fetch was
needed. This turned out to be due to the recursive nature of the rootfs
dependencies which include populate_lic.

We can explicitly whitelist these dependencies as being ones we don't
need to follow when installing sstate (the code defaults to being paranoid).

[YOCTO #8746]

(From OE-Core rev: 06c891f0fa6c7712eae233dd18612f1ab1889c45)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 22:30:57 +00:00
Richard Purdie 48aad516c2 populate_sdk_ext/sign_rpm/sign_package_feed: Add missing getVar parameter
We should always pass a parameter to getVar, add missing default value.

(From OE-Core rev: 31bc0a46a97d7dc98568a218c077c31d8b11dbd9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 22:30:56 +00:00
Alexander Kanavin 4ff00174aa mirrors: replace references to archive.apache.org
archive.apache.org does not contain current releases, only historical ones,
so upstream checks aren't accurate. It's replaced with official mirrors
containing current versions.

(From OE-Core rev: e9c85489ae354f52ff5b78f4d7fb6fafd0050522)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-12 23:42:54 +00:00
Andre McCurdy 95909bc788 kernel.bbclass: drop unnecessary 'eval' from kernel_do_configure()
KERNEL_CONFIG_COMMAND will be expanded by bitbake, so can be
used directly in kernel_do_configure()

(From OE-Core rev: f8a35d8e312df908c73724465035921856d30781)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-12 23:42:52 +00:00
Juro Bystricky ec79a19f24 insane: in libdir test allow libraries in libexecdir
Some recipes, such as sudo and gcc, put libraries into libexecdir.  Allow this
in the sanity test so that we don't have to whitelist the libdir check.

(From OE-Core rev: 9f5ad12a9532687a7a82e24b7c4506cabc13bba2)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-12 23:42:52 +00:00
Vladimir Zapolskiy 4515186bc4 package_ipk: allow to specify OPKG_ARGS in local.conf
If user specific parameters to opkg are set in local.conf, they are
rewritten in package_ipk.bbclass and ignored, instead append
package_ipk specific arguments to the user defined ones.

The change is needed, if a user has to pass an alternative path to a
temporary directory for opkg, e.g.

  OPKG_ARGS = "--tmp-dir=${TOPDIR}/tmp-opkg"

The default /tmp directory may be unusable for do_rootfs task, for
example if there is no enough space or /tmp is mounted with noexec
mount option, then an alternative path allows to complete do_rootfs
and fix the problems like this:

  ERROR: Unable to install packages.
  ...
  sh: /tmp/opkg-5jPLag/run-postinsts-UsUtaI/preinst: /bin/sh: bad interpreter: Permission denied
  sh: /tmp/opkg-5jPLag/base-files-4hFwQS/preinst: /bin/sh: bad interpreter: Permission denied
  sh: /tmp/opkg-5jPLag/run-postinsts-UsUtaI/preinst: /bin/sh: bad interpreter: Permission denied
  sh: /tmp/opkg-5jPLag/busybox-syslog-sJmfbw/preinst: /bin/sh: bad interpreter: Permission denied
  ...

(From OE-Core rev: 0be4c30aaf03de58d8037a89e342ca0b379a0881)

Signed-off-by: Vladimir Zapolskiy <vz@mleia.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:24 +00:00
Bob Ham 7cf7156b2c systemd.bbclass: Allow enabling of parameterised services
Currently the systemd.class will check whether a service exists when it is
requested to enabled it.  However, its check does not take into account that a
service like 'foo@eth0.service' can be enabled from a service named
'foo@.service'.  This patch alters the check function in systemd.class to look
for 'foo@.service' if the normal check fails.

(From OE-Core rev: 4ce15271dad3520e5de2500c609b05d5a511e453)

Signed-off-by: Bob Ham <bob.ham@collabora.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:24 +00:00
Ross Burton 551cda0bd5 base: check for existing prefix when expanding names in PACKAGECONFIG
When the DEPENDS are added as part of the PACKAGECONFIG logic the list of
packages are expanded so that any required nativesdk-/-native/multilib prefixes
and suffixes are added.

However the special handling of virtual/foo names doesn't check that the prefix
already exists, which breaks under nativesdk as in that situation there's an
explicit nativesdk- prefix *and* MLPREFIX is set to nativesdk-.  This results in
the same prefix being applied twice, and virtual packages such as virtual/libx11
ending up as virtual/nativesdk-nativesdk-libx11.

(From OE-Core rev: 55ca1fb8f0e81ff739b3c46897e43356d1f760c3)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:24 +00:00
Alexander Kanavin 642e92f2cd package_regex.inc: split entries with odd-even versioning into their own recipes
(From OE-Core rev: 5ddaba1fb833d0408cef5f58f786513b9293c30a)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-08 10:20:51 +00:00
Fang Jia 9d788d7330 toolchain-scripts.bbclass: unset command_not_found_handle
On Ubuntu-system, When sourcing the env.sh from an exported sdk, and
running a bogus linux command (for example "asd"), a core dump of
python is usually generated.

Unset the command_not_found_handle to fix it.

(From OE-Core rev: 473ccbebb426df757adb8955eaa5e191d88180d1)

Signed-off-by: Fang Jia <fang.jia@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-08 10:20:47 +00:00
Andre McCurdy 82ab99f7de waf.bbclass: remove unused parameter from get_waf_parallel_make()
(From OE-Core rev: f7062f54c2ad601d9838d13bfe7933be92a63214)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-08 10:20:47 +00:00
Richard Purdie 0c5d2398fc base: Improve handling of switching virtual/x providers
If you build virtual/kernel, then change PREFERRED_PROVIDER_virtual/kernel from say
"linux-yocto" to "linux-yocto-dev", you see errors from the sysroot about overlapping
files. The automatic uninstall logic doesn't trigger since the other recipes is
still technically parsed/buildable.

What we can do is look at the value of PREFERRED_PROVIDER_virtual/X and raise SkipRecipe
(skip parsing) if it provides this thing and its not selected. We skip cases no preferred
provider is set, or the value is in MULTI_PROVIDER_WHITELIST.We also inform the user
if they try to build something which conflicts with the configuration:

$ bitbake linux-yocto-tiny
ERROR: Nothing PROVIDES 'linux-yocto-tiny'
ERROR: linux-yocto-tiny was skipped: PREFERRED_PROVIDER_virtual/kernel set to linux-yocto, not linux-yocto-tiny

[YOCTO #4102]

(From OE-Core rev: 9a3b992b7339f7aa892f1dff7d159747630b045d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-08 10:20:47 +00:00
Alejandro Hernandez 2268a702f1 python3: Upgrade from 3.4.3 to 3.5
python3-native_3.4.3.bb -> python3-native_3.5.0.bb:
- changed version
- changed cheksum
- no license change, just dates

python3_3.4.3.bb -> python3_3.5.0.bb:
- changed version
- changed cheksum
- no license change, just dates

New:
- use_packed_importlib.patch: Fixes importlib on cross-compile environments

Rebased:
- Manifest
- 000-cross-compile.patch
- 020-dont-compile-python-files.patch
- 04-default-is-optimized.patch
- python-3.3-multilib.patch
- distutils3-base.bbclass
- distutils3-native-base.bbclass
- python3native.bbclass

Upstream:
- makerace.patch

Misc:
- pip2 is handled as default on major distros,
modified python3-pip to leave /usr/bin/pip available for pip2
- Fixed importing pip3 from python3 interpreter

(From OE-Core rev: 701ec1977ced1bb08461e6de98b4f63d21cba8a6)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:06 +00:00
Ross Burton b8ee7ae6b4 sanity: don't enforce DISPLAY for testimage
Now that qemurunner doesn't need DISPLAY set, let whether DISPLAY is set be up
to the user.

(From OE-Core rev: 259d1d1889b9349498d3f995a09466ce89bd3fcf)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:01 +00:00
Ross Burton 46755cc4b8 base: add automatic dependency on lzip-native for .lz SRC_URI
Now that the SRC_URI is parsed this can be done without false-positives.

(From OE-Core rev: 6fac3caf3d70823c830178f2299ceee275a1edd1)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:01 +00:00