Commit Graph

330 Commits

Author SHA1 Message Date
Rodrigo Chiossi da35877db8 lib/oe/terminal: add support for Terminology terminal emulator
Terminology is the default Enlightenment terminal emulator

(From OE-Core rev: f3da8570a977809fadb08315f384726b80f5ee01)

Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-03 14:53:54 +00:00
Chen Qi 4385b3cc9d package_manager.py: fix rootfs failure with multilib enabled
With the current code, if we use debian package backend and enable
multilib support, the do_rootfs process would always fail with error
messages like below.

    E: Unable to locate package packagegroup-core-boot

This patch fixes the above problem.

(From OE-Core rev: d140d556ae30b6dbd0ffce8882c3e22b17050820)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-03 14:53:54 +00:00
Alejandro Hernandez 57647c9cbb terminal.py: fixes launching multiple windows of gnome-terminal
When resolving a patch, a new process of gnome-terminal
is created for every patch to be resolved, it "waits"
for the previous one to end, instead of launching
multiple windows at the same time.

[YOCTO #7254]

(From OE-Core rev: b457e0b028418260aee25858b8af8938dd038fac)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-03 14:53:53 +00:00
Dan McGregor 491d1db8d1 image.bbclass: make kernel depmod data optional
This allows an image to skip the creation of kernel depmod
data. It is useful for creating an image that will run as a
container image inside a host with no knowledge of the parent's
kernel.

(From OE-Core rev: ca641aedff5f6bd155796ead02cb2eb871f8c17a)

Signed-off-by: Dan McGregor <dan.mcgregor@usask.ca>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-03 14:53:53 +00:00
Paul Eggleton ca232b5c7b lib/oe/image.py: add error checking for missing IMAGE_CMD
An invalid value in IMAGE_FSTYPES was triggering a traceback. Add a
check and a reasonable error message instead.

(From OE-Core rev: f38c35b925d6eab0d5a41b8592b0eb8b0a3b7b72)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-03 14:53:51 +00:00
Paul Eggleton 26e5b0f7b9 classes/image: ensure uninstalled packages do not appear in manifests
Since the rewrite of the image construction code in python a few
releases ago, we remove a couple of packages from the image as one of
the final steps when constructing the image (notably update-rc.d and
run-postinsts).  However, because of the order of operations, these
packages are still listed both in the buildhistory
installed_package*.txt files and in the manifest file created next to
the image, which is wrong.

There were two possible solutions to this: (1) change the order such
that the uninstallation occurs before calling ROOTFS_POSTPROCESS_COMMAND
or (2) add another hook variable in such that we can have the
package list collection code run at the right time. Because it's
currently possible (but very much not recommended) to install additional
packages within ROOTFS_POSTPROCESS_COMMAND, which may have postinstall
scripts and thus require the packages we would otherwise uninstall if we
were to take option 1, option 2 is really the least likely to cause
problems. Therefore, add ROOTFS_POSTUNINSTALL_COMMAND and make the image
and buildhistory classes use it.

Fixes [YOCTO #6479].

(From OE-Core rev: b198a189228648057c3be7d068598f50841b3bf9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-03 14:53:50 +00:00
Paul Eggleton 124f0419b9 lib/oe/rootfs.py: rename _uninstall_uneeded to fix spelling error
Rename it to _uninstall_unneeded.

(From OE-Core rev: cb217f442b8143d88112bebe4011051efbd905a3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-03 14:53:50 +00:00
Richard Purdie 3f15d8a807 kernel: Fix depmod for multilib
Using populate_sysroot for this data was a nice idea but flawed as it
doesn't work in multilib builds. Instead we can use PKGDATA_DIR since
this is consistent over multilib builds. It also turns out to be
slightly neater code too. Hopefully this resolves the problem once and
for all.

(From OE-Core rev: 6cccae3ca54c1177a1d91d23c3e151d74e735ee9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-29 15:36:52 +00:00
Richard Purdie 11ef55a93b kernel/image/depmodwrapper: Fixups for depmod
With the rpm package backend enabled, running:

bitbake <image>
bitbake virtual/kernel -c clean
bitbake <image> -c rootfs -f

results in an image with incorrect kernel module dependency information.
The problem is that the System.map and kernel-abiversion files are needed
for depmod and after the recent kernel changes, these are no longer in
sstate.

Its reasonable to require the kernel to unpack/build if you're
about to build a module against it. It is not reasonable to require this
just to build a rootfs.

Therefore stash the needed files specifically for depmod.

Also fix some STAGING_KERNEL_DIR references which were incorrect, found
whilst sorting through his change.

(From OE-Core rev: b851504dcf5e147c9efb1c7b6a4d22c1a1a87cd7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-23 14:32:02 +00:00
Richard Purdie 8bab098c36 lib/oe/package: Ensure strip breaks hardlinks
Normally, strip preserves hardlinks which in the case of the way our hardlink
rather than copy functionality works, is a disadvantage and leads to non-deterministic
builds. This adds a move into place after the strip operation to ensure hardlinks
are broken and we bring back build determinism.

(From OE-Core rev: 7c0fd561bad0250a00cef63e3d787573112a59cf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-23 11:36:31 +00:00
Pau Espin Pedrol 6f196670e8 lib/oe/sdk.py: Add SDKIMAGE_INSTALL_COMPLEMENTARY capabilities to DpkgSdk
Creating an SDK by means of do_populate_sdk, complementary packages
(SDKIMAGE_FEATURES = "dev-pkgs dbg-pkgs") are not installed when using
the deb packaging system.

The reason is that the call to install the complementary packages is missing
from the deb backend. This patch fixes that.

[YOCTO #7160]

(From OE-Core rev: 0bcca69ea97ac51acf290f8f1da1bde715ab51c4)

(From OE-Core rev: 6755935e58b0414870adb7c7abab1aa331596209)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-23 11:36:25 +00:00
Richard Purdie 1dfb9033e0 lib/oe/rootfs.py: Fix reference to abiversion file location
With the changes to kernel layout, we need to refer to the correct
location of the kernel abiversion file.

Thanks to Saul, Randy, Darren and Bruce to figuring out the issue.

(From OE-Core rev: 6541799c2e9a5a1586676c207d62f885c70e24dc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-16 23:16:15 +00:00
Otavio Salvador 01c7d083ef lib/oe/image.py: Handle compressed IMAGE_TYPEDEP values
When computing the dependency graph for the image generation, we need
to take into account the compression type and identify the base type
it relates to. This allow for a more robust graph generation even when
using composed image types.

(From OE-Core rev: 6e7d1de6cc99ed2def346dc40310573f5f0ce5ca)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-25 08:18:12 +00:00
Paul Eggleton db73bcc3b8 lib/oe: add recipeutils module
Add a module to help provide utility functions for dealing with recipes.
This would typically be used by external tools.

Substantial portions of this module were borrowed from the OE Layer
index code; other functions originally contributed by
Markus Lehtonen <markus.lehtonen@intel.com>.

(From OE-Core rev: fd970c3b42b39601f379300d83354d42d5a536ba)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-21 12:37:56 +00:00
Paul Eggleton 12bb32e766 lib/oe/patch: add support for extracting patches from git tree
When patches from a recipe have been written out to a git tree, we also
want to be able to do the reverse so we can update the patches next to
the recipe. This is implemented by adding a comment to each commit
message (using git hooks) which we can extract later on.

(From OE-Core rev: 765b7bad50eae5b79d13a3f4988dc440c3d9787f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-21 12:37:56 +00:00
Paul Eggleton 50e771d114 lib/oe/patch.py: abort "git am" if it fails
If we don't do this, you may still be in the git am resolution mode at
the end of applying patches, which is not desirable.

(From OE-Core rev: 630a393d01950a0d00b5d30ac376472911e50ff9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-21 12:37:56 +00:00
Paul Eggleton 37db9e62f0 lib/oe/patch: use --keep-cr with git am
Preserving carriage returns is important where the patch contains them.

(From OE-Core rev: 1cfcae0cd0bc776f5bb91a75bb8ffdad3d7bf200)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-21 12:37:56 +00:00
Paul Eggleton 400933b050 lib/oe/patch: auto-commit when falling back from git am
When PATCHTOOL = "git", if we're not able to use "git am" to apply a
patch and fall back to "git apply" or "patch", it is desirable to
actually commit the changes, attempting to preserve (and interpret) the
patch header as part of the commit message if present. As a bonus, the
code for extracting the commit message is callable externally in case it
is useful elsewhere.

(From OE-Core rev: 8c522846093809a8deb866079e73fa317266c80e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-21 12:37:56 +00:00
Paul Eggleton e03c04758b lib/oe/patch: fall back to patch if git apply fails
When PATCHTOOL = "git", git apply doesn't support fuzzy application, so
if a patch requires that it's better to be able to apply it rather than
just failing.

(From OE-Core rev: a8143f33d3104adcd10968e3b05df2024e723f5a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-21 12:37:56 +00:00
Paul Eggleton de8730ce2d classes/package: move read_shlib_providers() to a common unit
This allows us to use this function elsewhere in the code.

(From OE-Core rev: 657cff8a0f0e5db171b2ed9388a790ee0b135842)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-21 12:37:56 +00:00
Ross Burton 615fb3b2f2 package_manager.py: fix arguments to string format
Multiple arguments to string formats need to be in a tuple.

Reported by Lorenz <lqb.list@gmail.com>.

(From OE-Core rev: e30a4650beabac215b6d867070b7acdb3601a4d7)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-05 18:01:06 +00:00
Hongxu Jia 5007319cd2 package_manager.py: check the result of create_index
While invoking create_index failed, there was no error output
and didn't break the build until the package installation.
...
|ERROR: run-postinsts not found in the base feeds (qemux86 i586 x86
noarch any all).
...

The reason is we used multiprocessing to execute create_index, and
did not check its invoking result.

(From OE-Core rev: d8921e4ea68647dfcf02ae046c9e09bf59f3e6e4)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-03 12:23:59 +00:00
Jackie Huang 2cdcd4bfdf classextend: Do not extend for that already have multilib prefix
If a BSP supports two or more multilibs, for example:

    MULTILIBS = "multilib:lib32 multilib:lib64"

and a variable is already extended to include multilib variants,
for example in populate_sdk_base:

    commit 396371588c7fd2d691ca9c39cd02287e43cb665b
    Author: Richard Purdie <richard.purdie@linuxfoundation.org>
    Date: Thu Jul 24 22:09:09 2014 +0100

    populate_sdk_base: Extend TOOLCHAIN_TARGET_TASK to include multilib variants

    Most people expect the toolchain from a multilib build to contain multilib
    components. This change makes that happen and is easy for users to override
    should they want something different.

    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

The mapping clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK")
ends up with a wrong double extended package name like:

    lib32-lib64-packagegroup-core-standalone-sdk-target

This patch avoid such issues.

(From OE-Core rev: c4e9b2aa894d59fe951038b3b73795b6891df70a)

Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-12 15:38:31 +00:00
Aníbal Limón 1d19791828 package_manager: DpkgPM fix populate_sdk
DpkgPM change all_arch_list variable set from PACKAGE_ARCHS to passed
archs variable because is different when is executed from rootfs.py
and sdk.py.

Credits to: Ricardo Ribalda <ricardo.ribalda@gmail.com>

(From OE-Core rev: f6fb8c16f49fd9a2b124ad55f5c4fed82d7e6dca)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-09 10:21:22 +00:00
Alejandro Hernandez ed3e25379c image-buildinfo.bbclass: new class, writes build information to image
Writes build information to target filesystem on /etc/build such as enabled
layers, their current status and commit.

squashspaces was moved to oe/utils.py to make it available to different classes
and avoid code duplication.

[YOCTO #6770]

(From OE-Core rev: c9cc652e88ddedddf8a2f23fb9b8c024616526d7)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-09 10:21:22 +00:00
Hongxu Jia f948dbf2e7 rootfs.py: tweak inner warn message catching
The fix filters out irrelevant messages, and makes the catching more
accurate, the inner warn message in do_rootfs usually comes from the
output of complementary install, and pattern the format to catch it.

Here is the example of irrelevant messages:
...
|WARNING: log_check: There is a warn message in the logfile
|WARNING: log_check: Matched keyword: [warn]
|WARNING: log_check: `tmp/deploy/rpm/core2_64/pam-plugin-warn
-1.1.6-r5.0.core2_64.rpm' -> `tmp/work/intel_x86_64-wrs-linux/
wrlinux-image-installer/1.0-r0/rootfs/Packages.intel/./core2_64/
pam-plugin-warn-1.1.6-r5.0.core2_64.rpm'
...

(From OE-Core rev: 4ceb3b5f928af7f631294c83b83e3a3c89cbf890)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-04 10:27:13 +00:00
Paul Barker 365947e9d1 package_manager: Fix BAD_RECOMMENDATIONS for opkg
In package_manager.py, when using opkg as the packager, the command 'opkg <args>
info <pkg>' is called to get information about each pkg in BAD_RECOMMENDATIONS
in a format that can be written to the status file. The 'Status: ...' line is
modified and all other lines are passed through. Changing the verbosity level
argument for this command will change what it written into the status file.
Crucially, with the default verbosity level, no blank lines are being printed by
the opkg command and so no blank lines are being written to the status file to
separate each package entry.

The package parsing code in opkg expects package entries in the status file to
be separated by at least one blank line. If no blank line is seen, the next
package entry is interpreted as a continuation of the last package entry, but
the new values overwrite the old values.

So with the default verbosity level, a blank line follows some package entries
and these are parsed. The others are dropped due to the lack of blank lines. As
the verbosity increases, more debugging messages add blank lines and more
packages are parsed.

The solution to ensure that this works correctly regardless of the verbosity
level is simply add a blank line after the output of 'opkg info' is written to
the status file, ensuring that the next package is separated from the current
package.

[YOCTO #6816]

(From OE-Core rev: 3fa24eee41c26fecd5e4f680082288ec772d2de9)

Signed-off-by: Paul Barker <paul@paulbarker.me.uk>
Cc: Chris Carr <chris.carr@ge.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-04 10:27:10 +00:00
Hongxu Jia 8619d93726 multilib.bbclass/package_manager.py: fix <multilib>-meta-toolchain build failure
There is a failure to build lib32-meta-toolchain:
...
|ERROR: lib32-packagegroup-core-standalone-sdk-target not found in the base
feeds (qemux86_64 x86 noarch any all).
...

In package_manager.py, the variable 'DEFAULTTUNE_virtclass-multilib-lib32'
is used to process multilib image/toolchain. But for the build of lib32-
meta-toolchain, the value of 'DEFAULTTUNE_virtclass-multilib-lib32' is
deleted. In 'bitbake lib32-meta-toolchain -e', we got:
...
|# $DEFAULTTUNE_virtclass-multilib-lib32 [2 operations]
|#   set? /home/jiahongxu/yocto/build-20141010-yocto/conf/local.conf:237
|#     "x86"
|#   del data_smart.py:406 [finalize]
|#     ""
|# pre-expansion value:
|#   "None"
...

The commit 899d45b90061eb3cf3e71029072eee42cd80930c in oe-core deleted
it at DataSmart.finalize
...
Author: Richard Purdie <richard.purdie@linuxfoundation.org>
Date:   Tue May 31 23:52:50 2011 +0100

    bitbake/data_smart: Change overrides behaviour to remove
       expanded variables from the datastore
...

We add an internal variable 'DEFAULTTUNE_ML_<multilib>', assign it with the
value of 'DEFAULTTUNE_virtclass-multilib-lib32' before deleting.

For rpm backend in package_manager.py, we use DEFAULTTUNE_virtclass-multilib
-lib32 first, if it is not available, and try to use DEFAULTTUNE_ML_<multilib>

[YOCTO #6842]

(From OE-Core rev: 9c59d3d8b538d3a98ff4b5e5b189a4a23a85da2d)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-04 10:27:09 +00:00
Pascal Bach c97194b0b1 image.py: Fix error in graph sorting
The graph sorting algorithm for image dependencies does a look for an
occurrence of a searched string instead of comparing the chunk to the
searched string. This leads to the problem that ubifs is recognized as ubi aswell.

This fixes this by splitting up the string into chunks.

(From OE-Core rev: cec9725c540c2d54c27092e40d159694cea75b5f)

Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-30 13:01:22 +00:00
Chong Lu d7cb96f28d python-smartpm: Add checking for "rpm-ignoresize" option
The do_rootfs takes a very long time when build host has mounted many NFS
devices. syscall lstat() was being called on every filesystem mounted on the
build host during building.
The reason for the lstat() is that rpm is verifying that enough free disk space
is available to do the install. However, since the install is into the target
rootfs it should not matter how much free space there is in the host mounts.
Add checking for "rpm-ignoresize", by it, smart can make whether RPM skip
checking for diskspace when install a rpm package.

(From OE-Core rev: fc0668a019eca422540ceab3efcd2b2a27dd79e0)

Signed-off-by: Chong Lu <Chong.Lu@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-24 17:36:14 +01:00
Hongxu Jia ea8a7c5d34 sdk.py: fix write target sdk manifest failed based on ipk
bitbake meta-toolchain
ls tmp/deploy/sdk/poky-glibc-x86_64-meta-toolchain-i586-toolchain-1.7.manifest -al
...
|-rw-rw-r-- 1 root root 0 Oct 10 15:05 tmp/deploy/sdk/poky-glibc-x86_64-
meta-toolchain-i586-toolchain-1.7.manifest
...

The manifest is empty, the reason is target's ipk config path is
d.getVar('IPKGCONF_TARGET') rather than d.getVar('IPKGCONF_Target')

(From OE-Core rev: 81b3cc448f040dcb4c2f2b05983231ac53270663)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-10 16:44:33 +01:00
Anibal Limon c34775f886 deb_packaging: Added support for multilib
The Package Manager implementation for deb didn't
take a look about multilib enabled variants.

Changes are made for generate apt.conf, sources.list and
debian repo index Release and Packages files.

[YOCTO #1502]

(From OE-Core rev: b5fb879b351cc23977f3e441f758101551297566)

Signed-off-by: Anibal Limon <anibal.ezau.limon.belmares@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-30 14:33:31 +01:00
Hongxu Jia 3b92eb93ee rootfs.py: catch inner warn message
Package managements (smart/apt-get/opkg-cl) generate some warn messages
to stdout, and we need to catch them and output by bb.warn.

Here is an example, while invoking smart to attempt install doc packages,
if install failed, it generates warn message to stdout.
...
|warning: Can't install util-linux-doc-2.24.2-r1@i586: Can't
install util-linux-doc-2.24.2-r1@i586: no package provides info
...

The fix catches it and outputs:
...
|WARNING: log_check: There is a warn message in the logfile
|WARNING: log_check: Matched keyword: [warn]
|WARNING: log_check: warning: Can't install util-linux-doc-2.24.2-r1@
i586: Can't install util-linux-doc-2.24.2-r1@i586: no package provides
info
...

(From OE-Core rev: f8d725f49f2be4b854f523a5ee3a5c4357e67e30)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-30 14:10:34 +01:00
Hongxu Jia 280b6d0011 sstatesig: fix overrides behaviour to remove SIGGEN_LOCKEDSIGS_i586
Require a sig file which SIGGEN_LOCKEDSIGS_i586 is not null, but
the actual SIGGEN_LOCKEDSIGS_i586 is null.
Invoking 'bitbake -e' and we got:
...
 3935 # $SIGGEN_LOCKEDSIGS_i586 [2 operations]
 3936 #   set /path/to/locked-sigs.inc:8576
 3938 #   del data_smart.py:406 [finalize]
 3939 #     ""
...

It was caused by the following commit:
...
(Bitbake rev: 899d45b90061eb3cf3e71029072eee42cd80930c)
Author: Richard Purdie <richard.purdie@linuxfoundation.org>
Date:   Tue May 31 23:52:50 2011 +0100

    bitbake/data_smart: Change overrides behaviour to remove expanded
      variables from the datastore
...

We add prefix 't-' to type to workaround the overrides behaviour.

(From OE-Core rev: f6a39cc957bf85ff43513f0b76afc3b2c9c906b6)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-17 22:00:26 +01:00
Randy Witt f9723a36d7 sstatesig.py: Replace '_'s with '-'s in SSTATE_LOCKEDSIGS* names.
Using underscores in the "types" parts of the variable names can cause
unexpected issues with overrides. For example, if you have both
SSTATE_LOCKEDSIGS_x86_64 and SSTATE_LOCKEDSIGS_x86_64_i586, and i586
is in OVERRIDES, then you lose all of the contents in
SSTATE_LOCKEDSIGS_x86_64 and thus don't get some of the locked sstate.

Using '-'s in the variable names instead, eliminates these issues.

(From OE-Core rev: 6662c412a949a9f6b602c848e6303b19db7e5272)

(From OE-Core rev: 65f558a6f762fb13224091dc22903b58eeb9b392)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-17 22:00:26 +01:00
Hongxu Jia abf0e7e53f sstatesig: Improve the support for locked down sstate cache usage
Add code in the sstate hash validation code to ensure it really did
install these from sstate since if it didn't should to warn/abort
the build. The judgment condition is:
1) If a build is replaced by locked sstate-cache, it will triger a
   warn/error;
2) If objects are not used from the locked cache, it will triger a
   warn/error;
3) Use SIGGEN_LOCKEDSIGS_CHECK_LEVEL variable controls whether this
   is just a warning or a fatal error or nothing to report.

[YOCTO #6639]

(From OE-Core rev: 305912dce61c4fed0cbf631aa98a9e6f29db88e4)

(From OE-Core rev: 1683815695f39d4bad352348913f927ac8a1bcf5)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-17 22:00:25 +01:00
Richard Purdie a08d7dc9b2 sstatesig: Improve to handle locking of multiple machines
Instead of a single monolithic SIGGEN_LOCKEDSIGS, split this into
separate variables, one per sstate package architecture. Add in
a new SIGGEN_LOCKEDSIGS_TYPES variable which lists the package
architectures to load in.

SIGGEN_LOCKEDSIGS_TYPES is made machine specific using overrides.

Also sort the hashes in the lists by PN to make diffing them easier.

(From OE-Core rev: d8b0ce35981931a39e7db9d8e78de6e009b34688)

(From OE-Core rev: b42f305ce38b9e0f1a2b7cb9586bbabcd2d27429)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-17 22:00:25 +01:00
Richard Purdie c5cc4993f0 sstatesig/sstate: Add support for locked down sstate cache usage
I've been giving things some thought, specifically why sstate doesn't
get used more and why we have people requesting external toolchains. I'm
guessing the issue is that people don't like how often sstate can change
and the lack of an easy way to lock it down.

Locking it down is actually quite easy so patch implements some basics
of how you can do this (for example to a specific toolchain). With an
addition like this to local.conf (or wherever):

SIGGEN_LOCKEDSIGS = "\
gcc-cross:do_populate_sysroot:a8d91b35b98e1494957a2ddaf4598956 \
eglibc:do_populate_sysroot:13e8c68553dc61f9d67564f13b9b2d67 \
eglibc:do_packagedata:bfca0db1782c719d373f8636282596ee \
gcc-cross:do_packagedata:4b601ff4f67601395ee49c46701122f6 \
"

the code at the end of the email will force the hashes to those values
for the recipes mentioned. The system would then find and use those
specific objects from the sstate cache instead of trying to build
anything.

Obviously this is a little simplistic, you might need to put an override
against this to only apply those revisions for a specific architecture
for example. You'd also probably want to put code in the sstate hash
validation code to ensure it really did install these from sstate since
if it didn't you'd want to abort the build.

This patch also implements support to add to bitbake -S which dumps the
locked sstate checksums for each task into a ready prepared include file
locked-sigs.inc (currently placed into cwd). There is a function,
bb.parse.siggen.dump_lockedsigs() which can be called to trigger the
same functionality from task space.

A warning is added to sstate.bbclass through a call back into the siggen
class to warn if objects are not used from the locked cache. The
SIGGEN_ENFORCE_LOCKEDSIGS variable controls whether this is just a warning
or a fatal error.

A script is provided to generate sstate directory from a locked-sigs file.

(From OE-Core rev: 7e14784f2493a19c6bfe3ec3f05a5cf9797a2f22)

(From OE-Core rev: 884d4fa3e77cf32836f14a113c11489076f4a84d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-17 22:00:25 +01:00
Robert Yang d06839e7dd package_manager.py: use RPM_PREFER_ELF_ARCH
Use RPM_PREFER_ELF_ARCH to instead of RPM_PREFER_COLOR as rpm upstream
suggested, and use "4" to instead of "3" since it is a bit mask (not
enumeration), so we need "4" here.

(From OE-Core rev: 8dcfa7143a6690455c3bd5772621f142f2d9e559)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-10 11:33:22 +01:00
Kévin THIERRY e7a78bdc97 package_manager: RpmPkgsList: determine rpm version
Do not set the rpm_version to 5 if it is not provided, instead
determine the real rpm version.

(From OE-Core rev: 48beaee2e5b2b4ae35c596c19f8a38e0ff4427e9)

(From OE-Core rev: fe4ad726d0dc63be73f885598e216b8eb927668a)

Signed-off-by: Kévin THIERRY <kevin.thierry@open.eurogiciel.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-10 11:33:22 +01:00
Richard Purdie 45907b6d7f package_manager: Add rpm v4 support
Currently the package manager code only supports rpm v5. To be useful outside
of OE or with OE layers using v4, it makes sense to add in rpm v4 support. This
takes a patch from "Bartosh, Eduard" <eduard.bartosh@intel.com> and enhances
it to also include versions of the workarounds from poky-eurogiciel to
allow rpm v4 usage with the class for image construction.

(From OE-Core rev: fe21804c296bbb8b2b8b0c29e6e4890bc17f07fc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-03 11:09:03 +01:00
Ed Bartosh cc0a0614e8 package_manager.py: enable smart non-interactive mode
Added --quiet option to smart command line.
Without this option smart 1.4.1 turns into interactive mode, i.e.
start asking questions and expecting answers.

Internally within smart, this changes the default UI to one which
just prints to stderr, the naming of the parameter is a little
odd but does what we need.

(From OE-Core rev: 7a8d88b73d35ac86198a1092c49b33c378416a03)

Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-31 23:40:47 +01:00
Peter Kjellerstedt 3b47aa11c2 lib/oe/utils: Make multiprocess_exec() return anything
The variable "results" was accidentally used for multiple different
things at the same time, which unintentionally discarded anything that
was supposed to be returned from the function...

(From OE-Core rev: abf4eb613eba0892a5f240de7aa3a9a1b2879354)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29 23:44:37 +01:00
Richard Purdie c490ba0a93 sstatesig: Only squash dependencies for allarch packagegroups
The idea of squashing packagegroup dependencies was to avoid allarch
packages rebuilding upon tune/arch changes. Now that the allarch class
inclusion is conditional, we can narrow down the packagegroup squashing
to be specifically applied to allarch recipes.

(From OE-Core rev: bd8018792155ae842952432900c8431feeaffe30)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-25 08:57:44 +01:00
Richard Purdie be1b198076 lib/oe/utils: Add utils function for multiprocess execution
Our usage of multitprocessing is problematic. In particular, there is a bug
in python 2.7 multiprocessing where signals are not handled until command
completion instead of immediately.

This factors the multiprocess code into a function which is enhanced with
a workaround to ensure immediate signal handling and also better SIGINT
handling which should happen in the parent, not the children to ensure
clean exits. The workaround for the signals is being added to the core
bb.utils function so it can benefit all users.

package_manager is then converted to use the new code.

(From OE-Core rev: 72d153a3a90d31d9f4e41d77da24e44ccb33c56e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-23 09:31:41 +01:00
Christopher Larson d7fea14b1f oe.package_manager: fix use of PACKAGE_EXCLUDE for dpkg
It was iterating over the variable character-by-character rather than
word-by-word.

(From OE-Core rev: 207f2cc0b0fc53b22e1dedfa26905ab143fb0de6)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-11 17:43:44 +01:00
Robert Yang a9c8767a17 package_manager.py: set preferred ABI for rpm
When using the RPM packaging backend to generate a rootfs there needs to
be a way to configure the preferred ABI to resolve ELF file conflicts.

Currently RPM resolves ELF file conflicts with the last-installed wins.
Using SMART it's difficult to know what the last installed will be.

There are three specific policies that can be selected:
1: ELF32 wins
2: ELF64 wins
3: ELF64 N32 wins (mips64 or mips64el only)

Another option "0" is uncontrollable, which means that if two are being
installed at once Elf64 is preferred, but if they're being installed in
two different transactions, last in wins, so we don't document it.

Add RPM_PREFER_COLOR to let the user config the preferred ABI.

[YOCTO #4073]

(From OE-Core rev: f56d7be2c35cedcd763ba66913982aa4c425d561)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-02 09:26:16 +01:00
Richard Purdie addeef1dc9 lib/oe/rootfs: Improve error message whitespace
[YOCTO #6493]

(From OE-Core rev: 91ca6b1b2e009381d8e813906654c0958eee7efc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-28 12:21:24 +01:00
Richard Purdie 42470aa22d lib/oe/classextend: Avoid early expansion of PR values
Variables like RDEPENDS can contain EXTENDPKGV which in turn uses AUTOPR
based values. This gets set during do_package execution so we want to
defer expansion until then. The only way we can do this in the RDEPENDS
(and friends) mapping code is to subsitute a dummy value, then change it
back again. Horrible but I can't see any other way.

This resolves multilib build failures with inconsistent PR values.

(From OE-Core rev: 5aea553e6eaa3b9647f26944976d2a9da79cba42)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25 15:33:58 +01:00
Roxana Ciobanu 56bfda20f4 image.bbclass/rootfs.py: add variables to rootfs[vardeps]
Added base variables and package backend specific variables to
rootfs[vardeps] in order for rootfs to rebuild when changes are made.

Set some variables as [func] to inform bitbake that they are shell
scripts, so that it invokes its shell dependency parsing. Without
marking them as functions, changes in the actual function body would
not trigger rootfs rebuilds.

[YOCTO #6502]

(From OE-Core rev: b8b6214b885a0757f0e628937f8fe21c92c45155)

Signed-off-by: Roxana Ciobanu <roxana.ciobanu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-23 21:59:13 +01:00