With the current code, if we use debian package backend and enable
multilib support, the do_rootfs process would always fail with error
messages like below.
E: Unable to locate package packagegroup-core-boot
This patch fixes the above problem.
(From OE-Core rev: d140d556ae30b6dbd0ffce8882c3e22b17050820)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Multiple arguments to string formats need to be in a tuple.
Reported by Lorenz <lqb.list@gmail.com>.
(From OE-Core rev: e30a4650beabac215b6d867070b7acdb3601a4d7)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While invoking create_index failed, there was no error output
and didn't break the build until the package installation.
...
|ERROR: run-postinsts not found in the base feeds (qemux86 i586 x86
noarch any all).
...
The reason is we used multiprocessing to execute create_index, and
did not check its invoking result.
(From OE-Core rev: d8921e4ea68647dfcf02ae046c9e09bf59f3e6e4)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
DpkgPM change all_arch_list variable set from PACKAGE_ARCHS to passed
archs variable because is different when is executed from rootfs.py
and sdk.py.
Credits to: Ricardo Ribalda <ricardo.ribalda@gmail.com>
(From OE-Core rev: f6fb8c16f49fd9a2b124ad55f5c4fed82d7e6dca)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In package_manager.py, when using opkg as the packager, the command 'opkg <args>
info <pkg>' is called to get information about each pkg in BAD_RECOMMENDATIONS
in a format that can be written to the status file. The 'Status: ...' line is
modified and all other lines are passed through. Changing the verbosity level
argument for this command will change what it written into the status file.
Crucially, with the default verbosity level, no blank lines are being printed by
the opkg command and so no blank lines are being written to the status file to
separate each package entry.
The package parsing code in opkg expects package entries in the status file to
be separated by at least one blank line. If no blank line is seen, the next
package entry is interpreted as a continuation of the last package entry, but
the new values overwrite the old values.
So with the default verbosity level, a blank line follows some package entries
and these are parsed. The others are dropped due to the lack of blank lines. As
the verbosity increases, more debugging messages add blank lines and more
packages are parsed.
The solution to ensure that this works correctly regardless of the verbosity
level is simply add a blank line after the output of 'opkg info' is written to
the status file, ensuring that the next package is separated from the current
package.
[YOCTO #6816]
(From OE-Core rev: 3fa24eee41c26fecd5e4f680082288ec772d2de9)
Signed-off-by: Paul Barker <paul@paulbarker.me.uk>
Cc: Chris Carr <chris.carr@ge.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a failure to build lib32-meta-toolchain:
...
|ERROR: lib32-packagegroup-core-standalone-sdk-target not found in the base
feeds (qemux86_64 x86 noarch any all).
...
In package_manager.py, the variable 'DEFAULTTUNE_virtclass-multilib-lib32'
is used to process multilib image/toolchain. But for the build of lib32-
meta-toolchain, the value of 'DEFAULTTUNE_virtclass-multilib-lib32' is
deleted. In 'bitbake lib32-meta-toolchain -e', we got:
...
|# $DEFAULTTUNE_virtclass-multilib-lib32 [2 operations]
|# set? /home/jiahongxu/yocto/build-20141010-yocto/conf/local.conf:237
|# "x86"
|# del data_smart.py:406 [finalize]
|# ""
|# pre-expansion value:
|# "None"
...
The commit 899d45b90061eb3cf3e71029072eee42cd80930c in oe-core deleted
it at DataSmart.finalize
...
Author: Richard Purdie <richard.purdie@linuxfoundation.org>
Date: Tue May 31 23:52:50 2011 +0100
bitbake/data_smart: Change overrides behaviour to remove
expanded variables from the datastore
...
We add an internal variable 'DEFAULTTUNE_ML_<multilib>', assign it with the
value of 'DEFAULTTUNE_virtclass-multilib-lib32' before deleting.
For rpm backend in package_manager.py, we use DEFAULTTUNE_virtclass-multilib
-lib32 first, if it is not available, and try to use DEFAULTTUNE_ML_<multilib>
[YOCTO #6842]
(From OE-Core rev: 9c59d3d8b538d3a98ff4b5e5b189a4a23a85da2d)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The do_rootfs takes a very long time when build host has mounted many NFS
devices. syscall lstat() was being called on every filesystem mounted on the
build host during building.
The reason for the lstat() is that rpm is verifying that enough free disk space
is available to do the install. However, since the install is into the target
rootfs it should not matter how much free space there is in the host mounts.
Add checking for "rpm-ignoresize", by it, smart can make whether RPM skip
checking for diskspace when install a rpm package.
(From OE-Core rev: fc0668a019eca422540ceab3efcd2b2a27dd79e0)
Signed-off-by: Chong Lu <Chong.Lu@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Package Manager implementation for deb didn't
take a look about multilib enabled variants.
Changes are made for generate apt.conf, sources.list and
debian repo index Release and Packages files.
[YOCTO #1502]
(From OE-Core rev: b5fb879b351cc23977f3e441f758101551297566)
Signed-off-by: Anibal Limon <anibal.ezau.limon.belmares@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use RPM_PREFER_ELF_ARCH to instead of RPM_PREFER_COLOR as rpm upstream
suggested, and use "4" to instead of "3" since it is a bit mask (not
enumeration), so we need "4" here.
(From OE-Core rev: 8dcfa7143a6690455c3bd5772621f142f2d9e559)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do not set the rpm_version to 5 if it is not provided, instead
determine the real rpm version.
(From OE-Core rev: 48beaee2e5b2b4ae35c596c19f8a38e0ff4427e9)
(From OE-Core rev: fe4ad726d0dc63be73f885598e216b8eb927668a)
Signed-off-by: Kévin THIERRY <kevin.thierry@open.eurogiciel.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the package manager code only supports rpm v5. To be useful outside
of OE or with OE layers using v4, it makes sense to add in rpm v4 support. This
takes a patch from "Bartosh, Eduard" <eduard.bartosh@intel.com> and enhances
it to also include versions of the workarounds from poky-eurogiciel to
allow rpm v4 usage with the class for image construction.
(From OE-Core rev: fe21804c296bbb8b2b8b0c29e6e4890bc17f07fc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added --quiet option to smart command line.
Without this option smart 1.4.1 turns into interactive mode, i.e.
start asking questions and expecting answers.
Internally within smart, this changes the default UI to one which
just prints to stderr, the naming of the parameter is a little
odd but does what we need.
(From OE-Core rev: 7a8d88b73d35ac86198a1092c49b33c378416a03)
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Our usage of multitprocessing is problematic. In particular, there is a bug
in python 2.7 multiprocessing where signals are not handled until command
completion instead of immediately.
This factors the multiprocess code into a function which is enhanced with
a workaround to ensure immediate signal handling and also better SIGINT
handling which should happen in the parent, not the children to ensure
clean exits. The workaround for the signals is being added to the core
bb.utils function so it can benefit all users.
package_manager is then converted to use the new code.
(From OE-Core rev: 72d153a3a90d31d9f4e41d77da24e44ccb33c56e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was iterating over the variable character-by-character rather than
word-by-word.
(From OE-Core rev: 207f2cc0b0fc53b22e1dedfa26905ab143fb0de6)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using the RPM packaging backend to generate a rootfs there needs to
be a way to configure the preferred ABI to resolve ELF file conflicts.
Currently RPM resolves ELF file conflicts with the last-installed wins.
Using SMART it's difficult to know what the last installed will be.
There are three specific policies that can be selected:
1: ELF32 wins
2: ELF64 wins
3: ELF64 N32 wins (mips64 or mips64el only)
Another option "0" is uncontrollable, which means that if two are being
installed at once Elf64 is preferred, but if they're being installed in
two different transactions, last in wins, so we don't document it.
Add RPM_PREFER_COLOR to let the user config the preferred ABI.
[YOCTO #4073]
(From OE-Core rev: f56d7be2c35cedcd763ba66913982aa4c425d561)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For the meta-mingw layer, we need to process alternative SDK_OS, since this
is not a Linux based OS.
(From OE-Core rev: b31e015d2e379c24610948d345c5970545887468)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do not delete the __db.00* files in the PackageManager class. Leave this
operation up to the client classes. One side effect of this deletion was
the following message appearing in the output of the next rpm command
executed:
rpmdb: BDB1540 configured environment flags incompatible with existing
environment
We might also gain some time here by not deleting/creating those files
very often.
[YOCTO #6049]
(From OE-Core rev: 12e300f0af2a27c15d80298d3fbb27b092c35154)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The native sysroot should not be used as a store for the lists files since
multiple images running at once would conflict over this. Instead redirect
this to WORKDIR. This means some extra directories need to be created.
Also create apt.conf.d to silence some warnings.
(From OE-Core rev: dc4abfc8f99c08e0c1ac9d098ce17838d0eda028)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
a) There were missing parameters to the release and package commands (".")
b) The commands need to be executed as one block since they build upon each other
(From OE-Core rev: a3965b76ed4361455c89c982761263be03e1a8e5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Processing directories twice is both pointless and introduces a race condition.
When building the list, ensure duplicates (like "all" and "noarch") are handled
correctly.
(From OE-Core rev: 4c487543422ae471a01a573bab44e3f6a6d2497a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit creates a new class that has the only purpose to generate
various listings of installed packages in the rootfs.
Basically, the methods involved in listing the installed packages, that
were part of each backend PM class implementation, were moved to this
new class.
This change avoids instantiating a new PM object just to get the list of
installed packages in a certain rootfs.
(From OE-Core rev: a7290ed13378826723d1edc7e828eab848eaad10)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use apt-ftparchive to create a Release file compatible with SecureApt.
apt-ftparchive is also a more efficient replacement of
dpkg-scanpackages:
root@neopili:~/curro/qtec/qt5022/build-qt5022-cesium/build/tmp/deploy/deb/bobcat
_64# time PSEUDO_UNLOAD=1 apt-ftparchive packages . >/tmp/kkk
real 0m26.873s
user 0m20.968s
sys 0m1.212s
root@neopili:~/curro/qtec/qt5022/build-qt5022-cesium/build/tmp/deploy/deb/bobcat
_64# time PSEUDO_UNLOAD=1 dpkg-scanpackages . >/tmp/kkk
dpkg-scanpackages: info: Wrote 6022 entries to output Packages file.
real 0m59.721s
user 0m16.668s
sys 0m11.164s
apt-ftparchive is not compatible with libpseudo. The calls to ftw()
returns the path in absolute format instead of relative. This
produces
wrong Packages and Release files.
ie:
MD5Sum:
d20227a958f6870137ce0e41b7b84307 1453
/home/ricardo/curro/qtec/qt5022/build-qt5022-cesium/build/tmp/deploy/deb/all/Release
This is why it is called with PSEUDO_UNLOAD.
(From OE-Core rev: c9899a7605f15f7f1ae30c4624d53c7da825b00a)
Signed-off-by: Ricardo Ribalda Delgado <ricardo.ribalda@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding a common interface to add predefined package manager
channels to prebuilt rootfs:es.
Adding PACKAGE_FEED_URIS = "http://myre.po/repo/, will
assume repo directories named (rpm,ipk,deb) as subdirectories
and statically add them to the rootfs, using the same PKG_ARCHs
as the build which produced the images.
Tested with RPM, IPK and DEB.
deb feed functionality seem broken, is anyone using this ?
(From OE-Core rev: 9b8811045546ad67b4695d980f09636d5506e50c)
Signed-off-by: David Nyström <david.c.nystrom@gmail.com>
Signed-off-by: David Nyström <david.nystrom@enea.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a command executed with subprocess.check_output() fails, the
subprocess.CalledProcessError.output contains only STDOUT and the user
needs to check the log.do_rootfs to see any other details.
This commit forwards stderr to stdout so that, in case of failure, the
entire error output will be displayed in terminal.
[YOCTO #5902]
(From OE-Core rev: 4661f1441429952f30e19cebd93dc42ce31fd868)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure the channel was not already added, before attempting to add.
[YOCTO #5890]
(From OE-Core rev: 0574f5390fb8958db5864f41a7bd8037ac046ac5)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Converting a list to a set does not keep the order of the items, as they
were added, which might result in the wrong packages being installed in
the final image...
This reverts commit 12f47c23df.
[YOCTO #5890]
(From OE-Core rev: 495cce9c9e63484c740aa669facb995b0ef74f85)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous implementation was checking IMAGE_PKGTYPE and created the
index files just for the backend used to create the image. Apparently,
'bitbake package-index' should attempt to create the index files for all
backends specified in PACKAGE_CLASSES.
[YOCTO #5827]
(From OE-Core rev: 0521d48a1612bfc735e2c86acc9b685c1dc389ef)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the output from opkg-query-helper.py is empty, output.split('\n')
would result in a list containing one element which is an empty string
while iterating over each line in the output. An exception is then
thrown by the line:
pkg, pkg_file, pkg_arch = line.split()
with the message:
Exception: ValueError: need more than 0 values to unpack
To avoid this, we add a condition to only split the output if it isn't
empty.
(From OE-Core rev: ee7b75c895e77ab20f728423c8efc2ced92265e8)
Signed-off-by: Jonathan Liu <net147@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the following three functions to OpkgPM class:
- The 'dummy_install' is used to dummy install pkgs, and returns the log
of output;
- The 'backup_packaging_data' is used to back up the current opkg
database;
- The 'recover_packaging_data' is used to recover the opkg database
which backed up by the previous image creation;
Tweak 'remove' function in OpkgPM class, which the options for remove
with dependencies was incorrect.
Tweak 'handle_bad_recommendations' function in OpkgPM class:
- Fix none value check;
- Add the existance check of opkg status file;
[YOCTO #1894]
(From OE-Core rev: 5df18065e267a7e55a990ac3728414bb6e28a723)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* change the text a bit, because I have trouble parsing two
verbs at the beginning of sentence.
(From OE-Core rev: 9837d194682f46e3a0363765bd6b317180a4cda0)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Small error in the package list creation routine. Buildhistory was
supposed to use this but was never called. Hence, it escaped tests...
(From OE-Core rev: eaa1994ad22730cec8e8c57736915da6b45a416e)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its perfectly reasonable to call -c package-index with empty package
directories. This ensures the builds don't fail in such cases,
resolving autobuilder failures.
(From OE-Core rev: 11e2aaf2d751277e3e99ac2acbbeff2b7227be94)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the code from anonymous function in rootfs_rpm.bbclass has been
removed, MULTILIB_PREFIX_LIST variable was never set. Hence not all
directories got indexed.
This commit will move the multilib prefix list computation function from
RpmPM class to RpmIndexer, since the indexer needs it too. I was hoping
to avoid this but, unfortunately, I couldn't.
(From OE-Core rev: d3ba249aa1bf68aaeed226e934a4f4d5b7a19286)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Because the package-index.bb needs to create package indexes outside
do_rootfs environment, move the indexing capability out of
PackageManager class to a smaller Indexer class.
This commit:
* simply moves the indexing functions for ipk/deb with no changes;
* rewrites the RPM indexing function so that it can be easily moved out
of the PackageManager class;
* removes some RPM duplicate code, moves it into a method inside
RpmPM class and changes the RpmPM constructor so that the new method
is effective;
(From OE-Core rev: d339d6f48f81330e94162f333aad76f3c65d6bfd)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Print the entire command output in case of errors.
(From OE-Core rev: 2253c9ac2caa61dee0bd4fea04d4d77b79be7b36)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use python sets instead of lists, to avoid duplicates. When doing a
multilib build, "smart channel --add" fails because it tries to add
'all' channel twice.
(From OE-Core rev: 730d675090eec5c03e444f0448d96a52035d0bef)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're just attempting to install packages and the package list is
empty, just return.
(From OE-Core rev: 8bf5d80e3502a71fdda688aace0799b47ae19b3b)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using opkg as the PM backend, one has the option to provide custom
feeds to create the rootfs from.
This commit:
* fixes this in the refactored code;
* moves the custom config creation code to python;
* clean up the package-ipk.bbclass;
(From OE-Core rev: 19c538f57c8fa7c566e88a6dbe13ea4826d4f26c)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following commit:
rootfs_ipk: Ensure that BAD_RECOMMENDATIONS are honoured for all
architectures
changed the way BAD_RECOMMENDATIONS are handled. Make the change in the
new code too.
(From OE-Core rev: 7e518e399da51de3b159bd6804735b2f14c39357)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using stamps shortens rootfs/image creation because the package indexes
are not created again if no new package was installed in the deploy dir.
Unfortunately, there are some syncronization issues with
do_package_write_*() and the indexes are not properly created.
Will have to revisit this issue in the near future. For now, lose it.
(From OE-Core rev: 276f2a8a072ce243daa74210934ee63046e9c480)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit:
* adds missing mark_packages() function for OpkgPM. This is needed to
* manually change the packages installation status in the status file;
* fix OpkgPM remove_packaging_data() issue;
* fix OpkgPM handle_bad_recommendations() issue;
* improve OpkgPM/DpkgPM mark_packages() regex pattern;
* fix DpkgPM list_installed() problem;
* fix DpkgPM _create_configs() problem: status file should not be
truncated if it already exists/
(From OE-Core rev: d50a40100763dcbfabec2c147fdfcb46aa909329)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit:
* fixes a crash when handling interecept hook failures which happened when
MULTILIB_GLOBAL_VARIANTS was not set;
* convert dashes to underscores and use sets (so that we make sure the
items are unique) when creating RPM repos;
* uses a regex pattern to search for packages in the feeds list. The
old method could match also strings in the middle. For example: 'rpm'
matched 'kernel-module-lttng-probe-rpm" in qemux86_64 feeds;
* issue a bb.fatal if smart returns error while installing packages.
Otherwise we might end up with an incomplete image...
* fixes the /etc/rpm/platform file creation;
(From OE-Core rev: b98c7e4945f1c36a6e4f98144a3af4f3049450ae)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>