All tasks which implement a do_deploy should inherit this class to have the
changes in the deploy task staged.
Update recipes which include a do_deploy function to inherit this class and
to use DEPLOYDIR rather than DEPLOY_DIR_IMAGE.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Add pre clean hook
Add sstate_installpkg
sstate: Tie all pieces together with a setscene function
Use pythonic functions rather than os.system()
Implement sstate_clean function
package_ipk: clean shared state in prestate
packaged-staging2: implement fetching packages from mirror
Copy add staging_fetch() from packaged-staging.bbclass
packaged-staging2: make the output slightly friendlier
packaged-staging2: handle directories in the shared state
Add directories to the manifest when installing and remove them if empty when
cleaning.
Move most of the sstate code into the class itself removing the need for heavy boilderplate
packaged-staging2: Programatically ensure directories are last in the manifest
packaged-staging2: add cleanall
packaged-staging2.bbclass: Connect in cleanall function to do_clean
We want to ensure that directories appear at the end of the manifest so that
when we test to see if they should be deleted any contents added by the task
will have been removed first.
pstage2: Use oe.path.copytree
packaged-staging2: support lockfiles and plaindirs
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Signed-off-by: Joshua Lock <josh@linux.intel.com>
So far unlike prerm/postrm, update-rc.d has its own postinst method prepended
to pkg_postinst, which may result "System startup links for xxx already exist"
warning in the 1st boot of target image. Some pkg_postinst requires to run on
the target, and thus prepend here makes update-rc.d method executed twice: one
in rootfs creation and the other in the 1st target boot. So adjust the order
to append.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
currently staging.bbclass mangle *.la to ensure sysroot prefix attached to
dependency_libs if other *.la is referenced, however it only happens on
${libdir}. libpam is special with its libraries under ${base_libdir}. So
this extends mangle to ${base_libdir} too
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Add the ability to specify user level classes via the local.conf.
Use this new capability to add an image-prelink class that does an
image wide cross-prelink activity.
Signed-off-by: Mark Hatle <mhatle@windriver.com>
Fix an issue where pseudo could try to compile incorrectly on an
x86_64 host system. Now it verifies that it should be trying to build
a 32-bit library, and if so the necessary prereqs are available.
Also uprev to the latest version of pseudo changes to enable wrapping
of execl, local variable storage and misc fixes.
Signed-off-by: Mark Hatle <mhatle@windriver.com>
Move the environment-setup script generation into a separate class and adapt
meta-toolchain to use it.
Add a new dummy target, meta-ide-support, to install an appropriately
configured script to TMPDIR and ensure all of the required packages for target
development are available.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Change the pseudo integration:
* Uprev to latest open source version
* Restructure the patches to allow for many local DBs, as well as
pseudo specific lib dirs.
Signed-off-by: Mark Hatle <mhatle@windriver.com>
Use the virtual fakeroot program when installing a package and also
during packaging. This is important as it allows us to track full
permissions, owners, groups and special files generated by packages.
Signed-off-by: Mark Hatle <mhatle@windriver.com>
Once ipk meet errors, the log will show "Collected errors:\n xxxx ".
so keyword "Collected errors" is more general to catch errors.
Signed-off-by: Yu Ke <ke.yu@intel.com>
Fix parse error with packages such as spectrum-fw whose name has no
version string. Later we may have per-recipe option to disable automatic
check for those known with troubles, to reduce complexity in this part.
Signed-off-by Kevin Tian <kevin.tian@intel.com>
default cmake behavior for FIND_***, is to first search listed directories prefixed
with CMAKE_FIND_ROOT_PATH, and if not found then go to unprefixed directories. This
is not desired as build system may be referenced.
this commit refine the search path only on prefixed directories, and thus fix
libmusicbrains build failure on some systems
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Re implement the rootfs generation using rpm5. This also gets rid of the
need for yum, and handles all dep resolving internal to the script itself.
The new file scripts/rootfs_rpm-extract-postinst.awk comes from the original
yum integration work. It has been unchanged, but since yum is no longer used
we needed to move the script somewhere else.
Signed-off-by: Mark Hatle <mhatle@windriver.com>
Switch to RPM5 as our rpm provider of choice and update the recipe to the latest
stable release.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Modify the package_rpm.bbclass to understand the macro and command line changes
present in rpm5.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Enable changing the data directory on the fly from the environment and then use
this feature within poky to confine pseudo usage to each WORKDIR.
This fixes issues that could be seen under heavy inode reusage e.g.
with rm_work.
Work based mainly off a patch from Joshua Lock but finished by Richard
Purdie.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Per bpo#109 dependency failure during rootfs generation did not cause an error
despite that it would cause errors later on in the process. This patch updates
the key phrases searched for in the log checking function.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
this is one existing for stale reason when package's own Makefile doesn't
handle .pc correctly. After previous commits to remove unnecessary inheritage
on it (sqlite3, libpng), the only remaining is libgcrypt. Libgcrypt itself
doesn't ship a .pc and poky generates one. In this case, simply move the
manual installation to its own do_install.
Then it's safe to remove this class file
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Since scripts is now in PATH thanks to the layer functionality there is
no longer any need to have this recipe full of special cases, the scripts
can just be placed there.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
This package changed from autoconf to cmake during the last recipe
upgrade. This made the existing EXTRA_OECONF, which disabled wekbit
support among other things, non-functional, which in turn, broke the
build.
This patch disables webkit in a CMAKE way by adding EXTRA_OECMAKE to
the cmake class description, and setting it to "-DWEBKIT=no" in the
libproxy recipe and also reproduces the EXTRA_OECONF settings.
Signed-off-by: Jeff Dike <jdike@linux.intel.com>
You need to first set up the build directory by sourcing the poky build script,
after that builds can be run in that directory so long as bitbake is in $PATH
removing the need to source the init script for each build.
i.e:
$ . poky-init-build-env ~/my-build
$ bitbake some-image
<<later, in a different shell>>
$ cd ~/my-build
$ export PATH=/path/to/bitbake/bin:$PATH
$ bitbake an-image
This patch also removes use of OEROOT in recipes, etc.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Now perl version is in two digits like 12 in 5.12.1. Fix the old
function to work with multi digit version strings.
Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com>
If a program creates an empty directory during its build it's probably for a
good reason, so if it exists it seems sensible to stage it.
Some programs require localstatedir to operate so we should be sure to stage
it.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Now Warning includes the package name like zlib is printed bellow:
WARNING: zlib: Recipe file does not have license file information
(LIC_FILES_CHKSUM)
Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com>
Automation test is disabled by default. User need set TESTCLASS
to qemu in conf/local.conf and run bitbake command "bitbake
poky-image-xxx" or "bitbake poky-image-xxx -c qemuimagetest" to
trigger it. Currently only the sanity test with two testcases are
added.
To run the test, user need prepare a testing environment:
1) "expect" should be installed on system
2) NOPASSWD should be set for user to run bitbake
Signed-off-by Jiajun Xu <jiajun.xu@intel.com>
Only checking the license at packaging time means we don't check native builds,
so move the check to after do_configure
Signed-off-by: Joshua Lock <josh@linux.intel.com>
siteinfo_get_files() was broken for mips. It was missing the mips-common
site config file for all the mips targets. Fixed it by adding it.
This will also fix building of flex recipe on mips.
Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com>
both git/svn supports remote information query: 'git ls-remote', and
'svn info'. With them, now upstream version will be automatically
checked for git/svn packages.
In the meantime, manual latest version tagged in distro tracking
fields are also compared as one alternative if upstream check fails.
Also such check is one indicator whether tracking field is missing.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
* Switch from /usr/local/poky to /opt/poky
* Use a sysroots directory for both the "native" sdk binaries and the target
* Drop the meta-toolchain extras packages. These are replaced with packaged-staging.
* Change the nativesdk layout to match our usual filesystem layout
* Clean up various hardcoded prefix references
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Need to extend bitbake to provide this information rather than refer to a bitbake
internal variable.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
The OVERRIDES variable was being incorrectly set with the end result of the
runtime dependencies of the package not being encoded in it's package metadata.
This broke opkg-native in meta-toolchain.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
instead of exiting when once one md5 mismatch is seen. This would be
helpful to save time on filling md5sum for multiple license check
files.
Signed-off-by Kevin Tian <kevin.tian@intel.com>
The version information part of package dependency strings was not ending up
in the output packages correctly. This patch fixes this and ensures the version
information isn't lost.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
cross-canadian packages need to look for their SOLIBS in the nativesdk
sysroot so that dependencies are correctly picked up and meta-toolchains are
correctly built.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
* Remove additonal binaries known to cause "strip command failed"
errors during do_package on cross platforms.
Signed-off-by: Scott Garman <scott.a.garman@intel.com>
The 'is' keyword tests for object identity, returning True if the variables are
both referencing the same object. Changed the test to use the equality
operator, which compares the values of the objects.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
The opkg package manager uses /usr/lib/opkg to store lock files in, modify the
remove_package_data_files() function to create an empty directory once the data
files are removed so that the lock can be created.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Assembling an appropriate opkg.conf in the pstage_helper is difficult when
building for multiple target architectures in one Poky directory, work around
this by generating an appropriate opkg.conf for the TARGET_ARCH and using an
${TARGET_ARCH}-opkg.conf for packaged staging.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
In relocatable.bbclass and utility-tasks.bbclass we depend on functionality
that was only introduced with Pytrhon 2.6, so we now check that the build
system has it in the sanity class.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
We already have versioning of the local.conf file but it's entirely plausible
for the bblayers.conf and site.conf to change incompatibly so we should version
those too.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Automatic upstream version check can be triggered by "bitbake xxx -c checkpkg".
The rationale behind is to find a automatic way for version comparison between
current in-use one and upstream progress. The tricky thing is how to find out
useful version string from mass diverse information. Fortunately now it mostly
work for wget based protocol, except some sites (e.g. sourceforge) not providing
a directory service. Repo (git/svn/cvs) based protocols are handled in a very
simple manner, by always tagging as update required. This will be further
improved later. Use 'world' target to retrieve a full version check for all
active recipes.
Along with version check, other package information is also collected from
existing fields in .bb file
Signed-off-by Kevin Tian <kevin.tian@intel.com>
Some characters in the path for OEROOT are known to cause issues, so check for
them in the sanity checker.
Currerently we warn on +'s and spaces.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
This adds a new task (distro_check) for each recipe.
The task generates the source package list for Fedora OpenSuSE
Ubuntu Debian & Mandriva Linux distros.
As one recipe or source package can generate multiple target packages
the recipe name is compared with the source package name list of LInux
distributions.
Thread locking is used to avoid multiple threads racing for the
package list update.
Then the recipe name (PN) is checked if it exists in the package
list of distros. And if the DISTRO_PN_ALIAS then it is used to copmare
pacakge_name instead of the PN variable. Just for example the
DISTRO_PN_ALIAS can be defined in the recipe (.bb) files like this
In the file xset_1.0.4.bb:
DISTRO_PN_ALIAS = "Fedora=xorg-x11-server-utils;\
Ubuntu=x11-xserver-utils; Debian=x11-xserver-utils;Opensuse=xorg-x11"
The final results are stored in the tmp/log/distro_check-${DATETIME}.result
file.
FYI this command will generate the results for all recipies:
bitbake world -f -c distro_check
Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com>
Chris made some interesting changes to the patch handling in OE. Firstly the
patch and pnum parameters have been renamed, to better reflect their function,
to apply and striplevel. The apply parameter now takes either yes or no as its
value.
Therefore a patch line would be changed from:
file://some.patch;patch=1;pnum=2
to:
file://some.patch;apply=yes;striplevel=2
Secondly the apply parameter is inferred if not defined. Entries in SRC_URI
with diff or patch filename extensions will be automatically applied as patches
if the apply parameter is not set to no (or an unknown value).
Note: We have disabled the warnings when using the old style parameters for now
Signed-off-by: Joshua Lock <josh@linux.intel.com>
QEMU 0.12.x is relocatable so we no longer need these tests, which is good
because it doesn't work reliably with modern kernels.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
* Preserves scripts/ directory but excludes binaries known to cause
"strip command failed" errors during do_package on cross platforms
Signed-off-by: Scott Garman <sgarman@zenlinux.com>
The changes merged from upstream used the ASMDIR variable even when it might
not have been set. These changes may be incorrect but at the very least I can
now build a QEMU image (including v86d) with the 2.6.33 kernel now.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Kernels after 2.6.33 store the utrelease.h file, used to detect the kernels
version, in a different location.
Merged from oe.dev
Signed-off-by: Joshua Lock <josh@linux.intel.com>
The header installation for x86 was slightly different as to that employed
for other architectures, change them to match for consistency.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Changes merged from upstream to cope with changes to directory structures in
recent kernels when building for x86
Signed-off-by: Joshua Lock <josh@linux.intel.com>
If license file md5 information (LIC_FILES_CHKSUMS variable) is
missing in the recipe then just throw a warning instead of the build
failure.
Once enough recipes' LIC_FILES_CHKSUMS are filled then this warning
will be reverted back to the the fetal error.
If LIC_FILES_CHKSUMS field is present but invalid then the it still
causes a fetal build error.
Signed-Off-By: Nitin A Kamble <nitin.a.kamble@intel.com>
Added a new variable in recipe : LIC_FILES_CHKSUM
It is a required field for every recipe.
It describes license text location in the source files. And also stores
md5sum of that license text. Any change in this license text triggers build
error. Which enables developer to review any changes in the license and
update the license fields in the recipe accordingly.
For Example: contents of zlib_1.2.3.bb
LICENSE = "zlib"
LIC_FILES_CHKSUM = "file://README;md5=ae764cfda68da96df20af9fbf9fe49bd \
file://zlib.h;beginline=1;endline=30;md5=6ab03f03a5ee92d06b809797d4d5586d "
Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com>
Adapt modifications from upstream to make Poky classes use lib/oe for the
common Python functionality.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
The addition of BBLAYERS changes the build/conf directory to be incompatible
with the "old way" in the Purple release.
As such things are likely to occur in future we should bersion the build/conf
directory through local.conf
Signed-off-by: Joshua Lock <josh@linux.intel.com>
current_abi is the abi of the tmp directory, not the current abi specified in
the abi file...
Now the sanity check should work sanely.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Cross is no longer required so can go away, we now install cross packages into
the native sysroot and use them from there.
This patch includes updates to classes and some recipes which reference
CROSS_DIR. Others still need fixing an image can be built and run with this
patch applied.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Move the functionality into autotools and ensure all our Poky recipes are no
longer using it.
Keep the autools_stage class around for OE compatability but just have it
inherit autools.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
The generated path we create for the binaries RPATH can have a lot of directory
separators in. Use os.path.normpath() to tidy it up and only include the
required directory separators.
This patch is purely to appease my personal sense of niceness...
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Patch from OE.dev by Martin Jansa to workaround that we on modern Linux we can
no longer read this file.
We need a better fix but best option will be to update to a newer, relocatable,
QEMU release.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Cross scripts now live in sysroot/$arch-distro-os/crossscripts, this conveys
that they are no longer native system dependant and emphasises their purpose.
Bump the staging ABI and implement a simple migration from ABI 3 to ABI 4.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
The packages architecture field is incorrect, this patch changes it to:
native - build machine architecture
cross - build and target machine architectures
target - target machine architecture
Signed-off-by: Joshua Lock <josh@linux.intel.com>
The previous patch to do so was badly merged and didn't actually skip all
symlinks, only ones which where not an absolute path...
Signed-off-by: Joshua Lock <josh@linux.intel.com>
If this links to a file created by the package install we already handle it
when we process the sysroot. By ignoring symlinks here we don't cause a build
to fail when the symlink is to somewhere in the host OS.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
It's possible to have files in our sysroot which don't have the write (or in
some cases even the read) bit set. Test for these and if they are not set
temporarily set them so that we can chrpath the binaries.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Amend the default PSTAGE_SCAN_CMD, as used when scanning target packages, to
include binconfig scripts in the results and remove their hardcoded paths.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Make the processing of directories less naive so that it can handle a directory
with children that are directories. We now scan for and process binaries in all
directories below the scanned paths rather than only the top-level directory.
This patch moves the meat of the post-processing into a separate function which
is fed paths, process_dir (). Then when the function finds a subdirectory of
the passed path which is itself a directory it recursively calls itself.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
The .so files we ship also have a RPATH encoded so should be post-processed for
a relocatable RPATH too.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Python RDEPENDS are tricksy beasts and the runtime dependencies of Yum aren't
being pulled in for the RPM rootfs so explicitly add them to depends.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
This change, suggested by Chris Larson, ensures that the fix works even if the
definition of CROSS_DIR changes.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
packagedstaging_fastpath() was only copying the contents of CROSS_DIR to
PSTAGE_TMPDIR resulting in the folders contents being packaged and then
installed incorrectly at the top level of CROSS_DIR rather than in HOST_ARCH
specific sub directories.
This patch fixes that issue by copying the directory and its contents rather
than just the directory contents.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Extend the existing libtool hack so that native packages (native, cross, sdk)
have all non-binary files scanned and the STAGING_DIR fixed up.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
This patch removes the hard coded number of parent directory operators ( /..)
placed into the rpath and instead fully dynamically generates the rpath entries
based on the current rpath.
Theoretically this patch means we can now make cross packages relocatable but
this is *not* enabled as chrpath can only set a new rpath of the same length
or shorter than the existing rpath. Due to the amount of parent directory
jumps we need to encode in the rpath this can easily fail when TMPDIR is in a
short namespace (e.g. /usr/poky).
Signed-off-by: Joshua Lock <josh@linux.intel.com>
The initial pass at this class was pretty lame and broke on a lot of native
packages. This rewrite makes the code a lot more dynamic, removing use of hard
coded paths and improving the logic.
The class now runs a chrpath -l over the binary to determine what rpaths are
currently set. It then munges the output and determines relative versions of
each component of the rpath and uses chrpath -r to set them.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
The relocatable path will pre-process built binaries in SYSROOT_DESTDIR and
replace any harcoded dynamic link rpaths with relative paths.
Add an inherit of class in native.bbclass to make our native packages
relocatable and tweak the chrpath recipe so that the native package can make
itself relocatable with the just built chrpath binary.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Check the tarball stash on pokylinux.org before hitting the upstream
version control systems. This is friendlier to the upstream repositories than
us hammering their bandwidth.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Add simple support for trying to fetch staging packages from a http, https or
ftp mirror if they do not already exist in PSTAGE_DIR.
As documented in local.conf.sample
"Poky can try and fetch packaged-staging packages from a http, https or ftp
mirror. Set this variable to the root of a pstage directory on a server."
If the PSTAGE_MIRROR variable is not set, or the package cannot be found on
the mirror it will be built as usual.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Rename DEPLOY_DIR_PSTAGE to PSTAGE_DIR and move it outside of deploy.
Do soft assignment of a default variable inside bitbake.conf as
${OEROOT}/tmp/pstage.
Use a separate directory, PSTAGE_WORKDIR, to store transient packaged
staging files such as the stamp files and opkg.conf
Signed-off-by: Joshua Lock <josh@linux.intel.com>
The packaged-staging class already special cases native packages for ensuring
the package is only reused if the path doesn't change.
This patch ensures the special casing is also done for crosssdk packages.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Earlier commit had created a multiline string without
updating the python string quotation (""" for multiline).
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Only run the gettext section of the QA configure sanity check if
the package is not gettext as the gettext tools, shipped with
gettext, use the macro employed in the sanity check.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
The previous gettext check was broken resulting in many false positives.
This improved version works as follows:
* Adds to the existing os.walk() and builds a list of configure.in and
configure.ac files
* Tests whether DEPENDS includes an appropriate gettext
* If not greps the configure files in the list for instances of AM_GNU_GETTEXT
* If if an instance is found we bb.fatal with an appropriate message
Signed-off-by: Joshua Lock <josh@linux.intel.com>
When the package we are checking is a native package we need to check
for gettext-native rather than gettext in the DEPENDS.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Richard Purdie
Joshua Lock
Saul Wold
Kevin Tian
Nitin A Kamble
Mark Hatle
Yu Ke
Jeff Dike
Bruce Ashfield
Jiajun Xu
Scott Garman
Joe Sauer
Scott Garman
Richard Purdie