The path to where to install and find the sysroot components is used
in many places. This warrants it to get its own variable.
(From OE-Core rev: 70a84b525470f72339568409daf84845904e4cab)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've had a lot of users running into RSS issues where -initial recipes
were being installed into sysroots alongside their counterparts and
causing overlapping files issues.
In general this was through do_build dependencies. Such dependencies are
bad in general and I'd encourage people to compare the taskgraphs with
using a more specific dependency like do_populate_sysroot, do_image_complete
or do_deploy as often the more specific dependency will result in a much
cleaner build.
Regardless, we don't want -initial dependencies getting in the way like
this and there are cases a do_build dependency could make sense.
Deleting the do_build task in these cases makes sense since this is not
a build "endpoint" we'd ever want a user to use, its a behind the scenes
piece of bootstrappping.
Unfortunately to make this work, we need a newer bitbake version which
has a bitbake bug fixed.
(From OE-Core rev: 04c053d42ab05f77b2d1ca93a0fabae44073d57e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We used to need to build gcc-cross-initial against a bare sysroot to avoid
contamination. With RSS, we no longer need to do this since the recipe sysroot
is already bare. We can therefore simply point at that and drop this code.
(From OE-Core rev: f70603887f823c14030bb738c4951d7aa3f022db)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we stashed the gcc build directory for use in generating the various runtimes
we were being lazy and just used the staging directory. With recipe specific
sysroots this means we're copying a large chunk of data around with the cross
compiler which we don't really need in most cases.
Separate out the data into its own task and inject this into the configure
step. We have to do that here since autotools will wipe out ${B} if it thinks
we're rebuilding and we therefore have to time its recreation after that.
This also takes the opportunity to remove some pointless (as far as I can tell)
conditionals from the do_install code.
(From OE-Core rev: dcf15ccf3cc9d55e77228ba8d526f967fc9791b4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch is comparatively large and invasive. It does only do one thing, switching the
system to build using recipe specific sysroots and where changes could be isolated from it,
that has been done.
With the current single sysroot approach, its possible for software to find things which
aren't in their dependencies. This leads to a determinism problem and is a growing issue in
several of the market segments where OE makes sense. The way to solve this problem for OE is
to have seperate sysroots for each recipe and these will only contain the dependencies for
that recipe.
Its worth noting that this is not task specific sysroots and that OE's dependencies do vary
enormously by task. This did result in some implementation challenges. There is nothing stopping
the implementation of task specific sysroots at some later point based on this work but
that as deemed a bridge too far right now.
Implementation details:
* Rather than installing the sysroot artefacts into a combined sysroots, they are now placed in
TMPDIR/sysroot-components/PACKAGE_ARCH/PN.
* WORKDIR/recipe-sysroot and WORKDIR/recipe-sysroot-native are built by hardlinking in files
from the sysroot-component trees. These new directories are known as RECIPE_SYSROOT and
RECIPE_SYSROOT_NATIVE.
* This construction is primarily done by a new do_prepare_recipe_sysroot task which runs
before do_configure and consists of a call to the extend_recipe_sysroot function.
* Other tasks need things in the sysroot before/after this, e.g. do_patch needs quilt-native
and do_package_write_deb needs dpkg-native. The code therefore inspects the dependencies
for each task and adds extend_recipe_sysroot as a prefunc if it has populate_sysroot
dependencies.
* We have to do a search/replace 'fixme' operation on the files installed into the sysroot to
change hardcoded paths into the correct ones. We create a fixmepath file in the component
directory which lists the files which need this operation.
* Some files have "postinstall" commands which need to run against them, e.g. gdk-pixbuf each
time a new loader is added. These are handled by adding files in bindir with the name
prefixed by "postinst-" and are run in each sysroot as its created if they're present.
This did mean most sstate postinstalls have to be rewritten but there shouldn't be many of them.
* Since a recipe can have multiple tasks and these tasks can run against each other at the same
time we have to have a lock when we perform write operations against the sysroot. We also have
to maintain manifests of what we install against a task checksum of the dependency. If the
checksum changes, we remove its files and then add the new ones.
* The autotools logic for filtering the view of m4 files is no longer needed (and was the model
for the way extend_recipe_sysroot works).
* For autotools, we used to build a combined m4 macros directory which had both the native and
target m4 files. We can no longer do this so we use the target sysroot as the default and add
the native sysroot as an extra backup include path. If we don't do this, we'd have to build
target pkg-config before we could built anything using pkg-config for example (ditto gettext).
Such dependencies would be painful so we haven't required that.
* PKDDATA_DIR was moved out the sysroot and works as before using sstate to build a hybrid copy
for each machine. The paths therefore changed, the behaviour did not.
* The ccache class had to be reworked to function with rss.
* The TCBOOTSTRAP sysroot for compiler bootstrap is no longer needed but the -initial data
does have to be filtered out from the main recipe sysroots. Putting "-initial" in a normal
recipe name therefore remains a bad idea.
* The logic in insane needed tweaks to deal with the new path layout, as did the debug source
file extraction code in package.bbclass.
* The logic in sstate.bbclass had to be rewritten since it previously only performed search and
replace on extracted sstate and we now need this to happen even if the compiled path was
"correct". This in theory could cause a mild performance issue but since the sysroot data
was the main data that needed this and we'd have to do it there regardless with rss, I've opted
just to change the way the class for everything. The built output used to build the sstate output
is now retained and installed rather than deleted.
* The search and replace logic used in sstate objects also seemed weak/incorrect and didn't hold
up against testing. This has been rewritten too. There are some assumptions made about paths, we
save the 'proper' search and replace operations to fixmepath.cmd but then ignore this. What is
here works but is a little hardcoded and an area for future improvement.
* In order to work with eSDK we need a way to build something that looks like the old style sysroot.
"bitbake build-sysroots" will construct such a sysroot based on everything in the components
directory that matches the current MACHINE. It will allow transition of external tools and can
built target or native variants or both. It also supports a clean task. I'd suggest not relying on
this for anything other than transitional purposes though. To see XXX in that sysroot, you'd have
to have built that in a previous bitbake invocation.
* pseudo is run out of its components directory. This is fine as its statically linked.
* The hacks for wayland to see allarch dependencies in the multilib case are no longer needed
and can be dropped.
* wic needed more extensive changes to work with rss and the fixes are in a separate commit series
* Various oe-selftest tweaks were needed since tests did assume the location to binaries and the
combined sysroot in several cases.
* Most missing dependencies this work found have been sent out as separate patches as they were found
but a few tweaks are still included here.
* A late addition is that extend_recipe_sysroot became multilib aware and able to populate multilib
sysroots. I had hoped not to have to add that complexity but the meta-environment recipe forced my
hand. That implementation can probably be neater but this is on the list of things to cleanup later
at this point.
In summary, the impact people will likely see after this change:
* Recipes may fail with missing dependencies, particularly native tools like gettext-native,
glib-2.0-native and libxml2.0-native. Some hosts have these installed and will mask these errors
* Any recipe/class using SSTATEPOSTINSTFUNCS will need that code rewriting into a postinst
* There was a separate patch series dealing with roots postinst native dependency issues. Any postinst
which expects native tools at rootfs time will need to mark that dependency with PACKAGE_WRITE_DEPS.
There could well be other issues. This has been tested repeatedly against our autobuilders and oe-selftest
and issues found have been fixed. We believe at least OE-Core is in good shape but that doesn't mean
we've found all the issues.
Also, the logging is a bit chatty at the moment. It does help if something goes wrong and goes to the
task logfiles, not the console so I've intentionally left this like that for now. We can turn it down
easily enough in due course.
(From OE-Core rev: 809746f56df4b91af014bf6a3f28997d6698ac78)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When changing SDKMACHINE, we may encounter an error forcing us to wipe the TMP folder.
Since only SDK_ARCH is captured in the PN of the crosssdk recipes, changes to SDK_OS
result in conflicts. Eventually we hit the error:
ERROR: ...: The recipe <...> is trying to install files into a shared area when those files already exist.
The build has stopped as continuing in this scenario WILL break things
This patchset addresses the problem by SDK_SYS as the recipe name suffix instead
of SDK_ARCH.
[YOCTO #9281]
(From OE-Core rev: d2eccccb70e809d482c493922f23aef4409cfd82)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Configuring gcc with --enable-target-optspace (which causes gcc to
append "-g -Os" to the default CFLAGS_FOR_TARGET and so force libgcc
etc target libraries to always be optimised for size) dates back to
the very first commit in oe-core git in 2005 (for gcc 3.4.3).
Configuring gcc with --enable-target-optspace is not done widely
elsewhere (it's not used for Ubuntu or Fedora host gcc, the Linaro
binary toolchain or in Buildroot since early 2015). Sometime around
gcc 4.5.x it caused problems for powerpc and so was disabled for that
architecture:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=43810
This patch removes --enable-target-optspace completely (ie powerpc is
no longer a special case) and allows optimisation of libgcc etc to be
controlled directly by the flags present in TARGET_CFLAGS.
(From OE-Core rev: 686b266506a1a56fb68ab0f00d658787dd7fe4ce)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turns out writing the same list of packaging tasks multiple
times in multiple places is error prone. Move this to a new class
'nopackages", migrate existing users and add glibc-initial and
libgcc-initial since we don't want packages for those recipes.
This means the sstate for those recipes won't be installed, saving
small amounts of build time and bandwidth.
A reference to the old package_write task is also dropped.
(From OE-Core rev: cece583d58f82a50c3a4eac876eb326ac3b8f5e5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a project is configure to use sstate cache and has the host tool sysroot
cleaned, gcc-cross-initial may fail to be configured due to lack of
gnu-configize tool.
gcc-cross-initial recipe has autotools dependency inhibited, and the same flag
variable also excludes the gnu-config-native. Though there is an indirect
dependency through libmpc-native, it's not safe with sstate cache being used.
Moreover, gnu-config-native requires a perl package from autoconf-native to
run, otherwise it will fail with "Can't locate Autom4te/ChannelDefs.pm in @INC"
message.
This patch makes both dependencies explicit for gcc-cross-initial's
configuration.
(From OE-Core rev: 18a913e54b40a1654d0967290088be5e7fcdd6f6)
Signed-off-by: Yuanjie Huang <yuanjie.huang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The gcc default, bss-plt, will cause errors when using the prelinker. All
other distributions that I am aware of are using the the secure-plt. For an
explanation of the differences, the gcc docs:
Current PowerPC GCC accepts a `-msecure-plt' option that generates code
capable of using a newer PLT and GOT layout that has the security
advantage of no executable section ever needing to be writable and no
writable section ever being executable. PowerPC ld will generate this
layout, including stubs to access the PLT, if all input files (including
startup and static libraries) were compiled with `-msecure-plt'.
`--bss-plt' forces the old BSS PLT (and GOT layout) which can give
slightly better performance.
The security of the new PLT and ability to run the prelinker outweigh
any performance penalty.
The secure-plt is enabled by default. The old bss-plt can be enabled by
selecting 'bssplt' in the DISTRO_FEATURES.
(From OE-Core rev: 70c55aada1101a5c687cdaa79f370fa4530b39d9)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Various pieces of the code assume that the --sysroot option gets passed
into the compiler tools. By having a "sane" default, we don't always
spot when this occurs and this can later show up as breakage in sstate,
or in usage of the external toolchain.
We've long since talked about poisoning the default such that it will
break unless the correct option is specified. This patch does just that.
If this patch causes something to fail to build, it most likely means
the various compiler flags and commands are not correctly being passed
through to the underlying piece of software and that there is a real
problem that needs fixing, its not the fault of this patch.
(From OE-Core rev: 04b725511a505c582a3abdf63d096967f0320779)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
musl e.g. is configured to not use fixed-include
which is an improvement btw. but libgcc-initial configure
has tests which probe for limits.h and since we put
it in include-fixed/ dir and that dir does not appear
in gcc's internal default search path the configure tests
for CPP detection fail and libgcc-initial can not be compiled.
(From OE-Core rev: 3bdc225a9e622e9d594944833964fe396200db01)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This variable is used to ensure the proper version of --with-float=FOO
is passed to gcc's configure script. gcc also has a --with-fpu=FOO
option that means something different. To avoid confusion, change the
names to be consistent.
(From OE-Core rev: c17d883fa99b6967d83c3796d22fc0c1dbe704e6)
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Consistent use of whitespace in multi-line assignment, with special
focus on OECONF modifications. Quotes on separate lines, four-space
indentation, one value per line.
(From OE-Core rev: d971db8b2259e4c35b871cccf130fba193849560)
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We already indicate our intentions to use ld.bfd by
specifying it in configure using --with-ld which works
ok unless here where we manually create symlinks to
binutils-cross components, when we use ld-is-gold feature
default ld points to gold and this symlinking has to be
aware of the fact that we configured binutils and gcc-cross to use
gold as default ld but gcc-cross-initial uses BFD ld
This would be visible when using gold and rebuilding
eglibc
(From OE-Core rev: 77cab553ee6caa940e21cca46ff134f84e65c171)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows them to co-exist together in the native sysroot, with one
set of cross tools per target architecture.
(From OE-Core rev: a2c5509520d5c3e082f55844e6545d0309565f8f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The base_contains is kept as a compatibility method and we ought to
not use it in OE-Core so we can remove it from base metadata in
future.
(From OE-Core rev: d83b16dbf0862be387f84228710cb165c6d2b03b)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its useful to separate out the native (cross) binaries from the target
compilation. We already do this for libgcc, this now takes the same
approach for -initial.
(From OE-Core rev: 8a2aaf8b6bdca2d28c0047093c7f668750d57666)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If sstate-inputdirs and sstate-outputdirs don't match with ending '/'
characters, the manifest file can end up corrupted. This change
ensures the metadata is consistent in ending do_populate_root tasks
with this character to avoid manifest file corruption.
(From OE-Core rev: 3910eaf88d14904eef85b9e391387547df7fc54e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now glibc can be compiled with gcc-cross-initial therefore prepare
the stage to drop gcc-cross-intermediate
Also drop arm-nolibfloat.patch should not be needed anymore
half of changes in this patch are meant for OABI which we dont
use anymore
(From OE-Core rev: 30617bde61a3b0a0944b49a0c9fb7159dacbb19f)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we don't do this, a stale limits.h may be detected in STAGING_DIR_TARGET
which would result in a different limits.h getting generated by gcc-cross-initial
that references it. The referenced limits.h will then not get found by eglibc-initial
causing rather strange build failures.
The simplest solution is to create a temporary sysroot containing only the things
gcc-cross-initial should care about and this results in a correct limits.h file
regardless of what else may have been built.
(From OE-Core rev: 9c304eae0724474902fe2f3150adc6af115af9ba)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
currently gcc installs a limits.h which references to another
limits.h which it expects from target sysroot and that header
in target sysroot will come from eglibc. So we need to break
this catch-22 and hence we install a self sufficient limits.h
which is then happy when referenced and doesnt complain about
missing limits.h from target sysroot.
This is mostly used when eglibc-initial configure is run
(From OE-Core rev: eeb445ecbc651ad614a4fc492039bdad0747d45d)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using gold as default linker in final toolchain
gcc-cross-initial still needs to use BFD ld since it
will link eglibc-initial
(From OE-Core rev: f643f886b16f586426c744afde080a99d71a9d58)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If EXTRA_OECONF_FPU is left set, certain ARM variables related to hard-float
can get pulled in and trigger rebuilds of the crosssdk code. The best solution
is to simply force the variable to a known correct value for the SDK targets
currently supported in the same way as TARGET_FPU.
There is some slight rearrangement of the gcc code to ensure the variable is
always used to call the fpu function.
(From OE-Core rev: 410990445ada8cdcfaec4e6fa5791cee9a5b8983)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
this option by default points to /usr/local no matter
what so we cant let it sit on sidelines otherwise it
will access host machine's /usr/local which may not
be desired. So disable this option. This also helps in making
gcc's shared state more consistent
(From OE-Core rev: eee3658366e1ae9d3e429b3d3c968938d8d0f00e)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we stash the libgcc install tree and then reuse that
to populate libgcc recipe later. This mechanism does not work
for gcc 4.7/trunk since now libstdc++ needs access to build tree
of libgcc. This patch stashes the gcc-cross build tree
and then reuses this in libgcc as well as in gcc-runtime
recipe builds.
Now we build libgcc in the libgcc recipe instead of just
using the prebuilt install tree
core-image-minimal build/run tested on all qemu machines
(From OE-Core rev: 7cf9f0597648c0bdaa080976d74acfbfc4c8443d)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Right now for cross recipes e.g. gcc-cross and binutils-cross
we specify --disable-nls .... --enable-nls on configure cmdline
the --enable-nls coming from gettext bbclass.
So we disable nls for all cross inheriting recipes in gettext
bbclass and then we remove the extra --disable-nls in gcc-cross
and binutils-cross
This patch needs testing. Please help
(From OE-Core rev: d66b379f809b9c75981848fcc71ed5de13382bf7)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We force the C locale when running builds for determinstic error messages. We
therefore have no need to NLS support in binutils cross or gcc cross.
We also don't need the standard base/autotools dependencies for our
toolchain components since we don't autoreconf these.
This patch turns off nls and cleans up some of the dependencies resulting
in a slightly less convoluted set of build dependencies.
(From OE-Core rev: 54a3e2ee37003fc56af0339f857b0b6442790c26)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having one monolithic packages directory makes it hard to find things
and is generally overwhelming. This commit splits it into several
logical sections roughly based on function, recipes.txt gives more
information about the classifications used.
The opportunity is also used to switch from "packages" to "recipes"
as used in OpenEmbedded as the term "packages" can be confusing to
people and has many different meanings.
Not all recipes have been classified yet, this is just a first pass
at separating things out. Some packages are moved to meta-extras as
they're no longer actively used or maintained.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Peter Kjellerstedt
Richard Purdie
Juro Bystricky
Andre McCurdy
Yuanjie Huang
Mark Hatle
Khem Raj
Peter A. Bigot
Otavio Salvador
Dexuan Cui
Richard Purdie