2016-01-22 17:17:10 +00:00
|
|
|
UNINATIVE_LOADER ?= "${STAGING_DIR}-uninative/${BUILD_ARCH}-linux/lib/${@bb.utils.contains('BUILD_ARCH', 'x86_64', 'ld-linux-x86-64.so.2', 'ld-linux.so.2', d)}"
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
|
2016-01-22 13:05:21 +00:00
|
|
|
UNINATIVE_URL ?= "unset"
|
|
|
|
UNINATIVE_TARBALL ?= "${BUILD_ARCH}-nativesdk-libc.tar.bz2"
|
|
|
|
# Example checksums
|
|
|
|
#UNINATIVE_CHECKSUM[i586] = "dead"
|
|
|
|
#UNINATIVE_CHECKSUM[x86_64] = "dead"
|
2016-03-07 12:02:51 +00:00
|
|
|
UNINATIVE_DLDIR ?= "${DL_DIR}/uninative/"
|
2016-01-22 13:05:21 +00:00
|
|
|
|
2016-03-05 08:22:33 +00:00
|
|
|
# https://wiki.debian.org/GCC5
|
|
|
|
# We may see binaries built with gcc5 run or linked into gcc4 environment
|
|
|
|
# so use the older libstdc++ standard for now until we don't support gcc4
|
|
|
|
# on the host system.
|
|
|
|
BUILD_CXXFLAGS_append = " -D_GLIBCXX_USE_CXX11_ABI=0"
|
|
|
|
|
2016-03-12 08:57:07 +00:00
|
|
|
#
|
|
|
|
# icu configure defaults to CXX11 if no -std= option is passed in CXXFLAGS
|
|
|
|
# therefore pass one
|
|
|
|
BUILD_CXXFLAGS_append_pn-icu-native = " -std=c++98"
|
|
|
|
|
2016-03-04 16:48:37 +00:00
|
|
|
addhandler uninative_event_fetchloader
|
|
|
|
uninative_event_fetchloader[eventmask] = "bb.event.BuildStarted"
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
|
2016-03-04 16:48:37 +00:00
|
|
|
addhandler uninative_event_enable
|
|
|
|
uninative_event_enable[eventmask] = "bb.event.ConfigParsed"
|
|
|
|
|
|
|
|
python uninative_event_fetchloader() {
|
|
|
|
"""
|
|
|
|
This event fires on the parent and will try to fetch the tarball if the
|
|
|
|
loader isn't already present.
|
|
|
|
"""
|
2016-02-22 21:06:38 +00:00
|
|
|
|
2016-03-07 12:02:51 +00:00
|
|
|
chksum = d.getVarFlag("UNINATIVE_CHECKSUM", d.getVar("BUILD_ARCH", True), True)
|
|
|
|
if not chksum:
|
|
|
|
bb.fatal("Uninative selected but not configured correctly, please set UNINATIVE_CHECKSUM[%s]" % d.getVar("BUILD_ARCH", True))
|
2016-03-04 16:48:37 +00:00
|
|
|
|
2016-03-07 12:02:51 +00:00
|
|
|
loader = d.getVar("UNINATIVE_LOADER", True)
|
|
|
|
loaderchksum = loader + ".chksum"
|
|
|
|
if os.path.exists(loader) and os.path.exists(loaderchksum):
|
|
|
|
with open(loaderchksum, "r") as f:
|
|
|
|
readchksum = f.read().strip()
|
|
|
|
if readchksum == chksum:
|
|
|
|
return
|
|
|
|
|
|
|
|
import subprocess
|
2016-03-04 16:48:37 +00:00
|
|
|
try:
|
|
|
|
# Save and restore cwd as Fetch.download() does a chdir()
|
|
|
|
olddir = os.getcwd()
|
|
|
|
|
|
|
|
tarball = d.getVar("UNINATIVE_TARBALL", True)
|
2016-03-07 12:02:51 +00:00
|
|
|
tarballdir = os.path.join(d.getVar("UNINATIVE_DLDIR", True), chksum)
|
2016-03-04 16:48:37 +00:00
|
|
|
tarballpath = os.path.join(tarballdir, tarball)
|
2016-01-25 15:08:23 +00:00
|
|
|
|
|
|
|
if not os.path.exists(tarballpath):
|
2016-03-07 12:02:51 +00:00
|
|
|
bb.utils.mkdirhier(tarballdir)
|
2016-01-22 13:05:21 +00:00
|
|
|
if d.getVar("UNINATIVE_URL", True) == "unset":
|
|
|
|
bb.fatal("Uninative selected but not configured, please set UNINATIVE_URL")
|
|
|
|
|
2016-03-04 16:48:37 +00:00
|
|
|
localdata = bb.data.createCopy(d)
|
|
|
|
localdata.setVar('FILESPATH', "")
|
|
|
|
localdata.setVar('DL_DIR', tarballdir)
|
2016-02-22 21:06:38 +00:00
|
|
|
|
2016-03-30 19:48:58 +00:00
|
|
|
srcuri = d.expand("${UNINATIVE_URL}${UNINATIVE_TARBALL};sha256sum=%s" % chksum)
|
2016-03-04 16:48:37 +00:00
|
|
|
bb.note("Fetching uninative binary shim from %s" % srcuri)
|
2016-02-22 21:06:38 +00:00
|
|
|
|
2016-03-04 16:48:37 +00:00
|
|
|
fetcher = bb.fetch2.Fetch([srcuri], localdata, cache=False)
|
|
|
|
fetcher.download()
|
|
|
|
localpath = fetcher.localpath(srcuri)
|
|
|
|
if localpath != tarballpath and os.path.exists(localpath) and not os.path.exists(tarballpath):
|
2016-01-25 15:08:23 +00:00
|
|
|
os.symlink(localpath, tarballpath)
|
2016-03-04 16:48:37 +00:00
|
|
|
|
2016-03-06 10:17:20 +00:00
|
|
|
cmd = d.expand("mkdir -p ${STAGING_DIR}-uninative; cd ${STAGING_DIR}-uninative; tar -xjf ${UNINATIVE_DLDIR}/%s/${UNINATIVE_TARBALL}; ${STAGING_DIR}-uninative/relocate_sdk.py ${STAGING_DIR}-uninative/${BUILD_ARCH}-linux ${UNINATIVE_LOADER} ${UNINATIVE_LOADER} ${STAGING_DIR}-uninative/${BUILD_ARCH}-linux/${bindir_native}/patchelf-uninative ${STAGING_DIR}-uninative/${BUILD_ARCH}-linux${base_libdir_native}/libc*.so" % chksum)
|
2016-03-04 16:48:37 +00:00
|
|
|
subprocess.check_call(cmd, shell=True)
|
|
|
|
|
2016-03-07 12:02:51 +00:00
|
|
|
with open(loaderchksum, "w") as f:
|
|
|
|
f.write(chksum)
|
|
|
|
|
2016-03-07 12:06:53 +00:00
|
|
|
enable_uninative(d)
|
2016-03-04 16:48:37 +00:00
|
|
|
|
|
|
|
except bb.fetch2.BBFetchException as exc:
|
|
|
|
bb.warn("Disabling uninative as unable to fetch uninative tarball: %s" % str(exc))
|
|
|
|
bb.warn("To build your own uninative loader, please bitbake uninative-tarball and set UNINATIVE_TARBALL appropriately.")
|
|
|
|
except subprocess.CalledProcessError as exc:
|
|
|
|
bb.warn("Disabling uninative as unable to install uninative tarball: %s" % str(exc))
|
|
|
|
bb.warn("To build your own uninative loader, please bitbake uninative-tarball and set UNINATIVE_TARBALL appropriately.")
|
|
|
|
finally:
|
|
|
|
os.chdir(olddir)
|
|
|
|
}
|
|
|
|
|
|
|
|
python uninative_event_enable() {
|
|
|
|
"""
|
|
|
|
This event handler is called in the workers and is responsible for setting
|
|
|
|
up uninative if a loader is found.
|
|
|
|
"""
|
2016-03-07 12:06:53 +00:00
|
|
|
enable_uninative(d)
|
|
|
|
}
|
2016-03-04 16:48:37 +00:00
|
|
|
|
2016-03-07 12:06:53 +00:00
|
|
|
def enable_uninative(d):
|
2016-03-04 16:48:37 +00:00
|
|
|
loader = d.getVar("UNINATIVE_LOADER", True)
|
|
|
|
if os.path.exists(loader):
|
2016-02-22 21:06:38 +00:00
|
|
|
bb.debug(2, "Enabling uninative")
|
|
|
|
d.setVar("NATIVELSBSTRING", "universal")
|
|
|
|
d.appendVar("SSTATEPOSTUNPACKFUNCS", " uninative_changeinterp")
|
|
|
|
d.prependVar("PATH", "${STAGING_DIR}-uninative/${BUILD_ARCH}-linux${bindir_native}:")
|
2016-01-22 17:17:10 +00:00
|
|
|
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
python uninative_changeinterp () {
|
|
|
|
import subprocess
|
|
|
|
import stat
|
|
|
|
import oe.qa
|
|
|
|
|
|
|
|
if not (bb.data.inherits_class('native', d) or bb.data.inherits_class('crosssdk', d) or bb.data.inherits_class('cross', d)):
|
|
|
|
return
|
|
|
|
|
|
|
|
sstateinst = d.getVar('SSTATE_INSTDIR', True)
|
|
|
|
for walkroot, dirs, files in os.walk(sstateinst):
|
|
|
|
for file in files:
|
2016-02-04 08:31:38 +00:00
|
|
|
if file.endswith(".so") or ".so." in file:
|
|
|
|
continue
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
f = os.path.join(walkroot, file)
|
|
|
|
if os.path.islink(f):
|
|
|
|
continue
|
|
|
|
s = os.stat(f)
|
|
|
|
if not ((s[stat.ST_MODE] & stat.S_IXUSR) or (s[stat.ST_MODE] & stat.S_IXGRP) or (s[stat.ST_MODE] & stat.S_IXOTH)):
|
|
|
|
continue
|
|
|
|
elf = oe.qa.ELFFile(f)
|
|
|
|
try:
|
|
|
|
elf.open()
|
2016-02-24 13:31:40 +00:00
|
|
|
except oe.qa.NotELFFileError:
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
continue
|
2016-03-24 15:43:48 +00:00
|
|
|
if not elf.isDynamic():
|
|
|
|
continue
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
|
2016-02-23 21:10:13 +00:00
|
|
|
try:
|
|
|
|
subprocess.check_output(("patchelf-uninative", "--set-interpreter",
|
2016-03-24 15:43:46 +00:00
|
|
|
d.getVar("UNINATIVE_LOADER", True), f),
|
|
|
|
stderr=subprocess.STDOUT)
|
2016-02-23 21:10:13 +00:00
|
|
|
except subprocess.CalledProcessError as e:
|
2016-02-11 06:59:42 +00:00
|
|
|
bb.fatal("'%s' failed with exit code %d and the following output:\n%s" %
|
2016-02-23 21:10:13 +00:00
|
|
|
(e.cmd, e.returncode, e.output))
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
}
|