uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
NATIVELSBSTRING = "universal"
|
|
|
|
|
2016-01-22 17:17:10 +00:00
|
|
|
UNINATIVE_LOADER ?= "${STAGING_DIR}-uninative/${BUILD_ARCH}-linux/lib/${@bb.utils.contains('BUILD_ARCH', 'x86_64', 'ld-linux-x86-64.so.2', 'ld-linux.so.2', d)}"
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
|
2016-01-22 13:05:21 +00:00
|
|
|
UNINATIVE_URL ?= "unset"
|
|
|
|
UNINATIVE_TARBALL ?= "${BUILD_ARCH}-nativesdk-libc.tar.bz2"
|
|
|
|
# Example checksums
|
|
|
|
#UNINATIVE_CHECKSUM[i586] = "dead"
|
|
|
|
#UNINATIVE_CHECKSUM[x86_64] = "dead"
|
|
|
|
UNINATIVE_DLDIR ?= "${COREBASE}"
|
|
|
|
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
addhandler uninative_eventhandler
|
|
|
|
uninative_eventhandler[eventmask] = "bb.event.BuildStarted"
|
|
|
|
|
|
|
|
python uninative_eventhandler() {
|
|
|
|
loader = e.data.getVar("UNINATIVE_LOADER", True)
|
2016-01-22 13:05:21 +00:00
|
|
|
tarball = d.getVar("UNINATIVE_TARBALL", True)
|
|
|
|
tarballdir = d.getVar("UNINATIVE_DLDIR", True)
|
2016-01-25 15:08:23 +00:00
|
|
|
tarballpath = os.path.join(tarballdir, tarball)
|
|
|
|
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
if not os.path.exists(loader):
|
|
|
|
import subprocess
|
2016-01-22 13:05:21 +00:00
|
|
|
|
|
|
|
olddir = os.getcwd()
|
2016-01-25 15:08:23 +00:00
|
|
|
if not os.path.exists(tarballpath):
|
2016-01-22 13:05:21 +00:00
|
|
|
# Copy the data object and override DL_DIR and SRC_URI
|
|
|
|
localdata = bb.data.createCopy(d)
|
|
|
|
|
|
|
|
if d.getVar("UNINATIVE_URL", True) == "unset":
|
|
|
|
bb.fatal("Uninative selected but not configured, please set UNINATIVE_URL")
|
|
|
|
|
|
|
|
chksum = d.getVarFlag("UNINATIVE_CHECKSUM", d.getVar("BUILD_ARCH", True), True)
|
|
|
|
if not chksum:
|
|
|
|
bb.fatal("Uninative selected but not configured correctly, please set UNINATIVE_CHECKSUM[%s]" % d.getVar("BUILD_ARCH", True))
|
|
|
|
|
|
|
|
srcuri = d.expand("${UNINATIVE_URL}${UNINATIVE_TARBALL};md5sum=%s" % chksum)
|
2016-01-25 15:08:23 +00:00
|
|
|
localdata.setVar('FILESPATH', tarballdir)
|
|
|
|
localdata.setVar('DL_DIR', tarballdir)
|
2016-01-22 13:05:21 +00:00
|
|
|
bb.note("Fetching uninative binary shim from %s" % srcuri)
|
|
|
|
fetcher = bb.fetch2.Fetch([srcuri], localdata, cache=False)
|
|
|
|
try:
|
|
|
|
fetcher.download()
|
2016-01-25 15:08:23 +00:00
|
|
|
localpath = fetcher.localpath(srcuri)
|
|
|
|
if localpath != tarballpath and os.path.exists(localpath) and not os.path.exists(tarballpath):
|
|
|
|
os.symlink(localpath, tarballpath)
|
2016-01-22 13:05:21 +00:00
|
|
|
except Exception as exc:
|
|
|
|
bb.fatal("Unable to download uninative tarball: %s" % str(exc))
|
2016-01-22 17:17:10 +00:00
|
|
|
|
2016-01-25 15:07:18 +00:00
|
|
|
cmd = e.data.expand("mkdir -p ${STAGING_DIR}-uninative; cd ${STAGING_DIR}-uninative; tar -xjf ${UNINATIVE_DLDIR}/${UNINATIVE_TARBALL}; ${STAGING_DIR}-uninative/relocate_sdk.py ${STAGING_DIR}-uninative/${BUILD_ARCH}-linux ${UNINATIVE_LOADER} ${UNINATIVE_LOADER} ${STAGING_DIR}-uninative/${BUILD_ARCH}-linux/${bindir_native}/patchelf-uninative")
|
2016-01-22 13:05:21 +00:00
|
|
|
try:
|
|
|
|
subprocess.check_call(cmd, shell=True)
|
|
|
|
except subprocess.CalledProcessError as exc:
|
|
|
|
bb.fatal("Unable to install uninative tarball: %s" % str(exc))
|
|
|
|
os.chdir(olddir)
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
SSTATEPOSTUNPACKFUNCS_append = " uninative_changeinterp"
|
|
|
|
|
2016-01-22 17:17:10 +00:00
|
|
|
PATH_prepend = "${STAGING_DIR}-uninative/${BUILD_ARCH}-linux${bindir_native}:"
|
|
|
|
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
python uninative_changeinterp () {
|
|
|
|
import subprocess
|
|
|
|
import stat
|
|
|
|
import oe.qa
|
|
|
|
|
|
|
|
if not (bb.data.inherits_class('native', d) or bb.data.inherits_class('crosssdk', d) or bb.data.inherits_class('cross', d)):
|
|
|
|
return
|
|
|
|
|
|
|
|
sstateinst = d.getVar('SSTATE_INSTDIR', True)
|
|
|
|
for walkroot, dirs, files in os.walk(sstateinst):
|
|
|
|
for file in files:
|
2016-02-04 08:31:38 +00:00
|
|
|
if file.endswith(".so") or ".so." in file:
|
|
|
|
continue
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
f = os.path.join(walkroot, file)
|
|
|
|
if os.path.islink(f):
|
|
|
|
continue
|
|
|
|
s = os.stat(f)
|
|
|
|
if not ((s[stat.ST_MODE] & stat.S_IXUSR) or (s[stat.ST_MODE] & stat.S_IXGRP) or (s[stat.ST_MODE] & stat.S_IXOTH)):
|
|
|
|
continue
|
|
|
|
elf = oe.qa.ELFFile(f)
|
|
|
|
try:
|
|
|
|
elf.open()
|
2016-02-24 13:31:40 +00:00
|
|
|
except oe.qa.NotELFFileError:
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
continue
|
|
|
|
|
2016-02-23 21:10:13 +00:00
|
|
|
try:
|
|
|
|
subprocess.check_output(("patchelf-uninative", "--set-interpreter",
|
|
|
|
d.getVar("UNINATIVE_LOADER", True), f))
|
|
|
|
except subprocess.CalledProcessError as e:
|
2016-02-11 06:59:42 +00:00
|
|
|
bb.fatal("'%s' failed with exit code %d and the following output:\n%s" %
|
2016-02-23 21:10:13 +00:00
|
|
|
(e.cmd, e.returncode, e.output))
|
uninative: Add uninative - a way of reusing native/cross over multiple distros
These patches are the start of a new idea, a way of allowing a single set of
cross/native sstate to work over mutliple distros, even old ones.
The assumption is that our own C library is basically up to date. We build
and share a small tarball (~2MB) of a prebuilt copy of this along with a
patchelf binary (which sadly is C++ based so libstdc++ is in there). This
tarball can be generated from our usual SDK generation process through
the supplied recipe, uninative-tarball.
At the start of the build, if its not been extracted into the sysroot, this
tarball is extracted there and configured for the specified path.
When we install binaries from a "uninative" sstate feed, we change the
dynamic loader to point at this dynamic loader and C librbary. This works
exactly the same way as our relocatable SDK does. The only real difference
is a switch to use patchelf, so even if the interpreter section is too small,
it can still adjust the binary.
Right now this implements a working proof of concept. If you build the tarball
and place it at the head of the tree (in COREBASE), you can run a build from
sstate and successfully build packages and construct images.
There is some improvement needed, its hardcoded for x86_64 right now, its trivial
to add 32 bit support too. The tarball isn't fetched right now, there is just a
harcoded path assumption and there is no error handling. I haven't figured
out the best delivery mechanism for that yet. BuildStarted is probably not
the right event to hook on either.
I've merged this to illustrate how with a small change, we might make the
native/cross sstate much more reusable and hence improve the accessibility
of lower overhead builds. With this change, its possible the Yocto Project may
be able to support a configured sstate mirror out the box. This also has
positive implications for our developer workflow/SDK improvements.
(From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
|
|
|
}
|