generic-poky/meta/classes/uninative.bbclass

93 lines
4.0 KiB
Plaintext
Raw Normal View History

uninative: Add uninative - a way of reusing native/cross over multiple distros These patches are the start of a new idea, a way of allowing a single set of cross/native sstate to work over mutliple distros, even old ones. The assumption is that our own C library is basically up to date. We build and share a small tarball (~2MB) of a prebuilt copy of this along with a patchelf binary (which sadly is C++ based so libstdc++ is in there). This tarball can be generated from our usual SDK generation process through the supplied recipe, uninative-tarball. At the start of the build, if its not been extracted into the sysroot, this tarball is extracted there and configured for the specified path. When we install binaries from a "uninative" sstate feed, we change the dynamic loader to point at this dynamic loader and C librbary. This works exactly the same way as our relocatable SDK does. The only real difference is a switch to use patchelf, so even if the interpreter section is too small, it can still adjust the binary. Right now this implements a working proof of concept. If you build the tarball and place it at the head of the tree (in COREBASE), you can run a build from sstate and successfully build packages and construct images. There is some improvement needed, its hardcoded for x86_64 right now, its trivial to add 32 bit support too. The tarball isn't fetched right now, there is just a harcoded path assumption and there is no error handling. I haven't figured out the best delivery mechanism for that yet. BuildStarted is probably not the right event to hook on either. I've merged this to illustrate how with a small change, we might make the native/cross sstate much more reusable and hence improve the accessibility of lower overhead builds. With this change, its possible the Yocto Project may be able to support a configured sstate mirror out the box. This also has positive implications for our developer workflow/SDK improvements. (From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
NATIVELSBSTRING = "universal"
UNINATIVE_LOADER ?= "${STAGING_DIR}-uninative/${BUILD_ARCH}-linux/lib/${@bb.utils.contains('BUILD_ARCH', 'x86_64', 'ld-linux-x86-64.so.2', 'ld-linux.so.2', d)}"
uninative: Add uninative - a way of reusing native/cross over multiple distros These patches are the start of a new idea, a way of allowing a single set of cross/native sstate to work over mutliple distros, even old ones. The assumption is that our own C library is basically up to date. We build and share a small tarball (~2MB) of a prebuilt copy of this along with a patchelf binary (which sadly is C++ based so libstdc++ is in there). This tarball can be generated from our usual SDK generation process through the supplied recipe, uninative-tarball. At the start of the build, if its not been extracted into the sysroot, this tarball is extracted there and configured for the specified path. When we install binaries from a "uninative" sstate feed, we change the dynamic loader to point at this dynamic loader and C librbary. This works exactly the same way as our relocatable SDK does. The only real difference is a switch to use patchelf, so even if the interpreter section is too small, it can still adjust the binary. Right now this implements a working proof of concept. If you build the tarball and place it at the head of the tree (in COREBASE), you can run a build from sstate and successfully build packages and construct images. There is some improvement needed, its hardcoded for x86_64 right now, its trivial to add 32 bit support too. The tarball isn't fetched right now, there is just a harcoded path assumption and there is no error handling. I haven't figured out the best delivery mechanism for that yet. BuildStarted is probably not the right event to hook on either. I've merged this to illustrate how with a small change, we might make the native/cross sstate much more reusable and hence improve the accessibility of lower overhead builds. With this change, its possible the Yocto Project may be able to support a configured sstate mirror out the box. This also has positive implications for our developer workflow/SDK improvements. (From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
UNINATIVE_URL ?= "unset"
UNINATIVE_TARBALL ?= "${BUILD_ARCH}-nativesdk-libc.tar.bz2"
# Example checksums
#UNINATIVE_CHECKSUM[i586] = "dead"
#UNINATIVE_CHECKSUM[x86_64] = "dead"
UNINATIVE_DLDIR ?= "${COREBASE}"
uninative: Add uninative - a way of reusing native/cross over multiple distros These patches are the start of a new idea, a way of allowing a single set of cross/native sstate to work over mutliple distros, even old ones. The assumption is that our own C library is basically up to date. We build and share a small tarball (~2MB) of a prebuilt copy of this along with a patchelf binary (which sadly is C++ based so libstdc++ is in there). This tarball can be generated from our usual SDK generation process through the supplied recipe, uninative-tarball. At the start of the build, if its not been extracted into the sysroot, this tarball is extracted there and configured for the specified path. When we install binaries from a "uninative" sstate feed, we change the dynamic loader to point at this dynamic loader and C librbary. This works exactly the same way as our relocatable SDK does. The only real difference is a switch to use patchelf, so even if the interpreter section is too small, it can still adjust the binary. Right now this implements a working proof of concept. If you build the tarball and place it at the head of the tree (in COREBASE), you can run a build from sstate and successfully build packages and construct images. There is some improvement needed, its hardcoded for x86_64 right now, its trivial to add 32 bit support too. The tarball isn't fetched right now, there is just a harcoded path assumption and there is no error handling. I haven't figured out the best delivery mechanism for that yet. BuildStarted is probably not the right event to hook on either. I've merged this to illustrate how with a small change, we might make the native/cross sstate much more reusable and hence improve the accessibility of lower overhead builds. With this change, its possible the Yocto Project may be able to support a configured sstate mirror out the box. This also has positive implications for our developer workflow/SDK improvements. (From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
addhandler uninative_eventhandler
uninative_eventhandler[eventmask] = "bb.event.BuildStarted"
python uninative_eventhandler() {
loader = e.data.getVar("UNINATIVE_LOADER", True)
tarball = d.getVar("UNINATIVE_TARBALL", True)
tarballdir = d.getVar("UNINATIVE_DLDIR", True)
tarballpath = os.path.join(tarballdir, tarball)
uninative: Add uninative - a way of reusing native/cross over multiple distros These patches are the start of a new idea, a way of allowing a single set of cross/native sstate to work over mutliple distros, even old ones. The assumption is that our own C library is basically up to date. We build and share a small tarball (~2MB) of a prebuilt copy of this along with a patchelf binary (which sadly is C++ based so libstdc++ is in there). This tarball can be generated from our usual SDK generation process through the supplied recipe, uninative-tarball. At the start of the build, if its not been extracted into the sysroot, this tarball is extracted there and configured for the specified path. When we install binaries from a "uninative" sstate feed, we change the dynamic loader to point at this dynamic loader and C librbary. This works exactly the same way as our relocatable SDK does. The only real difference is a switch to use patchelf, so even if the interpreter section is too small, it can still adjust the binary. Right now this implements a working proof of concept. If you build the tarball and place it at the head of the tree (in COREBASE), you can run a build from sstate and successfully build packages and construct images. There is some improvement needed, its hardcoded for x86_64 right now, its trivial to add 32 bit support too. The tarball isn't fetched right now, there is just a harcoded path assumption and there is no error handling. I haven't figured out the best delivery mechanism for that yet. BuildStarted is probably not the right event to hook on either. I've merged this to illustrate how with a small change, we might make the native/cross sstate much more reusable and hence improve the accessibility of lower overhead builds. With this change, its possible the Yocto Project may be able to support a configured sstate mirror out the box. This also has positive implications for our developer workflow/SDK improvements. (From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
if not os.path.exists(loader):
import subprocess
olddir = os.getcwd()
if not os.path.exists(tarballpath):
# Copy the data object and override DL_DIR and SRC_URI
localdata = bb.data.createCopy(d)
if d.getVar("UNINATIVE_URL", True) == "unset":
bb.fatal("Uninative selected but not configured, please set UNINATIVE_URL")
chksum = d.getVarFlag("UNINATIVE_CHECKSUM", d.getVar("BUILD_ARCH", True), True)
if not chksum:
bb.fatal("Uninative selected but not configured correctly, please set UNINATIVE_CHECKSUM[%s]" % d.getVar("BUILD_ARCH", True))
srcuri = d.expand("${UNINATIVE_URL}${UNINATIVE_TARBALL};md5sum=%s" % chksum)
localdata.setVar('FILESPATH', tarballdir)
localdata.setVar('DL_DIR', tarballdir)
bb.note("Fetching uninative binary shim from %s" % srcuri)
fetcher = bb.fetch2.Fetch([srcuri], localdata, cache=False)
try:
fetcher.download()
localpath = fetcher.localpath(srcuri)
if localpath != tarballpath and os.path.exists(localpath) and not os.path.exists(tarballpath):
os.symlink(localpath, tarballpath)
except Exception as exc:
bb.fatal("Unable to download uninative tarball: %s" % str(exc))
cmd = e.data.expand("mkdir -p ${STAGING_DIR}-uninative; cd ${STAGING_DIR}-uninative; tar -xjf ${UNINATIVE_DLDIR}/${UNINATIVE_TARBALL}; ${STAGING_DIR}-uninative/relocate_sdk.py ${STAGING_DIR}-uninative/${BUILD_ARCH}-linux ${UNINATIVE_LOADER} ${UNINATIVE_LOADER} ${STAGING_DIR}-uninative/${BUILD_ARCH}-linux/${bindir_native}/patchelf-uninative")
try:
subprocess.check_call(cmd, shell=True)
except subprocess.CalledProcessError as exc:
bb.fatal("Unable to install uninative tarball: %s" % str(exc))
os.chdir(olddir)
uninative: Add uninative - a way of reusing native/cross over multiple distros These patches are the start of a new idea, a way of allowing a single set of cross/native sstate to work over mutliple distros, even old ones. The assumption is that our own C library is basically up to date. We build and share a small tarball (~2MB) of a prebuilt copy of this along with a patchelf binary (which sadly is C++ based so libstdc++ is in there). This tarball can be generated from our usual SDK generation process through the supplied recipe, uninative-tarball. At the start of the build, if its not been extracted into the sysroot, this tarball is extracted there and configured for the specified path. When we install binaries from a "uninative" sstate feed, we change the dynamic loader to point at this dynamic loader and C librbary. This works exactly the same way as our relocatable SDK does. The only real difference is a switch to use patchelf, so even if the interpreter section is too small, it can still adjust the binary. Right now this implements a working proof of concept. If you build the tarball and place it at the head of the tree (in COREBASE), you can run a build from sstate and successfully build packages and construct images. There is some improvement needed, its hardcoded for x86_64 right now, its trivial to add 32 bit support too. The tarball isn't fetched right now, there is just a harcoded path assumption and there is no error handling. I haven't figured out the best delivery mechanism for that yet. BuildStarted is probably not the right event to hook on either. I've merged this to illustrate how with a small change, we might make the native/cross sstate much more reusable and hence improve the accessibility of lower overhead builds. With this change, its possible the Yocto Project may be able to support a configured sstate mirror out the box. This also has positive implications for our developer workflow/SDK improvements. (From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
}
SSTATEPOSTUNPACKFUNCS_append = " uninative_changeinterp"
PATH_prepend = "${STAGING_DIR}-uninative/${BUILD_ARCH}-linux${bindir_native}:"
uninative: Add uninative - a way of reusing native/cross over multiple distros These patches are the start of a new idea, a way of allowing a single set of cross/native sstate to work over mutliple distros, even old ones. The assumption is that our own C library is basically up to date. We build and share a small tarball (~2MB) of a prebuilt copy of this along with a patchelf binary (which sadly is C++ based so libstdc++ is in there). This tarball can be generated from our usual SDK generation process through the supplied recipe, uninative-tarball. At the start of the build, if its not been extracted into the sysroot, this tarball is extracted there and configured for the specified path. When we install binaries from a "uninative" sstate feed, we change the dynamic loader to point at this dynamic loader and C librbary. This works exactly the same way as our relocatable SDK does. The only real difference is a switch to use patchelf, so even if the interpreter section is too small, it can still adjust the binary. Right now this implements a working proof of concept. If you build the tarball and place it at the head of the tree (in COREBASE), you can run a build from sstate and successfully build packages and construct images. There is some improvement needed, its hardcoded for x86_64 right now, its trivial to add 32 bit support too. The tarball isn't fetched right now, there is just a harcoded path assumption and there is no error handling. I haven't figured out the best delivery mechanism for that yet. BuildStarted is probably not the right event to hook on either. I've merged this to illustrate how with a small change, we might make the native/cross sstate much more reusable and hence improve the accessibility of lower overhead builds. With this change, its possible the Yocto Project may be able to support a configured sstate mirror out the box. This also has positive implications for our developer workflow/SDK improvements. (From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
python uninative_changeinterp () {
import subprocess
import stat
import oe.qa
if not (bb.data.inherits_class('native', d) or bb.data.inherits_class('crosssdk', d) or bb.data.inherits_class('cross', d)):
return
sstateinst = d.getVar('SSTATE_INSTDIR', True)
for walkroot, dirs, files in os.walk(sstateinst):
for file in files:
if file.endswith(".so") or ".so." in file:
continue
uninative: Add uninative - a way of reusing native/cross over multiple distros These patches are the start of a new idea, a way of allowing a single set of cross/native sstate to work over mutliple distros, even old ones. The assumption is that our own C library is basically up to date. We build and share a small tarball (~2MB) of a prebuilt copy of this along with a patchelf binary (which sadly is C++ based so libstdc++ is in there). This tarball can be generated from our usual SDK generation process through the supplied recipe, uninative-tarball. At the start of the build, if its not been extracted into the sysroot, this tarball is extracted there and configured for the specified path. When we install binaries from a "uninative" sstate feed, we change the dynamic loader to point at this dynamic loader and C librbary. This works exactly the same way as our relocatable SDK does. The only real difference is a switch to use patchelf, so even if the interpreter section is too small, it can still adjust the binary. Right now this implements a working proof of concept. If you build the tarball and place it at the head of the tree (in COREBASE), you can run a build from sstate and successfully build packages and construct images. There is some improvement needed, its hardcoded for x86_64 right now, its trivial to add 32 bit support too. The tarball isn't fetched right now, there is just a harcoded path assumption and there is no error handling. I haven't figured out the best delivery mechanism for that yet. BuildStarted is probably not the right event to hook on either. I've merged this to illustrate how with a small change, we might make the native/cross sstate much more reusable and hence improve the accessibility of lower overhead builds. With this change, its possible the Yocto Project may be able to support a configured sstate mirror out the box. This also has positive implications for our developer workflow/SDK improvements. (From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
f = os.path.join(walkroot, file)
if os.path.islink(f):
continue
s = os.stat(f)
if not ((s[stat.ST_MODE] & stat.S_IXUSR) or (s[stat.ST_MODE] & stat.S_IXGRP) or (s[stat.ST_MODE] & stat.S_IXOTH)):
continue
elf = oe.qa.ELFFile(f)
try:
elf.open()
except oe.qa.NotELFFileError:
uninative: Add uninative - a way of reusing native/cross over multiple distros These patches are the start of a new idea, a way of allowing a single set of cross/native sstate to work over mutliple distros, even old ones. The assumption is that our own C library is basically up to date. We build and share a small tarball (~2MB) of a prebuilt copy of this along with a patchelf binary (which sadly is C++ based so libstdc++ is in there). This tarball can be generated from our usual SDK generation process through the supplied recipe, uninative-tarball. At the start of the build, if its not been extracted into the sysroot, this tarball is extracted there and configured for the specified path. When we install binaries from a "uninative" sstate feed, we change the dynamic loader to point at this dynamic loader and C librbary. This works exactly the same way as our relocatable SDK does. The only real difference is a switch to use patchelf, so even if the interpreter section is too small, it can still adjust the binary. Right now this implements a working proof of concept. If you build the tarball and place it at the head of the tree (in COREBASE), you can run a build from sstate and successfully build packages and construct images. There is some improvement needed, its hardcoded for x86_64 right now, its trivial to add 32 bit support too. The tarball isn't fetched right now, there is just a harcoded path assumption and there is no error handling. I haven't figured out the best delivery mechanism for that yet. BuildStarted is probably not the right event to hook on either. I've merged this to illustrate how with a small change, we might make the native/cross sstate much more reusable and hence improve the accessibility of lower overhead builds. With this change, its possible the Yocto Project may be able to support a configured sstate mirror out the box. This also has positive implications for our developer workflow/SDK improvements. (From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
continue
try:
subprocess.check_output(("patchelf-uninative", "--set-interpreter",
d.getVar("UNINATIVE_LOADER", True), f))
except subprocess.CalledProcessError as e:
bb.fatal("'%s' failed with exit code %d and the following output:\n%s" %
(e.cmd, e.returncode, e.output))
uninative: Add uninative - a way of reusing native/cross over multiple distros These patches are the start of a new idea, a way of allowing a single set of cross/native sstate to work over mutliple distros, even old ones. The assumption is that our own C library is basically up to date. We build and share a small tarball (~2MB) of a prebuilt copy of this along with a patchelf binary (which sadly is C++ based so libstdc++ is in there). This tarball can be generated from our usual SDK generation process through the supplied recipe, uninative-tarball. At the start of the build, if its not been extracted into the sysroot, this tarball is extracted there and configured for the specified path. When we install binaries from a "uninative" sstate feed, we change the dynamic loader to point at this dynamic loader and C librbary. This works exactly the same way as our relocatable SDK does. The only real difference is a switch to use patchelf, so even if the interpreter section is too small, it can still adjust the binary. Right now this implements a working proof of concept. If you build the tarball and place it at the head of the tree (in COREBASE), you can run a build from sstate and successfully build packages and construct images. There is some improvement needed, its hardcoded for x86_64 right now, its trivial to add 32 bit support too. The tarball isn't fetched right now, there is just a harcoded path assumption and there is no error handling. I haven't figured out the best delivery mechanism for that yet. BuildStarted is probably not the right event to hook on either. I've merged this to illustrate how with a small change, we might make the native/cross sstate much more reusable and hence improve the accessibility of lower overhead builds. With this change, its possible the Yocto Project may be able to support a configured sstate mirror out the box. This also has positive implications for our developer workflow/SDK improvements. (From OE-Core rev: e66c96ae9c7ba21ebd04a4807390f0031238a85a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28 10:10:06 +00:00
}