socketserver.* should be part of python3-netserver.
(From OE-Core rev: 8aa71c7133f996f6ed09a3c094e553bed7f33c7a)
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows us to use html.py without importing misc.
(From OE-Core rev: d1fefcaa10b85837d18458e291a3df9093b3a6ea)
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit: 512334f102a33833d39af53467894315f0715d07
"python-3.5-manifest: Add imp to importlib"
added imp to importlib in the generated manifest, but not in the generator script.
(From OE-Core rev: bfe1d948ab034017622110c8365be7026e139fbf)
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
pkg-config currently only handles -I and -L correctly, but misses
-isystem, so we need to do this workaround to fix this issue.
There's a open bug in:
https://bugzilla.freedesktop.org/show_bug.cgi?id=99516
This appears when building lldpd and shows error:
ERROR: lldpd-0.9.6-r0 do_configure: This autoconf log indicates
errors, it looked at host include and/or library paths while
determining system capabilities.
Rerun configure task after fixing this.
ERROR: lldpd-0.9.6-r0 do_configure: Function failed: do_qa_configure
Change-Id: I83255b5952e19bb2f516b1186276cd03eb67eed3
(From OE-Core rev: 5bd23c01fd800c43ba3979b23585fe8ac30d7ec8)
Signed-off-by: Fabio Berton <fabio.berton@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Discovered on beaglebone black
(From OE-Core rev: 21fcb42d97e90eb76283b69ab876729f4cee7ad7)
Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise the common name udev-hwdb is only provided by systemd, meaning that
other recipes can't depend on a single name.
(From OE-Core rev: dea267d576695e23223a2f402f6c9c126b0413dc)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we switched to recipe specific sysroots (rss), performance took a nose dive. Its
easy to blame rss but it turns out not to be entirely at fault.
Three configurations are compared here:
a) Pre-RSS (revision 45df694a9f)
b) Post-RSS (revision 226a508da9)
c) as b) with this change
Overall build times:
a) 22794.25user 2687.88system 30:32.84elapsed 1390%CPU (0avgtext+0avgdata 919056maxresident)k
b) 22677.25user 3238.79system 36:16.68elapsed 1190%CPU (0avgtext+0avgdata 918896maxresident)k
c) 23571.84user 3383.65system 31:36.83elapsed 1421%CPU (0avgtext+0avgdata 919068maxresident)k
For the overall build and sstate directories, du -s shows:
a)
3992588 build-pre-rss/sstate-cache
30804484 build-pre-rss/tmp
b)
4013272 build-with-rss/sstate-cache
36519084 build-with-rss/tmp
c)
4014744 build-with-rss2/sstate-cache
35336960 build-with-rss2/tmp
However more worryingly:
$ du -s build-pre-rss/tmp/sysroots/
2506092 build-pre-rss/tmp/sysroots/
$ du -s build-with-rss/tmp/sysroots-components/
3790712 build-with-rss/tmp/sysroots-components/
$ du -s build-with-rss2/tmp/sysroots-components/
2467544 build-with-rss2/tmp/sysroots-components/
These numbers *should* be equivalent but as you can see, b) is ~1.2GB larger. The reason turned out
to be patchelf. Taking a specific binary from a specific recipe, bc from bc-native, in a) its 82kb
(stripped) yet in b) its 2.17MB.
$ ./patchelf --set-interpreter /bin/rp bc
warning: working around a Linux kernel bug by creating a hole of 2084864 bytes in ‘bc’
https://github.com/NixOS/patchelf/blob/master/src/patchelf.cc#L710 shows that this "hole" is just
padded zeros using memset, its not a proper sparse hole.
This patch copies files with cp --sparse=always after modifying them with patchelf, then replacing
the original file. The better fix will be to fix this in patchself itself and seek() there
when writing the new file but that means new uninative tarballs and will take a bit of work
so I'm proposing this workaround in the meantime.
Also, this patch drops error handling since subprocess check_output() tracebacks will print this
information if the command fails so we can simplify the code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used options.outdir instead of image_output_dir.
There is no sense to use extra variable for this.
(From OE-Core rev: c6013dcb158a84d48cc2677f1509681cf9e0a3cb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was a typo in current code: mktemp was used instead of
mkdtemp to create work directory. This is fixed by using
mkdtemp.
Create work directory as a subdirectory of output directory
to make sure both are on the same partition to make moving
of result image faster.
This also fixes possible disk space issues as mkdtemp uses
TMPDIR, TEMP or TMP environment variables to get default value
of its 'dir' parameter. Those variables are usually pointing
to /tmp, which is not the best location to create huge images.
(From OE-Core rev: e6579b4e007ce89000c46c09917fbd3c72f18a73)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure output directory exists before creating an image.
Create it if it doesn't exist.
(From OE-Core rev: e52bfb0e81d3fb2a474f08b2e2b8b89aadc61d14)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
libudev will be autodetected by the linkage, the intention here was to depend on
udev-hwdb to ensure that the USB ID lists are installed.
(From OE-Core rev: 04624464690da71b3ec904a48d5caa8087ca6381)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: c72d5acb9c2f4a7d4dfe0e78aae832b10aec4429)
Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently test_postinst_roofs_and_boot is building a full-cmdline image
this is taking a lot of time to execute the test, so a minimal image
will be build instead.
(From OE-Core rev: 3a6b0b9f94b7097a825edf3fe258a2f86b73edd6)
Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to prevent multiple instances of the script running at the same
time.
(From OE-Core rev: 96a194de890f7ef1e6e5e036b32848e0f9d1bcf5)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This module contains singe function serial_console_form_kargs, which
is used only by rootfs_pcbios_ext plugin. Moved it there and removed
syslinux module to make it easy to find and mainain plugin code.
[YOCTO #10619]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code deals with non-existing directory
and can be removed.
[YOCTO #10619]
(From OE-Core rev: e148884b8145fad212c4085f2ead5c3e5e3e42a2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This API is not used in wic code.
[YOCTO #10619]
(From OE-Core rev: 9d14cb8bb8cf9559b7ff0efa03427624c18e2506)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced parse_sourceparams function with list comprehension.
Used local variables instead of attributes.
Moved global variable to the local scope.
[YOCTO #10619]
(From OE-Core rev: 4adbac84046ff744f1452b5ff4d017d17d2d45e2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This file is not used anywhere in the wic code.
[YOCTO #10619]
(From OE-Core rev: e3c43ce3261663225aeba50b8c6229577574f9b7)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As this files is located in plugins/imager subdirectory it's
obvious that it's an imager plugin. Renamed to direct.py to
be consistent with plugin naming scheme.
[YOCTO #10619]
(From OE-Core rev: d5db8c2ee91bdd51bfbb2ebf61aea8ff0378d512)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed wrong continued indentation, unused import and
trailing new line pyling warnings.
[YOCTO #10619]
(From OE-Core rev: e78762a4b52c31357ef29a5b93c53db7190fa4f2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Synchronized attribure names in DirectImageCreator and
DirectPlugin for better readability. Simplified code,
removed unneeded global variable disk_methods.
(From OE-Core rev: b87b9ef84791615636424a224f74386a4aa0c2fa)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced class method __rootfs_dir_to_dict with a list
comprehension.
[YOCTO #10619]
(From OE-Core rev: 266fd31410771db4c06539f7368c196a6d03000a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This move simplifies directory structure and makes
further refactoring easier. The code from direct.py was used
only in direct_plugin, so it's safe to move it there.
[YOCTO #10619]
(From OE-Core rev: a8f5ebb26183faa9af6eb72f4dabfcf83aa1e8d4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed as they're not used anymore in wic code.
(From OE-Core rev: ffa2f3d7bf883d5add911b7c5d0be2b347733524)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a preparation to removing conf.py and config/wic.conf
from the codebase.
Got rid of using configmgr global object in direct_plugin and direct
modules. It was used to implicitly parse kickstart file and set
couple of variables.
Replaced usage of configmgr by passing parameters directly to the
DirectImageCreator.
[YOCTO #10619]
(From OE-Core rev: 79191119de010acb107f9392a991108728858441)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a preparation to removing conf.py and config/wic.conf
from the codebase.
confmgr object is complicated for no reason and almost
useless as all configuration info comes from command line and
bitbake variables. It's used it creator.py to store information
about output directory, logs and some never used functionality
like tmpfs for future use, which doesn't actually happen.
[YOCTO #10619]
(From OE-Core rev: 702ee7a1fe30d87d55ba9528ae89abff4e294fd9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extra configuration data should be write using the write_config
method instead of manually appending to the local.conf file
(From OE-Core rev: 65c26d51eedc687c92e8ade262d24dc53ca71cec)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed hddimg from FSTYPES in wic test suite as
wic doesn't depend on hddimg anymore.
[YOCTO #10835]
(From OE-Core rev: a332d9618c2d0f46b6c0c8fdeee8bec4cc73f2cf)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The sysroot of wic-tools is needed for wic, but if rm_work is enabled,
it will be removed before wic has a chance to use it, hence this fix.
(From OE-Core rev: fc59d35a8ceae6302e0a161bab04829fdfd157cd)
Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When building for the target, pkg-config uses the target glib-2.0 instead of
it's own minimal fork. To find this it needs to use pkg-config so ensure this
dependency exists in case it doesn't exist on the host already.
(From OE-Core rev: 2e55ecb93ecbbd8bfe626c07d3a032c2fe98efca)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This puts the dependencies on the correct task and removes pointless
noexec tasks allowing for a slightly cleaner task structure.
(From OE-Core rev: 8b80aee29150fcd50b11dedaab42cdec6e3f222b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The number of mkdir calls was showing up high on the profile charts since
it was getting called once per file which is excessive. Each call results
in one or more syscalls which is bad for performance. Cache which
directories we've seen to reduce the calls to a more reasonable number
and speed up recipe specific sysroots.
(From OE-Core rev: 680fb343be5d0a7f9abbe9c75ca232abe5871663)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
According to profile data, repeated calls to bb.debug and bb.note in
the extend_recipe_sysroot() codepath were accounting for 75% of the time
(1.5s) in calls from tasks like do_image_complete.
This batches up the log messages into one call into the logging system
which gives similar behaviour to disabling the logging but retains the
debug information.
Since setscene_depvalid is also called from bitbake's setscene code,
we have to be a little creative with the function parameters and leave
the other debug output mechanism in place. This should hopefully
speed up recipe specific sysroots.
(From OE-Core rev: 3b0af8dc0f796345d1f1ba77ea35bbd090a5feb3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need to hardcode a path to tail, follow the other tools examples and
don't specify a path since PATH is good enough for us.
(From OE-Core rev: 1a1e70304932cce5ad194e0a7ebb495da7b24c2e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't put target libs into a native/cross ${exec_prefix} but having
this in the default search path means all linker scripts have to be relocated.
This is a considerable chunk of files to create multiple copies of for no good
reason.
Instead, patch out the paths we don't need.
(From OE-Core rev: 20816eb5398512652c971a37589a2ca28ffd3d68)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Relocation of native .la files during recipe specific sysroot relocation
is probably the final straw in just killing these files off.
Change things so this class is inherited by default. If distros don't want to
do this, they can opt out but it seems like the best thing to do now since
.la files aren't needed on Linux.
(From OE-Core rev: 11d801f6a7319a95f824842df118c446f8da7a71)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The native .pc files currently have hardcoded paths in them meaning each has
to be relocated at final install time. pkg-config has built in functionality
to avoid this, namely the pcfiledir variable.
This function translates .pc files to use the variable meaning further
relocation later is unnecessary.
(From OE-Core rev: a3f4e9ff55c7d9eba175e83eb6c99cf349a74432)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst the path to the executable is dynamically determined, the passed in
environment variables or parameters are not relocatable and rely on the sstate
remapping code. In the recipe specific sysroot case this has become more costly.
This patch translates such paths into relocatable entries which means that
a sed replacement at final installation isn't needed.
(From OE-Core rev: 3e02a7cda0ba70e3f9886c5cce37d71361280b11)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Anders Darander
Fabio Berton
Alejandro del Castillo
Andreas Müller
Andre McCurdy
Ross Burton
Richard Purdie
Ed Bartosh
Ming Liu
Jose Perez Carranza
Markus Lehtonen
Leonardo Sandoval
Kristian Amlie