wic used native sysroot to get path to pseudo utility.
This approach doesn't work with recipe specific sysroots.
Using FAKEROOTCMD should fix the issue.
(From OE-Core rev: 03e051d594f285ea3c014d45e9b30028e683c602)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic used STAGING_DIR_NATIVE variable as a path to native sysroot.
This doesn't work with recipe specific sysroots as STAGING_DIR_NATIVE
points to the native sysroot of the current recipe.
Used RECIPE_SYSROOT_NATIVE variable of wic-tools recipe
to fix the issue.
(From OE-Core rev: de9d7d14cd03e4dfc5812890a53c79b706b56537)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildhistory interfers with various tests so error if the user has it enabled,
hence avoiding time spent running the tests only for them to fail.
For example, if there is an sstate cache and something comes from that cache,
you can get a "version went backwards" warning from buildhistory which would
then change the outcome of a test. Its safer/easier to disable it.
(From OE-Core rev: 941ec7c50c30052e346e72fef2920135dba89a21)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When extracting source for a recipe within devtool (for extract, modify
or upgrade) We need to redirect WORKDIR, STAMPS_DIR etc. under a
temporary directory so that:
(a) we pick up all files that get unpacked to the WORKDIR, and
(b) we don't disturb the existing build
However, with recipe-specific sysroots the sysroots for the recipe will
be prepared under WORKDIR, and if we used the system temporary directory
i.e. usually /tmp) as used by mkdtemp by default, then our attempts to
hardlink files into the recipe-specific sysroots will fail on systems
where /tmp is a different filesystem, and we'd have to fall back to
copying the files which is a waste of time. Put the temp directory under
the WORKDIR to prevent that from being a problem.
(From OE-Core rev: e10a973cd9390eacb13bdb99693a0622bd3695f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
[RP: Add needed mkdirhier call]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make it possible to store any bitbake config variables in the metadata.
Config values will be stored under a new config element in the xml report:
<config>
<variable name="MACHINE">qemux86</variable>
</config>
The value of MACHINE is moved there instead of having a dedicated
<machine> element.
[YOCTO #10590]
(From OE-Core rev: 6e7e6e37664b0a86111272f5f6f4a4e1d0f23302)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Revision is a bit vague and could point to a tag, for example. Git
commit objects are unambiguous and persistent so be explicit that the
element should contain git commit hash.
[YOCTO #10590]
(From OE-Core rev: a6521b89e86ddba6bd646dd3974d7b0390323b56)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's better just to not have the xml elements than to have elements with
faux data. One could have git branch named 'unknown', for example.
[YOCTO #10590]
(From OE-Core rev: fce531c21f5e56d0f416b3405a0b0fc5ba567679)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the same format, based on /etc/os-release, as for host distro
information.
[YOCTO #10590]
(From OE-Core rev: 0156ef46ccf5334ee72f0202f1089249c62af37b)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
$ runqemu nfs qemux86-64
[snip]
On your target please remember to add the following options for NFS
nfsroot=IP_ADDRESS:/path/to/nfsroot,nfsvers=3,port=,mountprog=,nfsprog=,udp,mountport=
[snip]
Note that the values are null, this is because their var names are
inconsistent.
[YOCTO #10519]
(From OE-Core rev: 10c136a382006c0ec2679a70f47ff2446c10372c)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When "runqemu /path/to/<file>.cpio.gz", it used the last suffix "gz" as
the fstype which was wrong. Check filename against self.fstypes firstly
can fix the problem.
(From OE-Core rev: 68c7589b67a83977331a04356b53aa51680a1d9d)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit 8c1c43b790 `wic: Create a logical partition
only when it is really mandatory` did not account for partitions that are not
present in partition table.
(From OE-Core rev: 57b05e924bba7b2fff07a34690474c0fa3046865)
Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'ksum.py' generates a combined summary of vmlinux and module sizes for
a built kernel, as a quick tool for comparing the overall effects of
systemic tinification changes. Execute from the base directory of the
kernel build you want to summarize. Setting the 'verbose' flag will
display the sizes for each file included in the summary.
(From OE-Core rev: 016b19c2589582d7ec3c8cac9cfa75a1edc716fe)
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In some cirsumstances the user doesn't want to supply a kernel, rootFS
or DTB to QEMU. This will occur more now that QEMU supports loading
images using a '-device loader' method.
Allow users to specify 'none' for QB_DEFAULT_FSTYPE or QB_DEFAULT_KERNEL
to avoid supplying these options to QEMU.
(From OE-Core rev: 2cc01c4e46b05b7ffcc8a11e7ebde6c43256c3c3)
Signed-off-by: Alistair Francis <alistair.francis@xilinx.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current code doesn't always show error output of the
external command and even ignores non-zero exit code.
Moved checking of exit code value to the lowest level
possible: to _exec_cmd. This should make wic to always
check exit code of the external command and issue
an error if it's not 0.
[YOCTO #10816]
(From OE-Core rev: 7f68001579c08509332d633b27b5c2ea9386b6c9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
part.rootfs_dir was used as an image name in the code.
However, when multi-rootfs feature is used this attribute
points to the name of the rootfs, e.g. if --rootfs command line
is rootfs1=core-image-minimal partf.rootfs_dir is 'rootfs1'.
The code also fails when image name is not provided in wic
commandline. For example, when wic is called with
--rootfs-dir=<path> part.rootfs_dir will contain path and
wic will crash trying to call bitbake -e <path> to get
value of ROOTFS_SIZE variable.
Fixed the code by getting image name properly and checking
if it's not a path.
[YOCTO #10815]
(From OE-Core rev: 2e05d9709f1308fc95d3406b8a409ea982c5b474)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This file has been moved to meta-yocto-bsp/wic/
(From OE-Core rev: efa7639b31c51e2874ba61fd68f9e2cb51145eaf)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The purpouse of this function was to check dependencies for building a
hybrid iso and build them using bitbake if not found. Calling bitbake in
this context means this wic plugin itself cannot be instrumented inside
bitbake recipes which is undesirable, the benefits of this are clear:
there is no need to maintain outside scripts to generate an iso using wic
and the isohybrid building logic can be further abstracted away into an
isohybrid.bbclass in the future which can be easily inherited or something
similar.
So remove the function and add all dependencies to NATIVE_RECIPES so that
wic can print useful errors when they're not built.
To automate building the isohybrid image dependencies, add the following
somewhere in your image build inheritence hierarcy (or maybe create a
bbclass in the future to do these sort of things automatically):
DEPENDS += "syslinux syslinux-native cdrtools-native e2fsprogs-native \
parted-native dosfstools-native mtools-native grub-efi-native"
(From OE-Core rev: ba4346069ab87f1cf942d1928f911eca6a9d65cd)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding http module from Python's standard library. This allow use
of the http module without installing all python-misc modules.
(From OE-Core rev: 3d0060f9703ba39fbdaafcbdf91e0c319b56f7b3)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current parsing code can wrongly interpret arbitrary lines
that are of 'key=value' format as legitimate bitbake variables.
Implemented more strict parsing of key=value pairs using
regular expressions.
(From OE-Core rev: f0ec387ad40fb9c098ac8d761993bc2bacc76e65)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the gummiboot project is no longer being maintained
and we are using systemd-boot as a replacement instead,
we can now clean up all remaining gummiboot files.
[YOCTO #10332]
(From OE-Core rev: 65eb3f51b70baaf24de871301a7247d5baed00ed)
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you delete the log file that the oe-selftest.log symlink points to
but not the symlink itself, because we were using os.path.exists() here
the code assumed that the symlink didn't exist when in fact it still
did. Use os.path.lexists() instead.
(From OE-Core rev: 263af91a0efd21e041ecdb0c40f9b2d4e735f67d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After systemd-boot was introduced, its been tested for a while with no major
issues being found until now, this patch completely replaces all gummiboot
instances with systemd-boot ones, taking the next step into cleaning
up systemd-boot/gummiboot.
[YOCTO #10332]
(From OE-Core rev: f9a61d3400ad9068a6d83b8eb6aefe3098c58e68)
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were a few straggling expansion parameter removals left for
getVar/getVarFlag where the odd whitespace meant they were missed
on previous passes. There were also some plain broken ussages such
as:
d.getVar('ALTERNATIVE_TARGET', old_name, True)
path = d.getVar('PATH', d, True)
d.getVar('IMAGE_ROOTFS', 'True')
which I've corrected (they happend to work by luck).
(From OE-Core rev: 688f7a64917a5ce5cbe12f8e5da4d47e265d240f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its possible something (like bitbake/tinfoil2) may mess around with the
environment and using the enviroment as a global variable store isn't
particularly nice anyway.
This patch changes the BUILDDIR usages so that the environment isn't used
as a global store and a global variable is used instead. Whilst that
is still not perfect, it does avoid the current double and triple backtraces
we're seeing where tinfoil2/bitbake has trampled the enviroment leading
to failures of failures making debugging even harder.
(From OE-Core rev: 689b676bbf2f1a5fadb04aeb41d5e68e35356545)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It may be necessary to override the parameters gathered for the qemu
invocation. For example, the qemux86 machine configuration sets "-vga
vmware", but when using OVMF as BIOS, only "-vga std" is supported.
By putting the parameters derived from custom runqemu parameters like
"qemuparams" after the parameters derived from the machine
configuration the user gets the possibility to override those.
(From OE-Core rev: b6feb7578d60289c8b6e376cfaac8a3ee45e72f9)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added kickstart file to produce partitioned image for
MPC8315 reference hardware.
[YOCTO #8719]
(From OE-Core rev: 8c873b4fd92b72adfc49d20bbfab4779857fb6c9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added new option --fixed-size to wks. The option can be used to indicate
the exact size of a partition. The option cannot be added together with
--size, in which case an error will be raised. Other options that
influence automatic partition size (--extra-space, --overhead-factor),
if specifiec along with --fixed-size, will raise an error.
If it partition data is larger than the amount of space specified with
--fixed-size option wic will raise an error.
(From OE-Core rev: fdd217ba874bd480e0180830fe2e6bd54dde19d9)
Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using devtool modify on the kernel, we have to do a bit of a dance
with tinfoil instances because we only find out that we're working on a
kernel recipe after tinfoil is initialised, but then we need to build
kern-tools-native which we're doing just by running bitbake directly.
With the tinfoil2 changes, a datastore for the recipe that we were
keeping around across the opening and closing of tinfoil is no longer
able to be used. Re-parse the recipe to avoid this problem.
(In future this whole thing will be able to be done in the same tinfoil
instance thanks to tinfoil2, but that refactoring is yet to be done.)
(From OE-Core rev: 06127d0115ba449bf04e2579cd1010065e0ed6e3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently wic looks for wks files in
<layer dir>/scripts/lib/wic/canned-wks/ directories.
This path is too nested and doesn't look consistent with the
naming scheme of layer directories.
Added <layer>/wic directory to the list of paths
to look for wks files.
(From OE-Core rev: 803b5fa798cf7580c7c10401eb04a1cccf51b8ab)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
git module is not included into standard Python
library and therefore causes import errors on the systems
where PythonGit is not installed.
As git module only used in the code implementing --repository
functionality it's better to import git only in the scope
that requires it.
[YOCTO #10821]
(From OE-Core rev: 66be32c1a075201d6ee0e9b9e10b84e6a2ace745)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When no --size is specified for the rootfs in the .wks, we want to obey the
rootfs size from the metadata, otherwise the defined IMAGE_ROOTFS_EXTRA_SPACE
and IMAGE_OVERHEAD_FACTOR will not be obeyed. In some cases, this can result
in image construction failure, if the size determined by du was insufficient
to hold the files without the aforementioned extra space.
This fallback from --size to ROOTFS_SIZE was already implemented when
--rootfs-dir is specified in the .wks, but it did not occur otherwise, neither
when --rootfs-dir= was passed to `wic create` nor when IMAGE_ROOTFS was used.
This made a certain amount of sense, as this fallback logic happened at such
a level that it wasn't able to identify which partitions were rootfs
partitions otherwise. Rather than doing it at that level, we can do it in
prepare_rootfs(), which is run by the rootfs source plugins.
Note that IMAGE_OVERHEAD_FACTOR and a --overhead-factor in the .wks will now
both be applied when --size isn't specified in the .wks. A warning is added
about this, though a user won't see it unless wic fails or they examine the
do_image_wic log.
Fixes [YOCTO #10815]
(From OE-Core rev: 1d50e11286722c4114c1ae0bc285f846cd85fc4c)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
getVarFlag() now defaults to expanding by default, thus remove the
True option from getVarFlag() calls with a regex search and
replace.
Search made with the following regex:
getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\)
(From OE-Core rev: 3e4806063fe11092b2307f113a6c0b0f04104091)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
(From OE-Core rev: 0a36bd96e6b29fd99a296efc358ca3e9fb5af735)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will add support to use qemu from the running host,
with this is possible to put qemu-native in ASSUME_PROVIDED
variable.
By default it will try to get qemu from the build sysroot,
and only if it fails will try to use the host's qemu.
(From OE-Core rev: fe7fd2cd3a9c4fb5b31bd3cab81c96a3b81cb540)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we don't catch this then attempting to run devtool in non-memres mode
when bitbake is already running will produce a traceback instead of just
an error message.
(From OE-Core rev: e01b75dff599ffa2b66e6608b28bbb3564365eee)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using the setVariable commands here followed by buildFile will result in
"basehash mismatch" errors, and that's expected since we are deviating
*at runtime* from what was previously seen by changing these variable
values. Set BB_HASH_IGNORE_MISMATCH to turn off the errors.
(From OE-Core rev: b0169796f294bbec0397b7eae86454a46b68cdc5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extracting the source for a recipe (as used by devtool's extract, modify
and upgrade subcommands) requires us to run do_fetch, do_unpack,
do_patch and any tasks that the recipe has inserted inbetween, and do so
with a modified datastore primarily so that we can redirect WORKDIR and
STAMPS_DIR in order to have the files written out to a place of our
choosing and avoid stamping the tasks as having executed in a real build
context respectively. However, this all gets much more difficult when in
memres mode since we can't call internal functions such as
bb.build.exec_func() directly - instead we need to execute the tasks on
the server. To do this we use the buildFile command which already exists
for the purpose of supporting bitbake -b, and setVariable commands to
set up the appropriate datastore.
(I did look at passing the modified datastore to the buildFile command
instead of using setVar() on the main datastore, however its use of
databuilder makes that very difficult, and we'd also need a different
method of getting the changes in the datastore over to the worker as
well.)
(From OE-Core rev: eb63b5339014fc72ba4829714e0a96a98e135ee2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The autotools code imports oe.package; we weren't experiencing a problem
with this probably due to OE itself adding that path previously.
(From OE-Core rev: a61d7bf8447b2d2c65eb34315c86086ff35c8bc9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If PATCHTOOL is "git", and PATCH_COMMIT_FUNCTIONS is set to "1", for
additional tasks between do_unpack and do_patch, make a git commit. This
logic was previously implemented in devtool itself, but it makes more
sense for it to be implemented in the patch class since that's where the
rest of the logic is for this (or in lib/oe/patch.py). It also makes
it possible for this to work with tinfoil2.
(From OE-Core rev: f24f59ea1d8bc335ea8576f6a346d0935f4a3548)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use Tinfoil.parse_recipe_file() and Tinfoil.parse_recipe() instead of
the recipeutils equivalents, and replace any local duplicate
implementations. This not only tidies up the code but also allows these
calls to work in memres mode.
(From OE-Core rev: f13b56266ee96dfab65a3a7db50e8051aa9f071a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
setup_tinfoil() already calls prepare(), we don't need to call it again
ourselves and doing so with tinfoil2 results in "ERROR: Only one copy of
bitbake should be run against a build directory". Calling prepare()
twice should probably still be allowed, so that ought to be fixed
separately, but in the mean time this code is still wrong so fix it
here.
(From OE-Core rev: 38b8a7d4aff096ea0a62f2ddf3fe2de1df591bf5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some variables in pkgdata files have a package-name override. When
the bare variable can not be found, try with the override-variant.
PKGSIZE is one such variable, and already had special code to handle this.
Test included.
(From OE-Core rev: 6df99cda894033cba68bc6ab91e47f67e0d788a5)
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Show usage text if script is not sourced.
Tested in bash, zsh and dash.
[YOCTO #10751]
(From OE-Core rev: ac7a905b18acb8bd9b2412b6682afbe1d7e18d7b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Created usage output for oe-find-native-sysroot script.
[YOCTO #10751]
(From OE-Core rev: 8ddfc48c7f3e2ca45c035cec492fdc31c6ad484f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Created usage output for oe-git-proxy script.
[YOCTO #10751]
(From OE-Core rev: ec0fdb5e896fc20dbafcc8ae507b17c011dc56fd)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Made usage output of oepydevshell-internal.py to look
similar to the output of other oe scripts.
[YOCTO #10751]
(From OE-Core rev: e6480af22a7a12c655efed14f8f1aea658f26b1c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Created usage output for oe-setup-builddir script.
[YOCTO #10751]
(From OE-Core rev: 77606455df7d45fd014c3603e1cf1b24efd37695)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Made usage output of oe-setup-rpmrepo to look similar to the
output of other oe scripts.
[YOCTO #10751]
(From OE-Core rev: 5423c9a412c680b781417a64b412838845b5d075)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Created usage output for oe-trim-schemas script.
[YOCTO #10751]
(From OE-Core rev: a9fac12157ed434a6711de1af77eda1f4a8e9e8a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Made usage output of oe-run-native to look similar to the
output of other oe scripts.
[YOCTO #10751]
(From OE-Core rev: e1c96125ea674509fbc9b36dc671b7a53bd848ac)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To allow recipetool plugins in one layer to shadow another in a well
defined way, first search BBPATH/lib/recipetool directories and then
scripts/lib/recipetool and load only the first found.
The previous search and load loop would load all found plugins with the
ones found later replacing any found before.
(From OE-Core rev: 7a8726a3662a3909dc6ef6e8d1029d0b1aa938c3)
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To allow devtool plugins in one layer to shadow another in a well
defined way, first search BBPATH/lib/devtool directories and then
scripts/lib/devool and load only the first found.
The previous search and load loop would load all found plugins with the
ones found later replacing any found before.
(From OE-Core rev: 1b2b8a0a80de17ea053002fdd124055d2798029a)
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Short variant of wic command line option --skip-build-check
is incorretly named -p. It's named -s in wic help and Yocto
documentation.
(From OE-Core rev: 8d4121459334349f10bfb0f4cda00a8461aab245)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't worth bother with logical partition on MBR partition type (aka
msdos) if disk image generated by wic should have 4 partitions.
(From OE-Core rev: 36a558fbdc96094626e7de1a3510691e30885368)
Signed-off-by: Alessio Igor Bogani <alessio.bogani@elettra.eu>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This new option allows to commit the result to a git repository,
along with the results it will add a metadata file for information
of the current selftest run, such as: hostname, machine, distro,
distro version, host version, and layers.
This implementation will have a branch per different hostname,
testing branch, and machine.
To use this feature use:
oe-selftest <options> --repository <repository_link>
[YOCTO #9954]
(From OE-Core rev: 758e4b7e619f9aaa79e3910b5539ff8fb2d1064a)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Yet another instance of us expecting a string back from subprocess when
in Python 3 what you get back is bytes. Just decode the output within
run_command() so we avoid this everywhere.
(From OE-Core rev: 103faae78cdff5280c7b7cdb7ca01e0868d02ec9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pre-processing /proc data during the build considerably reduces the
amount of data written to disk: 176KB instead of 4.7MB for a 20
minuted build. Parsing also becomes faster.
buildstats.bbclass only writes the reduced logs now, but support for
the full /proc files is kept around as reference.
(From OE-Core rev: b5e47df9af1ebbb477074587fdeae17eb2f55582)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The internal representation after parsing now matches exactly
what the drawing code needs, thus speeding up drawing a bit.
However, the main motivation is to store exactly that required
information in a more compact file.
(From OE-Core rev: ca06e67a0bb5820b38fda4c8dfee20764c1e59ae)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds a new, separate chart showing the amount of disk space used
over time for each volume monitored during the build. The hight of the
graph entries represents the delta between current usage and minimal
usage during the build.
That's more useful than showing just the current usage, because then a
graph showing changes in the order of MBs in a volume that is several
GB large would be just flat.
The legend shows the maximum of those deltas, i.e. maximum amount of
space needed for the build. Minor caveat: sampling of disk space usage
starts a bit later than the initial task, so the displayed value may
be slightly lower than the actual amount of space needed because
sampling does not record the actual initial state.
(From OE-Core rev: 263d189d066b578debf08b2bd07494a69b70f70d)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When matching fails, m.group(0) is invalid and can't be used in the
error message.
(From OE-Core rev: ddfea21e06a2e6b1a1e766969f8c134a6de7388a)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The only real change is the addition of two if checks that skips the
corresponding drawing code when there is no data.
(From OE-Core rev: 1658fd5e9ca1ba793cae604c2a395d54e3ec9056)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This enables rendering of the original bootchart charts for CPU, disk
and memory usage. It depends on the /proc samples recorded by the
updated buildstats.bbclass. Currently, empty charts CPU and disk usage
charts are drawn if that data is not present; the memory chart already
gets skipped when there's no data, which will also have to be added
for the other two.
(From OE-Core rev: 233d3e50b361feea07803a9c0f2a691e687c6cd5)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code did not handle x scaling correctly when drawing starts at
some time larger than zero, i.e. it worked for normal bootchart data,
but not for the system statistics recorded by buildstats.bbclass.
(From OE-Core rev: 166f8f9aaa1f01fc6d6a5451f8f06b815c51ffae)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Substracting curr_y when determining the hight of the process chart is
wrong because the height is independent of the position where the
chart is about to be drawn. It happens to work at the moment because
curr_y is always 10 when render_processes_chart() gets called. But it
leads to a negative height when other charts are drawn above it, and
then the grid gets drawn on top of those other charts.
Substracting some constant is relevant because otherwise the box is
slightly larger than the process bars. Not sure exactly where that
comes from (text height?); leg_s seems a suitable constant and happens
to be 10, so everything still gets rendered exactly as before.
(From OE-Core rev: b6bb690728c329ae448f89a1b68298c6dd8a573a)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When creating a patch set with cover letter using the
send-pull-request script, both the "In-Reply-To" and "References"
headers are appended twice in patch 2 and subsequent.
That's because git-format-patch already inserted them and then
git-send-email repeats that. Suppressing mail threading in
git-send-email with --no-thread avoids the problem and is the
right solution because it works regardless whether git-send-email is
called once or twicee.
Repeating these headers is a violation of RFC 2822 and can confuse
mail programs. For example, Patchwork does not detect a patch series
problem when there are these extra headers.
[YOCTO #10718]
(From OE-Core rev: 303a1aa3df43eb0b693d8602062fa33c4a08fdd6)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Scripts that produces script data to be consumed by gnuplot.
There are two possible plots depending if either the
-S parameter is present or not:
* without -S: Produces a histogram listing top N recipes/tasks versus
stats. The first stat defined in the -s parameter is the one taken
into account for ranking
* -S: Produces a histogram listing tasks versus stats. In this case,
the value of each stat is the sum for that particular stat in all recipes found.
Stats values are in descending order defined by the first stat defined on -s
EXAMPLES
1. Top recipes' tasks taking into account utime
$ buildstats-plot.sh -s utime | gnuplot -p
2. Tasks versus utime:stime
$ buildstats-plot.sh -s utime:stime -S | gnuplot -p
3. Tasks versus IO write_bytes:IO read_bytes
$ buildstats-plot.sh -s 'IO write_bytes:IO read_bytes' -S | gnuplot -p
(From OE-Core rev: 74408c19fba89de54c093fccf65b3a072d6a197b)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the patch that was applied in the python3 and python3-native
recipes to skip compilation of python modules.
Modify generate-manifest-3.5.py to match '__pycache__' directories in
FILES_*.
This is necessary because Python3 puts .pyc files in '__pycache__'
subdirectories one level below the corresponding .py files, whereas in
Python2 they used to be right next to the sources.
This change significantly reduces the startup overhead of Python3
scripts. For example, on a Cortex-A9, "python3 -c pass" took 0.40s
before, and 0.19s after.
(From OE-Core rev: bb4d689769703177dbb0df0935e15016b879f42b)
Signed-off-by: Dominic Sacré <dominic.sacre@gmx.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In previous implementation, a UnicodeDecodeError exception will be
raised if multi-byte encoded characters are printed by the subprocess.
As an example, the following command will fail in an en_US.UTF-8
environment because wget quotes its saving destination with '‘'(0xE2
0x80 0x98), while just the first byte is provided for decoding:
devtool add recipe http://example.com/source.tar.xz
The patch fixes the issue by avoiding such kind of incomplete decoding.
(From OE-Core rev: 1875ea92546d23abcab1b40b562477a0016f712d)
Signed-off-by: Jiajie Hu <jiajie.hu@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are many more stats on buildstats that 'Elapsed time', so make the script
more flexible to support all stats. Some cmd line examples:
$ buildstats.sh -s 'utime'
Buildstats' data covers proc's stats in different areas, including CPU times,
IO, program system resources and child program system resources. In order
to print values on each of these sets from command line, one can use the
following:
$ buildstats.sh -H -s 'TIME' | less
$ buildstats.sh -H -s 'IO' | less
and 'RUSAGE' and 'CHILD_RUSAGE' for program and program's child system
resources.
(From OE-Core rev: 81479b191287ccbf4cf94fa2d0ad46813091bca1)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The hello-mod recipe is unusual in that it has only local files in
SRC_URI and builds these out of ${WORKDIR}. When you use devtool modify
on it, devtool puts all of those files in an "oe-local-files"
subdirectory of the source tree, which is not ${S} (or ${B}) any more
and thus building the recipe afterwards fails. It's a bit of a hack, but
symlink the files in oe-local-files into the source tree (and commit the
symlinks with an ignored commit so that the repo is clean) to work
around the problem. We only do this at time of extraction, so any files
added to or removed from oe-local-files after that won't be handled, but
I think there's a limit to how far we should go to support these kinds
of recipes - ultimately they are anomalies.
I initially tried a hacky workaround where I set effectively set B =
"${WORKDIR}" and that allowed it to build, but other things such as the
LIC_FILES_CHKSUM checks still broke because they expected to find files
in ${S}. Another hack where I set the sourcetree to point to the
oe-local-files subdirectory works for hello-mod but not for makedevs
since whilst that is similar, unlike hello-mod it does in fact have
files in the source tree (since it has a patch that adds COPYING) and
thus the same issue occurred.
Also tweak one of the tests that tries devtool modify / update-recipe on
the makedevs recipe to try building it since that would have caught this
issue.
Fixes [YOCTO #10616].
(From OE-Core rev: 857c06d6a1d161bf5a01311d07758bd4241929a3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The .deb import feature did not import postinst, postrm, preinst, or
prerm functions. This change checks to see if those files exist, and
if so, adds the appropriate functions.
[ YOCTO #10421 ]
(From OE-Core rev: ebb73aa6ad920bfd6a23f8c20105d6bcf07dd3d5)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replicate bitbake and eforce en_US.UTF-8 locale so that ouptut of locale-aware
tools remains stable.
(From OE-Core rev: 17cd2cb99d3610fd77595ff116b2168188c250cd)
Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have a patch remotely fetched in a recipe (e.g. from an http
server) that needs updating then add a local version and substitute the
entry in SRC_URI to point to it.
One can argue about how desirable it is to be modifying patches fetched
in this way, but then one can argue about how desirable it is to have
such patches in the recipe in the first place - and in any case if
devtool update-recipe is to correctly transfer changes to such patches
made in the git repository within the source tree to the recipe then
there isn't much choice but to do it this way.
(From OE-Core rev: a19c26cc78a181f9dd2706dd42e7e450d7ad4082)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible to use gzip or bzip2 to compress patches and still refer
to them in compressed form in the SRC_URI value within a recipe. If you
run "devtool modify" on such a recipe, make changes to the commit for
the patch and then run devtool update-recipe, we need to correctly
associate the commit back to the compressed patch file and re-compress
the patch, neither of which we were doing previously.
Additionally, add an oe-selftest test to ensure this doesn't regress in
future.
Fixes [YOCTO #8278].
(From OE-Core rev: e47d21624dfec6f71742b837e91da553f18a28c5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As of the move to Python 3 and the fixes we applied at that time,
bb.process.run() will return a byte array of length 0 rather than an
empty string if the output is empty. That may be a bug that we should
fix, but for now it's easiest to just check the result here before
treating it as a string. This fixes running "devtool update-recipe" or
"devtool finish" on a recipe which has no source tree, for example
initramfs-framework.
Fixes [YOCTO #10563].
(From OE-Core rev: 66bf6978fc807ecc422fb6b6328f68bc3406cf15)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
xf86-input-libinput is RRECOMMENDed by xserver-xorg, these
legacy drivers are not needed.
(From meta-yocto rev: 715f72d7842b4a789a78e7889b2f01cd41f02df8)
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Running `oe-selftest --list-tests-by module wic` will produce the
following backtrace:
Traceback (most recent call last):
File "<snip>/poky/scripts/oe-selftest", line 668, in <module>
ret = main()
File "<snip>/poky/scripts/oe-selftest", line 486, in main
list_testsuite_by(criteria, keyword)
File "<snip>/poky/scripts/oe-selftest", line 340, in list_testsuite_by
ts = sorted([ (tc.tcid, tc.tctag, tc.tcname, tc.tcclass, tc.tcmodule) for tc in get_testsuite_by(criteria, keyword) ])
TypeError: unorderable types: int() < NoneType()
The root cause is that a test case does not necessarily have an ID
assigned, hence its value is None. Since Python 3 does not allow
comparison of heterogeneous types, TypeError is raised.
(From OE-Core rev: 71c6790689e2cbd3c4e882335b3b03e635ad46ed)
Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix typos in documentation of Image.add_partition() and
Image.__format_disks().
(From OE-Core rev: f5bf7bf253224912c66bab89f48ff63a73e0d698)
Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We explicitly check for --fstype if no source was provided for a
partition. However, this was not the case for rootfs partitions. Make
sure to raise an error if filesystem was left unspecified when preparing
a rootfs partition image.
(From OE-Core rev: b8c35fcad57810a87aa25ebeb533adf286eed565)
Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It seems that prepare_empty_partition_ext() and
prepare_empty_partition_btrfs() got broken in commit
c8669749e3, thus one could observe the
following backtrace:
Backtrace:
File "<snip>/poky/scripts/lib/wic/plugins/imager/direct_plugin.py", line 93, in do_create
creator.create()
File "<snip>/poky/scripts/lib/wic/imager/baseimager.py", line 159, in create
self._create()
File "<snip>/poky/scripts/lib/wic/imager/direct.py", line 290, in _create
self.bootimg_dir, self.kernel_dir, self.native_sysroot)
File "<snip>/poky/scripts/lib/wic/partition.py", line 146, in prepare
method(rootfs, oe_builddir, native_sysroot)
File "<snip>/poky/scripts/lib/wic/partition.py", line 325, in prepare_empty_partition_ext
os.ftruncate(sparse.fileno(), rootfs_size * 1024)
NameError: name 'rootfs_size' is not defined
(From OE-Core rev: 0bf686739a5e8a1193f5be5aa60afbf2ea1e5074)
Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The size field of Partition class is expected to be an integer and ought
to be set inside prepare_*() method. Make sure that this is always the
case.
(From OE-Core rev: a37838f995ae642b0b8bdd47a605a264fb3bf200)
Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Root device name in systemd-bootdisk.wks is 'sda'. This can cause
images, produced using this wks to refuse booting if real device
name is not 'sda'. For example, when booting MinnowBoard MAX from
MicroSD card the boot process stucks with this message on the boot
console output: Waiting for root device /dev/sda2...
This happens because real device name of MicroSD card on this device
is mmcblk1.
Used --use-uuid option for root partition. This should make
wic to put partiion UUID instead of device name into kernel command
line.
[YOCTO #10485]
(From OE-Core rev: 5b73d5f484cc844affe91ec19d881d42e187f30c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The function write_qemuboot_conf() in qemuboot.bbclass always inserts
the full path into QB_DEFAULT_KERNEL. Remove this path before using the
variable.
(From OE-Core rev: 7c0fdfa1316011b856a795d8e42c36ac8b5638b2)
Signed-off-by: Alistair Francis <alistair.francis@xilinx.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When you run devtool add on a source tree we attempt to figure out the
correct name and version for the recipe. However, despite our best
efforts, sometimes the name and/or version we come up with isn't
correct, and the only way to remedy that up until now was to reset the
recipe, delete the source tree and start again, specifying the name this
time. To avoid this slightly painful procedure, add a "rename"
subcommand that lets you rename the recipe and/or change the version.
(From OE-Core rev: 9303d8055c45a0f6af295d70a6f6a8b9d8d8a7c9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
recipetool sets the LICENSE value based on licenses detected from the
source tree. If there are multiple licenses then they were being
separated by spaces, but this isn't actually legal formatting and if
you're using "devtool add" you get a warning printed when devtool
parses the recipe internally.
Earlier I had made a conscious decision to do it this way since it's up
to the user to figure out whether the multiple licenses should all apply
(in which case they'd be separated with &) or if there is a choice of
license (in which case | is the correct separator). However, I've come
to the conclusion that we can just default to & and then the ugly
warning goes away, and it's the safest alternative of the two (and most
likely to be correct, since it's more common to have a codebase which is
made up of code with different licenses, i.e. all of them apply to the
combined work).
I've tweaked the comment that we add to the recipe to explicitly state
that we've used & and that the user needs to change that if that's not
accurate.
Fixes [YOCTO #10413].
(From OE-Core rev: ecac6aee8cf3313350b58c21012bcd67cfb915e4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you run devtool finish to move a recipe created in the workspace by
devtool add or devtool upgrade to a layer, and that layer is not
currently included in bblayers.conf (perhaps unintentionally), then the
recipe will no longer be visible to bitbake. In this scenario, show a
warning so that the user isn't surprised by the recipe "going missing".
(From OE-Core rev: 4da8a58e2997db4f24ae0cac0ba27259d7857a05)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If devtool finish is run on a recipe where the recipe file itself is in
the workspace (e.g. where devtool add / devtool upgrade has been used)
and the specified destination layer is not in bblayers.conf, then we
need to avoid running bitbake -c clean at the end because the recipe has
been moved, but the bbappend is still present in the workspace layer at
that point and so if we do it will fail due to the dangling bbappend.
It's difficult to do the clean at the point we'd want to because tinfoil
is holding bitbake.lock for most of the time, but in any case cleaning
the recipe is less important than it used to be since we started
managing the sysroot contents more strictly, so just disable cleaning
under these circumstances to avoid the problem.
Fixes [YOCTO #10484].
(From OE-Core rev: c6980307d43632f4172e79d9607004203af4e9c8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running devtool add, instead of hiding the recipetool create
output, change it so that it's appropriate to show in the devtool
context and show it in real-time. This means that you get status output
such as when a URL is being fetched (though currently no progress
information.) recipetool create now has a hidden --devtool option to
enable this display mode.
(From OE-Core rev: 219aec8803de4ef04c514c87ecfb15359c9424a6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced running of truncate utility with the standard library
call os.ftruncate
(From OE-Core rev: 1ba6101ceaee354816e690d44bc9a5dd8dcf4011)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 'task-time' Python script is used for simple manual analysis of
buildstats. It displays task timing information in the same format (and
using the same calculation) as the Bash 'time' builtin, and can
optionally sort tasks by real (wall-clock), user (user space CPU), or
sys (kernel CPU) time used.
The timing information comes from the getrusage(2) fields added by
commit adfdca4df1 ("buildstats: Improve to add getrusage data and
corrected IO stats"). That commit is required for the script to work.
Example 1: Running 'task-time' on a specific task buildstat:
$ task-time ./20161005235448/gettext-0.16.1-r6/do_compile
./20161005235448/gettext-0.16.1-r6/do_compile:
real 0m54.560s
user 0m46.028s
sys 0m2.772s
Example 2: Running 'task-time' on a directory, sorting on wall-clock
time:
$ task-time tmp/buildstats/20161018083535 --sort real
tmp/buildstats/20161018083535/bash-4.3.30-r0/do_fetch:
real 10m59.140s
user 0m1.152s
sys 0m0.320s
tmp/buildstats/20161018083535/readline-native-6.3-r0/do_fetch:
real 8m57.310s
user 0m0.860s
sys 0m0.288s
tmp/buildstats/20161018083535/perl-5.22.1-r0/do_compile:
real 4m28.840s
user 4m1.348s
sys 0m15.816s
...
Example 3: Running 'task-time' on all do_compile buildstats for a
particular build by using shell globbing, sorting on user space CPU
time:
$ task-time tmp/buildstats/20161018083535/*/do_compile --sort user
tmp/buildstats/20161018083535/qemu-native-2.7.0-r1/do_compile:
real 0m49.570s
user 21m45.236s
sys 1m44.380s
tmp/buildstats/20161018083535/linux-yocto-4.8+gitAUTOINC+03bf3dd731_67813e7efa-r0/do_compile:
real 0m49.530s
user 21m39.588s
sys 1m59.576s
tmp/buildstats/20161018083535/gcc-cross-i586-6.2.0-r0/do_compile:
real 1m8.130s
user 15m54.256s
sys 1m28.776s
...
Example 4: Comparing a task between two builds:
$ task-time 201610052{25856,35448}/gettext-0*/do_compile --sort real
20161005235448/gettext-0.16.1-r6/do_compile:
real 0m54.560s
user 0m46.028s
sys 0m2.772s
20161005225856/gettext-0.19.8.1-r0/do_compile:
real 0m41.520s
user 2m17.312s
sys 0m7.536s
(From OE-Core rev: 76dfad5b598e2937554bddeecf47482b14a854cd)
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This directory shouldn't contain local.conf and bblayers.conf - just
templates for them; except it doesn't have to contain those, it just has
to exist to pass this test. Change the error message accordingly, and
mention TEMPLATECONF so that the user has at least some context.
(From OE-Core rev: 61adaaa4348c670769f8750223977dbefe369ffb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using 'slirp' as a command line option to runqemu will start QEMU
with user mode networking instead of creating tun/tap devices.
SLIRP does not require root access. By default port 2222 on the
host will be mapped to port 22 in the guest. The default port
mapping can be overwritten with the QB_SLIRP_OPT variable e.g.
QB_SLIRP_OPT = "-net nic,model=e1000 -net user,hostfwd=tcp::2222-:22"
(From OE-Core rev: 80e6fc678f3dcd774d9376cdf2a6afcba2cd0b09)
Signed-off-by: Todor Minchev <todor.minchev@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"Shows" -> "Show", to be consistent with standard form of help output.
(From OE-Core rev: 5a7994df6cdb5af8d240e2802e6bb3d9671f17e3)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
mkefidisk.sh will soon be deprecated in favor of .wic images.
Added deprecation warning to the script to inform users that
this script will soon be removed from the codebase.
(From OE-Core rev: ccef84fac7b20d483df87aac8c620459fe31b6af)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If file_exclude matches a directory, os.unlink() got called with this
directory as an argument.
Filter out paths that end in a directory separator.
This still leaves the (then empty) directories, but this does not affect
the git commit.
(From OE-Core rev: e84b9185cc8f8e9f9df0e050543bb3a2c59426c3)
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* it reports at least 2 issues in every build (this file in
native and target sysroot) add it to whitelist
(From OE-Core rev: 798107887294072cacb23b668f446e151fe3c35f)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* format of bitbake tasks changed in:
2c88afb taskdata/runqueue: Rewrite without use of ID indirection
-ERROR: Task 4 (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb, do_fetch) failed with exit code '1'
+ERROR: Task /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) failed with exit code '1'
so strip not only '\.bb, .*' used before, but also '\.bb:.*' to drop
the task name to get recipe name.
* for more details see:
http://lists.openembedded.org/pipermail/openembedded-core/2016-June/123132.html
* without this change you can see test-dependencies.sh trying to rebuild packages
like:
Building recipe: fbprogress (6/21)
Building recipe: fbprogress.bb:do (7/21)
where the later of course doesn't exist as a recipe
(From OE-Core rev: 26ed215c30c183a3814889c145670220fd3b8bfa)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a branch/src-repository has the same name as a file/directory, git
since 1.4.0(?) gives an error like the one below:
ambiguous argument 'bitbake': both revision and filename
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Add two dashes to make the intent clear.
(From OE-Core rev: a84ec3ac15a59f72fcb46d97942009c8e459b5d0)
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When calling tempfile.NamedTemporaryFile().write(str()), at least on
Python 3.4.2 this fails with this error:
TypeError: 'str' does not support the buffer interface
Change the file-mode for all such files from binary to text mode.
(From OE-Core rev: d08f3882a35eec8a042d2501715684444e353605)
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new runqemu script assumes that if OECORE_NATIVE_SYSROOT is set then
it shouldn't try to run bitbake to find out the values of various
variables such as DEPLOY_DIR_IMAGE; this assumption is incorrect for the
extensible SDK. To work around this, clear OECORE_NATIVE_SYSROOT in the
environment when running runqemu.
Fixes [YOCTO #10447].
(From OE-Core rev: abff69a48bf3076ce8e21356accdc8d85d2c8dbf)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
According to UEFI specification all EFI platforms must support
GUID Partition Table(GPT) disk layout. Here is a list of advantages
of using GPT disk layout over the legacy MBR partitioning:
- Logical Block Addresses (LBAs) are 64 bits (rather than 32 bits).
- Supports many partitions (rather than just four primary partitions).
- Provides both a primary and backup partition table for redundancy.
- Uses version number and size fields for future expansion.
- Uses CRC32 fields for improved data integrity.
- Defines a GUID for uniquely identifying each partition.
- Uses a GUID and attributes to define partition content type.
- Each partition contains a 36 character human readable name.
Used GPT partitioning in all EFI kickstart files.
Tested result images on NUC, MinnowBoard MAX and MinnowBoard Turbot.
(From OE-Core rev: 66d4efeb0face5fc8a1755dcd2ed3a611997a04d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used ttyS0 console.
Removed usage of ttyPCH0 (FRI2 leftover)
Decreased bootloader timeout to 5 seconds
Removed 'vmalloc=256MB snd-hda-intel.enable_msi=0' as it's not
needed for any of reference BSPs.
(From OE-Core rev: ee16a283d695112236906338887c6eba109c9fb5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used ttyS0 console.
Removed usage of ttyPCH0 (FRI2 leftover)
Decreased bootloader timeout to 5 seconds
Removed 'vmalloc=256MB snd-hda-intel.enable_msi=0' as it's not
needed for any of reference BSPs.
(From OE-Core rev: 7b2443364a42c731bc9eefa6bb460262464e26b6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In modern kernels, ARCH=x86 defaults to 64 bits, so explicitly
unset this word length for qemu i386 architectures.
(From meta-yocto rev: 9750779bc78de3ea33d7f88e4972a5b21c3f811b)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used ttyS0 console.
Removed usage of ttyPCH0 (FRI2 leftover)
Decreased bootloader timeout to 5 seconds
Removed 'vmalloc=256MB snd-hda-intel.enable_msi=0' as it's not needed
for any of reference BSPs.
(From OE-Core rev: b46825a6ce5cecd24fed814bae48f9cfd0b97568)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Stopped using GPT partition table in mkefidisk.wks as it's not
supported by all reference hardware.
(From OE-Core rev: 07bb11b097a67ff89ae633fa1992db5494d75c0c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Disk identifier created by parted doesn't match the one we generated
and used in bootloader config. We need to rewrite it to make our image
bootable.
Modified involved API and data structures to access previously
generated disk identifiers after MBR is initialized. Written disk
identifiers to MBR.
(From OE-Core rev: 221d3bdd6e0ab8a4d25e2c96bd976cbec4e76681)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added generation of partition UUIDs for MSDOS partitions.
UUID for MSDOS partitions is <disk identifier>-<partition number>,
where disk identifier is a random 4 bytes long number. It's usually
generated when MBR/partition table is initialized.
As UUID is used to point to the root partition in bootloader config
we need to generate it before the MBR is initialized.
After MBR is created we need to rewrite system identifier to match
it with what is used in bootloader config. This will be implemented
in the next commit.
(From OE-Core rev: 6ecc6addf4080eda75a15af077816c81c6bf70a5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
sgdisk fails to set PARTUUID for msdos partitions as it's only
supported for GPT partitions.
Checked partition table format to run sgdisk --partition-guid
only for GPT partitions.
(From OE-Core rev: 0c1804821917d6d3a062ca19b63667d030138f21)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Makes it a bit more descriptive and potentially more discoverable. Most
people seemed to prefer an oe- prefix, so let's go with that.
(From OE-Core rev: 97e526ca10a00010987ffa3b90ec48337503a573)
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With recent changes to recipeutils, the list of local files returned
by get_recipe_local_files could possibly include source files. This
only happens when the recipe contains a SRC_URI using subdir= to put
files in the source tree. These files should be ignored when
populating the list of local files for oe-local-files directory.
[YOCTO #10326]
introduced in
OE-Core revision 9069fef5dad5a873c8a8f720f7bcbc7625556309
(From OE-Core rev: 31f1bbad248c36a8c86dde4ff57ce42efc664082)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
All kernel choices today (linux-yocto_4.* and custom) have the same xserver options,
so remove the duplicate lines.
(From meta-yocto rev: c456b5cf172e5ee1fca078383cad189325ea05f5)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Relying on that awk is installed on the target just to extract the
fourth column (i.e., the free volume size) from `df -P` is an
unnecessary dependency for devtool deploy-target. As it is already
using sed to mangle the output from `df -P`, this can easily be
modified to only extract the free volume size.
(From OE-Core rev: 7bab454b0bf0075fbb2a5de06286a9da1df2adc6)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user runs devtool add on an npm:// URL (or source tree that uses
node.js), and npm is not available, just build nodejs-native instead of
telling the user they need to do it; if that fails because there isn't
any such recipe (which would be the default, since it's not in OE-Core)
then produce a slightly more readable error message hinting at what the
user needs to do.
Note that this forces the use of nodejs-native rather than npm on the
host - this makes sense for two reasons: (1) we need it to be compatible
with nodejs for the target, and (2) we have to have a recipe for that
anyway, so allowing you to avoid having a recipe for the native version
isn't really beneficial.
There's a bit of a hack in here in order to allow this - for node.js
sources that aren't fetched via npm we don't know that they are that
until we've fetched and unpacked them, by which time we're inside
recipetool and have an active tinfoil instance that will prevent bitbake
being run. To avoid this being an issue, we allow recipetool to get to
the point where we know we need npm and then exit with a specific exit
code, at which point devtool can try to build it and then if that
succeeds, it will re-execute recipetool. This is definitely not ideal,
but it can't really be refactored and done properly until we do the
tinfoil2 refactoring; in the mean time though we still want to be
helpful to the user.
Fixes [YOCTO #10337].
(From OE-Core rev: f40662bde5aab158c4e4c3c3ff5e68665a4194a5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want to remove the -f/--fetch option at some point (as you can now
specify a URL as a positional argument instead) so display a warning
that it's deprecated if it is used.
(From OE-Core rev: 43476d77a91d50454ca26e016a3413b24e9f3aec)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were supposed to be printing out the specified recipe name here but I
forgot to specify a parameter for the string.
(From OE-Core rev: 87f844e533adfc229a5d26857a82cc6b125216c8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a preparation to use mkefidisk as a default wks for
genericx86* BSPs. This change enables usage of partition UUID
instead of device name to specify root partition in kernel
command line. It should make images to boot on devices with
boot device names that differ from what's mentioned in wks file.
(From OE-Core rev: 23cca700870230b46d251086441136e99659ef12)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Utilize the new return value (2) from oe-build-perf-test. Do not exit
with an error in case some individual tests fail. Even if some tests
fail we still want to complete successfully, that is, display and
archive the results and do cleanup. The individual tests do not depend
on each other anymore so test failures shouldn't affect the results of
successful tests.
(From OE-Core rev: e3c7d8a98a261a6a8c913e7fcd19264df501636d)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new return value '2' that indicates that some tests failed but
there were no fatal errors (i.e. configuration mistakes or bugs in the
tests themselves).
(From OE-Core rev: 194e95f3f068456f30c0e971eb8e6e775279427c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Display default values for '-a' and '-w' command line arguments in the
usage help text.
(From OE-Core rev: 580708398f22333bc4b5899e4129a8939fb7ce12)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Stricter checking of command line arguments. The script doesn't use any
positional arguments so don't accept any and error out if those are
found.
(From OE-Core rev: 4725ee8e4e4837446dfa3a319eb68cc9572c55eb)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add mipsel and mips64el as an option.
(From OE-Core rev: 072dd5b3b164ca7a5fd9dc969c991c15adeb0cbe)
Signed-off-by: Zubair Lutfullah Kakakhel <Zubair.Kakakhel@imgtec.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Makes it possible to average over multiple buildstats. If --multi is
specified (and the given path is a directory) the script will read all
buildstats from the given directory and use averaged values calculated
from them.
All of the buildstats must be from a "similar" build, meaning that no
differences in package versions or tasks are allowed. Otherwise, the
script will exit with an error.
(From OE-Core rev: 315f44ba39e9b13facacd0fd3796fa87329d9d69)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For comparing the elapsed wall clock time of tests. Default values for
--min-val and --min-absdiff are 5 seconds and 2 seconds.
(From OE-Core rev: 8e7a5beb2ce116bcd87111d190a4ac5d771e8884)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Two new options, making it possible to compare the number of filesystem
operations of tasks. Defaults for --min-val and --min-absdiff are set to
more or less arbitrary 500 and 50 operations, respectively.
(From OE-Core rev: 75292a1de1a59e19198d26b7c1291004a5ca92f3)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These are I/O counter values from /proc/<pid>/io and represent the
number of bytes read from / written to the storage layer. Default values
for --min-val and --min-absdiff limits are set to 512kB and 128kB,
respectively.
(From OE-Core rev: 24a12e40caeb05dac13c417f35733761af219f03)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A new command line option for choosing which "attribute" of the
buildstats to compare. At first, the already supported 'cputime' is the
only available option. But, refactoring done in this patch should make
it easy to add new attribute types.
(From OE-Core rev: 0782825138731b3f1e6a8e05d723c1d5cd60c90c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Dynamically adjust the width of all fields in task diff output. Makes
it easier to print other units than cputime, too.
(From OE-Core rev: 559b858f2a3712ec21debb71681593bd7cf55041)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New class representing buildstats data of a single task.
(From OE-Core rev: 472818a32f96699a6dc9c7c487f38d716678fd7a)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rename these arguments to --min-val and --min-absdiff in preparation for
supporting other "quantities" than just cputime.
(From OE-Core rev: 441336bc1750939c2da2d9e4dc5a6893b283bf68)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rename wks for systemd-boot per the suggestion from community.
Also amend description to distinguish it from others when
running "wic list images".
(From OE-Core rev: 6303dbbaa08214a37caf38e3b6b5a30a108bd3b7)
Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
QEMU produces a warning if drive format is not specified:
WARNING: Image format was not specified for
'tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic'
and probing guessed raw.
Automatically detecting the format is dangerous for raw images,
write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
Set image format to 'vmdk', 'qcow2' or 'vdi' for correspondent image
types. Set it to 'raw' for the rest of image types.
(From OE-Core rev: 5100bb36502ef7c81220a3c4809eb1b3ac83801f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This function was broken by the multi-config changes, and isn't needed anymore
now that recipeutils.pn_to_recipe can handle provides. Without this, the
newappend sub-command fails.
(From OE-Core rev: 4a5028dc3d1ab2f97465e63db5b05de73daebdfa)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the change of crosssdk to use SDK_SYS, we need to update the script
to match.
(From OE-Core rev: 8ab1f6073f86d05493bb32a8135c9d912d72f5f0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When devtool creates a new workspace, it produced a README with one very
long line and no space following 'bblayers.conf'. Add a line break as was
intended.
(From OE-Core rev: 4ad1bcfc3c88ced5b7fc80c950613e31becb40f3)
Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If runqemu-ifup fails hen running testimage, a rather cryptic error
regarding "no tty present" is displayed. If this step fails, we
should at least point the user at runqemu-gen-tapdevs. A quick search
of this term in the manual will lead them to "Enabling Runtime Tests
on QEMU" which should give them all the info they need.
(From OE-Core rev: 3b6494fad2b8b65e0d52cda0cdf500e93c72823a)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Preserving images for every partition doubles disk space
consumed by an image build. As those images are not used,
so it's better to remove them after assembling final image.
(From OE-Core rev: 51171b4aa10f2218c5e27d785ca7bf4f3949a4b4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When booting a system with the rootfs being of cpio* type the networking
setup should still work the same as for all other root filesystem types.
This change removes the clearing of the NETWORK_CMD variable allowing
for the slirp/tap setup to be provided to QEMU.
(From OE-Core rev: 7d01a9c80de0cdbac3831301dd996c7b61754c74)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Not all QEMU machines (outside of those available in OE-Core) are
capable of using the virtio-rng-pci device due to various machine models
not having a pci/virtio bus. This makes it such that the use of the
'-device virtio-rng-pci' flag to QEMU is machine specific.
This patch removes the general addition of the flag to all runqemu
targets and adds the flag into the QB_OPT_APPEND for all the qemu*
machines in OE-Core that support its use (which is all of them).
(From OE-Core rev: e890c05e66a21702e9e8ccce794b74cb7f5518ed)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added serial console to kernel command line to to make it
easier to boot wic images on devices without display.
Tested on MinnowBoard MAX.
(From OE-Core rev: 9a774e3bcd5dc4c85d642acc7bf26095b8c620e4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If DEPLOY_DIR_IMAGE doesn't exist during check_arg_machine() we
will attempt to guess a suitable value later when check_and_set()
calls validate_paths(), therefore this shouldn't raise an exception
(From OE-Core rev: ed8d6f391c567048bd50dc3234804915f8212cef)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a MACHINE value is passed we can't validate it by running bitbake
as the toolchain environment doesn't include the build system, we
must assume that the passed value for MACHINE is correct.
(From OE-Core rev: 2c569678566c49b3ea237ef2de0fbae782263449)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Emulate some logic from the prior, shell based, version of runqemu
to try and infer the correct setting for MACHINE from the kernel
and rootfs filenames.
(From OE-Core rev: a5adabe1414061d6864c5913dd5e66a4527838f1)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to validate and ensure all paths are set regardless of
whether runqemu was invoked with a .qemuboot.conf file or
otherwise. Split this logic out into a separate method called
during check_and_set()
(From OE-Core rev: e843b2d49a151c1fe0d2a7ba00c41d2a35775736)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Search rootfs in the following order:
- IMAGE_NAME*.FSTYPE
- IMAGE_LINK_NAME*.FSTYPE
* Search kernel in the following order:
- QB_DEFAULT_KERNEL
- KERNEL_IMAGETYPE
- KERNEL_IMAGETYPE*
* Search dtb in the following order:
- QB_DTB
- QB_DTB*
- *.dtb
* Fix DTB, it should only work with "-kernel" option.
[YOCTO #10265]
(From OE-Core rev: 32ff0974ed06f797c6b7d9092a8dc9ae50e9a572)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It creats /etc/runqemu-nosudo when creats taps, so should remove it when
remove taps.
(From OE-Core rev: 3d4bf5b0ea581e3e9b388328b086d03f9174fd61)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no STAGING_DIR_NATIVE or bitbake in a extracted sdk,
so check OECORE_NATIVE_SYSROOT and use it.
(From OE-Core rev: 93649edc034f2540ff55dc9b41638797209cfb9c)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A qemuboot conf file is a convenience but it should still be
possible to invoke runqemu without them, especially for examples
such as using the SDK with an extracted rootfs via NFS.
As read_qemuboot() is always called we need to be sure that function
can return cleanly, without throwing Exceptions, even if a qemuboot
conf file isn't found.
(From OE-Core rev: 3541c21f1976b517b79a19882240a8f36b970292)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the kernel or rootfs names written to the qemuboot.conf can't
be found, try and find the symlinked variant of the filename.
This will help usability of runqemu, for example where a user
downloads an image and associated files as the symlinked names
yet the qemuboot.conf variables point to the full, non-linked,
file names.
(From OE-Core rev: ca5a686c6e165a51f95cb6a834cd53f6f66d42d4)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make it clearer that we are looking for a file which ends with
qemuboot.conf
(From OE-Core rev: 2579e05269a14b53a54232a8bf4414ac2dfe6472)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a required binary cannot be found print some guidance pointing
to using a sourced OE build environment or a qemuboot.conf file,
based on a similar message from the previous shell-based runqemu.
(From OE-Core rev: 87cfb5165490cd4e7a8c2570ef5a62898db8395e)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that recipeutils.validate_pn() properly validates characters used in
the name, we can drop this bit checking for '/' since that's not
permitted by validate_pn(). (The FIXME comment here - that I myself
apparently wrote - is questionable since that function was clearly never
intended to allow '/', perhaps I was misled because it was broken and
did so).
(From OE-Core rev: e010d9be3709cf3c607ffc03c3188abe4e1e9eb4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In keeping with making recipetool create / devtool add as easy to use as
possible, users shouldn't have to know how to reformat git short form ssh
URLs for consumption by BitBake's fetcher (for example
user@git.example.com:repo.git should be expressed as
git://user@git.example.com/repo.git;protocol=ssh ) - instead we should
just take care of that automatically. Add some logic in the appropriate
places to do that.
(From OE-Core rev: 78c672a72f49c4b6cfd8c247efcc676b0ba1681a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Filter out a plain "Licensed under the XXXX license" statement, as seen
in the capnproto project (and no doubt others).
(From OE-Core rev: ba4aa319fd49ee02ce2e30c2db0f3988c0e8833c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
AX_PKG_SWIG is not the only commonly-used macro for detecting swig -
there's also AC_PROG_SWIG. As per AX_PKG_SWIG, add swig-native to
DEPENDS if AC_PROG_SWIG is found in configure.ac.
(From OE-Core rev: 847a1aa7153fc8a7b820353283a6f1e51d64f8de)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If python is required then we need to inherit pythonnative (or
python3native) otherwise do_configure will probably fail since it won't
be able to find python.
(From OE-Core rev: 63234cc45aee91b031657971f36997e1443f80ee)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When creating a recipe for an existing local git clone, we attempt to
use the fetcher to determine if it supports the SRCREV variable.
Unfortunately running this code does a network check to get the latest
revision as a direct result of us using '${AUTOREV}' as a default value.
If you don't have a network connection this will of course fail. Rather
than have this block creating the recipe, catch the exception and just
guess from the URL.
Ultimately this should probably be fixed in the fetcher but for now this
will at least resolve the issue on this end.
(From OE-Core rev: f7e43f931d7d6019a3b2509b2b2635978fbbae36)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I ran into an example where recipetool was getting the name/version
completely wrong:
https://bitbucket.org/sortsmill/libunicodenames/downloads/libunicodenames-1.1.0_beta1.tar.xz
>From this it would create a libunicodenames-1.1.0-beta1_1.1.0-beta1.bb
file (likely because it couldn't split the file name and therefore took
all of it, then got the version from one of the files inside the
tarball). When this happens it's just irritating because you then have
to delete the recipe / run devtool reset and then run recipetool create
/ devtool add again and specify the version manually.
This patch is the result of systematically running the
determine_from_filename() function over the files on the Yocto Project
source mirror and my local downloads directory and fixing as many of the
generic issues as reasonably practical - it now gets the name and
version correct much more often. There are still cases where it won't,
but they are now in the minority.
(From OE-Core rev: 7b018b1d493a8d10fd02b8cc220990b191c87fe5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Try to ensure that for Apache, GPL and LGPL where the values extracted
from the "Classifiers" field may not be version-specific, if there is a
versioned license in the free-form license field then use that instead.
Also insert the free-form license field as a comment in the recipe for
the user's reference.
(From OE-Core rev: 237f66042eedd906f654827b53bf9269738267ab)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make use of the extravalues dict to send back other variable values from
the python handling plugin, and enable passing back PV and PN. This not
only places variable values in the final recipe a bit more consistently
with other types of source, it also allows the name and version to be
picked up fron a local source tree and not just when the recipe is
fetched from a remote URL that happens to have those in it.
(From OE-Core rev: 3e7029f28c6ea9bb1d283bcdc3fdfee11455af8e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The open(self.lock, 'w') may fail when the lock is created by other
users, return false for this case to let it try other devices.
Fixed:
runqemu - INFO - Running /sbin/ip link...
runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap0.lock...
Traceback (most recent call last):
File "/buildarea/lyang1/poky/scripts/runqemu", line 972, in <module>
ret = main()
File "/buildarea/lyang1/poky/scripts/runqemu", line 963, in main
config.setup_network()
File "/buildarea/lyang1/poky/scripts/runqemu", line 810, in setup_network
self.setup_tap()
File "/buildarea/lyang1/poky/scripts/runqemu", line 761, in setup_tap
if self.acquire_lock():
File "/buildarea/lyang1/poky/scripts/runqemu", line 182, in acquire_lock
lock_descriptor = open(self.lock, 'w')
PermissionError: [Errno 13] Permission denied: '/tmp/qemu-tap-locks/tap0.lock'
(From OE-Core rev: f364f773a0381a75b5992c8c8a1d63a81dbd4422)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Shell functions in bitbake are executed with /bin/sh so should be POSIX
compliant and not use Bash extensions, or at least only use extensions that are
implemented in both dash and ash (busybox).
This tool will extract all of the shell scripts from all recipes and run them
through checkbashisms (it assumes that checkbashisms is on $PATH).
There is a whitelist to filter out false-positives such as the use of $HOSTNAME
(a bashism) in functions where we have defined it, or using the 'type' builtin
which is supported by ash/dash.
[ YOCTO #8851 ]
(From OE-Core rev: d77fe838ab7631a19e90ff4226f0712e54aa4e22)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New script for comparing buildstats from two separate builds. The script
has two modes: normally it prints the differences in task execution
(cpu) times but using --ver-diff option makes it just print the recipe
version differences without any cpu time data. Other command line
options are provided to alter the sort criteria of the data and to
filter out insignificant differences and/or short tasks.
(From OE-Core rev: e707718374ce1c95769a5f99aa3bfdfc0be685b2)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python coverage versions lower than 4.x have problems with some distros. Adding the 4.x version as requirement to continue with coverage tracking.
[YOCTO #10207]
(From OE-Core rev: a378b817504986173c4b0984a28aead247589b3f)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When 'git request-pull' fails it makes sense to remove output
directory. Otherwise create-pull-request will complain that
output directory already exists on the next run.
(From OE-Core rev: 8535784ba84060e0e7ca05e7771a58f9c70c69b9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There might be a race issue when multi runqemu processess are
running at the same time:
| Traceback (most recent call last):
| File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 920, in <module>
| ret = main()
| File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 911, in main
| config.setup_network()
| File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 760, in setup_network
| self.setup_tap()
| File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 697, in setup_tap
| os.mkdir(lockdir)
| FileExistsError: [Errno 17] File exists: '/tmp/qemu-tap-locks'
(From OE-Core rev: ec33043477a0b915b0911f7d7eacb24361e4aaa8)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow access to the snapshot option of qemu to simplify some of our runtime
testing to avoid copying images.
(From OE-Core rev: 8fec4a5a004f0e99734f8c0820c66522d08f213e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have problems where systems simply stop booting and hang. This is due
to a lack of entropy which means ssh keys and networking can't be brought
up. Adding in the virtio-rng passthrough support allows host entropy to
pass into the guess and avoids these hangs.
This is particularly problematic after the gnutls upgrade which starts
using /dev/random instead of /dev/urandom but was an issue we'd occasionally
seem before that.
It particualrly affected x86 and ppc machines for some reason.
(From OE-Core rev: 51b001909f1856c45cf87091d6e4446c266d5786)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current qemu instances all share the same MAC address. This shouldn't be an
issue as they are all on separate network interfaces, however on the slight
chance this is causing problems, its easy enough to ensure we use unique
MAC addresses based on the IP numbers we assign.
(From OE-Core rev: c01962bf88786dd84ad83cc1d315297607d29f7c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
testimage.bbclass uses runqemu to execute runtime tests on a qemu
target, this means that bitbake is already running and `bitbake -e`
can't be called to obtain bitbake variables.
runqemu tries to work around being unable to read values for
bitbake variables by inferring the MACHINE from the
DEPLOY_DIR_IMAGE setting, however if a user sets that variable in
a manner which doesn't follow the systems expectations (i.e. if
running `bitbake -c testimage` against a directory of pre-generated
images in a user-specified path) the inferring of the MACHINE name
from the DEPLOY_DIR_IMAGE location will fail.
It's possible that check_arg_machine() shouldn't cause runqemu to
fail and that runqemu should proceed with the user-supplied value
even if it can't be verified. This patch simply ensures that a
workflow where the user sets DEPLOY_DIR_IMAGE continues to work
without changing too much of the runqemu code.
[YOCTO #10238]
(From OE-Core rev: f94ac02f459e2ea0fc471463966997814a67e0ca)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When runqemu is invoked from a running bitbake instance it will be
unable to call `bitbake -e` due to the lock held by the calling
bitbake instance.
Our test code sets an OE_TMPDIR environment variable from which we
can infer/guess paths. Add code to do so when self.bitbake_e can't
be set, much as the sh version of runqemu did.
[YOCTO #10240]
(From OE-Core rev: 1e8165ea2f19aecdc03ccd102ee44ef0544f0f39)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the STAGING_*_NATIVE directories from the config file don't exist
and we're in a sourced OE build directory try to extract the paths
from `bitbake -e`
(From OE-Core rev: 9326af1c20636320c70caecebd47aedafb3f2d25)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When runqemu is started with a *.qemuboot.conf arg assume that image
artefacts are relative to that file, rather than in whatever
directory the DEPLOY_DIR_IMAGE variable in the conf file points to.
(From OE-Core rev: a6448371b87f754def669adfdc01b07d18003405)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, runqemu had hard coded machine knowledge, which limited its
usage, for example, qemu can boot genericx86, but runqemu can't, we need
edit runqemu/runqemu-internal a lot if we want to boot genericx86.
Now bsp conf files can set vars to make it can be boot by runqemu, and
qemuboot.bbclass will save these info to DEPLOY_DIR_IMAGE/qemuboot.conf.
Please see qemuboot.bbclass' comments on how to set the vars.
* Re-write it in python3, which can reduce lines from 1239 to about 750
lines
* All the machine knowledges are gone
* All of the TUN_ARCH knowledge are gone
* All the previous options are preserved, and there is a new way to run
runqemu: (it doesn't need run "bitake -e" in such a case)
$ runqemu tmp/deploy/images/qemux86
or:
$ runqemu tmp/deploy/images/qemuarm/<image>.ext4
or:
$ runqemu tmp/deploy/images/qemuarm/qemuboot.conf
* Fixed audio support, not limited on x86 or x86_64
* Fix SLIRP mode, add help message, avoid mixing with tap
* Fix NFS boot, it will extract <image>.tar.bz2 or tar.gz to
DEPLOY_DIR_IMAGE/<image>-nfsroot when no NFS_DIR, and remove it after
stop.
* More bsps can be boot, such as genericx86 and genericx86-64.
* The patch for qemuzynq, qemuzynqmp, qemumicroblaze has been sent to
meta-xilinx' mailing list.
* I can't find any qemush4 bsp or how to build it, so it is not
considered atm.
[YOCTO #1018]
[YOCTO #4827]
[YOCTO #7459]
[YOCTO #7887]
(From OE-Core rev: 60ca8a8d899b90a4693fd62b6ec97d0c76a9f6c5)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we output extra blank lines (because of some automated editing) then
it makes the output recipe look a bit untidy. You could argue that we
should simply have the editing code not do that, but sometimes we don't
have enough context there for that to be practical. It's simple enough
to just filter out the extra blank lines when writing the file, so just
do it that way.
(From OE-Core rev: cbebc9a2edf7d7a422ee5c71219e79e3b349de3b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have your own node.js application you may not publish it (or at
least not immediately) in an npm registry - it might just be in a
repository on github or on your local machine. Add support to recipetool
create for creating recipes to build such applications - extract their
dependencies, fetch them, and add corresponding npm:// URLs to SRC_URI,
and ensure that LICENSE / LIC_FILES_CHKSUM are updated to match. For
example, you can now run:
recipetool create https://github.com/diversario/node-ssdp
(I had to borrow some code from bitbake/lib/bb/fetch2/npm.py to
implement this functionality; this should be refactored out but now
isn't the time to do that refactoring.)
Part of the fix for [YOCTO #9537].
(From OE-Core rev: 4fb8b399c05a1b66986fc76e13525f6c5e0d9b58)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you make adjustments to the source tree (as create_npm.py will be)
then you will need to re-run the license variable handling code at the
end so that we get all of the files that should go into
LIC_FILES_CHKSUM if nothing else. Split out the license variable
handling to a separate function in order to allow this.
(From OE-Core rev: f0d6f4b7e87ea781ac0dffcc8d0310570975811b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For debugging it's useful to be able to tell recipetool to keep the
temporary directory.
(From OE-Core rev: 480a6b745a85b2881e5cc1a0bbb572e3235ca008)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure we fetch submodules and set SRC_URI correctly when pointing to a
git repository that contains submodules.
(From OE-Core rev: 65d5cc62d4ecfc78ce4b37b3886a7fe5aa05a75e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When trying to map python module dependencies to the packages that
provide them, if we're looking for .so files that satisfy
dependencies then we need to exclude files found under the .debug
directory, otherwise the dependency will get mapped to the python-dbg
package which isn't correct.
For example, this fixes creating a recipe for pyserial and not getting
python-fcntl in RDEPENDS_${PN}, leading to errors when trying to use the
serial module on the target.
(From OE-Core rev: 46a068ca35975988a8e9c0310f71fdcee55937a4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If AX_PKG_SWIG is found in configure.ac, then what's being looked for is
the swig binary, not swig for the target - so fix the dependency
accordingly.
(From OE-Core rev: 2600cd6f6c63ecf79804e2bc6eb6f198a012d5d6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's rare but there are recipes that have individual files (as opposed
to archives) in SRC_URI using subdir= to put them under the source tree,
the examples in OE-Core being bzip2 and openssl. This broke devtool
update-recipe (and devtool finish) because the file wasn't unpacked into
the oe-local-files directory and thus when it came time to update the
recipe, the file was assumed to have been deleted by the user and thus
the file was erroneously removed. Add logic to handle these properly so
that this doesn't happen.
(We still have another potential problem in that these files become part
of the initial commit from upstream, which could be confusing because
they didn't come from there - but that's a separate issue and not one
that is trivially solved.)
(From OE-Core rev: 9069fef5dad5a873c8a8f720f7bcbc7625556309)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The --log-file command line argument was slightly broken as {out_dir}
string replacement was not working as expected.
(From OE-Core rev: fc62f54e3d788cc79fd27664f05db7efccef23ab)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Not specifying -C caused oe-build-perf-test to try to commit results to
the build directory.
(From OE-Core rev: 6f4786f5522c366a7fd92f630c3f32629a9f9471)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should always shut down tinfoil when we're finished with it, either
by explicitly calling the shutdown() method or by using it as a
context manager ("with ...").
(From OE-Core rev: 5ec6d9ef309b841cdcbf1d14ac678d106d5d888a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The multiconfig changes altered some of the functions being called here,
so update the calls. Make use of the new Tinfoil.parse_recipe_file()
function to make parsing easier.
(From OE-Core rev: 95b6ceffd947271f315d8a7660797ab371adfbb9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using PATCHTOOL = "git", the user of the system is not really the
committer - it's the build system itself. Thus, specify "dummy" values
for username and email instead of using the user's configured values.
Various parts of the devtool code that need to make commits have also
been updated to use the same logic.
This allows PATCHTOOL = "git" and devtool to be used on systems where
git user.name / user.email has not been set (on versions of git where
it doesn't default a value under this circumstance).
If you want to return to the old behaviour where the externally
configured user name / email are used, set the following in your
local.conf:
PATCH_GIT_USER_NAME = ""
PATCH_GIT_USER_EMAIL = ""
Fixes [YOCTO #8703].
(From OE-Core rev: 765a9017eaf77ea3204fb10afb8181629680bd82)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rename the (main) log file of the oe-build-perf-test script from
'output.log' to 'oe-build-perf-test.log'. Also, add a new command line
option --log-file which makes it possible to use an alternative log file
name/path, if needed. Note that the file name/path is relative to the
output directory.
(From OE-Core rev: 4909fae1a6d1d068b33252088b41b8d82d1a836c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Write globalres log file and commit results to Git even if some tests
failed. Now that tests do not depend on each other there should be no
risk of bogus results caused by test failures.
(From OE-Core rev: 8036975b268fe209476e230555006facd3cbda71)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New command line argument '-w' may be used to specify work dir other
than the default <GIT_DIR>/build-perf-test.
(From OE-Core rev: 824284895f25146520a624b7b97f7475d0135814)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add new command line argument '-a' that can be used to define the
directory where results (tarballs) are archived. Giving an empty string
disables archiving which makes sense if you store results in Git.
(From OE-Core rev: d53cf92847aa80724be4412801c993948a09cd27)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add new command line argument '-C' that allows saving results in a Git
repository.
(From OE-Core rev: 3d06795d8cd9017b042a7283c16ac71d4f6317a6)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use getopts for parsing the command line. This changes the usage so that
if a commit (to-be-tested) is defined it must be given by using '-c',
instead of a positional argument.
(From OE-Core rev: b1f77ba41033397a2b25977963682b86f2f76471)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Makes it possible to create easily sortable tags. Also, the default tag
format is updated to use the new keyword.
(From OE-Core rev: e3161654d75dfc3b059c519205b38b26e3ffb215)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes it possible to create numbered tags, where the "basename" of
the tag is the same and the only difference is an (automatically)
increasing index number. This is useful if you do multiple test runs on
the same commit. For example, using:
--commit-results-tag {tester_host}/{git_commit}/{tag_num}
would give you tags something like:
myhost/decb3119dffd3fd38b800bebc1e510f9217a152e/0
myhost/decb3119dffd3fd38b800bebc1e510f9217a152e/1
...
The default tag format is updated to use this new keyword in order to
prevent unintentional tag name clashes.
(From OE-Core rev: cf2aba16338a147f81802f48d2e24a96c7133548)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a Git tag when committing results to a Git repository. This patch
also implements --commit-results-tag command line option for controlling
the tag name. The value
is a format string where the following fields may be used:
- {git_branch} - target branch being tested
- {git_commit} - target commit being tested
- {tester_host} - hostname of the tester machine
Tagging can be disabled by giving an empty string to
--commit-results-tag. The option has no effect if --commit-results is
not defined.
(From OE-Core rev: 60059ff5b81d6ba9ba344161d51d1290559ac2df)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do a pre-check on the path that is specified with --commit-results
before running any tests. The script will create and/or initialize a
fresh Git repository if the given directory does not exist or if it is
an empty directory. It fails if it finds a non-empty directory that is
not a Git repository.
(From OE-Core rev: 759357a3bdbe75a3409b9e58979ab8b45d9b6ae8)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A new command line option for defining the branch where results are
commited. The value is actually a format string accepting two field
names:
- {git_branch} expands to the name of the target branch being tested
- {tester_host} expands to the hostname of the tester machine
The option has no effect if --commit-results is not used.
(From OE-Core rev: b54b63395ec632748a57a702812c8a9a07af35ab)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implement a new command line option '--commit-results' which commits the
test results data into a Git repository. The given path must be an
existing initialized local Git repository.
(From OE-Core rev: b6f635513ca971402e7a970acc2168fb5d4a9476)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is safer as the current working directory may change.
(From OE-Core rev: 4b7bf7860713581ba351599fe32817ba24e8f8d0)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Makes it possible to run only a subset of tests.
NOTE: The tests currently have (unwritten) dependencies on each other so
use this option with care. Mainly for debugging.
(From OE-Core rev: be4373be54e5b84f951771b0e75140f212838020)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding argparse module from Python's standard library. This allow use
argparse without installing all python-misc modules. For compatibility,
add python3-argparse as RDEPENDS to python3-misc.
(From OE-Core rev: f2b96001e074d26f5eb8711c2217a695fb02de4c)
Signed-off-by: Fabio Berton <fabio.berton@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Queue module has been renamed to queue in Python 3.
(From OE-Core rev: e19a430da2ef60b2c6cf6a67210ec1a7b292c8ca)
Signed-off-by: Fabio Berton <fabio.berton@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The missing split() causes dev and dbg packages to match.
(From OE-Core rev: bf83e0f0a3d52958c4380599f1afc4b8e058afd7)
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The --wilcard-version flag was only used in the srcrev variant of the
update-recipe command.
(From OE-Core rev: d3057cba0b01484712fcee3c52373c143608a436)
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
check_patch function opens patch file in text mode. This causes
python3 to throw exception when calling readline():
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa7 in position
NNNN: invalid start byte
Opening file in binary mode and using binary type instead of strings
should fix this.
(From OE-Core rev: a7f1435c4c26237cdb55066c9f5408b4fdf016aa)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
$ python3
>>> import signal
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/path/to/sdk/sysroots/x86_64-pokysdk-linux/usr/lib/python3.5/signal.py", line 4, in <module>
from enum import IntEnum as _IntEnum
ImportError: No module named 'enum'
(From OE-Core rev: 6306dc8351c19059c4c2a8e75bb5733e64532732)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous attempt on this was a bit erroneous, dropping time stamps
completely although only the timestamp format should've been changed.
(From OE-Core rev: bafcff95e2b5e0b9a8c76ce46a62667bf6f49b00)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately to implenent multiconfig support in bitbake some APIs
had to change. This updates code in OE to match the changes in bitbake.
Its mostly periperhal changes around devtool/recipetool
[Will need a bitbake version requirement bump which I'll make when merging]
(From OE-Core rev: 041212fa37bb83acac5ce4ceb9b7b77ad172c5c3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With some recent changes in the kern tools, we can drop some changes in
the yocto-bsp and yocto-kernel tools that ensured proper patching and
BSP inheritance.
In particular, we no longer need to signify the start of patching, and
we must instruct the tools that we only want configuration fragments
via inheritance .. no patches (since they are already applied).
(From meta-yocto rev: 34ed5eebd0b5baab98b6b2d7b3f06ca40932b37d)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove timestamps from the stderr log in order to make the console
output more readable, i.e. more in line with the output from unittest
runner.
(From OE-Core rev: d28eeeabde9b4b7160a273445023a44fd50e29ab)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
So that the log file would not miss any records.
(From OE-Core rev: 9ce6e20ce239067896dc65f09e3fef1173293065)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prevent logger from writing to stderr when the tests are being run by
the TestRunner. During this time the logger output is only written to
the log file. This way the console output from the script is cleaner and
not mixed with possible logger records.
(From OE-Core rev: 36f58b5172d4e2e182aa447fb3ec4d1ac9f6820d)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Convert scripts/oe-build-perf-test to be compatible with the new Python
unittest based buildperf test framework.
(From OE-Core rev: 249d99cd7ec00b3227c194eb4b9b21ea4dcb7315)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new class is derived from unittest.TextTestResult class. It is
actually implemented by modifying the old BuildPerfTestRunner class
which, in turn, is replaced by a totally new simple implementation
derived from unittest.TestRunner.
(From OE-Core rev: 89eb37ef1ef8d5deb87fd55c9ea7b2cfa2681b07)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds the following packages: python3-enum (needed by python3-git),
python3-selectors (needed by python3-subprocess), python3-signal (needed by python3-subprocess),
and it also fixes the following ones with missing dependencies: python3-subprocess,
python3-compression, python3-datetime
[YOCTO #10127] [YOCTO #10124] [YOCTO #10122]
(From OE-Core rev: 0575e8c9fb52a7b594025fd20445a2edd06e3c69)
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the dot parser to the new networkx API (using pydotplus to parse).
Also, switch the path display to output the paths as they are found instead of
collecting them into a list, so output appears sooner.
(From OE-Core rev: c91898b07465fdd5f3629babb7ff9226454de24e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you set up a local mirror in SSTATE_MIRRORS then you can end up with
symlinks in SSTATE_DIR rather than real files. We don't want these
symlinks in the sstate-cache prodcued by gen-lockedsig-cache, so
dereference any symlinks before copying.
(From OE-Core rev: d65a6ee9e7a9c63b9a16bdb5025af8a7c6433c4f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you check out OE-Core and then run oe-init-build-env you get an error
about not having bitbake checked out in a "bitbake" subdirectory,
however it's possible to specify the bitbake path on the
oe-init-build-env command line, so hint at that in the error message
rather than implying it has to be in the default location.
(From OE-Core rev: 5a1efa91a418e3206b047564d0fd6d5bac22a8d3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support to direct boot Linux instead of just booting u-boot.
(From OE-Core rev: e5c6a78db46192800669f1b392351f6b52f3e20c)
Signed-off-by: Alistair Francis <alistair.francis@xilinx.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Set cover letter's subject automatically as the patch's subject when
there is only one patch.
[YOCTO #9410]
(From OE-Core rev: 162b80f8a4670befaf6ffd2c178671cf7370b767)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
So that we don't have specify "-u <contrib>" everytime, and
CPR_CONTRIB_REMOTE can be overrided by -u.
[YOCTO #9409]
(From OE-Core rev: 81c58fd33e725ce7dba693763646f4c30747bbd5)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before this patch, we need two steps to create PULL:
* Step 1, create branch:
$ git push <contrib> <local_branch>:<remote_branch>
* Step 2, create PULL:
$ create-pull-request -u <contrib> -l <local_branch> -b <remote_branch> -r <local_branch>~<n>
We can see that the args used in step 1 are in step 2, so we can use
"create-pull-request -a" or set CPR_CONTRIB_AUTO_PUSH in to create the
branch to simplify the steps.
[YOCTO #9408]
(From OE-Core rev: a569bec9219394703d1c1d9b28dd19bf5b058e7f)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This bash script prints list of modules uncovered by oe-selftest
or any other test that produces coverage report.
It expects coverage report on its stdin and a directory to look
for python modules as a command line parameter, e.g.
coverage report --rcfile=build/.coveragerc | ./scripts/contrib/uncovered bitbake/
should print list of uncovered python modules from bitbake/
directory tree to stdout.
[YOCTO #9809]
(From OE-Core rev: 00d9df2b70d00b9767f32f172192f33cbf3aca0e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
avoiding characters like ':' and making a clearer separation of the
fields that compose the filename. Changing from:
oe-selftest-2016-07-20_16:05:27.log
to:
oe-selftest-20160720-160527.log
(From OE-Core rev: e7b2362d723b5dcabb440cd513380bfe8a0badb2)
Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
if available, use the xmlrunner for exporting the test results to a
dir named the same than the log where the text results are stored.
this means creating a dir with the name of the log (without the .log)
and dumping there the xml files that indicate the results of each of
the tests.
if xmlrunner is not available then it will behave the same as before,
no xml exports.
[YOCTO#9682]
(From OE-Core rev: d51f9dd34d759c77b9e7050405cbb6a88a578f73)
Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The regex here needs to be anchored to the end or it'll match longer
URLs, which was exactly what I was trying to avoid. This regression was
introduced in OE-Core revision 7998dc3597657229507e5c140fceef1e485ac402.
Fixes [YOCTO #10023].
(From OE-Core rev: 9291c5d3c257d5ada7605dfe46ababda08f6d3c1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a comment to the recipe listing license files that were found but
not able to be identified, so that the user can find and examine them
by hand fairly easily.
Fixes [YOCTO #9882].
(From OE-Core rev: 4b7d1bf8172533e9ac91a49ade152a05e2ee4146)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the new oe-check-sstate to filter the sstate artifacts shipped with
the extensible SDK by effectively running bitbake within the produced
eSDK and and getting it to tell us which tasks it will restore from
sstate. This has several benefits:
1) We drop the *-initial artifacts from the minimal + toolchain eSDK.
This still leaves us with a reasonably large SDK for this
configuration, however it does pave the way for future reductions
since we are actually filtering by what will be expected to be there
on install rather than hoping that whatever cuts we make will match.
2) We verify bitbake's basic operation within the eSDK, i.e. that
we haven't messed up the configuration
3) We verify that the sstate artifacts we expect to be present are
present (at least in the sstate cache for the build producing the
eSDK). Outside deletion of sstate artifacts has been a problem up to
now, and this should at least catch that earlier i.e. during the
build rather than when someone tries to install the eSDK.
This does add a couple of minutes to the do_populate_sdk_ext time, but
it seems like the most appropriate way to handle this.
Should mostly address [YOCTO #9083] and [YOCTO #9626].
(From OE-Core rev: 4b7b48fcb9b39fccf8222650c2608325df2a4507)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a script to check which sstate artifacts would be installed by
building a given target - by default this is done with a separate
TMPDIR to ensure we get the "from scratch" result. The script produces a
list of tasks that will be restored from the sstate cache. This can also
be combined with BB_SETSCENE_ENFORCE* to check if sstate artifacts are
available.
The implementation is a little crude - we're running bitbake -n and
looking at the output. In future when we have the ability to execute
tasks from tinfoil-based scripts we can look at rewriting that part of
it to use that instead.
(From OE-Core rev: 4d059e02099e6244765027f2771192434764c606)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rawcopy plugin copies source files to build folder before using them
to assemble result image. After assembling the image wic renames
source files to <image>.p<partition number>. If the same source file
is used in multiple partitions wic breaks trying to rename file that
doesn't exist.
Added <line number> suffix to the files when copying them to the
build dir. This should make filename unique even if the same source
file is used for multiple partitions.
[YOCTO #9826]
(From OE-Core rev: 43a809bfe99024083b4ab4eb9895b084c9c4fa80)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For example:
$ oe-selftest --run-tests-by name hello world
2016-07-12 00:33:28,678 - selftest - ERROR - Failed to find test: hello
2016-07-12 00:33:28,679 - selftest - ERROR - Failed to find test: world
(From OE-Core rev: 665a0f93bde0d61e0c7ceab072ca3f1f22b2f700)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
gethostip comes from syslinux. It seems odd to depend on a bootloader
to clone a git repository.
Switch to using getent from the c-library, which should be available
on every system.
We now also support the case where a hostname resolves to more than
one IP address.
(From OE-Core rev: c91dbf3ca2faec95195c85b65aa6cab7de9bca2c)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a subcommand which will "finish" the work on a recipe. This is
effectively the same as update-recipe followed by reset, except that the
destination layer is required and it will do the right thing depending
on the situation - if the recipe file itself is in the workspace (e.g.
as a result of devtool add), the recipe file and any associated files
will be moved to the destination layer; or if the destination layer is
the one containing the original recipe, the recipe will be overwritten;
otherwise a bbappend will be created to apply the changes. In all cases
the layer path can be loosely specified - it could be a layer name, or
a partial path into a recipe. In the case of upgrades, devtool finish
will also take care of deleting the old recipe.
This avoids the user having to figure out the correct actions when
they're done - they just do "devtool finish recipename layername" and
it saves their work and then removes the recipe from the workspace.
Addresses [YOCTO #8594].
(From OE-Core rev: fa550fcb9333d59b28fc0e4aebde888831410f5c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This provides us with the information we need to remove the original
version recipe and associated files when running "devtool finish" after
"devtool upgrade".
(From OE-Core rev: 92eb42c347af919cd9f8739515fdf806c12b5ba8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will be called by "devtool finish" to allow it to update the recipe
or create the bbappend depending on the destination.
(From OE-Core rev: 5067cdc73483b53d46d9bf584723e41957c7ec54)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will be called by "devtool finish" to allow it to reset the recipe
at the end.
(From OE-Core rev: b8d398516556eaf97679e28ad58448f570984b52)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there are files in the oe-local-files directory which are identical
to the original version, then we shouldn't be copying them to the
destination layer. This is particularly important when using the -a
option to create a bbappend.
(From OE-Core rev: 9230bfcc839eb35630949f0a8ed058ca1fa944b1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool update-recipe was defaulting to the ${BPN} named directory when
adding patches next to a recipe, but that meant if you already had files
in a ${BP} named directory (i.e. name and version) or "files" then you'd
end up with two directories next to the recipe, which is usually not
what you want. To avoid this, look through FILESPATH and take the first
one that's the same level or one level down from the recipe and already
exists, if any.
(From OE-Core rev: c7a8190cf8bdf86ba850b6780b8e951e90232c06)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
zipfile.py has dependencies on importlib, threading, and shell
importlib has a dependency on lang
operator and contextlib added to the lang package instead of falling into misc
(From OE-Core rev: 769ad8e114fda1fe112d3747408edbeb7b066a85)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
According to wic documentation partition mount point is an optional
argument. Skipping mount point also makes sense in certain
configurations when one needs to specify a partition that is not mounted
by the running system, such as a recovery or a mirror partition (in dual
rootfs setups).
(From OE-Core rev: 5e063a4c6bb0e0623a4d25bb2bf6eecd9ad6b9f1)
Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This wks is just a copy of gummiboot wks with bootloader
changed to systemd-boot. A very basic boot test on a X86
target is done with a direct-boot image generated by:
wic create mksystemd-bootdisk -e core-image-minimal
Because we plan to replace gummiboot with systemd-boot at any
time in the future, we summarize history (as much as I can) of
the current gummiboot wks before it's gone:
-----------------------------------------------------------------
commit 7d4bb40905
Author: Ed Bartosh <ed.bartosh@linux.intel.com>
Date: Wed Sep 2 13:58:02 2015 +0300
wic: get rid of scripts/lib/image
Moved content of scripts/lib/image/ to scripts/lib/wic as
one directory with the same name as a tool is self-explanatory
and less confusing than two.
(From OE-Core rev: 5dc02d572794298b3362378cea3d7da654456c44)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
commit 77561e7191
Author: Ed Bartosh <ed.bartosh@linux.intel.com>
Date: Wed Sep 2 13:58:01 2015 +0300
wic: use ext4 in canned .wks files
Latest kernel doesn't have ext3 compiled in. Wic images produced
from canned .wks can't boot because of that. Switching to ext4
fixes this issue.
(From OE-Core rev: d281a65a81f369fc8d75023b8f911ce4106969c1)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
commit 072657ce98
Author: Tom Zanussi <tom.zanussi@linux.intel.com>
Date: Mon Aug 11 20:35:41 2014 -0500
wic: Add mkgummidisk kickstart file
This is the same as mkefidisk but uses gummiboot instead of grub-efi.
(From OE-Core rev: 5979409ebfab0bb07b3c2b2fcf14a722c441f07b)
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
-----------------------------------------------------------------
(From OE-Core rev: 546cd8352b8adce074831ec31cfa3bb2bf2f0084)
Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new systemd-boot enabled in OE is the old gummiboot
merged into systemd project. Our intention is to replace
gummiboot with systemd-boot in OE once every feature based
on gummiboot is supported with systemd-boot.
Before we can purge gummiboot, we temporarily keep both of
the two bootloaders supported.
Patch doesn't do replacement for every "gummi" occurrence.
We think cleaning can be done in background after we disable
people to use gummiboot, so we just keep change small and
safe this time.
(From OE-Core rev: daa5f8b886408eb6a17898b18ac97d5a0d76d2cc)
Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is currently possible to specify a file (e.g. a tarball) on the local
disk as the source, but you have to know to put file:// in front of it.
There's really no need to force users to jump through that hoop if they
really want to do this so check if the specified source is a file and
prefix it with file:// if that's the case.
Also ensure the same works for "devtool add" at the same time.
(From OE-Core rev: 71350003790c38e84b0e525a71a2fe5d24e3d083)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For a while now, Github hasn't been advertising a specific repository
URL since cloning the web URL with git works. Armed with this knowledge
and fully expecting people to just paste the github URL, we need to
handle this situation specially. If it looks like a github URL to the
root of a repository then treat it as a git repository instead of a
normal https URL to be fetched by the wget fetcher.
(From OE-Core rev: 7998dc3597657229507e5c140fceef1e485ac402)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
move graph-tool to python3
(From OE-Core rev: 0d0864ae0ff9e53623ad1c7146b071f2a046f21f)
Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These recipes can't be used with devtool because they can't be unpacked
in the normal way.
(From OE-Core rev: b2cf098969b8b800a78d650cf60c0b5ad31c85b5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If devtool returns exit code 4 then record the recipes as "skipped"
rather than "failed" - these are recipes we know cannot work (usually
because they don't provide any source).
(From OE-Core rev: 8fc109f1cb6eb437c12d2d11a6937de6f035e296)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Certain recipes cannot be used with devtool extract / modify / upgrade -
usually because they don't provide any source. Return a specific exit
code (4) so that scripts such as scripts/contrib/devtool-stress.py know
the difference between this and a genuine failure.
(From OE-Core rev: ffd295fed4ab81fc0bd00bb145ef4d72c49584bf)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were attempting to open the recipe file unconditionally here - we
need to account for the possibility that the recipe file has been
deleted or moved away by the user.
(From OE-Core rev: 47822a2aff56fd338c16b5ad756feda9f395a8a1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In OE-Core revision 7baf57ad896112cf2258b3e2c2a1f8b756fb39bc I changed
the default update-recipe behaviour to only update patches for commits
that were changed; unfortunately I failed to handle the --initial-rev
option which was broken after that point. Rework how the initial
revision is passed in so that it now operates correctly.
(From OE-Core rev: b2ca2523cc9e51a4759b4420b07b0b67b3f5ac43)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently gnome-terminal just returns straight away, opening a terminal in a new
separate process we have no insight into. For patch resolution, this leads to
spawning many different terminal windows, for pydevshell, it just flashes a window
up and then closes.
We need to block until the command completes but gnome-terminal gives us no way
to do this. We therefore write the pid to a file using a "phonehome" wrapper
script, then monitor the pid until it exits.
[YOCTO #7254]
(also fixing do_devpyshell)
(From OE-Core rev: 76e8ab47c936674b8bb9bf1c48de53b30f5bf74a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a typo in the logging parameters, "filname" is being used instead of "filename" for yocto-kernel, yocto-layer and wic scripts. This didn't cause issues before since python 2 didn't validate unused parameters but with python >= 3.4.3 an exception is thrown. This patch fixes this parameter name.
[YOCTO #9834]
(From OE-Core rev: 5d123a6ec0e97652b2ec4295428797a336ef357a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When detecting the second serial options we shouldn't append the
custom QEMU options otherwise we will end duplicating those.
(From OE-Core rev: 79798f20b2c0b98d84c3c4b14600635ff8ddfdad)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Code cleanup, no functional changes - this code was never used.
(From OE-Core rev: 397b76c7f26e38e761b94b1f7987aafd55048e10)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're opening source files with the default encoding (utf-8) but we
can't necessarily be sure that they are UTF-8 clean - for example,
recipetool create ftp://mama.indstate.edu/linux/tree/tree-1.7.0.tgz
prior to this patch resulted in a UnicodeDecodeError. Use the
"surrogateescape" mode to avoid this.
Fixes [YOCTO #9822].
(From OE-Core rev: 50fcd9d1b9a20d49bc873467a82a071f2f2f8b5a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As /dev/pty opened in binary mode its content has to
be decoded when reading from it and encoded when writing to it.
(From OE-Core rev: 211870ddbce5c966b2882e97cb2efe29b72a62a4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Opening text stream in unbuffered mode raises the following
exception In Python 3:
ValueError: can't have unbuffered text I/O
Fixed by leaving std* streams in text mode and flushing
stdout explicitly.
(From OE-Core rev: 732001cb268683f5b56e251e2964ec5b694a2147)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If BB_SETSCENE_ENFORCE is set to "1" and an sstate package fails to
download outside of the whitelist specified by
BB_SETSCENE_ENFORCE_WHITELIST, then fail immediately so you can tell
that the problem was caused by failing to restore the task from sstate.
Part of the implementation of [YOCTO #9367].
(From OE-Core rev: 9e711b54487c3141d7264b8cf0d74f9465020190)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the changes to migrate from optparse to argparse there was a syntax change that needs an update in the help. This is basically just the change of 'properties' and 'property' from positional arguments to options. This patch makes the required changes.
[YOCTO #8321]
(From meta-yocto rev: b171379b5ca54d55ea763421794a651e71bbda91)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Optparse is deprecated and should be avoided. The arparse library is better suited and has more tools to handling the parsing of arguments. This patch makes necessary changes to migrate to the better library and uses arparse subcommand feature to improve organization of this script.
[YOCTO #8321]
(From meta-yocto rev: 3f45993b96d4d960da0efe8672dc323c9db091a2)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The version option is not maintained and is useless inside this script.
There is no reason for this script to have an independent version value.
[YOCTO #8321]
(From meta-yocto rev: 8f8790e56d00f2eaaf6508fb1909335f1fbef5ff)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Format of properties file is expected to be a simple json detailing properties, if this format fails an exception is thrown. This patch adds a graceful error message to the case when the properties file has a wrong format.
[YOCTO #9750]
(From meta-yocto rev: 7e543aa19d0d4b2112e6316783fb31b76df3493e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python3 requires strings to be encoded as bytes before sending them through a subprocess pipe. The help.py file is not considering this and fails when issuing paged help commands. This patch adds this encoding to solve the problem.
[YOCTO #9868]
(From meta-yocto rev: 35b487a47f0cbb99fdee2ec9cc8b56b814c8860e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a typo in the logging parameters, "filname" is being used instead of "filename" for yocto-kernel, yocto-layer and wic scripts. This didn't cause issues before since python 2 didn't validate unused parameters but with python >= 3.4.3 an exception is thrown. This patch fixes this parameter name.
[YOCTO #9834]
(From meta-yocto rev: 844fbba63b146a2ded3fced0d62bf047bf844af3)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A shell script wrapper around oe-build-perf-test script. The purpose of
this wrapper is to act as a executor script, making it possible to run
the tests with a single command. The wrapper script initializes the
build environment, runs oe-build-perf-test and archives the results.
(From OE-Core rev: 946a076c2ce20dd8f7cfa1acbdab1268d406d3e1)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new option defines the output directory for the test result data.
(From OE-Core rev: bc865b5cb5a9a76048ee9c55a29f5e1a926bb543)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Makes it possible to guard that multiple tests are not run in parallel.
(From OE-Core rev: 181e92e7a1bccf678b3eb1bf547608a142784f97)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using this option the script appends test results into a 'global results
file'. A CSV-formatted output of the results. This option is to provide
compatibility with the old build-perf-test.sh.
(From OE-Core rev: e9f18e63220e452f2b0c878998e57d944ae83980)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new class is responsible for actually running the tests and
processing their results. This commit also adds a decorator function for
adding new tests. No automatic test discovery, at least yet.
(From OE-Core rev: bf90aecb7e150d6bfac7240286c797b79d26528b)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new utility class for dropping Linux kernel caches. It uses sudo
and tee to write to the drop_caches file. Checking if the user has the
permissions to drop caches (without a password) is done by trying to
writing an invalid value to the drop_caches file. This way, we will find
if writing (with tee) is possible but not really dropping caches, yet.
User can avoid giving the password by adding something like:
<user> ALL = NOPASSWD: /usr/bin/tee /proc/sys/vm/drop_caches
to the system sudoers file.
(From OE-Core rev: c9cb248429ced50c96d11ba5361c272d4c9b9323)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The script will be required to be run in an initialized bitbake build
environment.
(From OE-Core rev: 1bce7b10283255a4498d11ead920c1f3b1dec4de)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Initial wireframe for re-writing build-perf-test.sh in Python.
(From OE-Core rev: 764eb2d011305b84501cc183531a2a5353b0b5ab)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
GNU make looks for "makefile" and "GNUmakefile" in addition to
"Makefile", so add these other names to the heuristic for detecting a
make-based project.
(From OE-Core rev: 204d19b02265e5b2241888e4c92c0a730f3d3472)
Signed-off-by: Nathan Lynch <nathan_lynch@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* format of bitbake tasks changed in:
2c88afb taskdata/runqueue: Rewrite without use of ID indirection
-ERROR: Task 4 (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb, do_fetch) failed with exit code '1'
+ERROR: Task /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) failed with exit code '1'
so strip not only '\.bb, .*' used before, but also '\.bb;.*' to drop
the task name to get recipe name.
* for more details see:
http://lists.openembedded.org/pipermail/openembedded-core/2016-June/123132.html
* without this change you can see test-dependencies.sh trying to rebuild packages
like:
Building recipe: fbprogress (6/21)
Building recipe: fbprogress.bb:do (7/21)
where the later of course doesn't exist as a recipe
(From OE-Core rev: b7d6d4203cf2021ee2b9b84c8faf15198bfb536d)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With Python 2, argparse subparsers behaviour in Python 2 was to print
the usage information if the subparsers argument wasn't specified.
However, with Python 3.2.3 and later a subparsers argument is not
required by default, leading to errors when no arguments are specified:
AttributeError: 'Namespace' object has no attribute 'func'
Restore the previous desired behaviour of showing the help text for
devtool, recipetool and the devtool-stress script by setting
subparsers.required to True.
(From OE-Core rev: d36fdea1a7f32d97187e0e9e6d701ae8fa304e8f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow specifying more than one recipe on the devtool reset command line.
Also tweak the help text slightly.
(From OE-Core rev: ad92ed8e4f7f48a3d212962531d596b36f6b284f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Clarify slightly the intended usage of the workspace layer.
(From OE-Core rev: d9f3af6f4e6d3df30b411bbcc3c2b6f7f62c52ad)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit ab75075e79 broke the string formatting
for multiple arguments. Since parentheses were removed the formatting considers
only first argument instead of all of them. Fixed parentheses on the formatting.
[YOCTO #9723]
(From meta-yocto rev: d6d641d5b4bb0c8b9acd5cd1341a4a3214bff086)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Have to convert implicit relative imports to explicit imports since
implicit relative imports are not supported on python3.
[YOCTO #9723]
(From meta-yocto rev: 89cecc603d229768428f3cb68d21235dee80efda)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
According to https://docs.python.org/3/whatsnew/3.0.html python3
renamed 'raw_input' to 'input'. Making the appropiate changes for
this.
[YOCTO #9723]
(From meta-yocto rev: 76efeeeac7fb30b44ee7057b3b3d1fd84329dae1)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a typo in the logging parameters, "filname" is being used instead
of "filename". This didn't cause issues before since python 2 didn't
validate unused parameters but with python >= 3.4.3 an exception is
thrown. This patch fixes this parameter name.
[Yocto #9736]
(From meta-yocto rev: aae5c2c39f64be87152b2e5470b50681c0f54670)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If buildhistory-diff can't import git module it throws error
message suggesting to install it.
Due to the move to Python 3 the suggested package name and
version became outdated.
Changed package name to python3-git and version to >= 0.3.4
as GitPython supports Python 3 starting from 0.3.4.
[YOCTO #9741]
(From OE-Core rev: 0632c3e69070e340218c1c1c835b2343cd143d8d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New option can be used for displaying full information about binary
package(s), including name, full version, recipe name, recipe full
version and package size.
This information can be useful inside custom buildhistory class
to produce detailed image manifest for further analysis.
List of packages can be specified as command line arguments or can
be read from file (e.g. from existing image manifest).
Warning: In case of image manifest is used as input outside of build
process, be aware that pkgdata might change since the time image
was built.
Output format:
{PKG} [PKGE:]{PKGV}[-{PKGR}] {PN} [PE:]{PV}[-{PR}] {PKGSIZE}
(From OE-Core rev: bc0cdaa927124150a6c38cd47977ee4da8dd440e)
Signed-off-by: Alexander D. Kanevskiy <kad@kad.name>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current qemuzynq options were causing QEMU to exit instead of
booting as the QEMU Zynq-7000 machine does not have virtio or
display devices.
(From OE-Core rev: e76f62a862a895147a89cea2519679aad337a679)
Signed-off-by: Alistair Francis <alistair.francis@xilinx.com>
Reported-by: Arno <star@gmx.li>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default arparse behaviour in Python 2 and early versions of Python 3
was to print usage message if no subcommands provided by user.
In Python >= 3.2.3 subparser has to have attribute 'required' set to
True to behave similar way.
By default subparser.required attribute is set to False, usage message is
not printed and code continues to be executed.
oe-pkgdata-util code raises AttributeError exception in
this case:
AttributeError: 'Namespace' object has no attribute 'func'
Fixed this by setting subparser.required to True.
[YOCTO #9711]
(From OE-Core rev: 207dba5b6eb04c1ae2081bac06724b3b6a198cb3)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
swabber hasn't been used in years and never did work well in the first
place. Remove its recipes, class and configuration.
(From OE-Core rev: e18657df0b7e45a224fae17e68c447eae94258ac)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was broken with the python3 fixes.
(From OE-Core rev: 0eadd008d08f9a6bb17a1f641b4735d0d2012580)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
stdeout and stderr content returned by subprocess API has different
types in Python 3(bytes) and Python 2(string). Decoding it to
'utf-8' makes it unicode on both pythons.
(From meta-yocto rev: 1de9d0b4ad289c56907d082748cdc0111988cb4f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used items() as iteritems() doesn't exist in python 3.
(From meta-yocto rev: 0fd2e739cd14c6a02b6f4610176a0ad29b2ecc0e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used exec() builtin instead of 'exec' statement as
this statement doesn't exist in python 3.
(From meta-yocto rev: ceb6c6fac212c5e1e76bf58b793dc2b9420b4a66)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implicit relative imports within packages are not supported in
python 3. They have to be converted to explicit imports.
Used 'from .module import' syntax for relative imports.
(From meta-yocto rev: 8eb2befc544e29fd87ac1fc75886cc6d7f40ad90)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used metaclass=<metaclass> syntax instead old
__metaclass__ = <metaclass> as only new one is supported
in python 3.
(From meta-yocto rev: 70ba79981d904b4b67440c0b5755b5449cddb025)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Renamed raw_input to input as raw_input does not
exist in python 3.
(From meta-yocto rev: c9df9f2699885f2ba5b031c8761aefbf3c796067)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Encoded data before sending it through http as urllib expecting bytes.
Fixed TypeError: POST data should be bytes or an iterable of bytes. It
cannot be of type str.
(From OE-Core rev: b3f4de76d1b32c5079b0b857655cc2baad088519)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since mtools has been patched to live with filesystems with sizes
not divisible by sectors-per-track, we no longer need to try to
set the size based on our guess of the sectors-per-track dosfstools is
going to use.
(From OE-Core rev: 334e32af88b310ff1ed950d127a6dedeb460f8d0)
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
stdeout and stderr content returned by subprocess API has different
types in Python 3(bytes) and Python 2(string). Decoding it to
'utf-8' makes it unicode on both pythons.
(From OE-Core rev: ae4d36375683b6cfd48af25bfca70b2e77f7ade5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used tempfile.TemporaryFile() API instead of deprecated
os.tmpfile().
(From OE-Core rev: bf1b411eb1cd2cc960325d5fdb0cb4f4f7b1e40e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 3 ignores the __cmp__() method and doesn't have cmp() builtin
function. This caused sorted() call to raise
TypeError: unorderable types: Record() < Record()
Removing __cmp__ method and implementing __lt__ should solve the
problem as __lt__ is the only method needed for sort[ed] to work.
(From OE-Core rev: 391cd33720e7d7e8e261193199272739293ad881)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reduce is not a builtin function in python3.
It has to be imported from functools.
(From OE-Core rev: f9bda8b36decb5a837adcfeb1173092401988801)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used metaclass=<metaclass> syntax instead old
__metaclass__ = <metaclass> as only new one is supported
in python 3.
(From OE-Core rev: 067a2156f36a12c731d49f88cf9d0b0bab7a0dcb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed print_function and with_statement imports from __future__
as they're supported by python 3 by default.
(From OE-Core rev: 77ae2daad5d775d710b953cf0c623ce74cb2c274)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Renamed raw_input to input as raw_input does not
exist in python 3.
(From OE-Core rev: 32765150b860ecdea74b4494b9531f5bc40252bb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used join method instead of strings.join as stings.join
doesn't exist in python 3.
(From OE-Core rev: 10529d8fbc7254523f9749f4b35b07ebcccb6205)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some functions and classes have been moved from urllib[2]
to urllib.request and urllib.error in python 3.
Used new imports to make the code working in python 3.
(From OE-Core rev: ec3f1759e8b491a44a1fc1ecb6f89919dd30da97)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changed old syle except statements 'except <exception>, var'
to new style 'except <exception> as var' as old style is not
supported in python3.
(From OE-Core rev: 438eabc248f272e3d272aecaa4c9cec177b172d5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Converted return value of items() keys() and values() to
lists when dictionary is modified in the loop and when
the result is added to the list.
(From OE-Core rev: 874a269eb1d70060c2f3b3f8b70800e2aea789f4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch contains all the other misc pieces of the transition to
python3 which didn't make sense to be broken into individual patches.
(From OE-Core rev: fcd6b38bab8517d83e1ed48eef1bca9a9a190f57)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As most of oe-test cases work for devtool and recipetool
it makes sense to switch both tools to python 3 by
explicitly referring to python3 in their shebangs.
(From OE-Core rev: dad9617809c60ec5f11d4780b0afa1cffa1efed5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used urllib.parse instead of urlparse to make code
working in python 3.
(From OE-Core rev: 0a064f2216895db0181ee033a785328e704ddc0b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ConfigParser API was renamed to configparser in python 3.
Renamed ConfigParser -> configparser in scripts/ to make the
code working in python 3.
(From OE-Core rev: de6e98f272e623ce72e724e66920eecf10cb2d41)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved call of decode('utf-8') as close as possible to
call of subprocess API to avoid calling it in a lot of
other places.
Decoded binary data to utf-8 where appropriate to fix devtool
and recipetool tests in python 3 environment.
(From OE-Core rev: 30d02e2aa2d42fdf76271234b2dc9f37bc46b250)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 3 doesn't have basestring type as all string
are unicode strings.
(From OE-Core rev: e8cfab060f4ff3c4c16387871354d407910e87aa)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Encoded help text before sending it to pager.communicate as
it expects binary.
[YOCTO #9412]
(From OE-Core rev: 23c27d9d936efaa17da00525f1d2e2f98c53abc7)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switched scripts/wic to use python3 as a default
python interpreter.
(From OE-Core rev: ea6245d2383e2ba905ef9f1ba210e5dadc779ad8)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
xrange() no longer exists in python 3, use range()
(From OE-Core rev: d022b4335100612d6596cc4c4956cb98ed5873cc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.
(From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python3 standardises its use of iteration operations. Update
the code to match the for python3 requires.
(From OE-Core rev: 2476bdcbef591e951d11d57d53f1315848758571)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check that 'python' is a python v2 since that is what we assume everywhere
and upstream python devs recommend. We can need both python2 and python3
available since we don't control the software we might download and run.
Also check that python 3 is >= 3.4.0, our minimum version for bitbake.
(From OE-Core rev: 3dd26cd6b3d731f7698f6fbcd1947969f360cdc4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used print function instead of print statement to make
the code work in python 3.
(From meta-yocto rev: d6eea5a794dd8802b773a9186479a863847e6e55)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change shebang line to python3 and add parentheses to print
(From OE-Core rev: 85b6a53386382c0d92b5bea545c2db5e0204e629)
Signed-off-by: Allen Wild <allenwild93@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used print function instead of print statement to make
the code work in python 3.
(From OE-Core rev: 80fecc44761fa38ccf2e4dc6897b9f1f0c9c1ed0)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If fetching source from a git repository, typically within OpenEmbedded
we encourage setting SRCREV to a fixed revision, so change to do that by
default and add a -a/--autorev option to use "${AUTOREV}" instead.
(From OE-Core rev: 000480c42797dd2f03ebc3bc6d1dabfc6a7b75f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we use ${BP} for the subdirectory, the default value of S will work
rather than having to have an ugly value derived from the package
file name in both places. This does mean that we have to assume the
default though (we can't just let the normal logic work because the
value of BP is the default until later on, so the replacement doesn't
work).
(From OE-Core rev: 13bc2438d61c345a8f229b9d83bf36a14d08916f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extract the metadata from package files and use it to set variable
values in the recipe (including recipe name and version, LICENSE,
SUMMARY, DESCRIPTION, SECTION and HOMEPAGE). For LICENSE we take care
not to step on any value determined by our license file scan; if there
is one we simply add a comment above the LICENSE setting so the user can
resolve it.
(From OE-Core rev: 19e6b661d38c5ae9b19d6340762c289830baba59)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow plugins to set any variable value through the extravalues dict,
and use this to support extracting SUMMARY and HOMEPAGE values from spec
files included with the source; additionally translate "License:" to a
comment next to the LICENSE field (we have our own logic for setting
LICENSE, but it will often be useful to see what the spec file says if
one is present).
Also use the same mechanism for setting the same variables for node.js
modules; this was already supported but wasn't inserting the settings in
the appropriate place in the file which this will now do.
(From OE-Core rev: 91fc35ff5e89aa6d4c4ad945e45406fb4f71018e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh
Richard Purdie
Paul Eggleton
Markus Lehtonen
Robert Yang
Maciej Borzecki
Tom Zanussi
Chen Qi
Alistair Francis
Ioan-Adrian Ratiu
Derek Straka
Alejandro Hernandez
Patrick Ohly
Christopher Larson
Joshua Lock
Mariano Lopez
Ola x Nilsson
Alessio Igor Bogani
Leonardo Sandoval
Dominic Sacré
Jiajie Hu
Stephano Cetola
Jussi Kukkonen
Ulf Magnusson
Todor Minchev
Robert P. J. Day
Olaf Mandel
Martin Jansa
Tobias Hagelborn
Peter Kjellerstedt
Zubair Lutfullah Kakakhel
Jianxun Zhang
Joe MacDonald
Nathan Rossi
Ross Burton
Humberto Ibarra
Joshua Lock
Fabio Berton
Bruce Ashfield
Kyle Russell
Benjamin Esquivel
André Draszik
Maxin B. John
Otavio Salvador
Nathan Lynch
Alexander D. Kanevskiy
Allen Wild