When using PATCHTOOL = "git", the user of the system is not really the
committer - it's the build system itself. Thus, specify "dummy" values
for username and email instead of using the user's configured values.
Various parts of the devtool code that need to make commits have also
been updated to use the same logic.
This allows PATCHTOOL = "git" and devtool to be used on systems where
git user.name / user.email has not been set (on versions of git where
it doesn't default a value under this circumstance).
If you want to return to the old behaviour where the externally
configured user name / email are used, set the following in your
local.conf:
PATCH_GIT_USER_NAME = ""
PATCH_GIT_USER_EMAIL = ""
Fixes [YOCTO #8703].
(From OE-Core rev: 765a9017eaf77ea3204fb10afb8181629680bd82)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pass appropriate options to tar invocations in copytree() and
copyhardlinktree() to ensure that any extended attributes on the files
are preserved during the copy.
We have to drop the use cpio in "Copy-pass" mode in copyhardlinktree()
because cpio doesn't support extended attributes on files. Instead we
revert back to using cp with different patterns depending on whether
or not the directory contains dot files.
(From OE-Core rev: e591d69103a40ec4f76d1132a6039d9cb1555103)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current functionality allows for the removal of certain packages
based on the read-only image feature. This patch extends this
functionality by adding the FORCE_RO_REMOVE variable, which will
remove these packages regardless of any image features.
[ YOCTO #9491 ]
(From OE-Core rev: cfb869ffd4c37c3cc8e6b3eb732c1a7b7cfc3cb0)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
LICENSE should be a superset of all LICENSE_<pkg> values. That is,
LICENSE should contain all licenses and LICENSE_<pkg> can be used to
"filter" this on a per-package basis. LICENSE_<pkg> shouldn't contain
anything that isn't specified in LICENSE.
This patch implements simple checking of LICENSE_<pkg> values. It does
do not do advanced parsing/matching of license expressions, but,
checks that all licenses mentioned in LICENSE_<pkg> are also specified in
LICENSE. A warning is printed if problems are found.
(From OE-Core rev: 0f4163a12ea431d0ba6265880ee1e557333d3211)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately to implenent multiconfig support in bitbake some APIs
had to change. This updates code in OE to match the changes in bitbake.
Its mostly periperhal changes around devtool/recipetool
[Will need a bitbake version requirement bump which I'll make when merging]
(From OE-Core rev: 041212fa37bb83acac5ce4ceb9b7b77ad172c5c3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we don't have uninative enabled there's more merging to be done in
the default configuration (SDK_EXT_TYPE = "full" which by default means
SDK_INCLUDE_TOOLCHAIN = "1") and there are likely files that already
exist in the sstate feed we're assembling, so we need to take care to
merge the directory contents rather than just moving the directories
over. Additionally we now only run this if uninative genuinely isn't
enabled (i.e. NATIVELSBSTRING is different to the fixed value of
"universal".)
In the process of fixing this I discovered an unusual behaviour in
os.rename() - when we're merging these feeds we're dealing with
hard-linked sstate artifacts, and whilst os.rename() is supposed to
silently overwrite an existing destination (permissions allowing), if
you have the source and destination as hardlinks to the same file then
the os.rename() call will just silently fail. As a result the code now
just checks if the destination exists and deletes the source if so
(since we know it will be the same file, we don't need to check in this
case.)
(From OE-Core rev: 2b5b920c6b4f4d5c243192aa75beff402fd704d3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Give each rootfs its own RPM channel to use. This puts the RPM metadata
in a private subdirectory of $WORKDIR, rather than living in DEPLOY_DIR
where other tasks may race with it.
This allows us to reduce the time that the rpm.lock is held to only the
time needed to hardlink the RPMs, allowing the majority of the rootfs
operation to run in parallel.
Also, this fixes the smart tests by generating an index for all packages
at the time of the test, rather than using the one provided by the
rootfs process.
Original credit for the enhancement should go to Steven Walter
stevenrwalter@gmail.com.
(From OE-Core rev: a92c196449c516fe51786d429078bbb1213bb029)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Monitoring the process started by gnome-terminal was
spinning in a busy-loop. Insert some sleeping so that
we don't eat all the cpu.
(From OE-Core rev: 314937429d700204f296cfd1c0c5f215a2e5b939)
Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you supplied an empty file to patch_recipe() (or an empty list to
patch_recipe_lines()) then the result was IndexError because the code
checking to see if it needed to add an extra line of padding didn't
check to see if there were in fact any lines before trying to access the
last line.
Fixes [YOCTO #9972].
(From OE-Core rev: 92a73e870478ddb2a2d137e3fff28828809bec2e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the new oe-check-sstate to filter the sstate artifacts shipped with
the extensible SDK by effectively running bitbake within the produced
eSDK and and getting it to tell us which tasks it will restore from
sstate. This has several benefits:
1) We drop the *-initial artifacts from the minimal + toolchain eSDK.
This still leaves us with a reasonably large SDK for this
configuration, however it does pave the way for future reductions
since we are actually filtering by what will be expected to be there
on install rather than hoping that whatever cuts we make will match.
2) We verify bitbake's basic operation within the eSDK, i.e. that
we haven't messed up the configuration
3) We verify that the sstate artifacts we expect to be present are
present (at least in the sstate cache for the build producing the
eSDK). Outside deletion of sstate artifacts has been a problem up to
now, and this should at least catch that earlier i.e. during the
build rather than when someone tries to install the eSDK.
This does add a couple of minutes to the do_populate_sdk_ext time, but
it seems like the most appropriate way to handle this.
Should mostly address [YOCTO #9083] and [YOCTO #9626].
(From OE-Core rev: 4b7b48fcb9b39fccf8222650c2608325df2a4507)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're to completely replace the standard SDK with the extensible SDK,
we need to be able to provide the standard toolchain on install without
doing anything other than installing it, so that you can install the SDK
and then point your IDE at it. This is particularly applicable to the
minimal SDK which normally installs nothing by default.
NOTE: enabling this option currently adds ~280MB to the size of the
minimal eSDK installer. If we need to reduce this further we would have
to look at adjusting the dependencies and/or the sstate_depvalid()
function in sstate.bbclass which eliminates dependencies, or look at
reducing the size of the artifacts themselves.
Implements [YOCTO #9751].
(From OE-Core rev: ed0d8ed72370df694f720cc13897493478dc1de9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
tar's --no-recursion flag only applies to files mentioned after the
flag, which made it a no-op in this invocation of tar, because it was at
the end of the command line.
This is simple to verify with GNU tar 1.29:
| $ mkdir foo
| $ mkdir foo/dir
| $ touch foo/dir/file
| $ tar -cf - foo --no-recursion | tar t
| foo/
| foo/dir/
| foo/dir/file
| $ tar -cf - --no-recursion foo | tar t
| foo/
Modify the code so that it actually does what the comment says by moving
the flag in front of the --files-from argument.
(From OE-Core rev: d45f5e71fef5ffbd4408f69c5c179dc71a3eb452)
Signed-off-by: Clemens Lang <clemens.lang@bmw-carit.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oe.utils.getstatusoutput() is a wrapper for subprocess.getstatusoutput() which
uses Universal Newlines, so the output is a str() not bytes().
(From OE-Core rev: ce24d4c3632b71939ad198268a900ee823a89b27)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible in an attempt only install, that everything listed is not
available to be installed. This will have the effect of clearing the
package list. However, we only check for an empty package list at
the beginning of the function. We need to also check before running the
install, otherwise we can fail due to 'error: no package(s) given".
(From OE-Core rev: 9ae6a2830dacb3c335754a6da91bd5cc30546b31)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using PR service the buildhistory-diff output contains a lot of
PKGR changes: In practice the mass of PKGR updates hide other important
changes as they often account for 80% of all changes.
Skipped incremental and decremental changes of PKGR versions to reduce
amount of the script output. All changes are still included in the
output if script is run with -a/--report-all command line option.
[YOCTO #9755]
(From OE-Core rev: a343788b11f6c4f92ae8d2035fe8cb54f922227e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This task runs all functions in IMAGE_QA_COMMANDS after the image
construction has completed in order to validate the resulting image.
Image sanity checks should either be Python functions which raise
bb.build.FuncFailed on failure or shell functions with return a
non-zero exit code.
Python functions may instead raise an oe.utils.ImageQAFailed
Exception which takes an extra argument, a description of the
failure.
python image_check_python_ok () {
if True:
raise bb.build.FuncFailed('This check always fails')
else:
bb.note("Nothing to see here")
}
image_check_shell_ok () {
if true
exit 1
else
exit 0
fi
}
[YOCTO #9448]
(From OE-Core rev: c9bef2ecf1a30159d11781184829f41844a58c13)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a subcommand which will "finish" the work on a recipe. This is
effectively the same as update-recipe followed by reset, except that the
destination layer is required and it will do the right thing depending
on the situation - if the recipe file itself is in the workspace (e.g.
as a result of devtool add), the recipe file and any associated files
will be moved to the destination layer; or if the destination layer is
the one containing the original recipe, the recipe will be overwritten;
otherwise a bbappend will be created to apply the changes. In all cases
the layer path can be loosely specified - it could be a layer name, or
a partial path into a recipe. In the case of upgrades, devtool finish
will also take care of deleting the old recipe.
This avoids the user having to figure out the correct actions when
they're done - they just do "devtool finish recipename layername" and
it saves their work and then removes the recipe from the workspace.
Addresses [YOCTO #8594].
(From OE-Core rev: fa550fcb9333d59b28fc0e4aebde888831410f5c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This provides us with the information we need to remove the original
version recipe and associated files when running "devtool finish" after
"devtool upgrade".
(From OE-Core rev: 92eb42c347af919cd9f8739515fdf806c12b5ba8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Allow the function to be called with the base layer path (in which
case it will just return the same path)
* Ensure that the function doesn't recurse indefinitely if it's called
on a file that's not inside a layer
* Correct the doc comment for accuracy
(From OE-Core rev: 912026d85c2f535be2f60c45979162ea25c7f356)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
During a parallel build it's possible for unrelated shlib files to be removed if
the recipe they came from is about to be rebuilt. They can't be involved in the
dependency chains as otherwise they wouldn't be removed, so just silently handle
files disappearing.
[ YOCTO #8555 ]
(From OE-Core rev: 1e355da3fda742c78d99ddd2ee5caa9df52f92e1)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With Python 3, the encoding of a file is significant; several recipes in
OE-Core have patches which are not fully utf-8 decodable e.g. man,
lrzsz, and gstreamer1.0-libav, leading to errors when using devtool's
modify, upgrade or extract subcommands on these recipes. To work around
this, try reading the patch file as utf-8 first and if that fails try
latin-1 before giving up.
(From OE-Core rev: 7f4d7a6f51569954e204f110827a8ce256bcdc68)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the new pid monitoring code we have for recent versions of
gnome-terminal we can just drop the --disable-factory code now since
the other solution handles this case as well.
(From OE-Core rev: ed4957c444a2982c19e2f1f96d9afb2a992c1daf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently gnome-terminal just returns straight away, opening a terminal in a new
separate process we have no insight into. For patch resolution, this leads to
spawning many different terminal windows, for pydevshell, it just flashes a window
up and then closes.
We need to block until the command completes but gnome-terminal gives us no way
to do this. We therefore write the pid to a file using a "phonehome" wrapper
script, then monitor the pid until it exits.
[YOCTO #7254]
(also fixing do_devpyshell)
(From OE-Core rev: 76e8ab47c936674b8bb9bf1c48de53b30f5bf74a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was a copy-and-paste of subprocess.check_output() from when we supported
Python <2.7, so simply delete it and use subprocess.check_output() instead.
(From OE-Core rev: b1f2d9ed8d4dc89c9e669f43f546463ccc2a76b9)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're building the extensible SDK we don't need to see the "Writing
locked sigs" message; it's only necessary when the user explicitly runs
bitbake -S none <target>.
(From OE-Core rev: 440a351ee13920045c9d3e51882908f7b3f01d35)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the new task progress functionality to report progress during
do_rootfs. This is a little coarse and ideally we would have some
progress within the installation section, but it's better than
nothing.
(From OE-Core rev: 370f08d434480c1790950e40db8f7687da78cb14)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In RpmPM:insert_feeds_uris, the paths are kept in sets, which are unordered,
but they are later used to set the priority for the Smart channels, so
unexpected results could occur. Change the sets to lists and use the same
code as in create_configs() to add items to the list, rather than the set
operators.
[YOCTO #9717]
(From OE-Core rev: ce4137f4bb955207fede0c4ef338835d9a461f59)
Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Specify tmpdir for opkg via '-t' option so that opkg does not use
the default 'TMPDIR' which usually is '/tmp' on build host.
This would solve race problems like below.
sh: /tmp/opkg-rOG6Tl/opkg-intercept-iPoEp5/depmod: Permission denied
(From OE-Core rev: 5665e284b7e7ee071201664dc50a3412bd4ac8cd)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Order is not preserved in dict() and this code depends on the order of
these lists of package architectures used when multilibs are enabled.
This caused 'random' breakage where sometimes the correct order was present
and sometimes it wasn't.
Use collections.OrderedDict() to avoid this problem.
Kudos to Bill Randle and Alejandro Hernandez who did most of the work debugging
this, I simply took the problem they identified and wrote a patch to fix it.
This unblocks the M1 build but this code needs auditing as there are clearly
other ordering issues (e.g. the set() usage).
[YOCTO #9717]
(From OE-Core rev: 61a33582dfc964d612d20d34734a787d873e312c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
create_socket: Use urllib because urllib2 is now urllib in python3
and proxies as argument are deprecated so export them in the environ
instead.
get_links_from_url: Change usage of sgmllib for parsing HTML because
is deprecated in python 3, use instead bs4 that is already imported
in the bitbake tree.
[YOCTO #9744]
(From OE-Core rev: ee26ecf58277560459dd01992bb3f486f92c1531)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On python 3, bytes variable types must be decoded if these are intended to be
used as strings, otherwise we get the following error exception:
TypeError: Type str doesn't support the buffer API
(From OE-Core rev: b950539c911b7945d652b05616164828e711ac7f)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds decoding needed by some commands output later used as
strings.
[YOCTO #9702]
(From OE-Core rev: 0440b5ace411c61f802376d4e1c9eac93e72d65f)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
string.rstrip() doesn't exist in Python 3, so use the .rstrip method on the
object itself instead.
(From OE-Core rev: 554719e3f65e468f89a96d4869766d3ff7a8012e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The read method of the data_stream File object now returns bytes,
not a str, so we must decode it.
(From OE-Core rev: cfae302c4996c49a8754497ea9f13f8331d6975d)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bytes type variables are threat different as strings variables in python3,
so convert bytes to strings. This was found when using the
script buildhistory-diff, where the interpreter was yielding this error
TypeError: Type str doesn't support the buffer API
(From OE-Core rev: 3064d36186b47954eb94095217f7bb37e3fce651)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch contains all the other misc pieces of the transition to
python3 which didn't make sense to be broken into individual patches.
(From OE-Core rev: fcd6b38bab8517d83e1ed48eef1bca9a9a190f57)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved call of decode('utf-8') as close as possible to
call of subprocess API to avoid calling it in a lot of
other places.
Decoded binary data to utf-8 where appropriate to fix devtool
and recipetool tests in python 3 environment.
(From OE-Core rev: 30d02e2aa2d42fdf76271234b2dc9f37bc46b250)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
xrange() no longer exists in python 3, use range()
(From OE-Core rev: d022b4335100612d6596cc4c4956cb98ed5873cc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.
(From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python3 standardises its use of iteration operations. Update
the code to match the for python3 requires.
(From OE-Core rev: 2476bdcbef591e951d11d57d53f1315848758571)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In python3, strings are unicode by default. We need to encode/decode
from command pipelines and other places where we interface with the
real world using the correct locales. This patch updates various
call sites to use the correct encoding/decodings.
(From OE-Core rev: bb4685af1bffe17b3aa92a6d21398f38a44ea874)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The syntax for octal values changed in python3, adapt to it.
(From OE-Core rev: 737a095fcde773a36e0fee1f27b74aaa88062386)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add some more variables in appropriate places in recipe_progression such
that the patch_recipe_* functions are able to insert variables in the
right place within a recipe.
(From OE-Core rev: c33ba6cc5d14b1da96f6d906836c50e0346dcf06)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This function was assuming that what you wanted was that output lines
had trailing newline characters. If you're just outputting each line
verbatim to a text file then that's fine, but sometimes you start with
the assumption that the lines don't have trailing newlines; thus we
shouldn't allow for the possibility that the caller doesn't want them
and add a parameter to control it.
(From OE-Core rev: fb2bb509ff5c7bd71b41a1dcba3b1bff1d18cf5d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Split out a function from patch_recipe_file() that takes just the lines
as input so we can edit recipe lines in memory. This will be used within
recipetool to ensure we insert new values in the right place.
(From OE-Core rev: d780642f950fb3a9699f466a405a2710d870dd08)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having the package path with all the other package info allows to
reuse more code and have this information outside the package manager,
without additional processing.
[YOCTO #8536]
(From OE-Core rev: 343f762792cbfccffaf3aa901289f9bb0f8cef3d)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update older exception syntax to modern one required by python 3.
Compatible with python 2.7.
(From OE-Core rev: d13f0ac614f1d1e2ef2c8ddc71cbfcf76a8dc3f2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
file() API doesn't exist in python 3, convert to open(). Also handle
some cases where files aren't closed. Compatible with python 2.7.
[Contributions from Ed and Richard]
(From OE-Core rev: 0f4ec13e11bb8abe21aba2a28547dfb9372bc377)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used print function instead of print statement to make
the code work in python 3.
[Changes from both Ed and Richard]
(From OE-Core rev: ced1995694c394d92cb82fb9c25a33dc027a3b69)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In commit 20ea6d274bb0a9a5addb111f32793de49b907865, debugfs generation
for images using opkg, which included openssl was fixed.
However, that broke the generation of the opkg-based images, that lacks
openssl. The error is a python stack trace, showing that shutil.copytree
tries to copy a non-existing directory.
This relates to [YOCTO #9040].
(From OE-Core rev: f6b0b260ce18a30d04edfb0afb7942b9f9a5480b)
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a common _log_check_common() function (based on the old
_log_check_warn() function) to implement the logic for both
_log_check_warn() and _log_check_error().
The main benefit of this is that now all error messages will be
reported again, not just the first one found. Additionally the output
will now look the same for both error and warning messages.
This removes the context for the error messages. However, since there
was no indication in the output that some of the lines were context,
they were more confusing than helping.
(From OE-Core rev: dd73dcac36b80b3b886a2e9bf575b91c4f60d039)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For each warning found in the log, _log_check_warn() would output a
line stating that it had found a warning, then the actual warning and
finally an empty line. This is quite excessive when there are many
warnings in the log.
With this change the output is instead a line stating how many
warnings were found, followed by the warnings. This makes the output
much more compact and actually much more readable.
(From OE-Core rev: d6e3477749b1d09d40a773e0ac857a24d5851984)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will make _log_check_warn() exclude the same lines as
_log_check_error() does.
(From OE-Core rev: 85f64c68278f797c6f73f002f63d7f46fe80aef4)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before there were three different ways to exclude a line from being
searched for error messages in _log_check_error(). Now there is only
one: an array of regular expressions. This should make it easy to add
more excludes if nedded.
(From OE-Core rev: 321df88088fbfa657b61b2bae32751f03daec46f)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In commit 0387d095 lines with "NOTE:" in them were excluded from the
log check for warnings. However, those lines were only there in the
first place since the regular expression that is used to find warning
messages explicitly included those lines...
(From OE-Core rev: 865ab39f18a52ed84217df56d0e65113e2894d02)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The fact that this function was overridden in the RpmRootfs class
seems to have led to a number of misstakes when changes have been made
to the base function in the Rootfs class. E.g., this change will
properly solve ticket 7789, which was supposedly solved in 38871dc0,
but that change had no effect in practice as the log_check_regex that
was modified for RpmRootfs class was not used by the RpmRootfs version
of _log_check_error()...
The only thing _log_check_error() in RpmRootfs did that the base
function in Rootfs did not do was to skip lines in the log that begin
with a + sign. This has now been moved to the base function instead.
[YOCTO #7789]
(From OE-Core rev: 1eb0a46502fca4b2ee30ccd2508f4e21a40c25ca)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The reverted commit added a warning with the log line that triggered
the log check for error messages before the warning that states that
an error has been found in the log. However, the error line is output
by the call to bb.fatal() that follows immediately after the original
warning, which makes it redundant. Additionaly, having two warnings
contradicts the intent of commit 8dfdd329 where the log warnings were
tidied up.
This reverts commit f9cf31525fc885e1a0f65bd55654631257f87078.
(From OE-Core rev: e509a83c8eb5a572aad7e7451d019fa2469d70c3)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This new method extract the content of RPM file to a tmpdir,
without actually installing the package.
[YOCTO #9569]
(From OE-Core rev: 5f5c2a0fac5ad2baca162902410064375e8c610c)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes it is needed to have the content of a package outside
the recipe context. This new method extract the content of an
IPK/DEB file to a tmpdir, without actually installing the package.
A new OpkgDpkgPM class was added to share the code for opkg and dpkg.
There were need some changes to opkg_query() in order to use it
with apt-cache output. Also set default values to avoid UnboundLocalError
[YOCTO #9569]
(From OE-Core rev: 7d214b34e11dc57316ed5c1c7747c4601286f6d2)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using the opkg and apt-get package managers the function
opkg_query() can be useful when query for package information.
This change moves the function outside the Indexer class so
the Indexer, OpkgPM, DpkgPM can benefit from it.
[YOCTO #9569]
(From OE-Core rev: 799bc1d1c747aad02b6d844bf55abfbd3ecc034c)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ordering of dependency variables needs to be deterministic to avoid task checksums
changing. Use an OrderedDict to achieve this.
(From OE-Core rev: 855a2d21503856af392ab2d54ccfa270505ba142)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When enabling extra DEBUGFS image generation with opkg, the following error is
seen when openssl-cnf is included in the image.
Collected errors:
* file_md5sum_alloc: Failed to open file /mnt/cs-builds/anders/oe-build/build-ccu/tmp/work/ccu-oe-linux-gnueabi/ccu-image/1.0-r0/rootfs/usr/lib/ssl/openssl.cnf: No such file or directory.
Lots of similar issues was fixed by an earlier commit in oe-core,
5084ed9401250ed269a49d27b303806ab173c5d5, but openssl-cnf is outside of that fix.
Followup to [YOCTO #9490]
(From OE-Core rev: 20ea6d274bb0a9a5addb111f32793de49b907865)
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible for a Node.js module to have node_modules subdirectories
that contain no package.json file (e.g. iotivity-node has such a
directory). It appears these should simply be ignored, or else with the
way the current code works we will get errors later.
(From OE-Core rev: 8c522f1f536270e195c8c73f5c72801495e7b33b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is not enabled by default, as there are still limitations and possible
issues with opkg (and rpm?) packaging data containing broken symlinks for
local indexes:
http://cgit.openembedded.org/openembedded-core/commit/?id=c8e0ec2da9ad4ce1c103966906a85f68c15400dd
There are other use cases for the packaging data to be available in SDK,
since it provides comprehensive info about SDK's contents and in the case of
opkg and dpkg is all text-based and can be easily parsed by simple scripts.
Introduce new "package-management" flag for SDKIMAGE_FEATURES list (similar
to the one already used for IMAGE_FEATURES) that controls presence of the
packaging data in resulting SDK, while unifying this behavior across the
board for supported pkg managers - rpm, opkg, dpkg.
(From OE-Core rev: 9ab934e4aecb759c922049245888dcd2a8c55477)
Signed-off-by: Denys Dmytriyenko <denys@ti.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When enabling extra DEBUGFS image generation with opkg, errors are seen like:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Cannot get the installed packages list. Command '/media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin/opkg -f /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/opkg.conf -o /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs --force_postinstall --prefer-arch-to-version status' returned 0 and stderr:
Collected errors:
* file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/syslog-startup.conf.busybox: No such file or directory.
* file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/fstab: No such file or directory.
basically for all CONFFILES in the image. This is due to the file rearranging
the rootfs generation code does. If we preserve the /etc directory,
the avoids the problem.
We need to tell copyfile to preserve symlinks since some are present in /etc.
[YOCTO #9490]
(From OE-Core rev: 5084ed9401250ed269a49d27b303806ab173c5d5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a recipe sets SRC_URI and then appends more items to it with +=
(such as the current rpm recipe in OE-Core), the code in
patch_recipe_file() was failing with a traceback. Work around the
problem for now by dropping the existing lines if we understand the
operation, else just set the value outright at the end. This leaves
something to be desired as it either doesn't respect the existing
structure or leaves a mess but it's better than the current
breakage.
We'll need to come up with a better solution later. Part of the problem
is the existing code structure doesn't allow for patch_recipe_file() to
know what's being added or removed - it only knows the final value that
the caller wants set.
Fixes [YOCTO #9458].
(From OE-Core rev: da5cd84794e1ab5c4efcd6250d3b93a46f610744)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When creating an eSDK ensure that any .git directories are not included.
[ YOCTO #9426 ]
(From OE-Core rev: 6a5e2b2196e5654fc54ba5b2e51a390c966fd1b7)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Package names cannot contain underscores yet some npm modules use them as part
of the name, replace them with hyphens in the package name.
(From OE-Core rev: fea932c79c8201e3e7649f4443874ea540e33461)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, ROOTFS_POSTINSTALL_COMMAND was run only after the opkg rootfs
install post processing phase. This patch makes it generic so it is run fter
any package manager specific rootfs post processing, but before _run_intercepts().
[YOCTO #9248]
(From OE-Core rev: 8f2571881cd01ace88d282a63ad802cab4ab6940)
Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The distribution identifier is often used to create filenames, so it needs to be
safe to use as a filename. Whilst most distributions have e.g. Fedora or Debian
as their name, it is possible that the name contains special characters.
To ensure this doesn't cause a problem strip out any non-alphanumerics from the
distribution name before returning it.
[ YOCTO #9443 ]
(From OE-Core rev: 8a96a7207561e00eb92e4fb69e7340f20bfa2053)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to support workflows using devtool where a user might want to
modify tasks that exist in locked-sigs.inc, there must be a way to unlock
recipes.
This patch adds that support by allowing the user to add recipes to
SIGGEN_UNLOCKED_RECIPES. Recipes that exist in that variable will have
all their tasks unlocked, as well as any tasks that depend on that
recipe.
For example if foo->bar->baz, if you unlock baz, it will also unlock bar
so that foo can be rebuilt without explicitly specifying bar as being
unlocked.
[YOCTO #9195]
(From OE-Core rev: 8a8fc54d824767a6a94d12a4ace98b0bdbb1aa25)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous message when signatures didn't match between the metadata
and the locked signatures file, the message output was a bit confusing.
Now the message should be of the form:
The zlib-native:do_install sig is computed to be
53531910a2a7848432da89def942a91a, but the sig is locked to
d25ba9035f7ccb308e51bbe1066e8d27 in SIGGEN_LOCKEDSIGS_t-x86-64
which will hopefully be more useful in understanding the problem.
[YOCTO #9195]
(From OE-Core rev: 49eeabfff8bbea69401db41f7219e29acf47af73)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the SIGGEN_LOCKEDSIGS_TASKSIG_CHECK and
SIGGEN_LOCKEDSIGS_SSTATE_EXISTS_CHECK variables to replace
SIGGEN_LOCKEDSIGS_CHECK_LEVEL.
SIGGEN_LOCKEDSIGS_TASKSIG_CHECK will no control whether there is a
warning or error if a task's hash in the locked signature file doesn't match
the computed hash from the current metadata.
SIGGEN_LOCKEDSIGS_SSTATE_EXISTS_CHECK will control whther there is a
warning or error if a task that supports sstate is in the locked
signature file, but no sstate exists for the task.
Previously you could only have warning/errors for both controlled by
SIGGEN_LOCKEDSIGS_CHECK_LEVEL. This was an issue in the extensible sdk,
because we know sstate won't exist for certain items in the reverse
dependencies list for tasks. However, we still want to error if task
signatures don't match.
[YOCTO #9195]
(From OE-Core rev: 0fe2a5e5ffd01e926d0f3d4c78ad9910296e2d1a)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
opkg does not return a non-zero exit code even if it found
errors. When that happens, parsing the output leads to strange
follow-up errors.
To avoid this we need to check explicitly for non-empty
stderr. Reporting only that on a failure also leads to shorter error
messages (stdout may be very large).
(From OE-Core rev: 7d9e915224a9bc451fddfbbfad533d9b06e9987d)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Show the actual command that failed when raising a CmdError. Makes
figuring out what actually failed much easier.
[YOCTO #9344]
(From OE-Core rev: 8e9c03df1810daab7171733f1713ef94d3a18ab2)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
So that the packagedata module can be used externally to the core OE
environment, add a missing import.
(From OE-Core rev: da4df2313c8df92cf321a7631a9a389f895d4615)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously the list of packages that are considered unneeded for a
read-only rootfs was hardcoded. This made it impossible to, e.g., have
shadow installed on a system with a read-only rootfs, but where /etc
is mounted writable.
This also lists ${VIRTUAL-RUNTIME_update-alternatives} rather than
update-alternatives (as was previously the case) since this should
actually remove the intended package.
(From OE-Core rev: e3b881d4168e5b02ff00f5c470ba472ab8bbc747)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this, do_package_write_rpm doesn't depend on rpm-native which
it really should since that is needed to build rpms.
[YOCTO #8047]
(From OE-Core rev: 3fab4f9920d004fe13fb01434d4c7f3b8bbd7895)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's useful to know if a binary is statically or dynamically linked, so add a
method to determine this.
(From OE-Core rev: 96813445e6618fd8442600d81e53c448310b6e8b)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The python-expect is not installed on the distro such as Ubuntu by
default, and we can get rid of it.
Use RPM_GPG_PASSPHRASE to replace of RPM_GPG_PASSPHRASE_FILE which is
more straightforward.
(From OE-Core rev: 4a8a74c62836a20610daf029d4cec0b3087758b2)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Simplify the RPM code by removing support for RPM 4 now that we've
dropped the RPM 4 recipe.
(From OE-Core rev: 7db6f0a402948ce489bafadf2e389802f764f122)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When writing the index using ipk packages there could be a race condition
when populate the index. This happens because the architectures
are repeated (specially all) and the commands generated to write the index
run in parallel.
This change avoid the duplication of commands using a set instead of a list.
[YOCTO #8924]
(From OE-Core rev: 74adb14b0002e20099cc2c34e01862e8ddb8e013)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signing package feeds will default to ascii armored signatures (ASC) the
other option being binary (BIN). This is for both rpm and ipk backends.
(From OE-Core rev: 862a3892feb2628282e1d6f2e4498a7a3bd60cbf)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create gpg signed ipk package feeds using the gpg backend if configured
(From OE-Core rev: a2ee831cfb688bc64c071f75a1dff8a963abe287)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for multiple types of signatures (binary or ascii)
in export_pubkey(). There is no change in behaviour for the function,
the previous implicit default is the new parameter "armor" default.
(From OE-Core rev: 95ba4a982b887444908207e3180fe4bc46281d3b)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Starting from v2.1 passing passwords directly to gpg does not work
anymore [1], instead a loopback interface must be used otherwise
gpg >2.1 will error out with:
"gpg: signing failed: Inappropriate ioctl for device"
gpg <2.1 does not work with the new --pinentry-mode arg and gives an
invalid option error, so we detect what is the running version of gpg
and pass it accordingly.
[1] https://wiki.archlinux.org/index.php/GnuPG#Unattended_passphrase
(From OE-Core rev: 0413bd8e294ca8ac972ac68662b43a981952f5ae)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implement ipk signing inside the sign_ipk bbclass using the gpg_sign
module and configure signing similar to how rpm does it. sign_ipk uses
gpg_sign's detach_sign because its functionality is identical to package
feed signing.
IPK signing process is a bit different from rpm:
- Signatures are stored outside ipk files; opkg connects to a feed
server and downloads them to verify a package.
- Signatures are of two types (both supported by opkg): binary or
ascii armoured. By default we sign using ascii armoured.
- Public keys are stored on targets to verify ipks using the
opkg-keyrings recipe.
(From OE-Core rev: a40f27aa7802e8a0bd87a5417e35adbface62d05)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than rolling all of an npm module's dependencies into the same
package, split them into one module per package, setting the SUMMARY and
PKGV values from the package.json file for each package. Additionally,
mark each package with the appropriate license using the license
scanning we already do, falling back to the license stated in the
package.json file for the module if unknown. All of this is mostly in
aid of ensuring all modules and their licenses now show up in the
manifests for the image.
Additionally we set the main LICENSE value more concretely once we've
calculated the per-package licenses, since we have more information at
that point.
(From OE-Core rev: 8226805f83d21e7c1d2ba21969f3e8ee4b137496)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_shared_workdir is not a proper sstate task, it always reruns if
needed, so special-case it in warnings when checking locked sigs.
(From OE-Core rev: 4b08f982a2b15bff9092f60f7957301bb2d2108b)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
meta/lib/oe/package_manager.py was also updated. This ensures that any
diagnostic messages are ignored from the output of rpmresolve.
The patches have been split into bug fixes (things that belong upstream)
and local changes that are OE specific.
The following patches are obsolete and have been removed:
rpm-remove-sykcparse-decl.patch
fstack-protector-configure-check.patch
rpm-disable-Wno-override-init.patch
rpm-lua-fix-print.patch
rpm-rpmpgp-fix.patch
verify-fix-broken-logic-for-ghost-avoidance-Mark-Hat.patch
(From OE-Core rev: ee97e53fcceabc6ef4ddc68f38c5fa0e05c5d9a8)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prelink contains some hardcoded assumptions about the path layout of
the target system. Unfortunately if the system doesn't match, prelink
doesn't work. This breaks:
a) prelink of those images
b) the unsafe-references-in-binaries QA test (which uses prelink-rtld)
One way to work around this is to construct an ld.so.conf file which
lists the library paths in question. We do this in sanity QA check and
in the rootfs prelink code, being careful not to trample any existing
target ld.so.conf.
There is an additional problem that $LIB references in RPATHs won't be
handled correctly, I've not see any system use these in reality though
so this change at least improves things.
(From OE-Core rev: 7fd1d7e639c2ed7e0699937a5cb245c187b7c811)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a build-sdk command which is only available within the extensible
SDK that builds a derivative extensible SDK. The idea is recipes in the
workspace become a part of the new SDK - for example, this allows taking
a vendor provided SDK, adding a few libs and then producing a new SDK
with those included.
When normally building the extensible SDK, the workspace is excluded;
here we need to copy into the new SDK (renaming it in the process); the
recipes' task signatures become locked and thus the sources are no
longer needed, so they are removed along with the workspace bbappends
which would interfere with the locked signatures. Additionally we need
to just copy the configuration files (i.e. local.conf and auto.conf)
rather than filtering and appending to them since that work has already
been done when constructing the original SDK. The extra sstate artifacts
from workspace recipes are also determined and copied into the new SDK
in minimal mode (on the assumption that you won't set up a new sstate
mirror).
This reuses some code from build-image, so that needed to be
generalised to allow that.
Implements [YOCTO #8892].
(From OE-Core rev: 59e207ff6dd4b50a8905e14bc9292cf2794f4e7a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously the keys were put into the os-release package. The package
indexing code was also deploying the keys rather than only using the keys.
This change makes signing-keys.bb the only publisher of the keys and also
uses standard tasks that already have sstate.
(From OE-Core rev: 1e38068ac38dfd067655dfd41464e28439179306)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows directly giving the passphrase, instead of reading from a
file.
[YOCTO #9006]
(From OE-Core rev: fd55c6e86b38b33f62006324e73678a13a534220)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Refactor the LocalSigner class. Do not store keyid or passphrase file in
the signer object as they are only needed for some of the methods. For
example, the newly added verify() method does not need any key
parameters and export_pubkey only uses keyid.
(From OE-Core rev: e2412294b6b1d3a80ee97a0706613349edc51d33)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A new method for verifying detached signatures.
[YOCTO #9006]
(From OE-Core rev: 4445c645c71151bd56ff7e133343a6f1e30cc3b3)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of raising a generic Exception that can't be handled specifically, raise
a ValueError. Also update the callers so any unexpected exceptions are not
ignored.
Also, rename isBigEngian() to isBigEndian().
(From OE-Core rev: c136652f9c0b35aafa393e63567daf029ae03929)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When opening an ELF file check that a filename points to a normal file before
attempting to open it, as if the file turns out to be something more exotic like
a FIFO it could hang forever.
(From OE-Core rev: 4b3576bc30d8f8cdcde25189def8b059fc92b27c)
(From OE-Core rev: d3af2058e2753516b9aaf7f6d71162363eea11d4)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It seems that if the --workdir option is not set konsole does open in the users
home directory. By setting --workdir . konsole opens in the recipes work
directory. This is the same behavior as observed for other consoles.
(Tested with Konsole 2.14.2 on Debian Jessie).
(From OE-Core rev: bd06944249c3de3f629c013e14f446464441c4da)
Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When extracting source for a recipe, if there are additional custom
tasks run that make changes to the source, create a commit in the
generated git branch so they are contained. This is particularly
useful for tasks that come before do_patch since otherwise the changes
might get incorporated in the first applied patch, but otherwise it
helps avoid the tree being dirty at any point.
Fixes [YOCTO #7626].
(From OE-Core rev: 997a77d9b20af1778b804778e5d8c8a7424f7582)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prevents crash in signing if GPG_PATH is defined.
(From OE-Core rev: 0ecd748258abfe5ed9e9e2505aeb42b232389968)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This if branch is causing the following error during do_rootfs:
"Exception: UnboundLocalError: local variable 'cfg_file' referenced
before assignment". This happends because the cfg_file variable is
defined at a deeper nesting level (just above the if branch).
(From OE-Core rev: 555981c10e6b7f5c2be7f2c3fefb89b1add72cab)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the nativelsb argument is not used, then create_locked_sstate_cache()
can get collisions when moving the files from the input_sstate_cache
to the output_sstate_cache.
The specific case where this was encountered was when a "universal"
nativelsb directory already existed in the input_sstate_cache.
(From OE-Core rev: 760f7178e0267f930c8af9cb59039e317149f944)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In some cases, we may have some kind of download layers in BBLAYERS, so
that we can set BB_NO_NETWORK to "1". This results in extremely large
extensible SDK. And we actually don't need these download layers in the
SDK.
Add a new variable, SDK_LAYERS_EXCLUDE, to enable users to explicitly
exclude some layers when generating the extensible SDK.
[YOCTO #8878]
(From OE-Core rev: acf1148bf3f4e489e9e2b0b8745753e1311ee812)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This sets the scene for removing the default False for expansion
from getVarFlag. This would later allow True to become the default.
On the most part this is an automatic translation with:
sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, True):g' -i `grep -ril getVar *`
In this case, the default was False, but True was used since in most
cases here expansion would be expected.
(From OE-Core rev: 42a10788e89b07b14a150ced07113566cf99fcdd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove opkg package feed lists after generating the rootfs. The
lists have been generated by the local feed but are no longer
necessary after image generation. This still leaves the package
management fully usable (and hence is different from omitting
package-management from IMAGE_FEATURES).
(From OE-Core rev: 25477874ef737777cedc623a8e1c5aedf2f4bae1)
Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than modifying files in .git/hooks, which can be read-only
(e.g., if it is a link to a directory in /usr/share), move away the
entire .git/hooks directory temporarily.
(From OE-Core rev: a88d603b51a9ebb39210d54b667519acfbe465c3)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new Python module (oe.gpg_sign) for handling GPG signing
operations, i.e. currently package and package feed signing. The purpose
is to be able to more easily support various signing backends and to be
able to centralise signing functionality into one place (e.g. package
signing and sstate signing). Currently, only local signing with gpg is
implemented.
[YOCTO #8755]
(From OE-Core rev: 9b3dc1bd4b8336423a3f8f7db0ab5fa6fa0e7257)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OE-Core rev: f2b64f725803ad8be7c2876c531e057a4fe5ca7c (poky
1362986886) unintentionally broke opkg/dpkg
multilib support within the SDK by making things not honour
self.install_order. This reinstates that code for opkg/dpkg but
not rpm where the original problem was.
(From OE-Core rev: 98b585120137a3db07ed742a8f18223883ad6dc5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This prevents an error in do_populate_sdk when building
buildtools-tarball with ipk as the package manager:
Exception: UnboundLocalError: local variable 'pkg' referenced
before assignment
(From OE-Core rev: f971ae290cbbc51ec3669b57c55b2b7c0bd9bc87)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When constructing the sstate-cache directory for the extensible SDK,
we were copying in any matching native sstate packages, and as the
signature doesn't actually change when the distro changes (since
NATIVELSBSTRING is just a path separator for the artifacts and is not
part of the signature) we ended up copying duplicated packages when the
distro changed e.g. upon host distro upgrade. Only search in the
NATIVELSBSTRING-named subdirectory for native packages and the issue
goes away.
Fixes [YOCTO #8885].
(From OE-Core rev: 6c6baf6aa1823b8b20123f505e45c2768a193ad5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a variable SDK_INCLUDE_PKGDATA which you can set to "1" to include
pkgdata for all recipes in the world target. There are a couple of uses
for this:
1) If you use "devtool add" to add a recipe that builds something which
depends on anything in world, the dependency can then be correctly
mapped to the recipe providing it and that recipe can be added to
DEPENDS, since we have the pkg-config and shared library dependency
data within pkgdata.
2) You'll be able to search for these recipes and any files they
package for the target with "devtool search" since that also uses
pkgdata
This of course assumes you've tailored world through EXCLUDE_FROM_WORLD
to only include recipes you'd want built in your distro, but I think
that's a reasonable assumption; failing that there is a
WORLD_PKGDATA_EXCLUDE variable that you can set to exclude any recipes
you don't want.
Note that this patch relies on functionality implemented in a recent
BitBake patch and will not work without it.
Implements [YOCTO #8600].
(From OE-Core rev: 67149ea097d6fab7496b43e85a40853f40bd527e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that the method list() is not used anymore, remove it.
[YOCTO #7427]
(From OE-Core rev: 01e7dfbce972cfb926668e5ee194c83838e1e1b6)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch changes the use list_pkgs() instead of list()
from class RpmPkgsList. The change is in two functions,
image_list_installed_packages from rootfs.py and
sdk_list_installed_packages from sdk.py.
With this change the functions calling the functions
listed above, must format the output as they required.
The formatting can be done using format_pkg_list() from
oe.utils.
The classes calling the afected functions are changed too
with this patch, to keep the same functionality using the
new data structure.
[YOCTO #7427]
(From OE-Core rev: 983ea373362514e5888bd1d7d9c4f136c94b00f2)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The class PkgsList returns a dictionary with all the installed
packages, because the data structure is a dictionary there is
needed to format the data in order to write to a file.
The function format_pkg_list returns a formated sting with all
packages installed. The output will depend on the requested format
when calling the function.
[YOCTO #7427]
(From OE-Core rev: 25725e6e5fff8017aaf3a6fcd9b1b893c22630b5)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the class PkgList returns a formated string of the
installed packages. It would be more clean to pass a standard
data structure to the callers instead to format the output
inside PkgsList class.
This patch adds list_pkgs() method to PkgsList class to get the
all the information for installed packages and return a dictionary
with the info.
[YOCTO #7427]
(From OE-Core rev: 6cbb144a86a8188fad102bb281fd9e8d0a4b9142)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rootfs with dpkg was failing due to false-positive in logcheck_error
because current logic of DpkgPM handles missing dependencies failure
using apt-get -f install [1][2].
This support was broken due to addition of logcheck and don't take into
account dpkgpm cases, in order to fix add an attr for specify expected
errors regex'es by package manager.
[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/rootfs.py#n659
[2] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/package_manager.py#n2038
(From OE-Core rev: fa7a5ebef87883755491b847c2f4e1a7b021d585)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* tmp/deploy/deb subdirectories do not get hyphens replaced
with underscores, so don't do that translation when building
the sources list.
* Fix MULTILIB_VARIANTS handling to be more general and
work for all architectures
* Also include a fix for a warning generated by apt
due to missing apt/preferences.d directory.
(From OE-Core rev: c3ee9b12ae704eadf02ff288d8035b5885d6218e)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Right now, the locked task hashes list for the extensible SDK locks
down only the sstate tasks.
Whilst asthetically pleasing, this gives two problems:
* Half the task are left floating meaning checksum mismatches
are a pain to debug
* The later code which copies relavent data files out the sstate
cache can't use any of this data.
This patch modifies things so all the checksums are listed in the locked
file. An exclusion of tasks probably makes more sense for the library
function rather than an allowed list.
The only sstate task being deliberaly excluded here was do_package
so add in a function to explictly exclude those sstate object files.
The net result of this that siginfo files for all tasks are included in
the SDK, which means commands like "bitbake -S printdiff" now function.
(From OE-Core rev: 6b70479e47b8a8743d8b410d6bc08da1607a318e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch splits the code in lib/oe/image into separate tasks, one per
image type. This removes the need for the simple task graph code and defers
to the bitbake task management code to handle this instead.
This is a good step forward in splitting up the monolithic code and starting
to make it more accessible to people.
It should also make it easier for people to hook in other tasks and processes
into the rootfs code.
Incidentally, the reason this code was all combined originally was due to
limitations of fakeroot where if you exited the session, you lost permissions
data. With pseudo this constraint was removed.
We did start to rework the rootfs/image code previously and got so far with
untangling it however we did prioritise some performance tweaks over splitting
into separate tasks and in hindsight, this was a mistake and should have been done
the other way around. That work was suspended due to changes in the people working
on the project but this split has always been intended, now is the time to finish
it IMO.
There were some side effects of doing this:
* The symlink for the manifest moves to the rootfs-postcommands class and into
the manifest function.
* There is no seperate "symlink removal" and "symlink creation", they are merged
* The date/time stamps of the manifest and the built images can now be different since
the tasks can be run separately and the datetime stamp will then be different
between do_rootfs and the do_image_* tasks.
(From OE-Core rev: c2dab181c1cdabac3be6197f4b9ea4235cbbc140)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the next step in splitting up do_image, move the pre and post processing
commands to separate tasks. This also creates the do_image_complete task
which acts as the end marker task for image generation.
(From OE-Core rev: 800528eaa421d451b596545125cb218e08989151)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Often, bitbake -S printdiff would show that there was a checksum not found
which would turn out to be from gcc-source. This is due to it being a
shared-workdir recipe.
For now, hardcode the special case into the sstatesig code to stop people
(including me) puzzling over this.
If/as/when we add any other shared workdir recipes, we'll need to rethink
this.
(From OE-Core rev: f11342f0c838b520828927c9d69f7c17309c1b48)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, smart is invoked with --log-level=warning, which hides all
output from pre/post install scriptlets. That makes it hard to debug
scriptlet failure or why they get postponed to first-boot via
/etc/rpm-postinst.
The new ROOTFS_RPM_DEBUG variabled is expected to be set to an integer in
local.conf an incrementally adds more output:
0 = default, only warnings
1 = --log-level=info (includes information about executing scriptlets and their output)
2 = --log-level=debug
3 = --log-level=debug plus dumps of scriplet content and command invocation
The default behavior is not changed yet, but it seems that level 1 would
be a better default.
(From OE-Core rev: 5cb597a19dbfe825e5b26d828e18644c9ee58f86)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the current state some of the base utils (update-rc.d,
base-passwd, shadow, and update-alternatives) are unistalled
when there is no package manager in the image. Checking for
previous commits, the unistall of these utils were to be
done in a read-only filesystem.
It is a valid option to have these utils without a package
manager, and also make sense to remove them when building a
read-only filesystem.
This changes the check logic from having a package mananger
to if is a read-only filesystem to remove the utils.
Another change implemented with this patch is that delayed
post installs now doesn't depend if there is a package manager.
Also it is a valid option to have post install scripts without
package manger.
[YOCTO #8235]
(From OE-Core rev: 5aae19959a443c6ac4b0feef10715c8acf3c6376)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Run remove_packaging_data() on both host and target sysroots to wipe
opkg state after install, similar to what RpmSdk does.
Use case:
Opkg may download local package indexes (I.e. file:// URLs) by
sym-linking them into /var/lib/opkg/lists/ instead of copying [1].
This leaves behind broken symlinks under the lists directory when
using opkg to build SDK sysroots.
The -h option may be set via SDKTAROPTS in some configurations to create
symlink-less SDK archives for Windows file systems. Sysroots containing
broken symlinks will fail to archive under this configuration.
Testing:
Verified /var/lib/opkg/ is empty after running populate_sdk() in a Fido
based distribution.
[1] http://git.yoctoproject.org/cgit/cgit.cgi/opkg/commit/?h=opkg-0.3.x&id=f9022a8520fcde8f1b71424d26a652c218fce685
(From OE-Core rev: c8e0ec2da9ad4ce1c103966906a85f68c15400dd)
Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Reviewed-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Natinst-ReviewBoard-ID: 119065
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use bb.utils.edit_metadata() to replace some of the logic in this
function; this avoids us effectively having two implementations of the
same thing. In the process fix the following issues:
* Insert values before any leading comments for the next variable
instead of after them
* Insert overridden variables (e.g. RDEPENDS_${PN}) in the correct place
* Properly handle replacing varflag settings (e.g. SRC_URI[md5sum])
(From OE-Core rev: 0f81b83fc5fd908efa7f6b837137830ca65f6ed6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that the user specifies just the name portion instead of a file
name with extension. (We can't just look for . since there are recipe
names such as "glib-2.0" that legitimately contain .).
(From OE-Core rev: a4c6af737811adb2bca87e3896f8710738dd255e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For patches that we have to extract the header information by hand (i.e.
will not apply with "git am"), make the following improvements:
* If we can't extract author/date/subject, then try to do so from the
commit that added the patch in git (assuming the metadata is tracked
by git)
* Take only first Signed-off-by line instead of last
* Accept any case for "Signed-off-by" in case author has typed it by
hand
* Improve conditional - we can skip the other cases if one matches
Implements [YOCTO #7624].
(From OE-Core rev: 13ec296b5c35aefa2c44f64f8bd1ef54c4a0a731)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If multiple images are being built simultaneously against the same
sysroot then the call to os.makedirs in Image._write_wic_env can fail
with:
File: '.../meta/lib/oe/image.py', lineno: 341, function: _write_wic_env
0337: """
0338: stdir = self.d.getVar('STAGING_DIR_TARGET', True)
0339: outdir = os.path.join(stdir, 'imgdata')
0340: if not os.path.exists(outdir):
*** 0341: os.makedirs(outdir)
0342: basename = self.d.getVar('IMAGE_BASENAME', True)
0343: with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
0344: for var in self.d.getVar('WICVARS', True).split():
0345: value = self.d.getVar(var, True)
File: '/usr/lib/python2.7/os.py', lineno: 157, function: makedirs
0153: if e.errno != errno.EEXIST:
0154: raise
0155: if tail == curdir: # xxx/newdir/. exists if xxx/newdir exists
0156: return
*** 0157: mkdir(name, mode)
0158:
0159:def removedirs(name):
0160: """removedirs(path)
0161:
Exception: OSError: [Errno 17] File exists: '.../tmp-glibc/sysroots/cheetah/imgdata'
Using bb.utils.mkdirhier() protects against this.
There's also little point in checking to see if the directory already
exists - we might as well just try and create it regardless.
Once the directory has been created, there's no race on the actual file
since the filename contains IMAGE_BASENAME.
(From OE-Core rev: e3f216eef1b7a8ecfd00317e60704cff472238ba)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* without this the do_rootfs task doesn't respect OPKGLIBDIR and
info, status are created in different directory than opkg on
target expects
* people who modify OPKGLIBDIR need to make sure that opkg.conf included
in opkg package also sets info_dir and status_file options
(From OE-Core rev: 11f1956cf5d732cfed6d28c8c292b5053966895c)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Creating a file for every image containing a few variables isn't
necessary if wic is not being used, so don't write the file if WICVARS
is empty.
(From OE-Core rev: d40779a7d3ce4f326c29ec7971731cab1f505a37)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have a local workspace layer enabled when building the
extensible SDK, we explicitly exclude that from the SDK (mostly because
the SDK has its own for the user to use). Adjust the message we print
notifying the user of this so it's clear that we're excluding it from
the SDK, and scale it back from a warning to a note printed with
bb.plain().
(From OE-Core rev: 90f46f74a088a7b965d2205eceb9eff6f276dd38)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If an old version is detected (<1.9), create a new window instead of split:
the reason is that there is no easy way to get the active pane height if no
nested formats are supported.
(From OE-Core rev: 457bd6297ae99627c5f596c3c09086d787d5a5ab)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change removes the file installed_pkgs.txt after it
was used for installation of complementary packages. This
file was causing confusion when left in the WORKDIR after
the build.
(From OE-Core rev: d0f3f3a294d509560bd12b93b26eeec65cfee314)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The method _uninstall_unneeded uses the file installed_pkgs.txt,
this file is left after the build and can cause confusion. This
changes allow to get the installed packages using functions of
rootfs instead of the installed_pkgs.txt file. With this change
now is possible to remove the file without breaking anything.
[YOCTO #8444]
(From OE-Core rev: bf935ac16f6175673417dda92a619946b52fac87)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the proxies dictionary has a proxy set to None urllib will throw an exception
instead of not using a proxy (abridged stack):
File: '/home/ross/Yocto/poky/meta/lib/oe/distro_check.py', lineno: 43, function: get_links_from_url
*** 0043: with create_socket(url,d) as sock:
0044: webpage = sock.read()
File: '/home/ross/Yocto/poky/meta/lib/oe/distro_check.py', lineno: 5, function: create_socket
0003:def create_socket(url, d):
0004: import urllib
*** 0005: socket = urllib.urlopen(url, proxies=get_proxies(d))
File: '/usr/lib/python2.7/urllib.py', lineno: 87, function: urlopen
0086: if data is None:
*** 0087: return opener.open(url)
File: '/usr/lib/python2.7/urllib.py', lineno: 203, function: open
0201: else:
0202: proxy = None
*** 0203: name = 'open_' + urltype
0204: self.type = urltype
0205: name = name.replace('-', '_')
Exception: TypeError: cannot concatenate 'str' and 'NoneType' objects
Filter out unset values so that the dictionary only has valid assignments in.
(From OE-Core rev: 2d91290ab5608dd1297d1c26ab807fc4574a8a6b)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The value isn't a string so don't try and expand it.
(From OE-Core rev: ab87d3649c39326938d82d623efafb76905f770d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This doesn't cause any issues right now but it make sense to standardise
on consistently using strings in the data store.
(From OE-Core rev: 99203fbe5ad470ef65cff93cec9d7f332883b5ee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some intercepts may want to access files in the native sysroot that are not on
$PATH (such as something in $libexecdir) but any use of STAGING_DIR_NATIVE in
the postinst that calls the intercept will be "baked" into the package, so if
sstate is reused it will use paths that may not exist.
Solve this by exporting the location of the native sysroot in the environment so
the postinst and intercept can use an environment variable instead of a bitbake
variable.
(From OE-Core rev: 646599a1f3d39238ba8009993750a46ae220e762)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the log_check_regex can potentially be false positive it
makes sense to print the whole line where error is found.
This way user will be able to see the error and understand
if it's valid or not.
[YOCTO: #7789]
(From OE-Core rev: f9cf31525fc885e1a0f65bd55654631257f87078)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed package feeds are not yet implemented for these package formats.
(From OE-Core rev: 49a5c8700deddac744ccfa033bebf7971f92e14b)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If defined, use GPG_PATH as the gpg home directory when signing package
feeds. This setting is only used by package_manager if package feed
singning has been enabled, i.e. PACKAGE_FEED_SIGN="1".
(From OE-Core rev: 2c94853e4c8e92ac5fe4af7b25e2ce8853a4025b)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton
Joshua Lock
Stephano Cetola
Markus Lehtonen
Richard Purdie
Jacob Kroon
Clemens Lang
Ross Burton
Mark Hatle
Ed Bartosh
Bill Randle
Chen Qi
Aníbal Limón
Leonardo Sandoval
Mariano Lopez
Jussi Kukkonen
Anders Darander
Peter Kjellerstedt
Denys Dmytriyenko
Brendan Le Foll
Randy Witt
Patrick Ohly
Robert Yang
Ioan-Adrian Ratiu
Pascal Bach
Stefan Agner
Matt Madison
Haris Okanovic
Mike Crowe
Martin Jansa
Dariusz Pelowski