Commit Graph

134 Commits

Author SHA1 Message Date
Paul Eggleton 54efca180b classes/license: drop erroneous sha256 parameter in LIC_FILES_CHKSUM
In OE-Core commit a48fea275b, a check on
the value of a "sha256" parameter was added, however there was no
mention of this in the commit message and no corresponding code to
actually verify the checksum as sha256 was added along with it either,
so there's no point in getting the value. Additionally it was assuming
that a sha256 value would be present without checking first, with the
result that if you leave out the md5 value in a recipe intentionally in
order to get it to tell you the correct value on the next build, you got
a traceback instead of the appropriate error containing the information.
Drop this entirely - if we want to implement this we need to do it
properly.

(From OE-Core rev: e9eaa7d15fe7ab643ab19556dab84051f8f1974e)

(From OE-Core rev: fcedfce43adad8659a1376bb28b1ed41e621dd53)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11 22:15:59 +01:00
Juro Bystricky 950be7ae8d license.bbclass: improve reproducibility
Two identical builds can end up having deploy/licenses folders
that differ. This is observed in cases where there are several
different license files of the same name in different folders,
i.e. "COPYING". In those case we have to differentiate the files
somehow and we do it via file expensions such as COPYING.0, COPYING.1.
However, which file will get which extension is presently random.
This means, for example, that COPYING.0 in one build is the same
as COPYING.1 in the other (and vice versa).
Although there is mothing wrong with this, for the sake of binary
reproducibility it is preferable to have a deterministic outcome.

(From OE-Core rev: 007ebc84979b1bc8b7520097793c7ab6d646c243)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:43 +01:00
Ross Burton 0ed537f95b license: don't assume source files are UTF-8
We can't assume that source files are entirely UTF-8, so when copying the
license blocks open the file as binary instead of text.

[ YOCTO #11135 ]

(From OE-Core rev: b606e1430c36f1ad528fbfbbf9b8b6243390b879)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-10 14:50:11 +00:00
Markus Lehtonen 6e1aad4596 license.bbclass: run chown only under pseudo
The copy_license_files() function in license.bbclass is called in two
different contexts. First, it is run as part of the do_populate_lic
task. In addition, it is called from do_package task when
LICENSE_CREATE_PACKAGE is enabled. The function has code that changes
the owner of license files to root which is meant only to happend in the
latter case - i.e. under do_package which is run under pseudo.
Previously, the code was blindly always running chown and just ignored
errors that happened when running from do_populate_lic. This patch
changes it to be more intelligent, only doing chown when running under
pseudo.

[YOCTO #10897]

(From OE-Core rev: 19118a1408f32bb24d95ab3d7d7faed58cbae900)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01 23:27:09 +00:00
Aníbal Limón 129fbf324a classes/license.bbclass: Don't copy unneeded licenses by package
Usually a recipe only provides one package but when provides more
than one package the LICENSE variable per package (i.e. linux-firmware)
needs to take into account to avoid unnecesary copy of licenses into
packages.

The patch validates if LICENSE exists in package LICENSES in order to
don't copy unneeded licenses.

As result of this patch some packages will not contain licenses there
are not into LICENSE variable.

For example:

acl contains GPLv2+ instead of GPLv2+ and LGPLv2.1+
libacl contains LGPLv2+ instead of GPLv2+ and LGPLv2.1+

This behaviour is declared on the acl recipe as:

SUMMARY = "Utilities for managing POSIX Access Control Lists"
HOMEPAGE = "http://savannah.nongnu.org/projects/acl/"
SECTION = "libs"

LICENSE = "LGPLv2.1+ & GPLv2+"
LICENSE_${PN} = "GPLv2+"
LICENSE_lib${BPN} = "LGPLv2.1+"

[YOCTO #10325]

(From OE-Core rev: 8c8c8edea9c9015e21f47f3d10e6f45446a2823b)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01 23:27:06 +00:00
Aníbal Limón 5a2a1ec16b license.bbclass: Don't copy again LICENSE already handled as no-generic
The NO_GENERIC_LICENSE mapping was added [1] to enable copy LICENSES
from upstream source code into recipe licenses, previously that only
common-licenses was processed.

This result on copy twice the NO_GENERIC_LICENSE specified because there
is a mapping between license in LIC_CHKSUM and NO_GENERIC_LICENSE.

In order to avoid double copy one as generic_ and other as LICENSE. keep
track of licenses already copied.

For linux-firmware the result will be only generic_ licenses into
common-licenses.

[YOCTO #10325]

[1] http://lists.openembedded.org/pipermail/openembedded-core/2015-April/104222.html

(From OE-Core rev: 95b9e2cd26c7cae265ff52af90480b75251f00e5)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01 23:27:06 +00:00
Mike Crowe 0b8d85a094 kernel, license, sstate, rootfs.py: Remove deploy directory README
It isn't clear that the README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
file in the deploy directory warrants the complexity it brings elsewhere.
Let's just remove it entirely.

In particular, if two do_image_complete tasks run in parallel they risk
both trying to put their image into ${DEPLOY_DIR_IMAGE} at the same time.
Both will contain a README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
file. In theory this should be safe because "cp -alf" will just cause one
to overwrite the other. Unfortunately, coreutils cp also has a race[1]
which means that if one copy creates the file at just the wrong point the
other will fail with:

 cp: cannot create hard link ‘..../tmp-glibc/deploy/images/pantera/README_-_DO_NOT_DELETE_FILES_IN_THIS_D.txt’ to
+‘..../tmp-glibc/work/rage_against-oe-linux-gnueabi/my-own-image/1.0-r0/deploy-my-own-image-complete/README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt’: File exists

[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25680

(From OE-Core rev: 71e9e88847d7000781642ea6187ebd8f40dfdcfe)

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01 11:17:45 +00:00
Joshua Watt 8279d6f257 classes/license: Respect beginline and endline
Recipes have the option of specifying a 'beginline' and 'endline'
options in LIC_FILES_CHKSUM which will cause the license bbclass to
only extract those specific lines from the license file when generating
the license database.

(From OE-Core rev: 515f4c129ad27e9502621738d0bf7f5552627f19)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:41 -08:00
Richard Purdie 022a8b58c8 meta/scripts: Various getVar/getVarFlag expansion parameter fixes
There were a few straggling expansion parameter removals left for
getVar/getVarFlag where the odd whitespace meant they were missed
on previous passes. There were also some plain broken ussages such
as:

d.getVar('ALTERNATIVE_TARGET', old_name, True)
path = d.getVar('PATH', d, True)
d.getVar('IMAGE_ROOTFS', 'True')

which I've corrected (they happend to work by luck).

(From OE-Core rev: 688f7a64917a5ce5cbe12f8e5da4d47e265d240f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:11 +00:00
Joshua Lock 3c59b1bf93 meta: remove True option to getVarFlag calls
getVarFlag() now defaults to expanding by default, thus remove the
True option from getVarFlag() calls with a regex search and
replace.

Search made with the following regex:
getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\)

(From OE-Core rev: 2dea9e490a98377010b3d4118d054814c317a735)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Joshua Lock c4e2c59088 meta: remove True option to getVar calls
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.

Search made with the following regex: getVar ?\(( ?[^,()]*), True\)

(From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Ross Burton e35775c598 classes/license: copy licenses even if there are no checksummed files
Previously do_populate_lic would bail writing the license files (such as the MIT
license text) to deploy/licenses/${PN}/ if there were no files listed in
LIC_FILES_CHKSUM.  In general this isn't a problem since LIC_FILES_CHKSUM was
effectively a required field, and for packages which don't have traditional
content was generally set to a separate license such as
${COREBASE}/meta/COPYING.MIT.

However LIC_FILES_CHKSUM is now only required if there are sources in SRC_URI,
so oe-core b18fa5f removed these now redundant values. The unexpected side
effect of this is that that recipes that generate their content (such as
os-release) or are otherwise "interesting" (such as perf) don't have their
license files copied over, resulting in warnings from do_rootfs.

Resolve this by not returning early if LIC_FILES_CHKSUM is empty, and always
copy the references license files.

(From OE-Core rev: 94f5ac0651249cb98fb8d1308b7c0d10605d5a1e)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-06 23:35:36 +00:00
Paul Eggleton 6879d524c1 classes/license: fix handling of symlinks pointed to in LIC_FILES_CHKSUM
If you set LIC_FILES_CHKSUM to point to a relative symlink then you'll
get "Could not copy license file" warnings in copy_license_files() since
the symlink won't be valid after it's copied. If the source is a symlink
then we need to dereference it first.

I encountered this when I used recipetool on the sources for capnproto,
where the c++ directory contains a LICENSE.txt symlink to the LICENSE
file in the parent directory, and this symlink ends up being pointed to
in LIC_FILES_CHKSUM.

(From OE-Core rev: c4d3b1e9c37b920444e53d3231552da18d101882)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-06 23:35:32 +00:00
Ulf Magnusson 0aaa51e1e5 license.bbclass: Use bb.fatal() instead of raising FuncFailed
This sets a good example and avoids unnecessarily contributing to
perceived complexity and cargo culting.

Motivating quote below:

< kergoth> the *original* intent was for the function/task to error via
           whatever appropriate means, bb.fatal, whatever, and
           funcfailed was what you'd catch if you were calling
           exec_func/exec_task. that is, it's what those functions
           raise, not what metadata functions should be raising
< kergoth> it didn't end up being used that way
< kergoth> but there's really never a reason to raise it yourself

FuncFailed.__init__ takes a 'name' argument rather than a 'msg'
argument, which also shows that the original purpose got lost.

(From OE-Core rev: 8e9255763674703ea16651da64fe794e5359f16e)

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-04 16:29:04 +01:00
Markus Lehtonen 831e983251 license.bbclass: do not process LICENSE_pn variables
The loop iterating over LICENSE_pn variables has never worked. In
addition, the LICENSE variable is supposed to contain all licenses
defined in LICENSE_pn variables. Thus, it is simpler just to use LICENSE
as the data we get is essentially the same.

[YOCTO #9499]

(From OE-Core rev: d7229489c7dfd35164fd107d7944f3c273776118)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-25 22:54:49 +01:00
Markus Lehtonen a48fea275b license: improve handling of license files with identical basenames
Previously, find_license_files() in license.bbclass just blindly assumed
that all different licenses specified in LIC_FILES_CHKSUM have unique
filenames. As a consequence, only the last one of these similarly named
license files was copied and the rest were "lost". This patch changes
the behavior so that all license files get copied. However, if multiple
identically named files are found, they are renamed to <file>.0,
<file>.1 etc.

The patch also changes the handling of NO_GENERIC_LICENSE slightly.
Previously, only basenames of NO_GENERIC_LICENSE and LIC_FILES_CHKSUM
were compared when searching for the correct license file. After this
patch NO_GENERIC_LICENSE must have the full path, matching what is
specified in LIC_FILES_CHKSUM. This is required in order to be able
to handle identical filenames (basenames) consistently. For example, if
you have:
LICENSE = "my-custom-license"
LIC_FILES_CHKSUM = "file://src/LICENCE;md5=d41d8cd98f00b204e9800998ecf8427e"

you must specify:
NO_GENERIC_LICENSE[my-custom-license] = "src/LICENCE"

[YOCTO #9663]

(From OE-Core rev: d5e1375884e509ec745bac43f1f7f7950f62f280)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-10 10:46:32 +01:00
Ross Burton 4fdfee4a78 classes/license.bbclass: don't set [dirs] and [cleandirs]
There's no need to set these as the restore from sstate will create the
directories as required.

(From OE-Core rev: 3ab3ebc06d22f0776091e39237235ea50c4503b2)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Manuel Huber 66c5a4ad62 classes/license: handle EXDEV if hard link to license fails
Hard links can still fail even if st_dev is the same for source
and destination. In case of EXDEV error, fall back to copying.

(From OE-Core rev: c00423d6bab9849e331beadf4d3cee90e04fe295)

Signed-off-by: Manuel Huber <manuel.h87@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-15 18:05:22 +01:00
Richard Purdie 44e9a0d2fa classes/lib: Update to explictly create lists where needed
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.

(From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:00 +01:00
Richard Purdie 8587bce564 classes/lib: Update to match python3 iter requirements
python3 standardises its use of iteration operations. Update
the code to match the for python3 requires.

(From OE-Core rev: 2476bdcbef591e951d11d57d53f1315848758571)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:00 +01:00
Markus Lehtonen a9b503b268 license.bbclass: make sure that image manifest dir exists
Previously, write_deploy_manifest() was relying on
write_package_manifest() to create the subdirectory for the manifest
file. However, do_rootfs may be an empty function so that
write_package_manifest() will not be called and the manifest
subdirectory will not be created, causing a build failure. This patch
fixes that by creating the directory hierarchy inside
write_deploy_manifest().

[YOCTO #9446]

(From OE-Core rev: e2dbe5eb869b8336b91023b83d7ca866197efa73)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-22 16:28:57 +01:00
Markus Lehtonen 98bf7dea84 license.bbclass: do write_deploy_manifest in image postprocessing
Call write_deploy_manifest() in image postprocessing phase, instead of
rootfs postprocessing. The reason being that not necessarily all
do_deploy tasks are dependencies of the do_rootfs and we might miss
something.

[YOCTO #9446]

(From OE-Core rev: 1f35b05d60f4e84554c5ee837cb071815e12c91c)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-19 21:11:24 +01:00
Bjørn Forsman f7fed7ce75 license.bbclass: fix warnings when run in unprivileged "container" env
An unprivileged "container" environment like this[1] doesn't have root
account (uid 0) which causes tons of "Invalid argument" warnings:

  $ bitbake ...
  ...
  WARNING: Could not copy license file [src] to [dest]: [Errno 22] Invalid argument: '[src]'
  WARNING: Could not copy license file [src] to [dest]: [Errno 22] Invalid argument: '[src]'
  WARNING: Could not copy license file [src] to [dest]: [Errno 22] Invalid argument: '[src]'
  ...

Fix it by handling EINVAL similar to existing handling of EPERM (which
was added for when not running under pseudo).

[1]: The real environemnt is buildFHSUserEnv from NixOS/nixpkgs, but a
  demonstration of the issue can be done like this:

    $ touch f
    $ unshare --user --mount chown 0:0 f
    chown: changing ownership of ‘f’: Invalid argument

(From OE-Core rev: d00b2250a6afebd7d1373c04b4006290f0cd4043)

Signed-off-by: Bjørn Forsman <bjorn.forsman@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-06 10:29:23 +01:00
Jagadeesh Krishnanjanappa b051a953fc license.bbclass: fix host contamination warnings for license files
We get below host contamination warnings of license files for
each recipe, when we try to create a separate ${PN}-lic package (which
contains license files), by setting LICENSE_CREATE_PACKAGE equal to "1"
in local.conf.

-- snip --
WARNING: QA Issue: libcgroup: /libcgroup-lic/usr/share/licenses/libcgroup/generic_LGPLv2.1 is owned by uid 5001, which is the same as the user running bitbake. This may be due to host contamination [host-user-contaminated]
WARNING: QA Issue: attr: /attr-lic/usr/share/licenses/attr/libattr.c is owned by uid 5001, which is the same as the user running bitbake. This may be due to host contamination [host-user-contaminated]
WARNING: QA Issue: bash: /bash-lic/usr/share/licenses/bash/COPYING is owned by uid 5001, which is the same as the user running bitbake. This may be due to host contamination [host-user-contaminated]
-- CUT --

Since the license files from source and OE-core, are populated in a normal
shell environment rather in pseudo environment (fakeroot); the ownership of
these files will be same as host user running bitbake. During the do_package
task (which runs in pseudo environment (fakeroot)), os.link preserves the
ownership of these license files as host user instead of root user.
This causes license files to have UID same as host user id and resulting in
above warnings during do_package_qa task.

Changing ownership of license files to root user (which has UID and GID as 0)
under pseudo environment will solve above warnings, and on exiting pseudo
environment the license files will continue to be owned by host user. Perform
this manipulation within try/except statements, as tasks which are not exected
under pseudo (such as do_populate_lic) result in OSError when trying to
change ownership of license files.

(From OE-Core rev: a411e96c3989bc9ffbd870b54cd6a7ad2e9f2c61)

Signed-off-by: Jagadeesh Krishnanjanappa <jkrishnanjanappa@mvista.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 16:58:12 +00:00
Ross Burton ce1bed7d3d license.bbclass: add LICENSE_CREATE_PACKAGE to perform_packagecopy vardeps
This class has a perform_packagecopy_prepend() that uses LICENSE_CREATE_PACKAGE,
but the BitBake variable dependency code doesn't know that oe.data.typed_value()
is a getter.  Until BitBake knows this, add an explicit vardeps.

[ YOCTO #5635 ]

(From OE-Core rev: 0450c74e36b5547e3e903fdabe0f52675ac94dea)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:39:02 +00:00
Richard Purdie 7fa6eeba1c classes/lib: Add expand parameter to getVarFlag
This sets the scene for removing the default False for expansion
from getVarFlag. This would later allow True to become the default.

On the most part this is an automatic translation with:

sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, True):g'  -i `grep -ril getVar *`

In this case, the default was False, but True was used since in most
cases here expansion would be expected.

(From OE-Core rev: 42a10788e89b07b14a150ced07113566cf99fcdd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:39:00 +00:00
Mariano Lopez 6ebda8e659 lib/oe/rootfs: Use list_pkgs() instead of list()
This patch changes the use list_pkgs() instead of list()
from class RpmPkgsList. The change is in two functions,
image_list_installed_packages from rootfs.py and
sdk_list_installed_packages from sdk.py.

With this change the functions calling the functions
listed above, must format the output as they required.
The formatting can be done using format_pkg_list() from
oe.utils.

The classes calling the afected functions are changed too
with this patch, to keep the same functionality using the
new data structure.

[YOCTO #7427]

(From OE-Core rev: 983ea373362514e5888bd1d7d9c4f136c94b00f2)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:15 +00:00
Jackie Huang 320a319220 license.bbclass: fix license manifest
The license manifest is wrong when there is no spaces before and after separates |&()
and we can get warning like:
WARNING: The license listed Artistic-1.0GPLv1+ was not in the licenses collected for recipe

(From OE-Core rev: 20d6a4ce4e395251604d92111ce5ecacde5ce172)

Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:08 +00:00
Richard Purdie 388e5802df license: Fix BB_TASKDEPDATA references
We shouldn't try and expand what is a python dict object
and we don't want it to influence the task hashes, task
dependencies are already accounted for.

(From OE-Core rev: 697072130a3298aede7166f125aa982be23793ca)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-16 11:56:30 +00:00
Mariano Lopez d95df11131 license.bbclass: Create image license manifest
This change adds the license_deployed_manifest function
that will create the manifest for the packages deployed
next to the image but not installed in rootfs. Some
examples of these recipes would be the bootloaders, or
the kernel.

This new function was added to ROOTFS_POSTPROCESS_COMMAND
so it will run after every rootfs task.

This change also modify the write_license_files because
the image manifest is different from the root manifest.

[YOCTO #6772]

(From OE-Core rev: a58e8e164b888a6e4747ef62a7a43825c4f47c32)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:01 +00:00
Mariano Lopez efdab52624 license.bbclass: Add function get_deployed_files
This function will get the files that were deployed using
the sstate-control manifest file. This will give a better
view of what was deployed next to the image.

[YOCTO #6772]

(From OE-Core rev: f1a0a86ea2db6f16d9b643636a8973aaeb85e4a6)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:01 +00:00
Mariano Lopez cc0d044109 license.bbclass: Added function get_deployed_dependencies
This change introduce a new function to get the dependencies
that were deployed. It uses BB_TASKDEPDATAto get all the
dependencies of the current task, so it is possible to get
different packages depending at what point this function is
called.

[YOCTO #6772]

(From OE-Core rev: 2be96279580eac2b03970131dcd81b13c7f7f7d5)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:01 +00:00
Mariano Lopez d45e10e1cf license.bbclass: Added get_boot_dependencies function
This function gets the dependencies from the classes that
create a boot image, this is required because sometimes
the bootloader dependecy is in these classes. The current
classes covered are bootimg and bootdirectdisk because
these are the only clases that add dependencies.

[YOCTO #6772]

(From OE-Core rev: b165c2d3349115546c9db2f084eb6def4913b82d)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:01 +00:00
Mariano Lopez 8b1e7bcef2 license.bbclass: Split license create manifest
This changes moves the writing of the licenses to a
separated function that could be called for other packages.

With these change it will be easier to reuse the writing of
the license for the packages deployed but not installed in
the rootfs.

[YOCTO #6772]

(From OE-Core rev: 71fb5da7a3a8e61b15be4efc05b968c0c9920d2f)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:00 +00:00
Mariano Lopez 1a210e6374 license.bbclass: Write recipeinfo file in license folder
Currently there is no way to get the recipe version when
creating the rootfs. It is needed because the manifest
file for the image has to contain this important piece
of information.

This change writes a new file in the license folder for
every recipe. This file is called "recipeinfo" and have
the information used to write the manifest file for the
recipes deployed next to the image.

[YOCTO #6772]

(From OE-Core rev: 2d97d82fb51281fe130e8d135f3cf9a78423dd94)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:00 +00:00
Paul Eggleton 9167f200eb classes/license: fix intermittent license collection warning
Fixes the following warning sometimes appearing during image builds:

WARNING: The license listed ABC was not in the licenses collected for recipe xyz

The files being looked for here, which runs during do_rootfs,
are written out by the do_populate_lic task for each recipe. However,
there was no explicit dependency between do_rootfs and all of the
do_populate_lic tasks to ensure they had run - only an implicit link via
do_build, so it is possible that sometimes they had not depending on how
the tasks were scheduled. Add an explicit set of dependencies to fix
this.

(From OE-Core rev: ef7dc532e800d9b170246550cbc8703adf624beb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:50:26 +00:00
Alex Franco 9b6bf6a41f Empty image: package list splitting and iteration
A few short fixes to splitting/iteration done over package lists
in license.bbclass, package_manager.py and rootfs.py.

[YOCTO #7664]

(From OE-Core rev: a902e98c5938f52ec960e0518e0ceaf8f5ee610c)

Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-29 13:38:05 +01:00
Aníbal Limón b850da9224 license_class: Fix choose_lic_set into incompatible license
Use canonical_license when doing evaluation of license expresion since
INCOMPATIBLE_LICENSE are already canonized.

[YOCTO #8080]

(From OE-Core rev: 799b25e09a149303edc1c098935f38a438878a95)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 07:34:08 +01:00
Richard Purdie 86d30d756a meta: Add explict getVar param for (non) expansion
Rather than just use d.getVar(X), use the more explict d.getVar(X, False)
since at some point in the future, having the default of expansion would
be nice. This is the first step towards that.

This patch was mostly made using the command:

sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *`

(From OE-Core rev: ab7c1d239b122c8e549e8112c88fd46c9e2b061b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:25 +01:00
Aníbal Limón 45bf783a23 license_class: Add explicity avoid of CLOSED license in validations
The CLOSED license isn't a generic license it is a set and can
be any closed source license.

[YOCTO #7752]

(From OE-Core rev: f33651a46d12bb763a4e3c13438d084411406849)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-22 13:29:47 +01:00
Aníbal Limón 2d69dd2caf license_class: license_create_manifest improvment
Use image_list_installed_packges instead of open manually the package
manifest and iterate over it.

(From OE-Core rev: 7ffe10df73cc20d10fcd41b121074445273bd60e)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-09 22:28:14 +01:00
Aníbal Limón b394cd4c7e license: Add support for handle INCOMPATIBLE_LICENSE in manifest creation
When INCOMPATIBLE_LICENSE's is specified it need to be removed from
license.manifest and also avoid copy to target image.

Add ManifestVisitor that walk the license string searching for
INCOMPATIBLE_LICENSE's if found remove it.

[YOCTO #6765]

(From OE-Core rev: d1278570041029d7c9fc6ce657e9a1701a421841)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-09 22:28:14 +01:00
Aníbal Limón 8c7082de0a license_class: Generalize license_ok function
Add dont_want_licenses as parameter to license_ok function and move it
to oe.license module in order to use in other modules.

(From OE-Core rev: 243fe3a4583a21ad6c0b2a26196ed18d41740f7a)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-09 22:28:14 +01:00
Aníbal Limón b971eb0313 license_class: Reimplemented manifest creation in python
Reimplemented license_manifest_create from shell to python for
INCOMPATIBLE_LICENSE handle using oe.license module.

Optimizations are made to avoid license copy now uses a hardlink
and symbolic link this helps to save space during build.

(From OE-Core rev: a7976cbae34dc1bd08395afb8d720d43c5f0062e)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-09 22:28:14 +01:00
Jackie Huang 9c2e9616c2 license.bbclass: allow copying license not in common licenses
Some package like linux-firmware has many licenses that aren't in any
way common, and new ones will be added from time to time, in order to
avoid adding bunch of such common license files that are only applicable
to a specific package, NO_GENERIC_LICENSE is added to allow copying license
not in common licenses, it should be used in the recipe as:

NO_GENERIC_LICENSE[<license_name>] = "<license file in fetched source>"

e.g.
NO_GENERIC_LICENSE[Firmware-Abilis] = "LICENCE.Abilis.txt"

(From OE-Core rev: 56930227128d55dab22f79138152b29cf040ceff)

Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-30 23:04:16 +01:00
Chen Qi d70c981b83 license.bbclass: skip license checking if the package contains no file
If the package doesn't contain any file, then the license isn't relevant
as far as the final image is concerned. So we skip the license checking
in license_create_manifest if such case.

(From OE-Core rev: f7b6684d2248f4a56e70b292891f9a454275449d)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-22 08:50:03 +00:00
Robert Yang 05a69edeb7 license.bbclass: set dirs for do_populate_lic_setscene
Fixed:
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
  File "bitbake/lib/bb/build.py", line 497, in exec_task
    return _exec_task(fn, task, d, quieterr)
  File "bitbake/lib/bb/build.py", line 437, in _exec_task
    exec_func(func, localdata)
  File "bitbake/lib/bb/build.py", line 212, in exec_func
    exec_func_python(func, d, runfile, cwd=adir)
  File "/home/nxadm/nx/ala-blade44.1/builds-2015-03-09-163005/qemuppc_world_oe_bp/bitbake/lib/bb/build.py", line 237, in exec_func_python
    os.chdir(cwd)
OSError: [Errno 2] No such file or directory: 'bitbake_build/tmp/work/ppc7400-wrs-linux/taglib/1.9.1-r0/build'

When running setscene, the cwd is $B which maybe removed by
autotools.bbclass or cmake.bbclass when rebuild.

(From OE-Core rev: b3262af11094f93c9588f2e3edc70a5058491e6d)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-16 17:41:55 +00:00
Martin Jansa 2f8e5a8be1 license.bbclass: fix unexpected operator for LICENSE values with space
* add quotes around pkged_lic so that it works correctly with spaces
* fixes following error:
  run.license_create_manifest.50601: 193: [: GPLv2: unexpected operator

(From OE-Core rev: 2bb8b2abb689d91b7b7e28e6bd528747bde94dd2)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-17 22:35:04 +00:00
Paul Eggleton 9f03969994 oe-pkgdata-util: improve command-line usage
* Use argparse instead of optparse for standardised help output, options
  and a much cleaner code structure
* Look up pkgdata directory automatically so the user doesn't have to
  specify it
* Use standard logging

NOTE: this does mean a slight change in syntax - if you do want to
specify the pkgdata directory (usually only necessary if you're calling
it from within the build process) you need to use the parameter -p (or
 --pkgdata-dir) and specify this before the command, not after it.

Examples:

oe-pkgdata-util find-path /sbin/mke2fs
oe-pkgdata-util lookup-recipe libelf1
oe-pkgdata-util read-value PKGSIZE libc6
oe-pkgdata-util -p /home/user/oe/build/tmp/sysroots/qemux86-64/pkgdata read-value PKGSIZE libc6

(From OE-Core rev: 04dc571ac7c26f0dcf1a1fcd466482e22519998d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-14 08:41:01 +00:00
Paul Eggleton 30a5c9edc6 classes/license: tweak license format messages
Strictly speaking not all of these characters are operators, so reword
the message to describe them as separators. Also use the standard
"recipename: message" format.

(From OE-Core rev: 9e5da3e83f2a5d43620e07a31728b7427329d8f4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-08 08:00:25 +00:00