Commit Graph

475 Commits

Author SHA1 Message Date
Markus Lehtonen bb971577ab meta/lib: new module for handling GPG signing
Add a new Python module (oe.gpg_sign) for handling GPG signing
operations, i.e. currently package and package feed signing. The purpose
is to be able to more easily support various signing backends and to be
able to centralise signing functionality into one place (e.g.  package
signing and sstate signing). Currently, only local signing with gpg is
implemented.

[YOCTO #8755]

(From OE-Core rev: 9b3dc1bd4b8336423a3f8f7db0ab5fa6fa0e7257)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 22:31:58 +00:00
Richard Purdie a1c35f302d lib/oe/sdk: Partially revert "sdk.py: fix conflicts of packages"
OE-Core rev: f2b64f725803ad8be7c2876c531e057a4fe5ca7c (poky
1362986886) unintentionally broke opkg/dpkg
multilib support within the SDK by making things not honour
self.install_order. This reinstates that code for opkg/dpkg but
not rpm where the original problem was.

(From OE-Core rev: 98b585120137a3db07ed742a8f18223883ad6dc5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:34 +00:00
Joshua Lock 0aeb33f6e0 lib/oe/package_manager: prevent testing an undefined variable
This prevents an error in do_populate_sdk when building
buildtools-tarball with ipk as the package manager:

Exception: UnboundLocalError: local variable 'pkg' referenced
before assignment

(From OE-Core rev: f971ae290cbbc51ec3669b57c55b2b7c0bd9bc87)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton 477fa84390 gen-lockedsig-cache: copy correct native sstate into ext SDK
When constructing the sstate-cache directory for the extensible SDK,
we were copying in any matching native sstate packages, and as the
signature doesn't actually change when the distro changes (since
NATIVELSBSTRING is just a path separator for the artifacts and is not
part of the signature) we ended up copying duplicated packages when the
distro changed e.g. upon host distro upgrade. Only search in the
NATIVELSBSTRING-named subdirectory for native packages and the issue
goes away.

Fixes [YOCTO #8885].

(From OE-Core rev: 6c6baf6aa1823b8b20123f505e45c2768a193ad5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton d3a4f72896 classes/populate_sdk_ext: add option to bring in pkgdata for world
Add a variable SDK_INCLUDE_PKGDATA which you can set to "1" to include
pkgdata for all recipes in the world target. There are a couple of uses
for this:

1) If you use "devtool add" to add a recipe that builds something which
   depends on anything in world, the dependency can then be correctly
   mapped to the recipe providing it and that recipe can be added to
   DEPENDS, since we have the pkg-config and shared library dependency
   data within pkgdata.
2) You'll be able to search for these recipes and any files they
   package for the target with "devtool search" since that also uses
   pkgdata

This of course assumes you've tailored world through EXCLUDE_FROM_WORLD
to only include recipes you'd want built in your distro, but I think
that's a reasonable assumption; failing that there is a
WORLD_PKGDATA_EXCLUDE variable that you can set to exclude any recipes
you don't want.

Note that this patch relies on functionality implemented in a recent
BitBake patch and will not work without it.

Implements [YOCTO #8600].

(From OE-Core rev: 67149ea097d6fab7496b43e85a40853f40bd527e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:32 +00:00
Mariano Lopez 571289df7b lib/oe/package_manager.py: Remove list() from PkgsList class
Now that the method list() is not used anymore, remove it.

[YOCTO #7427]

(From OE-Core rev: 01e7dfbce972cfb926668e5ee194c83838e1e1b6)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:15 +00:00
Mariano Lopez 6ebda8e659 lib/oe/rootfs: Use list_pkgs() instead of list()
This patch changes the use list_pkgs() instead of list()
from class RpmPkgsList. The change is in two functions,
image_list_installed_packages from rootfs.py and
sdk_list_installed_packages from sdk.py.

With this change the functions calling the functions
listed above, must format the output as they required.
The formatting can be done using format_pkg_list() from
oe.utils.

The classes calling the afected functions are changed too
with this patch, to keep the same functionality using the
new data structure.

[YOCTO #7427]

(From OE-Core rev: 983ea373362514e5888bd1d7d9c4f136c94b00f2)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:15 +00:00
Mariano Lopez 03075f671c lib/oe/utils: Add function format_pkg_list()
The class PkgsList returns a dictionary with all the installed
packages, because the data structure is a dictionary there is
needed to format the data in order to write to a file.

The function format_pkg_list returns a formated sting with all
packages installed. The output will depend on the requested format
when calling the function.

[YOCTO #7427]

(From OE-Core rev: 25725e6e5fff8017aaf3a6fcd9b1b893c22630b5)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:15 +00:00
Mariano Lopez c708411f20 lib/oe/package_manager: Add list_pkgs() to PkgsList class
Currently the class PkgList returns a formated string of the
installed packages. It would be more clean to pass a standard
data structure to the callers instead to format the output
inside PkgsList class.

This patch adds list_pkgs() method to PkgsList class to get the
all the information for installed packages and return a dictionary
with the info.

[YOCTO #7427]

(From OE-Core rev: 6cbb144a86a8188fad102bb281fd9e8d0a4b9142)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:15 +00:00
Aníbal Limón b8ebac9794 DpkgRootfs: Fix logcheck_error false-positive when use multilib
Rootfs with dpkg was failing due to false-positive in logcheck_error
because current logic of DpkgPM handles missing dependencies failure
using apt-get -f install [1][2].

This support was broken due to addition of logcheck and don't take into
account dpkgpm cases, in order to fix add an attr for specify expected
errors regex'es by package manager.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/rootfs.py#n659
[2] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/package_manager.py#n2038

(From OE-Core rev: fa7a5ebef87883755491b847c2f4e1a7b021d585)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:31 +00:00
Matt Madison 4aeb69d35f package_manager.py: fixes for multilib deb packaging builds
* tmp/deploy/deb subdirectories do not get hyphens replaced
  with underscores, so don't do that translation when building
  the sources list.

* Fix MULTILIB_VARIANTS handling to be more general and
  work for all architectures

* Also include a fix for a warning generated by apt
  due to missing apt/preferences.d directory.

(From OE-Core rev: c3ee9b12ae704eadf02ff288d8035b5885d6218e)

Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:31 +00:00
Richard Purdie e306d5495a populate_sdk_ext: Change to include siginfo and non sstate task sigs
Right now, the locked task hashes list for the extensible SDK locks
down only the sstate tasks.

Whilst asthetically pleasing, this gives two problems:

* Half the task are left floating meaning checksum mismatches
  are a pain to debug
* The later code which copies relavent data files out the sstate
  cache can't use any of this data.

This patch modifies things so all the checksums are listed in the locked
file. An exclusion of tasks probably makes more sense for the library
function rather than an allowed list.

The only sstate task being deliberaly excluded here was do_package
so add in a function to explictly exclude those sstate object files.

The net result of this that siginfo files for all tasks are included in
the SDK, which means commands like "bitbake -S printdiff" now function.

(From OE-Core rev: 6b70479e47b8a8743d8b410d6bc08da1607a318e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie 0a4e1f968a image: Create separate tasks for rootfs construction
This patch splits the code in lib/oe/image into separate tasks, one per
image type. This removes the need for the simple task graph code and defers
to the bitbake task management code to handle this instead.

This is a good step forward in splitting up the monolithic code and starting
to make it more accessible to people.

It should also make it easier for people to hook in other tasks and processes
into the rootfs code.

Incidentally, the reason this code was all combined originally was due to
limitations of fakeroot where if you exited the session, you lost permissions
data. With pseudo this constraint was removed.

We did start to rework the rootfs/image code previously and got so far with
untangling it however we did prioritise some performance tweaks over splitting
into separate tasks and in hindsight, this was a mistake and should have been done
the other way around. That work was suspended due to changes in the people working
on the project but this split has always been intended, now is the time to finish
it IMO.

There were some side effects of doing this:

* The symlink for the manifest moves to the rootfs-postcommands class and into
  the manifest function.
* There is no seperate "symlink removal" and "symlink creation", they are merged
* The date/time stamps of the manifest and the built images can now be different since
  the tasks can be run separately and the datetime stamp will then be different
  between do_rootfs and the do_image_* tasks.

(From OE-Core rev: c2dab181c1cdabac3be6197f4b9ea4235cbbc140)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie fdced52387 image: Move pre/post process commands to bbclass
As the next step in splitting up do_image, move the pre and post processing
commands to separate tasks. This also creates the do_image_complete task
which acts as the end marker task for image generation.

(From OE-Core rev: 800528eaa421d451b596545125cb218e08989151)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie 1ee08426d7 sstatesig: Handle special case of gcc-source shared-workdir for printdiff
Often, bitbake -S printdiff would show that there was a checksum not found
which would turn out to be from gcc-source. This is due to it being a
shared-workdir recipe.

For now, hardcode the special case into the sstatesig code to stop people
(including me) puzzling over this.

If/as/when we add any other shared workdir recipes, we'll need to rethink
this.

(From OE-Core rev: f11342f0c838b520828927c9d69f7c17309c1b48)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:47 +00:00
Patrick Ohly c91078954e package_manager.py: add debugging support for rpm scriptlet execution
By default, smart is invoked with --log-level=warning, which hides all
output from pre/post install scriptlets. That makes it hard to debug
scriptlet failure or why they get postponed to first-boot via
/etc/rpm-postinst.

The new ROOTFS_RPM_DEBUG variabled is expected to be set to an integer in
local.conf an incrementally adds more output:
  0 = default, only warnings
  1 = --log-level=info (includes information about executing scriptlets and their output)
  2 = --log-level=debug
  3 = --log-level=debug plus dumps of scriplet content and command invocation

The default behavior is not changed yet, but it seems that level 1 would
be a better default.

(From OE-Core rev: 5cb597a19dbfe825e5b26d828e18644c9ee58f86)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:19 +00:00
Mariano Lopez 9c0186fd57 rootfs.py: Change logic to unistall packages
In the current state some of the base utils (update-rc.d,
base-passwd, shadow, and update-alternatives) are unistalled
when there is no package manager in the image. Checking for
previous commits, the unistall of these utils were to be
done in a read-only filesystem.

It is a valid option to have these utils without a package
manager, and also make sense to remove them when building a
read-only filesystem.

This changes the check logic from having a package mananger
to if is a read-only filesystem to remove the utils.

Another change implemented with this patch is that delayed
post installs now doesn't depend if there is a package manager.
Also it is a valid option to have post install scripts without
package manger.

[YOCTO #8235]

(From OE-Core rev: 5aae19959a443c6ac4b0feef10715c8acf3c6376)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-12 23:42:51 +00:00
Haris Okanovic 639cadd6f9 sdk.py / OpkgSdk: remove_packaging_data() after install
Run remove_packaging_data() on both host and target sysroots to wipe
opkg state after install, similar to what RpmSdk does.

Use case:

Opkg may download local package indexes (I.e. file:// URLs) by
sym-linking them into /var/lib/opkg/lists/ instead of copying [1].
This leaves behind broken symlinks under the lists directory when
using opkg to build SDK sysroots.

The -h option may be set via SDKTAROPTS in some configurations to create
symlink-less SDK archives for Windows file systems. Sysroots containing
broken symlinks will fail to archive under this configuration.

Testing:

Verified /var/lib/opkg/ is empty after running populate_sdk() in a Fido
based distribution.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/opkg/commit/?h=opkg-0.3.x&id=f9022a8520fcde8f1b71424d26a652c218fce685

(From OE-Core rev: c8e0ec2da9ad4ce1c103966906a85f68c15400dd)

Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Reviewed-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Natinst-ReviewBoard-ID: 119065
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:25 +00:00
Leonardo Sandoval 6650357f13 lib/oe/package_manager: Introducing PACKAGE_FEED_BASE_PATHS/PACKAGE_FEED_ARCHS
The recently introduced PACKAGE_FEED_PREFIX is not flexible enough for
constructing URIs, because the same PREFIX is used for all PACKAGE_FEED_URIS.
Also, the string 'PREFIX' is confusing because it is not at the beginning of
the URI. The variable PACKAGE_FEED_BASE_PATHS replaces PACKAGE_FEED_PREFIX,
allowing multiple base paths to be appended on each PACKAGE_FEED_URIS. In the
other hand, a new variable called PACKAGE_FEED_ARCHS, similar in concept to
PACKAGE_BASE_PATHS, defines package architectures defined by the user.

To demonstrate the usage of the PACKAGE_FEED_URIS, PACKAGE_FEED_BASE_PATHS and
PACKAGE_FEED_ARCHS, let's assume these variables are set on local.conf

    PACKAGE_FEED_URIS = "https://example.com/packagerepos/release \
                         https://example.com/packagerepos/updates"
    PACKAGE_FEED_BASE_PATHS = "rpm rpm-dev"
    PACKAGE_FEED_ARCHS = "all core2-64"

the resulting feeds would be

    https://example.com/packagerepos/release/rpm/all
    https://example.com/packagerepos/release/rpm/core2-64
    https://example.com/packagerepos/release/rpm-dev/all
    https://example.com/packagerepos/release/rpm-dev/core2-64
    https://example.com/packagerepos/updates/rpm/all
    https://example.com/packagerepos/updates/rpm/core2-64
    https://example.com/packagerepos/updates/rpm-dev/all
    https://example.com/packagerepos/updates/rpm-dev/core2-64

(From OE-Core rev: 229723a20095e80bde29e4b3398047f62f972170)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:25 +00:00
Paul Eggleton 1fbd76093d lib/oe/recipeutils: refactor patch_recipe_file() to use edit_metadata()
Use bb.utils.edit_metadata() to replace some of the logic in this
function; this avoids us effectively having two implementations of the
same thing. In the process fix the following issues:

* Insert values before any leading comments for the next variable
  instead of after them
* Insert overridden variables (e.g. RDEPENDS_${PN}) in the correct place
* Properly handle replacing varflag settings (e.g. SRC_URI[md5sum])

(From OE-Core rev: 0f81b83fc5fd908efa7f6b837137830ca65f6ed6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:04 +00:00
Paul Eggleton 21481bc17a lib/oe/recipeutils: check in validate_pn() for names instead of filenames
Ensure that the user specifies just the name portion instead of a file
name with extension. (We can't just look for . since there are recipe
names such as "glib-2.0" that legitimately contain .).

(From OE-Core rev: a4c6af737811adb2bca87e3896f8710738dd255e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:04 +00:00
Paul Eggleton a360fa7d51 lib/oe/patch: improve extraction of patch header
For patches that we have to extract the header information by hand (i.e.
will not apply with "git am"), make the following improvements:

* If we can't extract author/date/subject, then try to do so from the
  commit that added the patch in git (assuming the metadata is tracked
  by git)
* Take only first Signed-off-by line instead of last
* Accept any case for "Signed-off-by" in case author has typed it by
  hand
* Improve conditional - we can skip the other cases if one matches

Implements [YOCTO #7624].

(From OE-Core rev: 13ec296b5c35aefa2c44f64f8bd1ef54c4a0a731)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:03 +00:00
Mike Crowe 804f5b83f3 image.py: avoid mkdir race when building multiple images
If multiple images are being built simultaneously against the same
sysroot then the call to os.makedirs in Image._write_wic_env can fail
with:

File: '.../meta/lib/oe/image.py', lineno: 341, function: _write_wic_env
     0337:        """
     0338:        stdir = self.d.getVar('STAGING_DIR_TARGET', True)
     0339:        outdir = os.path.join(stdir, 'imgdata')
     0340:        if not os.path.exists(outdir):
 *** 0341:            os.makedirs(outdir)
     0342:        basename = self.d.getVar('IMAGE_BASENAME', True)
     0343:        with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
     0344:            for var in self.d.getVar('WICVARS', True).split():
     0345:                value = self.d.getVar(var, True)
File: '/usr/lib/python2.7/os.py', lineno: 157, function: makedirs
     0153:            if e.errno != errno.EEXIST:
     0154:                raise
     0155:        if tail == curdir:           # xxx/newdir/. exists if xxx/newdir exists
     0156:            return
 *** 0157:    mkdir(name, mode)
     0158:
     0159:def removedirs(name):
     0160:    """removedirs(path)
     0161:
Exception: OSError: [Errno 17] File exists: '.../tmp-glibc/sysroots/cheetah/imgdata'

Using bb.utils.mkdirhier() protects against this.

There's also little point in checking to see if the directory already
exists - we might as well just try and create it regardless.

Once the directory has been created, there's no race on the actual file
since the filename contains IMAGE_BASENAME.

(From OE-Core rev: e3f216eef1b7a8ecfd00317e60704cff472238ba)

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:03 +00:00
Martin Jansa 312862f12a package_manager.py: define info_dir and status_file when OPKGLIBDIR isn't the default
* without this the do_rootfs task doesn't respect OPKGLIBDIR and
  info, status are created in different directory than opkg on
  target expects
* people who modify OPKGLIBDIR need to make sure that opkg.conf included
  in opkg package also sets info_dir and status_file options

(From OE-Core rev: 11f1956cf5d732cfed6d28c8c292b5053966895c)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:02 +00:00
Mike Crowe b00f73429a image.py: Avoid creating empty .env file in _write_wic_env
Creating a file for every image containing a few variables isn't
necessary if wic is not being used, so don't write the file if WICVARS
is empty.

(From OE-Core rev: d40779a7d3ce4f326c29ec7971731cab1f505a37)

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:02 +00:00
Dariusz Pelowski a88505b450 lib/oe/terminal: use C locale when determining version
(From OE-Core rev: 5e5bf4f7e242494ed9c2cbc13bec7f21494db17c)

Signed-off-by: Dariusz Pelowski <dariusz.pelowski@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:02 +00:00
Paul Eggleton 2948169291 classes/populate_sdk_ext: tweak reporting of workspace exclusion
If you have a local workspace layer enabled when building the
extensible SDK, we explicitly exclude that from the SDK (mostly because
the SDK has its own for the user to use). Adjust the message we print
notifying the user of this so it's clear that we're excluding it from
the SDK, and scale it back from a warning to a note printed with
bb.plain().

(From OE-Core rev: 90f46f74a088a7b965d2205eceb9eff6f276dd38)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:00 +00:00
Leonardo Sandoval e864f71f4c terminal: Open a new window instead of split on older tmux versions (<1.9)
If an old version is detected (<1.9), create a new window instead of split:
the reason is that there is no easy way to get the active pane height if no
nested formats are supported.

(From OE-Core rev: 457bd6297ae99627c5f596c3c09086d787d5a5ab)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-25 08:08:08 +00:00
Mariano Lopez 49a465cd23 package_manager.py: Delete installed_pkgs.txt file
This change removes the file installed_pkgs.txt after it
was used for installation of complementary packages. This
file was causing confusion when left in the WORKDIR after
the build.

(From OE-Core rev: d0f3f3a294d509560bd12b93b26eeec65cfee314)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:42 +00:00
Mariano Lopez ace895d162 rootfs.py: Stop using installed_pkgs.txt
The method _uninstall_unneeded uses the file installed_pkgs.txt,
this file is left after the build and can cause confusion. This
changes allow to get the installed packages using functions of
rootfs instead of the installed_pkgs.txt file. With this change
now is possible to remove the file without breaking anything.

[YOCTO #8444]

(From OE-Core rev: bf935ac16f6175673417dda92a619946b52fac87)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:42 +00:00
Ross Burton ccb1616e54 lib/oe/distro_check: don't set empty proxy keys
If the proxies dictionary has a proxy set to None urllib will throw an exception
instead of not using a proxy (abridged stack):

File: '/home/ross/Yocto/poky/meta/lib/oe/distro_check.py', lineno: 43, function: get_links_from_url
 *** 0043:    with create_socket(url,d) as sock:
     0044:        webpage = sock.read()
File: '/home/ross/Yocto/poky/meta/lib/oe/distro_check.py', lineno: 5, function: create_socket
     0003:def create_socket(url, d):
     0004:    import urllib
 *** 0005:    socket = urllib.urlopen(url, proxies=get_proxies(d))
File: '/usr/lib/python2.7/urllib.py', lineno: 87, function: urlopen
     0086:    if data is None:
 *** 0087:        return opener.open(url)
File: '/usr/lib/python2.7/urllib.py', lineno: 203, function: open
     0201:        else:
     0202:            proxy = None
 *** 0203:        name = 'open_' + urltype
     0204:        self.type = urltype
     0205:        name = name.replace('-', '_')
Exception: TypeError: cannot concatenate 'str' and 'NoneType' objects

Filter out unset values so that the dictionary only has valid assignments in.

(From OE-Core rev: 2d91290ab5608dd1297d1c26ab807fc4574a8a6b)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:41 +00:00
Richard Purdie 8137a846fb lib/oe/copy_buildsystem: Don't expand BB_TASKDEPDATA
The value isn't a string so don't try and expand it.

(From OE-Core rev: ab87d3649c39326938d82d623efafb76905f770d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:41 +00:00
Richard Purdie fecb07755c classes: Ensure pass setVar/setVarFlag strings, not integers
This doesn't cause any issues right now but it make sense to standardise
on consistently using strings in the data store.

(From OE-Core rev: 99203fbe5ad470ef65cff93cec9d7f332883b5ee)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:50:26 +00:00
Ross Burton d18d902fae lib/oe/rootfs: tell intercepts where the native sysroot is
Some intercepts may want to access files in the native sysroot that are not on
$PATH (such as something in $libexecdir) but any use of STAGING_DIR_NATIVE in
the postinst that calls the intercept will be "baked" into the package, so if
sstate is reused it will use paths that may not exist.

Solve this by exporting the location of the native sysroot in the environment so
the postinst and intercept can use an environment variable instead of a bitbake
variable.

(From OE-Core rev: 646599a1f3d39238ba8009993750a46ae220e762)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:30 +00:00
Ed Bartosh 8a555fe74a rootfs.py: add more info to the warning message
Since the log_check_regex can potentially be false positive it
makes sense to print the whole line where error is found.
This way user will be able to see the error and understand
if it's valid or not.

[YOCTO: #7789]

(From OE-Core rev: f9cf31525fc885e1a0f65bd55654631257f87078)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:25 +00:00
Markus Lehtonen 579e2547b8 package_manager: fail if signed feeds are enabled for ipk or dpkg
Signed package feeds are not yet implemented for these package formats.

(From OE-Core rev: 49a5c8700deddac744ccfa033bebf7971f92e14b)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:25 +00:00
Markus Lehtonen dbb9af6b0b package_manager: support GPG_PATH variable
If defined, use GPG_PATH as the gpg home directory when signing package
feeds. This setting is only used by package_manager if package feed
singning has been enabled, i.e.  PACKAGE_FEED_SIGN="1".

(From OE-Core rev: 2c94853e4c8e92ac5fe4af7b25e2ce8853a4025b)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:24 +00:00
Benjamin Esquivel 409e6e02ff populate SDK: prepare calling of bb.utils for exceptions
bb.utils.remove, bb.utils.movefile and bb.utils.mkdirhier can throw
exceptions that need handling and proper error messages
more work is required for these methods to handle properly the
exceptions that can be raised within the various OS calls they make
but this is a start to at least not hide the errors in the requested
operations

[YOCTO#8213]

(From OE-Core rev: 2e81dbdce6f92908c4d4c980af032516581178de)

Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-21 22:56:03 +01:00
Paul Eggleton db55d31dc2 devtool: handle virtual providers
For modify / extract / upgrade, if the specified "recipe" is not
actually a recipe but a virtual target such as virtual/kernel, map it
correctly to the actual recipe and make sure we use that name within the
workspace. Thanks to Chris Larson for reminding me this was still broken
and for a hint on how to fix it.

(From OE-Core rev: 1f7752282ffb47d2621030ddb2fa42a5e491d6d2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-21 22:56:03 +01:00
Otavio Salvador 556c0ea92e lib/oe/image.py: Fix dependency handling for compressed types
The dependency code needs to also include the dependency of base
types. For example:

 - sdcard.gz image with ext4

The dependency chain needs to include:

 - sdcard
 - ext4
 - gz

Until this change, the ext4 dependency were not being taken into
account when using the compressed one.

(From OE-Core rev: 10e5df3503632a6e1c54612055b19f7258c3ae2f)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-14 18:08:37 +03:00
Leonardo Sandoval f385ed1a06 lib/oe/distro_check: Remove '_proxy' on dict values used by urllib.open
There are two ways to pass the proxies values into urllib.open: exporting
the environment variables *_proxy and passing as dictionary into the
urllib.open call. The latter is currenty used and values should not have
the '_proxy' string, so removing it. Also, ignore uppercase proxies, these
are not taken into account by the library.

Tested on a network with proxy, adding 'inherit +="distrodata"' on local.conf
with the following commands

    for distrotask in distrodataall distro_checkall checklicenseall; do
        bitbake universe -c $distrotask
    done

[YOCTO #7567]

(From OE-Core rev: 92fb327b08e4f9cfb2400610c2983afe579b3592)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-12 14:36:30 +01:00
Mark Hatle a0d9d2d727 lib/oe/image.py: Add image generation for companion debug filesystem
The companion debug filesystem, enabled with IMAGE_GEN_DEBUGFS, was
creating the companion filesystem but was missing the code to actually
package it into a usable filesystem.

The code (and associated documentation) will allow the debugfs to generate a
companion tarball or other image.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-07 00:11:16 +01:00
Andre McCurdy 8ee9a933a5 package_manager.py: sort output of OpkgPkgsList().list
Without explicit sorting, the output generated by OpkgPkgsList().list
follows the order of packages in /var/lib/opkg/status, which appears
to be "random". Add sorting to make OpkgPkgsList().list behaviour
consistent with that of RpmPkgsList().list.

(From OE-Core rev: f06fb68a07b82e4b8f25d5cdf556cf8893ddf208)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-07 00:09:11 +01:00
Leonardo Sandoval 12cd705b28 distrodata: Take account proxies on distrodata tasks
Proxies defined in the enviroment where not taken into account
on the distrodata tasks. This commit implied passing the datastore
into the distro_check library and context manager for the
urllib.urlopen function.

One way to run distrodata tasks is using 'universe' as target and the
'all' distrodata task:

      $ bitbake universe -c distrodataall
      $ bitbake universe -c distro_checkall
      $ bitbake universe -c checklicenseall

Logs are located under TMPDIR/log

[YOCTO #7567]

(From OE-Core rev: 7d1c3470bb06e43245ccb7f067121de606506430)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01 15:07:48 +01:00
Richard Purdie 2c196955a8 lib/oe/sstate: Add tasks_resolved handler for virtual/xxx mappings
In SIGGEN_EXCLUDERECIPES_ABISAFE and SIGGEN_EXCLUDE_SAFE_RECIPE_DEP
we really need to be able to use virtual/xxx namespaces but this
currently doesn't work.

To make this work, we need to translate them into the resolved
providers. After such a hook was added to bitbake, we can add
this translation here.

(From OE-Core rev: 0a6d0d040ab7f885b667a34f4ddcc775d135c07c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01 07:43:35 +01:00
Richard Purdie c9ea0c619d lib/oe/package_manager: Handle empty package list in opkg case
If you build buildtools-tarball with opkg as the package manager, it
passes in an empty target packages list and fails.

This allows the code to cope with an empty package list (in sync
with the rpm backend).

(From OE-Core rev: b460afb12bb16a4b56d800c953c5f5c7da0bff84)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01 07:43:35 +01:00
Markus Lehtonen 31c3078a63 oe.patch.GitApplyTree: add paths argument to extractPatches
Makes it possible to define which paths are included in the patches.

(From OE-Core rev: 640e57b423e5a8f0e4572eac985f87139780f247)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01 07:43:31 +01:00
Markus Lehtonen d5e2dd47db recipeutils: implement get_recipe_local_files()
(From OE-Core rev: c94cf6e6e8a0645c822a708a432901dcb5ce5bf4)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01 07:43:31 +01:00
Alex Franco 2bbec5645e Empty image:rootfs.py:handle empty PACKAGE_INSTALL
rootfs.py: added a test in DpkgOpkgRootfs which skips the
get package post installs step (get_pkgs_postinsts), when
PACKAGE_INSTALL is empty.

[YOCTO #7664]

(From OE-Core rev: 2d5fb43ec647e8445fc8e8a5caba771c0361a539)

Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-28 12:00:28 +01:00
Alexander Kanavin 5cc44fe4fd recipeutils.py: don't hardcode the upstream version as 1.0 when SRC_URI is empty or absent
Instead, simply return the version of the recipe, so that the upstream
version checker will declare that they match.

(From OE-Core rev: 10c948a67546dd4ed2ac00b5210ef4b3eeca9401)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-28 12:00:24 +01:00