Commit Graph

4442 Commits

Author SHA1 Message Date
Brendan Le Foll 3c66619d50 bitbake: fetch2/npm: fix ud.registry so that alternative registries can be handled
Fixes [YOCTO #9231]

npm when given an invalid registry URL with --registry actually goes and
fetches from the default registry, but this commit makes sure it goes to the
specified one.

(Bitbake rev: 7c849be7c70a5db4f66fe3041486abb923b5e4ee)

Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-12 09:35:52 +00:00
Lucas Dutra Nunes d53413d3a8 bitbake: server/process: Try connecting 4 times before giving up
Instead of trying one time with a timeout of 20 seconds try 4 times with
a timeout of 5 seconds, to account for a slow server start.

(Bitbake rev: 4a7fe63126dd8177baa5ad21e59e0bebeea8c596)

Signed-off-by: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 22:47:29 +00:00
Michael Wood 0f0105932c bitbake: toaster: models List only have the specified project's imported layers
When returning the compatible layers make sure that we are only
listing: All the layers which are for this release && configuration
layers (i.e. aren't part of the build history) and which aren't an
imported layer OR are this project's imported layer(s).

[YOCTO #8944]

(Bitbake rev: de8baedaccb451c12bc3f642449db3f64aed6bf7)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 22:47:29 +00:00
Elliot Smith 0dcab0258e bitbake: toaster: rework task buildstats storage and display
The data available from buildstats is now more fine grained than
previously, so take advantage of that to enrich the data we save
against tasks:

* Store the CPU usage for user and system separately, and display
them separately.
* Disk IO is now measured in bytes, not ms. Also store the
read/write bytes separately.
* Store started and ended times, as well as elapsed_time. This
will enable future features such as showing which tasks were
running at a particular point in the build.

There was also a problem with how we were looking up the Task
object, which meant that the buildstats were being added to
new tasks which weren't correctly associated with the build. Fix
how we look up the Task (only looking for tasks which match the
build, and the task and recipe names in the build stats data) so
the build stats are associated with the correct task.

[YOCTO #8842]

(Bitbake rev: efa6f915566b979bdbad233ae195b413cef1b8da)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 22:45:16 +00:00
Sujith H cc74a8ae26 bitbake: toaster: use force_bytes to display non-ascii project names
When user enters a non-ascii character in the project
name of toaster, the build doesn't get triggered.
Use force_bytes to fix this.

Also deal with non-ascii project names when logging the
build request in runbuilds.

[YOCTO #9071]

(Bitbake rev: b6141c4d170885d3bdf63074afcb1e41fde0a8f0)

Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 22:45:16 +00:00
Peter Kjellerstedt aebc22dbfa bitbake: fetch2: Make SRC_URI[md5sum] and SRC_URI[sha256sum] expand their values
For some reason, the values for SRC_URI[md5sum] and SRC_URI[sha256sum]
were not being expanded. That lead to the following code not working
as expected:

SRC_URI = "http://.../${PN}-${PV}.tar.gz"

MD5SUM = "123abc..."
SHA256SUM = "abcd1234..."

SRC_URI[md5sum] = "${MD5SUM}"
SRC_URI[sha256sum] = "${SHA256SUM}"

(Bitbake rev: ba011470df0ea8bd89f01c0b02ec4b3969e60ce7)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 22:45:16 +00:00
Ed Bartosh d405f97af3 bitbake: xmlrpc: fix bug in setting XMLRPCServer.single_use
XMLRPCServer.single_use attribute was always set to False.
This caused xmlrpc server to keep running after build is done as
BitBakeServerCommands.removeClient only shuts down server if its
single_use attribute is set to True.

(Bitbake rev: 0a60b0928a0a746a60d2c2f294ff1903963c7086)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 22:45:16 +00:00
Paul Eggleton c50bdb300b bitbake: fetch2/npm: add missing URL argument to ParameterError
Without this you get a rather odd traceback instead of the proper
exception message.

(Bitbake rev: 2fe1826d3077eeda6cde433d3a1e6620f74e08dd)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 22:45:16 +00:00
Paul Eggleton fbf27c4c5d bitbake: fetch2/npm: properly handle npm dependencies
The output of "npm view dependencies" isn't entirely JSON if there are
multiple results, but the code here was just discarding the output if
the entire thing didn't parse as JSON. Split the output into lines and
iterate over it, parsing JSON fragments as we find them; this way we end
up with the last package's dependencies since it'll be last in the
output.

Digging further, it seems that the dependencies field reported by "npm
view" also includes optional dependencies. That wouldn't be a problem
except some of these optional dependencies may be OS-specific; for
example the "chokidar" module has "fsevents" in its optional
dependencies, but fsevents only works on MacOS X (and is only needed
there). If we erroneously pull in fsevents, not only is it unnecessary
but it causes "npm shrinkwrap" to throw a tantrum. In the absence of a
better approach, look at the os field and discard the module (along with
any of its dependencies) if it isn't for Linux.

As part of this, we can reduce the calls to npm view to one per package
since we get the entire json output rather than querying twice for two
separate fields. Overall the time taken has probably increased since we
are being more thorough about dependencies, but it's not quite as bad as
it could have been.

(Bitbake rev: 436d67fe7af89ecfbd11749a6ae1bc20e81f2cc8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 22:45:16 +00:00
Paul Eggleton ef6a4518e4 bitbake: fetch2/npm: fix errors with some version specifications
"2 || 3" is a valid version specification for a dependency in an npm
package.json file, but of course that looks like something else when
sent to a shell. Quote the version value to avoid this.

(Bitbake rev: bea0246831a46d943d2e27d6b38f6e498bd3413c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 22:45:16 +00:00
Ed Bartosh 3322fa7670 bitbake: toasterui: fix warning 'Unknown event'
Continue after processing BuildStarted event to fix
WARNING: Unknown event: <bb.event.BuildStarted object at 0x2554150>

(Bitbake rev: 12f1fb8c9b70fea0c9145f881bcceb8af32df6af)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:04 +00:00
Ed Bartosh 621cbc86c9 bitbake: toasterui: exit on final events
Toasterui exits only if bitbake observer shuts down.
In build mode it should exit when build is done.

Made toasterui exit on bb.command.CommandCompleted,
bb.command.CommandFailed and bb.command.CommandExit events
when it's running in build mode.

(Bitbake rev: b11f9d6d3c2eb615335901e1dcea699daf3afb4c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:04 +00:00
Ed Bartosh 8e138b7f2a bitbake: toasterui: make toasterui to work in build mode
Currently toasterui works only in observer mode. This is
artificial limitation which was made to support current toaster
design. As we decided to stop using bitbake server we'll
need to run toasterui also in build mode.

[YOCTO #7880]

(Bitbake rev: d4b5796899c3ca5c7becd7322291afd8afb35a31)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:04 +00:00
Ed Bartosh 0a6130697e bitbake: toasterui: check if setEventMask succeeded
Currently toasterui ignores return value of setEventMask
command, which created confusing difference between set of
events set by this command and the real set used in the code.

Checked if setEventMask succeeded. Print error message and
exit if it's not.

(Bitbake rev: 6e3f13ffb47102b5df2da91fbc3f5da3179245b2)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:04 +00:00
Ed Bartosh ac941ac0a2 bitbake: command: make setEventMask readonly
Executing setEventMask command when bitbake server is in readonly
mode causes runCommand to fail with the following error:
 'Not able to execute not readonly commands in readonly mode'

Set readonly attribute for setEventMask command to make it working
for Toaster UI. This should not do any harm as this command doesn't
influence cooker state.

(Bitbake rev: 8a47d30b2555255fbf6049c5ed69b29664c32b17)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:04 +00:00
Ed Bartosh dd3da9aa47 bitbake: toasterui: update list of events
Removed events not used in the code from the list.
Added events that are used in the code.

(Bitbake rev: 16b14ec16049cc2040a60ad5fc95f6e19dda91a6)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:04 +00:00
Ed Bartosh f56fa5dfd4 bitbake: toasterui: reformat list of events
Reformatted and reordered list of events to make changes
easily and see them clearly in the diffs.

(Bitbake rev: 42a2d1115f2b23dc063a3172285ca3be73cf70bb)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:04 +00:00
Ed Bartosh a71d32ac53 bitbake: toaster: remove sshbecontroller module
The code of this module is broken for a long time.
The functionality of it can be easily achieved by running
'manage.py runbuilds.py' on remote machine.

[YOCTO #8806]

(Bitbake rev: 975081eefdd7041a6b4bef6842c1bac9799a1b44)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:03 +00:00
Ed Bartosh 3db71b4087 bitbake: toaster: don't use sshbecontroller
Removed usage of sshbecontroller from bbcontroller, models, tests
and database schema.

(Bitbake rev: 3ee06eb7e96de5dba539ad52201867e77d06a53e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:03 +00:00
Ed Bartosh 790b2d1387 bitbake: toaster: raise NotImplementedError
Raised NotImplementedError instead of Exception to be
able to catch it.

This is a preparation for removing sshbecontroller module.
It has to be done as code in bldcontrol/tests.py imports custom
NotImplementedException from sshbecontroller.

(Bitbake rev: c243ab6c83fe12d84777e4c3a18fd393827b9327)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:03 +00:00
Belen Barros Pena 96535ba720 bitbake: toaster: bring back the strict directive
This patch e0fd96442a removed the
"use strict" directive from the projectpage.js file by mistake.

This patch makes amends.

(Bitbake rev: b8044ce60af3f0b064cfba76c577503cc896e358)

Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:03 +00:00
Belen Barros Pena 5b8b3993e4 bitbake: toaster: change 'revision' to 'Git revision'
I've received some feedback on the 'Revision' label we use in the import
layer page. It is not quite communicating that what's required is a Git
revision. Changing it to 'Git revision' to make it a bit more specific.

The change applies not only to the import layer page, but to all pages
showing revision information in the project configuration section.

For more on the feedback received, check

https://bugzilla.yoctoproject.org/show_bug.cgi?id=8429#c3

(Bitbake rev: 09392f36a4f115c2432302125e8cac48a9aa304f)

Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:03 +00:00
Michael Wood 07ead9869a bitbake: toaster: views api Package info return both kinds of RDEPENDS
As we do with the popover snippet which shows dependencies inline in the
table also show dependencies which have both TYPE_TRDEPENDS and
TYPE_RDEPENDS.

Also remove obsolete comment

(Bitbake rev: d3b5f3b7ba4550e7cd03a37ca19ccd2fc0042b2d)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:03 +00:00
Dave Lerner 9cda2ab8d5 bitbake: toaster: fixup dependency excludes for customimage
[YOCTO #9156]

For a customized image when adding a dependent package X that depends on
dependency package Y, in addition to adding X to appends_set and Y to
includes_set, make sure that Y is no longer in the excludes_set. Y may
have been added to the excludes_set by a prior package removal.

(Bitbake rev: 6b29d3297de2ae48a3ac5529ba0d22f895276b56)

Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:03 +00:00
Paul Eggleton a54cebe929 bitbake: fetch2/npm: ignore unknown headers in tarballs
Tarballs that are fetched down via npm repositories seem to often have
unknown headers. This doesn't affect our ability to extract the contents
though so we don't really care to see those warnings.

(Bitbake rev: b38975103e52a0c25e9ad9032c8cca1c47cbdcc2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:03 +00:00
Paul Eggleton 0cd1be1f09 bitbake: fetch2/npm: handle alternative dependency syntax
npm allows you to specify a dependency as a Github username+path, or
omit the version entirely. You can hit these if you don't use a
shrinkwrap file, with the result that the code later fails due to the
output of "npm view" being empty; so handle this lazily by just ignoring
this part of the dependency if it's not really a version.

(Bitbake rev: 7b7a65c44dbdd5ba9366d4e2093f76df8758d546)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:03 +00:00
Paul Eggleton d9999279d9 bitbake: fetch2/npm: fix indentation
No code changes, just fix to use four spaces.

(Bitbake rev: 66a9ee7d54ca9c25209f72da079f260ccdcc872a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:02 +00:00
Scott Rifenbark 353b755177 bitbake: bitbake-user-manual: Added expand() function to list.
Fixes [YOCTO #9147]

Added the d.expand(expression) fuction to the list of functions
in the "Accessing Datastore Variables Using Python" section.

(Bitbake rev: 19507b80f35d37dc4b1614bd390b8e261dd4a2bd)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-03 17:34:14 +00:00
Scott Rifenbark 638ad177aa bitbake: bitbake-user-manual: Added note for Python variable ref expansion.
Fixes [YOCTO #9148]

Added a note about Variable expressions (e.g. ${X}) are no longer
expanded within Python functions.

(Bitbake rev: 4ebe55cb8c8db1c5cf5d127e213487c5a453a68a)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-03 17:34:14 +00:00
Scott Rifenbark da22add33b bitbake: bitbake-user-manual: Enhance environment variable discussion.
Fixes [YOCTO #8567]

Updated the variable list describing the variables that affect
how environment variables are handled.  Also updated the section
on how those variables are passed.

(Bitbake rev: 91cb52b1e77bba9d046239933b5c0513d01e6824)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-03 17:34:14 +00:00
brian avery 937ecd07d3 bitbake: toaster: cleanup of bin/toaster startup code
This cleanup fixes a few issues:

    1) Removes the superflous code to set toasterconf.json
       - this isn't used and referenced meta-yocto
    2) Changes exit to return so we don't surprise the user by exiting their shell
       - this is necessary because it is being sourced
    3) Removes the last references to the old TOASTER_MANAGED variable
       - this is historical and no longer used.
    4) Adds -t parameter to lsof
      - This stops it from dying on odd filesystems and is much
        faster since all we are using are processes anyway
    5) Handles start and stop as params
      - it was easy to confuse the script especially
        if we were calling it with parameters.
      - if start/stop isn't specified, it will still toggle

(Bitbake rev: 88fddbe80f56828026bf93560037af52b5dab628)

Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 22:41:23 +00:00
brian avery a7d1b95c96 bitbake: ui: remove the puccho ui
This ui does not work in master, nor has it been updated for several years.

[YOCTO #9178]

(Bitbake rev: 9fad1d13eed1f725971e6d12d3977cd31e07019a)

Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 22:41:23 +00:00
bavery a9dc72f08b bitbake: hob: removal of hob ui and associated ui files
We've been gearing up the Toaster web UI to replace the Hob (GTK+ based) UI
for some time now; Hob has basically been on life support for the past few
releases. As of late last month in master, Toaster has the capability to
select the packages in an image, removing the last thing that Hob could do
that Toaster couldn't.

To recap, the reasons why Hob is being removed include:

- The code is tightly woven into BitBake, making it fragile. This means it
needs significant QA and maintenance on an ongoing basis.

- Some of the implementation is not ideal; we'll be able to remove some cruft
from BitBake and OE-Core at the same time.

- It's GTK+ 2 based, not the current GTK+ 3.

- Toaster is now a much more capable UI and is being actively maintained

The discussion about removing hob can be found at:
http://lists.openembedded.org/pipermail/openembedded-architecture/2016-February/000082.html

(Bitbake rev: be2cceea159c6ca9111eff3df87b98513eab6d72)

Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 22:41:23 +00:00
Brendan Le Foll 27468dbc3a bitbake: fetch2/npm: Add missing ParameterError import
(Bitbake rev: b583a40c9086b3587065995d8cee45b7ea36c1b3)

Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 22:41:22 +00:00
Brendan Le Foll 44e3461969 bitbake: npm: in cases where shrinkwrap resolved a git URL, ignore it and grab dist.tarball
npm-shrinkwrap will sometimes resolve a git URL which instead of a http url, in
this case go and grab the dist.tarball via npm instead of using the resolved
URL.

(Bitbake rev: eb53b927ff59aa19cf28bc46beb9f9a185a59990)

Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 22:41:22 +00:00
Richard Purdie 2a7318133b bitbake: fetch2: Fix unpack for absolute file urls
The previous commit breaks absolute pathnames in file:// urls, this
fixes it.

(Bitbake rev: b8113a1800687a37a26ac28deafdbafd74cc138e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 22:41:22 +00:00
Alexander Shashkevich 865d2feff6 bitbake: fetch2: fixes copying of file://dir; subdir=foo, bug 6128 and bug 6129
When in SRC_URI appears file://dir;subdir=foo unpacker copies 'dir' to ${WORKDIR}, not
${WORKDIR}/foo as it should be.

These changes are fixing following bugs as well:
Bug 6128 - Incorrect wildcard unpack behaviour in fetcher
Bug 6129 - Local directories unpack to a different location than local files

(Bitbake rev: e659a3b0c2771679057ee3e13cd42e6c62383ff2)

Signed-off-by: Alexander Shashkevich <alex@stunpix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 22:41:22 +00:00
Richard Purdie 2e27c4bf23 bitbake: fetch2/npm: Enable fetcher
For some reason the enablement piece of the patch went missing, add it.

(Bitbake rev: 0270b5a3873ed0aeca3a66198c87a6164fb644b8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 12:01:07 +00:00
Richard Purdie 030d920e2d bitbake: providers: Fix PREFERRED_VERSION lookup for '_' in PN
PN can contain '_', e.g. gcc-cross-x86_64 and an override cannot
hence we do this manually rather than use OVERRIDES.

(Bitbake rev: 7a6baf02617d1edced4eaff235e73d746e2a3b68)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:34:38 +00:00
Richard Purdie c679a3dd41 bitbake: fetch2: Skip lockfiles and donestamps for local files
For local files, there are no races with downloads, we don't need ".done"
stamps and we don't need lockfiles.

This considerably cleans up DL_DIR and all the pointless ".done" files
as well as removes stalls over local files with the same name.

(Bitbake rev: 48e903745db578d9b9b425a8d411c1369df0eb94)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:34:38 +00:00
Richard Purdie d01042eda9 bitbake: fetch2/__init__.py: Error if lockfile path invalid
Rather than create ".lock" and ".done" files with no name, error,
forcing us to fix the cases where this is a problem.

(Bitbake rev: 81158071508cc68c39db7d501370872f44d335cc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:34:38 +00:00
Richard Purdie ab7b7bf94d bitbake: fetch2/__init__: Fix decodeurl to better handle urls without paths
If we specify urls such as npm://somehost;someparams the fetcher currently
does a poor job of handling mirrors of these urls due to deficiencies in the
way decodeurl works. This is because "somehost" is returned as a path, not
a host.

This tweaks the code so that unless its a file url, the host is returned
correctly.

This patch also adds test cases for these urls to the exist set of test
urls.

We need to tweak the URI() class since this thinks this is a relative url
which is clearly isn't. We also need to handle the fact that encodeurl will
error if passed a url of this form (it would want the path to be '/'.

(Bitbake rev: 83203cd2e677706e0111892a7843b83263cb8bd9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:34:38 +00:00
Richard Purdie 06b4d8ffe4 bitbake: fetch2/wget: Set localfile for directories
If we wget something which looks directory like we end up with lock files
and done stamps without names, they also all use the same lockfile.

This change ensures that we use separate lock files based on the url
and avoid creating the mysterious ${DL_DIR}/.done files.

(Bitbake rev: 20bc82086018832e047345a672d74b6c1c113650)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:34:38 +00:00
Belen Barros Pena e0fd96442a bitbake: toaster: change 'delete layer' to 'remove layer'
I have received quite a few complaints about the use of the word
'delete' for layer removal, so change it to 'remove'. That also matches
the language we use for packages in image customisation.

[YOCTO #9165]

(Bitbake rev: 3c5ac2ddfb3f5ecd3f3218de0d6564e5f3842b98)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-26 17:20:38 +00:00
Belen Barros Pena 6e8282063f bitbake: toaster: rename 'run again' button
I quite dislike the 'Run again' label we use in the button that rebuilds
things. Changing it to 'Rebuild', which is shorter and more specific.

(Bitbake rev: 865a2015e86a1bc5cc7d63308f27c292d1ca98eb)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-26 17:20:36 +00:00
Dave Lerner c8dd72c797 bitbake: toaster: fix banner after customimage package add
Correct formatting of the banner message after adding a package to a
custom image.

[YOCTO #9101]

(Bitbake rev: da233005eb8cfa7842cd1a768c16e42aaaa55fad)

Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-26 17:20:35 +00:00
Belen Barros Pena 149f57470c bitbake: toaster: custom breadcrumb for the default project
The default "Command line builds" project does not have a Configuration
page. It therefore needs a custom breadcrumb where the project name goes
to the project builds page, instead of the project configuration page.

(Bitbake rev: 5545acf6703a25ee46776138bbbd804615add89c)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-26 17:20:33 +00:00
Haris Okanovic 4a12865c7b bitbake: prserv: Add dump_db()
Returns a script (string) that reconstructs the state of the
entire database at the time this function is called. The script
language is defined by the backing database engine, which is a
function of server configuration.
Returns None if the database engine does not support dumping to
script or if some other error is encountered in processing.

The SQLite3 implementation in db.py calls iterdump() [1] to generate
a script. iterdump() is the library equivalent of the `sqlite3 .dump`
shell command, and the scripts are compatible. Execute the script in
an empty SQLite3 database using the sqlite3 utility to restore a backup
of prserv.

Use case: Backup a live PR server database in a non-racy way, such
that one could snapshot the entire database after a set of bitbake
builds all using a shared server. I.e. All changes made prior to
the start of a dump_db() operation should be committed and captured
in the script. Subsequent changes made during the backup process are
not guaranteed to be captured.

Testing: ~7MB database backs up in ~1s while PR server is under load
from 32 thread bitbake builds on two separate machines.

[1] https://docs.python.org/2/library/sqlite3.html#sqlite3.Connection.iterdump

(Bitbake rev: 004003daf6bd0f0233ce5c2d95f1d7d64ab91bb3)

Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Reviewed-by: Ken Sharp <ken.sharp@ni.com>
Reviewed-by: Bill Pittman <bill.pittman@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-26 17:20:31 +00:00
Dave Lerner bdb51abb3c bitbake: toaster: remove custom images from Image Recipes
Fix the view of 'Image Recipes' under 'Configuration' to only show
image recipes that are not customised since custom images have their
own page.

[YOCTO #9111]

(Bitbake rev: 18a93b360301a5497d5c8ef74ab71f374f2ad210)

Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-26 17:20:29 +00:00
Elliot Smith 98d462c8e6 bitbake: toaster: show suffix for image files and basename for artifact files
The build dashboard doesn't show image and artifact files correctly,
as it shows the full filename for images and the filename plus
path relative to DEPLOY_DIR for artifacts.

Instead, show just the suffix for image files, and the basename
for artifact files.

(Bitbake rev: 8084dcdc283b4dc170f066c202f89d56ce1abbef)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-26 17:20:28 +00:00