Commit Graph

526 Commits

Author SHA1 Message Date
Kevin Tian eaa188f3b9 siggen.py: fix comparison around runtaskdeps
two dep lists should be sorted before the comparison, or else it just
causes false-positive confusion.

Also fix 'taskdeps' to 'runtaskdeps' when printing out the difference.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-11-15 23:04:06 +00:00
Joshua Lock 9545130576 bitbake-runtask-strace: simple shell script to wrap tasks in a strace call
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-11-14 21:08:27 -08:00
Richard Purdie 05ba6fc7cb bitbake: Rewrite profiling code so its functional for both none and xmlrpc backends
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 21:23:54 +08:00
Richard Purdie 89929e1f28 bitbake/fetch: Allow checking of a single url at a time (massive sstate speed improvement)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 20:10:53 +08:00
Richard Purdie caa7fe2284 bitbake/fetch: Fix handling of mirrors when checking for url validity
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 20:10:04 +08:00
Richard Purdie 7333328c48 Revert "bitbake/fetch/local: Also check DL_DIR for files since they could already exists there"
These changes were incorrect.

This reverts commit ae98f7eacb.
2010-11-13 20:03:13 +08:00
Richard Purdie aa81dda4d4 Revert "bitbake/fetch/local: Fix os.exists reference"
These changes were incorrect.

This reverts commit f8e3397935.
2010-11-13 20:02:38 +08:00
Richard Purdie f8e3397935 bitbake/fetch/local: Fix os.exists reference
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 10:23:15 +08:00
Richard Purdie ae98f7eacb bitbake/fetch/local: Also check DL_DIR for files since they could already exists there
[BUGID #533]

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 07:28:55 +08:00
Richard Purdie 5ae5a145dd bitbake.runqueue.py: Fix no exec task handling to correct counter housekeeping
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-07 20:33:12 +00:00
Richard Purdie b80a8d28f8 bitbake: Add support for 'noexec' tasks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-06 14:33:05 +00:00
Richard Purdie a59935fb3a bitbake/siggen: Allow siggen code to control final stampfile format
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-06 12:20:33 +00:00
Richard Purdie 88f6f3fadf bitbake/siggen: Provide better warnings about empty tasks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-02 13:17:06 +00:00
Richard Purdie dc13a6197f bitbake/siggen: Fix finalise call arguments
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-01 10:23:09 +00:00
Richard Purdie c5fab99a6f bitbake/fetch: When fetchers return errors, ensure any partial download is cleared
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-23 00:22:54 +01:00
Richard Purdie 535a77a9b6 bitbake/fetch: Make URL checking slightly less verbose (distracting with the sstate code)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-22 14:48:54 +01:00
Richard Purdie 36f1ae42fe pseudo/fakeroot: Move the pseudo directory creation into bitbake
If sstate was used to accelerate a build, the pseudo directory might not have
been created leading to subsequent task failures.

Also, sstate packages were not being installed under pseudo context meaning
file permissions could have been lost.

Fix these problems by creating a FAKEROOTDIRS variable which bitbake ensures
exists before running tasks and running the appropriate setscene tasks under
fakeroot context.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-22 13:00:40 +01:00
Richard Purdie 08cec50330 bitbake/fetch/git: Ensure fullclone repositories are fully fetched
The git fetcher was failing to pull in new branches into a git
repository mirror tarball as the git fetch command being used didn't
add new remote branches.

This patch uses "git fetch --all" for fullclones to ensure any
new remote branches are cloned correctly.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-19 17:17:41 +01:00
Richard Purdie 05c8ee60f1 bitbake/fetcher: Deal with a ton of different bugs
The more we try and patch up the fetcher code, the more things break. The
code blocks in question are practically unreadable and are full of corner
cases where fetching could fail. In summary the issues noticed included:

a) Always fetching strange broken urls from the premirror for "noclone"
   git repositories
b) Not creating or rewriting .md5 stamp files inconsistently
c) Always fetching git source mirror tarballs from the premirror even
   if they already exist but the checkout directory does now
d) Passing "None" values to os.access() and os.path.extsts() checks under
   certain circumstances
e) Not using fetched git mirror tarballs if the preexist and always
   try and fetch them.

This patch rewrites the sections of code in question to be simpler and
more readable, fixing the above problems and most likely other odd
corner cases.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-19 12:47:45 +01:00
Richard Purdie 1077021f70 bitbake/fetch/git.py: Fix git fetcher to correctly use mirror tarballs
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-19 00:19:34 +01:00
Richard Purdie 229b6630af bitbake-runtask: Ensure logging settings take effect in the worker process (and pass verbose setting)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-14 20:07:15 +01:00
Joshua Lock 812f8835d2 bitbake/runqueue: make the runtask program configurable
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-10-12 13:32:27 +01:00
Richard Purdie d14f9bf6c4 bitbake/build.py: Write logfiles on a per task basis, not per function
Per function logging made it hard to track down what was happening so
switch to logging on a per task basis.

[BUGID #383]

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-11 12:53:54 +01:00
Richard Purdie ff539b11ac bitbake-runtask: Ensure logging options are inherited from the parent
[BUGID #291]

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-11 12:52:57 +01:00
Richard Purdie 115514b5eb bitbake/fetch: Ensure SRCREV is still set correctly if the pn-X override has been expanded
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-09 17:24:16 +01:00
Richard Purdie bee7c21f6e bitbake/runqueue.py: Fix invalid variable reference fixing the -f option with setscene tasks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-07 15:45:02 +01:00
Richard Purdie 24ad63979f bitbake/fetch/__init__.py: Abstract mirror variable handling and enhance to accept \n delimitation in variables
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-07 15:42:15 +01:00
Darren Hart 0a20caf090 Export ALL_PROXY for bitbake commands
We already export http,ftp,https proxy environment variables. Some environments,
GNOME for instance, place the socks proxy in ALL_PROXY and all_proxy. Export it
as well.

Signed-off-by: Darren Hart <dvhart@linux.intel.com>
2010-10-07 10:18:25 +01:00
Richard Purdie 35d095c282 bitbake/sstate: Implement a lookup function to speed up setscene processing
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-05 22:26:33 +01:00
Joshua Lock 0ee7a9e6bd bitbake/fetch: if mirror fetching fails, ensure exception is raised
We catch any exception raised by the fetchers go() method and attempt to work
around it by trying any (post) mirrors which are configured. However, should
the mirrors fail the exception is lost and the fetch is assumed to have
completed successfully.

Instead, save the exception and if the local file does not exist after trying
the mirrors re-raise the exception.

Fixes [BUGID #362]

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-10-05 16:54:10 +01:00
Mark Hatle 8e429accb8 build.py: Add a symlink for the logging
[BUGID #375]

When configuring for the log file output, we generate a symlink to the
target filename.  This link uses the same file naming, but without the
active pid, making it easier to see which log file is the last one
generated.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
2010-10-01 18:45:32 +01:00
Richard Purdie 1d52fa5269 bitbake/bitbake-sigdiff: Extend to handle dumping single state siginfo files
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-30 18:02:35 +01:00
Richard Purdie 1c91f32f5e bitbake/siggen: Fix issue where excluded dependencies caused exceptions
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-30 18:02:35 +01:00
Richard Purdie 624924bdcd bitbake/bitbake-runtask: Signatures are needed in task context and have little overhead so enable in worker context
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-29 12:12:26 +01:00
Richard Purdie 9747616600 bitbake/siggen: Ensure full signature data is not held unless needed, reducing memory consumption
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-29 10:47:17 +01:00
Richard Purdie ec4d6b989a bitbake: Pass task hash information to subprocesses
Pass task has informaiton to work processes, allowing full manipulation of
the hash data in the task context allowing checksums to be usable.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-28 15:34:27 +01:00
Joshua Lock 15ceaaaaf7 bitbake/fetch: fix logic to prevent fetches when the file already exists
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-28 14:07:08 +01:00
Richard Purdie d85dc37b73 codeparser.py: Fix storing of hash values as object references can be corrupted
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-28 12:05:15 +01:00
Richard Purdie fe0f86e15c bitbake/runqueue: Remove now unneeded indentation
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 21:00:10 +01:00
Richard Purdie 474c2a2817 bitbake/runqueue.py: Optimise delay values to avoid unneeded delays
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 16:57:13 +01:00
Richard Purdie 19d9435a32 bitbake/siggen: Allow generation of siggen data from task context
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 15:40:31 +01:00
Richard Purdie 89c7260931 bitbake/build.py: Set BB_FILENAME to represent the .bb file being built (including any virtual prefix)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 15:39:07 +01:00
Richard Purdie d94f0f74c1 bitbake/siggen.py: Improve debugging of checksums
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 12:17:33 +01:00
Richard Purdie 4aec2155a5 bitbake/data.py: Allow variables to suplement dependencies using the vardep flag
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 12:16:46 +01:00
Joshua Lock c01e8760ba bitbake/fetch: ensure the go() method completes when not using premirrors
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-10 18:36:35 +01:00
Zhai Edwin 566de26cf5 bitbake: Add proxy variables to standard export list
With "no_proxy" exported to bitbake, both internal and externel file mirror can
be used. "https_proxy" enable fetching "https://" file through proxy.

Signed-off-by: Zhai Edwin <edwin.zhai@intel.com>
2010-09-10 11:47:38 +01:00
Richard Purdie f7b1c1056e bitbake/__init__.py: Fix named SRCREVS variable name ordering with backwards compatibility
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-09 15:09:50 +01:00
Joshua Lock 3e7db72216 bitbake: save the initial PATH for use when starting a subprocess
It was possible for bitbake-runtime to be run against a semi-installed
python-native resulting in tracebacks with ImportError's.

To prevent this we stash the initial PATH in the BBConfiguration when bitbake
is started and then set this in the env when launching bitbake-runtask through
subprocesses Popen() call.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-08 19:22:37 +01:00
Joshua Lock 04b8511614 bitbake/fetch/git: fix try_premirror() definition .Method takes 4 parameters.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-08 15:14:07 +01:00
Joshua Lock 3341e47b52 bitbake/fetch: fix by moving try_premirror() to the right place
The previous commit didn't define the method as part of the Fetch object

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-08 15:09:07 +01:00