Commit Graph

506 Commits

Author SHA1 Message Date
Richard Purdie 229b6630af bitbake-runtask: Ensure logging settings take effect in the worker process (and pass verbose setting)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-14 20:07:15 +01:00
Joshua Lock 812f8835d2 bitbake/runqueue: make the runtask program configurable
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-10-12 13:32:27 +01:00
Richard Purdie d14f9bf6c4 bitbake/build.py: Write logfiles on a per task basis, not per function
Per function logging made it hard to track down what was happening so
switch to logging on a per task basis.

[BUGID #383]

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-11 12:53:54 +01:00
Richard Purdie ff539b11ac bitbake-runtask: Ensure logging options are inherited from the parent
[BUGID #291]

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-11 12:52:57 +01:00
Richard Purdie 115514b5eb bitbake/fetch: Ensure SRCREV is still set correctly if the pn-X override has been expanded
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-09 17:24:16 +01:00
Richard Purdie bee7c21f6e bitbake/runqueue.py: Fix invalid variable reference fixing the -f option with setscene tasks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-07 15:45:02 +01:00
Richard Purdie 24ad63979f bitbake/fetch/__init__.py: Abstract mirror variable handling and enhance to accept \n delimitation in variables
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-07 15:42:15 +01:00
Darren Hart 0a20caf090 Export ALL_PROXY for bitbake commands
We already export http,ftp,https proxy environment variables. Some environments,
GNOME for instance, place the socks proxy in ALL_PROXY and all_proxy. Export it
as well.

Signed-off-by: Darren Hart <dvhart@linux.intel.com>
2010-10-07 10:18:25 +01:00
Richard Purdie 35d095c282 bitbake/sstate: Implement a lookup function to speed up setscene processing
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-05 22:26:33 +01:00
Joshua Lock 0ee7a9e6bd bitbake/fetch: if mirror fetching fails, ensure exception is raised
We catch any exception raised by the fetchers go() method and attempt to work
around it by trying any (post) mirrors which are configured. However, should
the mirrors fail the exception is lost and the fetch is assumed to have
completed successfully.

Instead, save the exception and if the local file does not exist after trying
the mirrors re-raise the exception.

Fixes [BUGID #362]

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-10-05 16:54:10 +01:00
Mark Hatle 8e429accb8 build.py: Add a symlink for the logging
[BUGID #375]

When configuring for the log file output, we generate a symlink to the
target filename.  This link uses the same file naming, but without the
active pid, making it easier to see which log file is the last one
generated.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
2010-10-01 18:45:32 +01:00
Richard Purdie 1d52fa5269 bitbake/bitbake-sigdiff: Extend to handle dumping single state siginfo files
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-30 18:02:35 +01:00
Richard Purdie 1c91f32f5e bitbake/siggen: Fix issue where excluded dependencies caused exceptions
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-30 18:02:35 +01:00
Richard Purdie 624924bdcd bitbake/bitbake-runtask: Signatures are needed in task context and have little overhead so enable in worker context
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-29 12:12:26 +01:00
Richard Purdie 9747616600 bitbake/siggen: Ensure full signature data is not held unless needed, reducing memory consumption
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-29 10:47:17 +01:00
Richard Purdie ec4d6b989a bitbake: Pass task hash information to subprocesses
Pass task has informaiton to work processes, allowing full manipulation of
the hash data in the task context allowing checksums to be usable.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-28 15:34:27 +01:00
Joshua Lock 15ceaaaaf7 bitbake/fetch: fix logic to prevent fetches when the file already exists
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-28 14:07:08 +01:00
Richard Purdie d85dc37b73 codeparser.py: Fix storing of hash values as object references can be corrupted
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-28 12:05:15 +01:00
Richard Purdie fe0f86e15c bitbake/runqueue: Remove now unneeded indentation
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 21:00:10 +01:00
Richard Purdie 474c2a2817 bitbake/runqueue.py: Optimise delay values to avoid unneeded delays
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 16:57:13 +01:00
Richard Purdie 19d9435a32 bitbake/siggen: Allow generation of siggen data from task context
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 15:40:31 +01:00
Richard Purdie 89c7260931 bitbake/build.py: Set BB_FILENAME to represent the .bb file being built (including any virtual prefix)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 15:39:07 +01:00
Richard Purdie d94f0f74c1 bitbake/siggen.py: Improve debugging of checksums
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 12:17:33 +01:00
Richard Purdie 4aec2155a5 bitbake/data.py: Allow variables to suplement dependencies using the vardep flag
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 12:16:46 +01:00
Joshua Lock c01e8760ba bitbake/fetch: ensure the go() method completes when not using premirrors
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-10 18:36:35 +01:00
Zhai Edwin 566de26cf5 bitbake: Add proxy variables to standard export list
With "no_proxy" exported to bitbake, both internal and externel file mirror can
be used. "https_proxy" enable fetching "https://" file through proxy.

Signed-off-by: Zhai Edwin <edwin.zhai@intel.com>
2010-09-10 11:47:38 +01:00
Richard Purdie f7b1c1056e bitbake/__init__.py: Fix named SRCREVS variable name ordering with backwards compatibility
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-09 15:09:50 +01:00
Joshua Lock 3e7db72216 bitbake: save the initial PATH for use when starting a subprocess
It was possible for bitbake-runtime to be run against a semi-installed
python-native resulting in tracebacks with ImportError's.

To prevent this we stash the initial PATH in the BBConfiguration when bitbake
is started and then set this in the env when launching bitbake-runtask through
subprocesses Popen() call.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-08 19:22:37 +01:00
Joshua Lock 04b8511614 bitbake/fetch/git: fix try_premirror() definition .Method takes 4 parameters.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-08 15:14:07 +01:00
Joshua Lock 3341e47b52 bitbake/fetch: fix by moving try_premirror() to the right place
The previous commit didn't define the method as part of the Fetch object

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-08 15:09:07 +01:00
Joshua Lock 7117a4458b bitbake/fetch: add try_premirror method and special case git fetcher
Add a new method, try_premirror, to the Fetch object which checks to see
whether the file needs to be fetched from a premirror.
Override this in the Git fetcher to only require a pre-mirror fetch when the
clone directory does not exist.

Fixes [BUGID 290]

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-08 14:25:24 +01:00
Richard Purdie 7b580b488c bitbake/bitbake-runtask: Ensure signals to the parent don't pass to the children directly
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-07 23:54:15 +01:00
Richard Purdie 647713050b bitbake/codeparser: Deal with functions with trailing whitespace
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-07 14:33:53 +01:00
Joshua Lock b78d5368cc fetch: fix setting of localpath in SRC_URI parameters
When setting localpath in the SRC_URI parameters the basename should be set
explicitly as the localpath() method is not called.

Fixes [BUGID #81]

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-06 14:32:31 +01:00
Richard Purdie 8d764de25f bitbake: Add missing signature files
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-03 16:11:45 +01:00
Richard Purdie 43595fabbe bitbake: Implement signatures
Includes functionality to find out what changes between two different singature data dumps.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-03 16:08:32 +01:00
Richard Purdie 453d8f49ac bitbake/codeparser: Ensure cached sheel entries return the correct dependencies
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-03 16:08:32 +01:00
Richard Purdie e8ed1454ae bitbake/ast: Ensure function definitions are always added to the dictonary
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-03 16:08:31 +01:00
Joshua Lock 232ef8430e bitbake/fetch: ensure the mirrored repository is updated as required
If we fetch a tarball from a mirror it's entirely possible that the
mirror will have been from before the required tag/branch/etc was included
in the repository. To that end use forcefetch() as a way of testing whether
the repository is up to date and if not fetch updates.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-01 11:14:57 +01:00
Joshua Lock 5c229f921f bitbake/git: define a forcefetch method
The git fetcher should force a fetch if the required tag is not present in
the local clone, or if the fullclone parameter is set.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-01 11:14:41 +01:00
Joshua Lock b54e95fd06 bitbake/fetch: Respect forcefetch even when pulling from a mirror
When pulling from a premirror we would prefer a local tarball even when the
caller had specified the forcefetch parameter.
Add an extra parameter 'force' to try_mirrors, defaulting to False. If set
the mirrors will be tested even if the file exists locally.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-01 11:14:23 +01:00
Richard Purdie 9897b81e56 bitbake/codeparser: Implement persistent cache
For a given input to this code, the output doesn't change to implement a persistent
cache of the data to speed up parsing.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-31 14:49:14 +01:00
Richard Purdie 332c33af18 bitbake/data.py: Add emit_func() and generate_dependencies() functions
These functions allow generation of dependency data between funcitons and
variables allowing moves to be made towards generating checksums and allowing
use of the dependency information in other parts of bitbake.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-31 12:47:41 +01:00
Richard Purdie 3492bff64a bitbake: Add codeparser for parsing shell and python functions
This commit is derived from Chris Larson's checksum work, turned into a
standalone piece of code for parsing python and shell functions.

The deindent code has been replaced with code to work around indentation
for speed. The original NodeVisitor in the ast was replaced with a faster
class walk call.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-31 12:41:23 +01:00
Richard Purdie 13fdd4ae5d bitbake/BBHandler: Save python functions into the dictonary
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-31 12:08:51 +01:00
Richard Purdie f7627e4f67 bitbake/data_smart.py: Allow the data expand function to keep track of references (including those from python code)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-31 12:06:18 +01:00
Richard Purdie 53ff069611 bitbake/runqueue.py: Ensure rqexe always exists and that empty task lists cause a graceful exit
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-25 00:59:13 +01:00
Richard Purdie 71a46a44a2 utils.py: Fix bb.copyfile to change the permissions of the file back correctly
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-23 13:01:05 +01:00
Mark Hatle 87dba4254b utils.py: Add a new extend_deps() to easily merge two dependency lists
Add a new extend_deps function to more easily merge two dependency lists.
This avoids adding duplicates, unless the value of the dependency is
different.

Signed-off-by: Mark Hatle <mhatle@windriver.com>
2010-08-20 23:46:08 +01:00
Richard Purdie 75b57d573c bitbake/utils.py: Allow copyfile to copy files which aren't readable
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-20 15:11:44 +01:00