Commit Graph

57 Commits

Author SHA1 Message Date
Patrick Ohly 431c1e171f combo-layer: handle empty commits during "init --history"
When importing the components during the "combo-layer init" with full
history and relocation into a destination directory, components with
empty commits were not handled because the "mv" command was invoked
with just one parameter.

Replacing that with a find/xargs pair avoids the problem and should
also fix the handling of filenames with special characters (in
particular spaces).

(From OE-Core rev: fd9d08b8ba512151615a03fda61a103760a0683c)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 22:39:43 +00:00
Richard Purdie e6f27619af combo-layer: Stop using filterdiff
I ran into an issue where a patch just deleting a single file
within the repository (meta/recipes-devtools/m4/m4/make.patch)
would get skipped by combo-layer.

It turns out this has the patch header (commented to avoid breaking scripts):

: diff --git a/meta/recipes-devtools/m4/m4/make.patch b/meta/recipes-devtools/m4/m4/make.patch
: deleted file mode 100644
: index 79fb415..0000000
: --- a/meta/recipes-devtools/m4/m4/make.patch
: +++ /dev/null
: @@ -1,42 +0,0 @@

and this is classed as > 5 headers in filterdiff. When we piped the path
through filterdiff, the --- line disappears, then the second time we pass
through filterdiff, it shows no lines changed and the patch is assumed
to be empty and skipped.

Changing MAX_HEADERS in filterdiff is one way to fix this, another would
be to grep out "deleted file mode" lines. Instead, we can use new
git syntax to exclude files from the git format-patch instead and avoid
filterdiff entirely.

(From OE-Core rev: 296c70afeef75396dea9ae436058314d406dc257)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:18 +00:00
Alexander D. Kanevskiy aa1844e847 combo-layer: introduce ability to exclude component from mass update
There is no ability at the moment for situations where users would like
to keep section in combo-layer.conf but don't update it, unless explicitly
specified.

Now, by adding "update = no" to desired section would exclude
that repository from "combo-layer update" command.

It is still possible to explicitly update it by "combo-layer update $section".

By default, all repositories are assumed as "update = yes"

(From OE-Core rev: 74ecf1a0e5c8230e55cb1a309f711d98fa63ce79)

Signed-off-by: Alexander D. Kanevskiy <kad@kad.name>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:39:31 +00:00
Patrick Ohly 8b847ffd04 combo-layer: fix action_pull for unknown branch
When reconfiguring the branch to something not already fetched,
action_pull fails with
   error: pathspec '<new branch name>' did not match any file(s) known to git.

It is the "git checkout" which fails like that. To solve this,
try the faster "git checkout + git pull" first and only if that fails,
fall back to the slow "git fetch + git checkout".

In the conf.hard_reset case, do the checkout always after the git fetch.

(From OE-Core rev: ad4d3b1da190cf08c6ac5f9a94a2a1c4980a184d)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-09 00:13:57 -07:00
Patrick Ohly b2a0c2fbd1 combo-layer: enhance output in action_pull
The git operations can fail, for example when the branch is unknown
or misconfigured.

Better move the info message and extend it such that it is printed
first and provides the necessary context, because otherwise the
CalledProcessError exception gets dumped without mentioning for which
component it occurred.

(From OE-Core rev: 00d01468a692faf4272894dd328e8c532bcf8b49)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-09 00:13:57 -07:00
Alexander Kanevskiy 405bb5b588 combo-layer: fix AttributeError traceback.
Commit c908a423f85a84ddd8249abd00254f29d47df74b introduced a new
issue in combo-layer that leads to a traceback as
args.hard_reset is an unknown variable. This change defines an
appropriate destination for the command args parser and fixes the
reference.

(From OE-Core rev: ac4ef551d506a5767946cf2fb717c2fd7f86df9c)

Signed-off-by: Alexander Kanevskiy <kad@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-31 12:09:42 +01:00
Markus Lehtonen b0079ec5e9 combo-layer: implement --hard-reset option
This option causes combo-layer to do git fetch and hard reset instead of
git pull in the component repositories. This makes sure that the local
component repositories are always in sync with the remote - tolerating
force pushes and overriding any locally made changes.

(From OE-Core rev: c908a423f85a84ddd8249abd00254f29d47df74b)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-16 15:09:17 +01:00
Markus Lehtonen d1a1dbb8a5 combo-layer: only allow fast-forward when pulling
Only allow fast-forward merges in the component repositories when doing
git-pull. This makes it possible to spot problems (i.e. rewriting of
history) in the component upstream . Also, this change prevents the
creation of local-only merge commits in the component repositories.
These merges cause "last_revision" field of the combo-layer config to
point to a git commit that is only present in the users local component
repository but nowhere in upstream.

(From OE-Core rev: 5207169b9c6c4b05c6b043745905a5498adca1a3)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-16 15:09:17 +01:00
Puustinen, Ismo daec53e977 combo-layer: modified to generate better commit messages.
This patch includes support for a global section in combo-layer.conf
called [combo-layer-settings]. Supported in this section is key
"commit_msg"; its value is the template for the git commit message
that updates the last_revision. The template can include substitution
for the updated component list: ${components}. The substituted value
will either be a comma-separated list of components or "all components",
if combo-layer was invoked without component list argument.

If the key is not present, the old default value is used for the commit
message.

Configuration file example:

[combo-layer-settings]
commit_msg = pulled in the latest changes for ${components}.

(From OE-Core rev: fe84747f961772b61031af59d44e54b178148379)

Signed-off-by: Ismo Puustinen <ismo.puustinen@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-08 00:01:22 +01:00
Richard Purdie 20a3a36547 scripts/combo-layer: Fix exit codes and tty handling
If combo-layer is called from a non-interactive context we should exit
with a correct error code rather than try and drop to a shell.

This patch cleans up a few error case exit codes as well as
detecting and handling non-interactive usage.

(From OE-Core rev: 3b1d89a51445cf526ca84eb5b53de434f9585d6e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-05 16:42:17 +01:00
Patrick Ohly 45b8e62439 combo-layer: handle unset dest_dir in sanity_check()
The previous "clean up dest_dir checking" patch (f8cdbe7497) improved
handling of empty dest_dir but made handling of unset dest_dir worse:
instead showing the "Option dest_dir is not defined for component ..."
error, it fails with a Python exception.

Avoid that by providing a sane fallback for the unset case. With that
change, dest_dir is no longer strictly required, but the check for it
is kept to ensure that a combo-layer.conf also works with older
combo-layer versions.

[Yocto #7773]

(From OE-Core rev: d4bf858b2c15bef128fd6d606b08203a318e2d4c)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-22 13:29:47 +01:00
Patrick Ohly dcb8f89723 combo-layer: improve merge commit handling
When the head of a branch is a merge commit, combo-layer did not
record that commit as last_revision because it only considers applied
patches, and the merge commit never gets applied.

This causes problems when the merge commit leads to multiple patches
and the commit id that gets recorded only reaches some of these
patches. The next run then will try to re-apply the other patches.

This special case is now detected and dealt with by bumping
last_revision to the branch commit. The behavior where the head is a
normal commit is intentionally not changed, because some users might
prefer the traditional behavior.

(From OE-Core rev: 701bcd52c208f22a9a6c48a11a35bcf1c4e413df)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-09 22:26:45 +01:00
Patrick Ohly 240dadea12 combo-layer: fix file_exclude for empty commits
The code detecting empty patches after removing files with
file_exclude failed for commits which were already empty before (like
the initial commit in some repos): such patches are completely empty
files, without a From line.

Detect that case and just let the normal empty patch detection deal
with it.

(From OE-Core rev: 3e310b8cd603539a2de115b2b73e1db35403b426)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-31 22:23:17 +01:00
Patrick Ohly 3775c6e714 combo-layer: fix file_exclude for dest_dir = .
"filterdiff -x ./some/file" does not remove changes for some/file.
We must be more careful about constructing the path name and
only add the prefix when it really means a directory.

While at it, also better normalize the path in copy_selected_files()
early on, to handle double slashes. Useful should the function ever
gets used for something other that dest_dir (which gets normalized in
sanity_check()).

(From OE-Core rev: 8ea63c6c920c39e5a4ba176223fe472f92e2632a)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-31 22:23:16 +01:00
Patrick Ohly 5508950045 combo-layer: clean up dest_dir checking
Empty dest_dir is basically undocumented behavior. The sample conf
only mentions using just a dot for the current directory. In practice,
the empty string does not work because of code like this:

def action_splitpatch(conf, args):
    ...
        if dest_dir != ".":
            filerange_root = '%s -x "%s/*"' % (filerange_root, dest_dir)

However, the empty string was not explicitly checked for, leading to
strange errors when trying to apply patches:

[12:50:23] Applying: foobar: xyz
fatal: unable to stat newly created file '/foobar': No such file or directory

This patch turns the empty string into an alias for the dot. This seems
more user-friendly than throwing an error. This alias is intentionally
not document in the sample conf, because the dot is clearer and works also
with older copies of combo-layer.

Instead of checking for both all the time and normalizing the path when
needed (as done in some places), rewrite the value in sanity_check()
and then only check for '.'.

(From OE-Core rev: f8cdbe749755dc769150d3a6c2c54318c80e1562)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-31 22:23:16 +01:00
Richard Purdie c5e9654ac9 scripts/combo-layer: Handle update with no repo/revision specified
Running an update operation with no repo/revision specified was failing.
This fixes that code path which worked until the change from:

http://git.yoctoproject.org/cgit.cgi/poky/commit/scripts/combo-layer?id=3592507a149b668c0a925e176535f7f2002fa543

(From OE-Core rev: 411a6a3694a9fcd563c5b5320597314fbec75cc2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 23:56:06 +00:00
Patrick Ohly e0d3cabc5a combo-layer: partial import for '--history init'
The new "since_revision" property can be used to cut off the imported
history at some point. This is useful to keep the resulting repository
smaller while still preserving enough history that "git annotate"
reports the right author and commit for most lines.

The initial, squashed import commit shows up with "unknown" as author
in the "git annotate" output. It has the repository name as prefix
in the subject line; importing that commit works best with a
layer hook which does not add the repository name again when
it is already present. Adding it here is useful for hooks
which do not extend the subject line.

(From OE-Core rev: 74f4c9e3bcdb3c4ca919623086e92a9379bd81ff)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:21:24 +00:00
Patrick Ohly 14eac2a474 combo-layer: combine trees via replacement objects
Instead of local graft entries rooting the imported branches in the
shared root commit, use replacement objects.

The advantage is that they get moved around by "git push" and "git
fetch", so everyone has the same, nicer view with everything starting
at the beginning of the combined repository.

If undesired, these objects can be removed with "git replace".

(From OE-Core rev: 220d816cda3c9b8d888288cc03eb74be5e71cc59)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:21:24 +00:00
Patrick Ohly dd985a241c combo-layer: init with full history
The new --history parameter enables a new mode in "combo-layer init"
where it copies the entire history of the components into the new
combined repository. This also imports merge commits.

Moving into a destination directory and applying commit hooks
is done via "git filter-branch" of the upstream branch. File
filtering uses the same code as before and just applies it
to that filtered branch to create the final commit which
then gets merged into the master branch of the new repository.

When multiple components are involved, they all get merged
into a single commit with an octopus merge. This depends
on a common ancestor, which is grafted onto the filtered
branches via .git/info/grafts.

These grafts are currently left in place. However, they do not get
pushed, so the local view on the entire history (all branches rooted
in the initial, empty commit, temporarily diverging and then
converging) is not the same as what others will see (branches starting
independently and converging). Perhaps "git replace" should be used
instead.

The final commit needs to be done manually, as before. A commit
message with some tracking information is ready for use as-is. This
information should be sufficient to implement also "combo-layer
update" using this approach, if desired. The advantage would be that
merge commits with conflict resolution would not longer break
the update.

(From OE-Core rev: 9e40cb1ab77029df7f2cf1e548a645ff6a62c919)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:21:23 +00:00
Patrick Ohly b4326bf85a combo-layer: update() also instance property
The Configuration class mirrors all properties in local hashes.
When updating the configuration, also update these properties
to remain consistent.

(From OE-Core rev: c575ff183ae5c35e0f7e8d90c222994574ce73c9)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:21:23 +00:00
Patrick Ohly 96b98451c4 combo-layer: exclude files
Some combined repos intentionally do not include certain files.
For example, Poky does not include bitbake's setup files and
OE-core's sample files under meta/conf.

When these files get modified in the upstream repository, applying the
patches fails and requires manual intervention. That is merely a
nuisance for someone familiar with the problem, but a real show
stopper when having the import run automatically or by someone less
experienced.

Therefore this change introduces "file_exclude", a new per-repo list
of file patterns which removes all matching files when initializing or
updating a combined repository. Because fnmatch is used under the hood
to match full path strings, removing entire directories must be done
with a pattern ending in a '/*' (in contrast to file_filter).

For Poky, the additional configuration looks like this:

[bitbake]
...
file_exclude = classes/base.bbclass
	conf/bitbake.conf
	.gitignore
	MANIFEST.in
	setup.py
	TODO

[openembedded-core]
...
file_exclude = meta/conf/bblayers.conf.sample
	meta/conf/local.conf.sample
	meta/conf/local.conf.sample.extended
	meta/conf/site.conf.sample

(From OE-Core rev: a51b37f4db6b144386d1bd5789ec91acc78a0bd8)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:21:23 +00:00
Patrick Ohly 627e524238 combo-layer: runcmd() with separate output
Allow the caller to specify a separate output stream. stderr is always
a temporary file opened by runcmd(), so read from that to capture
output for error reporting *and* the return value.

The reasoning for the latter is a) that this preserves the traditional
behavior when out=None and b) if the caller wants the content of
stdout, it can read from the stream itself, which is not possible for
the temporary stderr.

(From OE-Core rev: a084162a9dc4718ab453723f1f28aefc55100e2e)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:21:22 +00:00
Patrick Ohly 58c5bb28b5 combo-layer: make Signed-off-by optional
It depends on the diligence of the person running the combo-layer tool
whether the Signed-off-by line added to each commit actually indicates
that the person was involved in validating the change.

When the import is purely automatic, it is better to not add the line,
because the history is more useful without it (searching for the person
really only lists changes he or she was involved with) and it would
be a false statement.

The 'signoff' property can be set per repository, like every
other property. But setting it in the special [DEFAULT] section
is more useful, so that is what the example shows.

(From OE-Core rev: dba34e5a6f0a4c726266654dc5f26e0f7f13db0f)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:21:22 +00:00
Patrick Ohly b1408ba706 combo-layer: let user choose where properties get updated
When updating a property (at the moment, only 'last_revision'),
first check whether the user already added it to the main config.
If so, update there even if there is a local config.

This way, 'last_revision' can be shared between developers
as part of the repository while still configuring per-developer
repo paths outside of the git repository in a local config.

An example of a repository which is set up like that is tizen-distro,
with instructions for such a collaborative maintenance found here:
https://review.tizen.org/git/?p=scm/bb/tizen-distro.git;a=blob;f=README;hb=refs/heads/tizen-ivi

(From OE-Core rev: a3b246ee928d2ec59c8b6a775d9309f5be33ecc7)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:21:21 +00:00
Markus Lehtonen 3592507a14 combo-layer: support updating up to arbitrary commit
Support defining the top commit up to which to update. In other words,
this makes it possible to update up to certain point other than the
branch head. The update point (git commitish) is given on the command
line by appending the component name(s) with a colon and the commitish,
e.g.
 $ combo-layer update my_component:sha1

Only the "update" action supports this.

(From OE-Core rev: b7ee7cf5bbfd0f305a75e4df9455f9968aa67878)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-29 15:36:52 +00:00
Markus Lehtonen 62d94a44ae combo-layer: minor refactor
Change get_repos() to assume a list of repository names instead of full
list of command line arguments.

(From OE-Core rev: d2d8f3c2a7570bb74db713ddc83059d3b3bd4b2e)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-29 15:36:51 +00:00
João Henrique Ferreira de Freitas 1ef8100d59 combo-layer: fix revlist taking into account file_filter
If file_filter is set, git format-patch takes account but git rev-list
does not. So revlist is going to get with wrong revisions. And last_revision
will be updated with wrong revision. The next time that user run
combo-layer it complain about applying patches.

So ensure that 'git rev-list' are using file_filter as 'git format-patch'.

(From OE-Core rev: c22e40900d8e33c2c884e714c11ddb771b86923f)

Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-01 14:29:30 +01:00
Saul Wold 51cccb7fb9 combo-layer: Make directories only if they don't exisit yet
(From OE-Core rev: 24188967209bad291545909ddb89af35ab3f6021)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 00:52:37 +00:00
Richard Purdie 214ad32077 scipts/combo-layer: Fix check_rev_branch() for cases where the revision is on more than one branch
If a revision is in more than one branch, the check_rev_branch() function can't
cope with it and the tool returns incorrect errror messages. This patch
ensures it copes with this situation.

(From OE-Core rev: 14bd101c6a86dd048da98817f47694fb21504209)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-10-18 12:13:42 +01:00
Paul Eggleton a2eab6bb16 scripts/combo-layer: ensure we validate branch/revision on init
If both branch and last_revision are specified for a component when
combo-layer init is run, ensure that the specified revision is actually
on the specified branch and error out if not. Also ensure that the error
message mentions the component.

(From OE-Core rev: e498257ecbec94cec181d73bda57d44335b4dee0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-09-14 09:50:29 +01:00
Paul Eggleton 7ddf64d58d scripts/combo-layer: use last_revision if specified in init
If last_revision is specified for a component when running combo-layer
init, then use that revision instead of the latest revision on the
branch. Also, remove unnecessary git checkout during init since we
specify the revision to all calls to git when dealing with the component
repositories.

Fixes [YOCTO #3040].

(From OE-Core rev: ff8277cd133e9a02b131977078cff61fa587a1af)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-09-14 09:50:28 +01:00
Paul Eggleton eab2efc596 scripts/combo-layer: specify branch when getting current revision
Handle the case during update where the configured branch is not
currently checked out in the component repository by just specifying
it in the places where it was not previously.

(From OE-Core rev: a3ddf39af7e7cd47156677d8ae753964ee582745)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-23 11:43:33 +01:00
Paul Eggleton 74aac73d69 combo-layer: ensure init works with split local config
If the local configuration is already split out, ensure the init action
works properly and does not error in the case that last_revision is not
set. Additionally, if the local configuration is within the repository,
prevent it from being committed and add it to .gitignore.

(From OE-Core rev: de339b0cb201035e27df1128ccf526937b8325ec)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-31 11:59:06 +01:00
Paul Eggleton de3abcf78a combo-layer: allow splitting out local config
Allow splitting the local parts of the configuration (mostly
local_repo_dir and last_revision, although there is no limitation) to
a side-by-side -local.conf file, with component sections optionally
tagged with the combo layer branch name. This effectively allows you to:

 * avoid polluting the history by committing the updated last revision
   to the combo repository for every update
 * avoid putting local repo paths into the combo repository
 * manage multiple branches of the combo repository whilst avoiding the
   possibility of mixing the configuration for one branch with another.

An example split configuration (note, values may be artificial):

------------------- combo-layer.conf -------------------
[bitbake]
src_uri = git://git.openembedded.org/bitbake
dest_dir = bitbake
hook = scripts/combo-layer-hook-default.sh

[oe-core]
src_uri = git://git.openembedded.org/openembedded-core
dest_dir = .
hook = scripts/combo-layer-hook-default.sh
--------------------------------------------------------

---------------- combo-layer-local.conf ----------------
[bitbake]
local_repo_dir = ../repos/bitbake

[oe-core]
local_repo_dir = ../repos/oe-core

[bitbake|master]
branch = master
last_revision = db689a99beffea1a285cdfc74a58fe73f1666987

[oe-core|master]
branch = master
last_revision = 121a1499a81706366acc0081272a6bff634d4d62

[bitbake|denzil]
branch = 1.12
last_revision = 24b631acdaa143a4de39c6e1328849660c66f219

[oe-core|denzil]
branch = denzil
last_revision = 741146fa90f28f7ce8d82ee7f7e254872d519724
--------------------------------------------------------

It is assumed that the local config file will be added to .gitignore.

(From OE-Core rev: f0065d7a6973628803a17c57f2265512aba3234c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-31 08:02:11 +01:00
Paul Eggleton 6bea863704 combo-layer: check that last_revision is valid
If the user edits the configuration file by hand and sets last_revision,
we need to ensure that the revision is valid and on the specified
branch.

(From OE-Core rev: 05382932257257247b8c18bc06e9c0039d134d06)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-31 08:02:11 +01:00
Paul Eggleton 8140c5b7ee combo-layer: improve patch list handling and output
* Ignore blank lines in patch list
* Don't fail in interactive mode if patch list is deleted
* Show patch counter
* Show relative path for patches
* Print headings before applying patch list for each component

Also change to using a "with" block to read the patch list so it gets
closed properly when we're finished.

Fixes [YOCTO #2455].

(From OE-Core rev: 65461d7c35fdadb5b008052798731dce19ed187f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-31 08:02:10 +01:00
Paul Eggleton 2dc0b337bc combo-layer: drop to a shell when apply fails during update
If applying a patch fails during the update process, drop to a shell
instead of exiting; at that point the user can manually apply the patch,
do nothing and "exit" to skip it, or "exit 1" to abort the process.

(From OE-Core rev: c82b28982c4f630c130c827a7da3ac0454cd93b6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-31 08:02:10 +01:00
Paul Eggleton f558ecf90e combo-layer: ignore patch-* temp directories in dirty check
Make the dirty repo check somewhat less strict by ignoring old
patch directories created by this tool.

(From OE-Core rev: fea477ac55e2555c5bb0aad36db641aaa27aa915)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-31 08:02:10 +01:00
Paul Eggleton 2ed3f63b02 combo-layer: allow component pull to be done separately
* Add a -n option to disable component repo pull during update
* Add a 'pull' action to pull the component repos only

(From OE-Core rev: 61983b2191253b24117b63f586d5aac00c7eb48e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-31 08:02:10 +01:00
Paul Eggleton dad01a055b combo-layer: remove &> bashism
&> does not work with dash - use > xxxx 2>&1 instead.

(From OE-Core rev: 5582cc58e42402c4680877e669d8c1ce058e7098)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-31 08:02:10 +01:00
Paul Eggleton ab69a16b21 scripts/combo-layer: handle diffs in commit messages
A few recent commits in the OE-Core repository contain diffs in their
commit messages, which totally confuses git-am when applying them to the
combo repository during update. Add some code to detect and indent any
diff text in the commit message so that this does not happen (and show a
warning).

(From OE-Core rev: 6e70c95dc69be6708c3bc231cc2a99eac1360815)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-23 12:13:08 +00:00
Paul Eggleton fd3f7d5d6a scripts/combo-layer: limit component repo dirty check
If one or more components are specified for update, only check if their
repository/repositories are dirty rather than checking all of the
configured repositories.

(From OE-Core rev: b03e710c88ad0c66cf731647f26e8441d0074cae)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-23 12:13:08 +00:00
Paul Eggleton c3b3a681f7 scripts/combo-layer: allow updating a specific component(s) only
If you specify one or more components to update immediately following
the "update" command, only these components will be updated as opposed
to the default behaviour of updating all of them.

(From OE-Core rev: 1779afce024b31edcf846329da705ed5aa9fe63e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-21 17:00:47 +00:00
Paul Eggleton 8e7cd329b1 scripts/combo-layer: avoid saving last revision if unchanged
If we are running an update and the last revision hasn't changed since
the last update, don't write to the configuration file. This avoids
committing the config file with no changes other than spontaneous
reordering of sections, which sometimes occurs due to the behaviour of
the internal dictionary in Python's ConfigParser class. (This can be
fixed properly but the fix is only easy in Python 2.7+ due to the
availability there of the collections.OrderedDict class, and we
currently want to be compatible with 2.6.x as well.)

(From OE-Core rev: 4592f238de3fe6b0384c334774be9fcfc0985e4f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-21 17:00:46 +00:00
Paul Eggleton f273c04175 scripts/combo-layer: avoid error when config file is outside repo
Avoid displaying the error from the "git status" command we use to check
the status of the config file if the config file is outside of the
repository (a situation that is already handled).

(From OE-Core rev: 9d3e54057673348e85ba479705c4ffaad56973d6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-21 17:00:45 +00:00
Paul Eggleton ba5537d768 scripts/combo-layer: skip empty commits
If a commit is empty (for example, commits brought over from svn where
only properties were changed) then attempting to apply it with "git am"
will result in the error "Patch format detection failed", so skip it
instead.

(From OE-Core rev: 5754bc7bcc2f57b6b56e67e2900eeaab5bc184d8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-11-07 14:05:37 +00:00
Paul Eggleton c04305e695 scripts/combo-layer: improve some messages
Fix some grammar.

(From OE-Core rev: cc34f1bf136409d2e31c443dd5334c3a35d208af)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-11-07 14:05:36 +00:00
Paul Eggleton f82983034e scripts/combo-layer: auto-commit updated config file
If the config file is tracked within the combo-layer repository and it
is updated at the end of the "update" operation (because last_revision
has been changed), then automatically commit the file. This ensures that
multiple people can perform updates on different machines without the
last revision information going missing.

(If the file is outside the repository or is masked via .gitignore, this
will do nothing.)

(From OE-Core rev: 14f89acb43b0d9bec09b94668fbb056c7b5b1456)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-11-07 14:05:36 +00:00
Paul Eggleton 948d28e801 scripts/combo-layer: fix dirty repo check
Fix the dirty repository check to detect any changes, including
untracked files and uncommitted changes in the index.

(From OE-Core rev: f03fa8bbafa82ce6b09c08bf86011b6c51eb3531)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-11-07 14:05:36 +00:00
Paul Eggleton 25fc202e21 scripts/combo-layer: make component repo branch configurable
Add an optional per-component branch setting to allow specifying the
branch instead of always using master.

(From OE-Core rev: 8e2b8b05607103acd539808c5ab0cc80c0d481fc)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-11-07 14:05:36 +00:00