sysmo-bts
/
generic-poky
9
0
Fork 0
Code Pull Requests Releases Activity

documentation/poky-ref-manual/usingpoky.xml: Removed comments 

Removed some comments that were buried in the file that were
notes for working on the sstate section.

(From yocto-docs rev: bd03315031bbb1b682dcd2253f85fc184822a28e)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is contained in:
Scott Rifenbark 2011-12-07 15:11:32 -08:00 committed by Richard Purdie
parent c2494d3014
commit e7cfb3b469
1 changed files with 0 additions and 99 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

99
documentation/poky-ref-manual/usingpoky.xml
View File

@ -298,107 +298,8 @@
<ulink url='http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
</note>
</section>
<!--
<section id="considering-shared-state-cache">
<title>Considering Shared State Cache</title>
<para>
What is shared state in general.
Benefits?
How we handle things
(reference https://lists.yoctoproject.org/pipermail/yocto/2011-March/001157.htm),
which is RP's dissertation on how YP solved it.
We need to talk a bit about checksum generation for tasks and how the
sstate code uses them to figure out what needs rebuilt and what can be re-loaded
from the sstate cache.
Need to tell about cases where an implicit change can mess things up and under
normal situations the state in the sstate cache would be used but it shouldn't be.
This is the scenario described by bug 1500 - typical case.
Then we talk about how we can invalidate parts of the cache on a per-class basis
if needed.
there is a discussion at
https://lists.yoctoproject.org/pipermail/yocto/2011-March/001157.htm
that talks about sstate and how the YP team attacked and solved the problem.
This is probably a good place to get information from to broach the whole
sstate concept.
YP, by default, builds from scratch.
This is good but it means spending a lot of time rebuilding things that don't
necessarily need rebuilding.
The SSTATE_DIR variable points to the directory for the shared state cache that
is used during a build.
A task's inputs have a checksum or signature associated with them.
If the checksum changes on an input as compared to a prior build, the task must be rerun.
The shared state (sstate) code keeps track of what output is generated by which tasks.
So if a task's inputs have not changed then the output associated with the task can
be yanked from some place and re-used. No re-build required for that particular task.
A "run" shell script is created for each task.
You can create a checksum for the task based on the inputs to the task.
When you have this checksum, the code will look at it and compare it to the previous
checksum to see if the task's inputs have changed.
If so, the task needs to be re-run.
Python tasks have python functions that access variables.
Python functions will call other python functions as well.
The solution was to figure out the variable and function dependencies and create
a checksum value for the data coming into the python task.
Here is a conversation with Mark Hatle regarding bug 1500 (638 is related):
(01:23:34 PM) scottrif: mark - you have a minute?
(01:34:05 PM) Mark Hatle: sure..
(01:34:11 PM) Mark Hatle: might be a bit slow to respond, but I'm here
(01:34:45 PM) scottrif: Hi - I am looking at bug 1500 and trying to get a bit of better understanding. Here is the link to the bug - http://bugzilla.pokylinux.org/show_bug.cgi?id=1500
(01:35:25 PM) scottrif: It seems that the key for the user here is to when to "Know" when to put some comments into a function to invalidate certain areas of sstate.
(01:35:49 PM) Mark Hatle: what the issue is, if you make changes to something that is not normally calculated in the checksums for sstate, then you can get package mismatches..
(01:35:50 PM) scottrif: This trick of "knowing"... does it need to be explained?
(01:36:25 PM) Mark Hatle: The developer will have to know they made such a change.. Then to deal with this, they can use a patch like what is referenced to cause specific steps to be invalidated and various steps re-run..
(01:37:01 PM) scottrif: so my question is will the developer know when they make a change like this?
(01:37:04 PM) Mark Hatle: In this case, we change part of the back-end packaging mechanisms.. which changed internal dependency generation. The sstate code does not checksum the internal dependency generation, it assumes that is code that doesn't change behavior
(01:37:24 PM) Mark Hatle: They should understand the ramifications of their changes — and thus know they need to do this.
(01:37:46 PM) Mark Hatle: Examples of times you need to do this. Back end packaging changes occur — i.e. you change the format of dependency generation..
(01:38:38 PM) scottrif: do you have any other examples?
(01:38:39 PM) Mark Hatle: when you change a recipe itself, source code.. it is -not- necessary to do this
(01:38:49 PM) Mark Hatle: RP might be better at examples of when to do it..
(01:39:11 PM) scottrif: right - If I change a recipe then every thing dependent further down the line gets regenerated right?
(01:39:17 PM) Mark Hatle: This should never be necessary when a recipe changes.. it will only be necessary when some classes or back-end (packaging frameworks) change..
(01:39:21 PM) Mark Hatle: ya
(01:39:33 PM) Mark Hatle: Another way to think of this is implicit dependencies..
(01:40:01 PM) Mark Hatle: I change RPM.. If you build something that produces an RPM package.. the assumption is the RPM package won't change, even if the RPM binary changes..
(01:40:10 PM) Mark Hatle: If the format of the package changes.. you would need to do this
(01:40:53 PM) Mark Hatle: RP can probably give you an idea of the various implicit dependencies, and which ones this type of change is needed for
(01:41:26 PM) scottrif: okay. I am struggling a bit with how to word it. what I will do is write something up and send it out to you and RP for a look
(01:41:47 PM) Mark Hatle: ya, I understand.. it's an odd set of situations that can cause this — but we definitely need to document it
(01:42:01 PM) scottrif: I just want the information to help the user understand the conditions when they will want to invalidate parts of the sstate
(01:42:18 PM) scottrif: I will likely use the RPM example as the case to illustrate it
(01:42:26 PM) scottrif: as it seems pretty straight forward
(01:42:28 PM) Mark Hatle: yup. Key thing is it's only needed on implicit dependencies.. Normal case is back end packaging format changes..
(01:42:31 PM) Mark Hatle: yup
(01:42:47 PM) scottrif: ok - thanks Mark
Here is what RP wants to address 1500:
If its desired to change the checksum of a given subset of tasks, maybe
due to a change which isn't directly visible in the code itself (e.g. a
tool changed its output) its possible to do this by changing a function
comments since the sstate checksums include the body of functions. To
invalidate package sstate files for example, do_package or one of the
functions it calls can be changed, even if its just a cosmetic change to
the commends.
http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54
is an example of a commit which does this.
-->
</section>
<section id='usingpoky-install'>
<title>Installing and Using the Result</title>