Commit Graph

8 Commits

Author SHA1 Message Date
Leonardo Sandoval 70f99b427b bb-perf: plot histograms base on buildstats data
Scripts that produces script data to be consumed by gnuplot.
There are two possible plots depending if either the
-S parameter is present or not:

    * without -S: Produces a histogram listing top N recipes/tasks versus
      stats. The first stat defined in the -s parameter is the one taken
      into account for ranking
    * -S: Produces a histogram listing tasks versus stats.  In this case,
      the value of each stat is the sum for that particular stat in all recipes found.
      Stats values  are in descending order defined by the first stat defined on -s

EXAMPLES

1. Top recipes' tasks taking into account utime

    $ buildstats-plot.sh -s utime | gnuplot -p

2. Tasks versus utime:stime

    $ buildstats-plot.sh -s utime:stime -S | gnuplot -p

3. Tasks versus IO write_bytes:IO read_bytes

    $ buildstats-plot.sh -s 'IO write_bytes:IO read_bytes' -S | gnuplot -p

(From OE-Core rev: 74408c19fba89de54c093fccf65b3a072d6a197b)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:05 +00:00
Leonardo Sandoval 4bc5353c92 scripts: Specify the stats to take into account
There are many more stats on buildstats that 'Elapsed time', so make the script
more flexible to support all stats. Some cmd line examples:

$ buildstats.sh -s 'utime'

Buildstats' data covers proc's stats in different areas, including CPU times,
IO, program system resources and child program system resources. In order
to print values on each of these sets from command line, one can use the
following:

$ buildstats.sh -H -s 'TIME' | less

$ buildstats.sh -H -s 'IO' | less

and 'RUSAGE' and 'CHILD_RUSAGE' for program and program's child system
resources.

(From OE-Core rev: 81479b191287ccbf4cf94fa2d0ad46813091bca1)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:14 +00:00
Leonardo Sandoval f0af24eedf buildstats: Outputs 'task recipe elapsed-time' from each buildstats' recipe
Given a 'buildstats' path (created by bitbake when setting
USER_CLASSES ?= "buildstats" on local.conf) and task names, outputs
'<task> <recipe> <elapsed time>' for all recipes. Elapsed times are in
seconds, and task should be given without the 'do_' prefix.

Some useful pipelines

1. Tasks with largest elapsed times

    $ buildstats.sh -b <buildstats> | sort -k3 -n -r | head
    do_compile perl-5.20.0-r1 221.82
    do_configure gettext-native-0.19.4-r0 140.34
    do_compile openssl-native-1.0.2a-r0 107.48
    do_compile openssl-1.0.2a-r0 102.10
    do_configure perl-native-5.20.0-r0 90.70
    do_configure gettext-0.19.4-r0 88.17
    do_compile gcc-cross-i586-4.9.2-r0 83.98
    do_configure m4-native-1.4.17-r0 83.44
    do_compile qemu-native-2.2.0-r1 71.69
    do_compile glibc-2.21-r0 60.88

2. Min, max, sum per task

    $ buildstats.sh | datamash -t' ' -g1 min 3 max 3 sum 3 | sort -k4 -n -r
    do_configure 0.03 140.34 1968.66
    do_compile 0.01 221.82 1664.44
    do_install 0.03 40.31 330.45
    do_populate_sysroot 0.11 34.45 229.23
    do_unpack 0.01 36.1 193.54
    do_patch 0.01 9.2 62.07
    do_fetch 0.01 6.66 32.13
    do_populate_lic 0.09 1.65 30.7

(From OE-Core rev: 29fa8ee01ef3254272bcbdd13a8c7244548639a3)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-12 22:48:30 +01:00
Peter Kjellerstedt 4137f9a996 bb-matrix-plot: Use interpolation for sparse data
If not every combination of BB_NUMBER_THREADS and PARALLEL_MAKE have
been tested by bb-matrix.sh, e.g., by using BB_RANGE="04 08 10 12 16"
and PM_RANGE="04 08 10 12 16", then the graph that gnuplot generates by
default looks very jagged due to the missing data points. By using
splines to interpolate the missing data the graph looks a lot better.

This should not change graphs where all data points are available in any
way, only improve sparse graphs.

(From OE-Core rev: 9642c1314da64c70254f6b012aa73ef37bbaa33f)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:21:00 +01:00
Peter Kjellerstedt 643252f889 bb-matrix: Clean before, rather than after, building
This makes sure the the first build starts from a clean state. Otherwise
one could have the first build affected by any leftover state from
a previous build.

This also leaves a working state behind after the final build.

(From OE-Core rev: f8f86ac88aa1bba99ba28762cfbd97d3721da7d9)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:20:59 +01:00
Darren Hart 4287f70341 bb-matrix: Fix min and max calculations
The BB and PM ranges were originally intended to use leading 0s to
ensure all the values were the same string length, making for nice log
filenames and columnar dat files. However, not everyone will do this -
especially if it isn't documented.

Document the intent. Make the generation and parsing of dat files robust
to either method.

(From OE-Core rev: 90dc44c8246f2a580fe4a41ce28c201e52307500)

Signed-off-by: Darren Hart <dvhart@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-21 10:22:08 +00:00
Darren Hart fa4bcfdb73 bb-matrix: correct BB and PM number canonicalization
The bash string operation ${BB##*0} was greedy and in addition to converting
"02" to "2", also converted "20" to "", causing all builds for a BB value ending
in 0 to run with BB_NUMBER_THREADS=1.

(From OE-Core rev: b975de5ea76c5f8827fb48c0c3c29902872ad3d6)

Signed-off-by: Darren Hart <dvhart@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-14 22:23:09 +01:00
Darren Hart b8f6c225cd bb-matrix: initial scripts to record TIME(1) metrics for BB and PM combinations
The bb-matrix.sh script will run a bitbake command, building core-image-minimal
by default, for various combinations of BB_NUMBER_THREADS and PARALLEL_MAKE. It
records all relevant metrics of the TIME(1) command for each combination in a
data file.

The bb-matrix-plot.sh script can be used to visualize each of these metrics via
a 3d surface plot, either solid surface or wireframe with a value-map
projection on the XY plane.

(From OE-Core rev: 50fdf562ce5c41782ff1bdea43a20e769e61eb92)

Signed-off-by: Darren Hart <dvhart@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-12 15:08:21 +01:00