This patch contains all the other misc pieces of the transition to
python3 which didn't make sense to be broken into individual patches.
(From OE-Core rev: fcd6b38bab8517d83e1ed48eef1bca9a9a190f57)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.
(From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the proxies dictionary has a proxy set to None urllib will throw an exception
instead of not using a proxy (abridged stack):
File: '/home/ross/Yocto/poky/meta/lib/oe/distro_check.py', lineno: 43, function: get_links_from_url
*** 0043: with create_socket(url,d) as sock:
0044: webpage = sock.read()
File: '/home/ross/Yocto/poky/meta/lib/oe/distro_check.py', lineno: 5, function: create_socket
0003:def create_socket(url, d):
0004: import urllib
*** 0005: socket = urllib.urlopen(url, proxies=get_proxies(d))
File: '/usr/lib/python2.7/urllib.py', lineno: 87, function: urlopen
0086: if data is None:
*** 0087: return opener.open(url)
File: '/usr/lib/python2.7/urllib.py', lineno: 203, function: open
0201: else:
0202: proxy = None
*** 0203: name = 'open_' + urltype
0204: self.type = urltype
0205: name = name.replace('-', '_')
Exception: TypeError: cannot concatenate 'str' and 'NoneType' objects
Filter out unset values so that the dictionary only has valid assignments in.
(From OE-Core rev: 2d91290ab5608dd1297d1c26ab807fc4574a8a6b)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are two ways to pass the proxies values into urllib.open: exporting
the environment variables *_proxy and passing as dictionary into the
urllib.open call. The latter is currenty used and values should not have
the '_proxy' string, so removing it. Also, ignore uppercase proxies, these
are not taken into account by the library.
Tested on a network with proxy, adding 'inherit +="distrodata"' on local.conf
with the following commands
for distrotask in distrodataall distro_checkall checklicenseall; do
bitbake universe -c $distrotask
done
[YOCTO #7567]
(From OE-Core rev: 92fb327b08e4f9cfb2400610c2983afe579b3592)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Proxies defined in the enviroment where not taken into account
on the distrodata tasks. This commit implied passing the datastore
into the distro_check library and context manager for the
urllib.urlopen function.
One way to run distrodata tasks is using 'universe' as target and the
'all' distrodata task:
$ bitbake universe -c distrodataall
$ bitbake universe -c distro_checkall
$ bitbake universe -c checklicenseall
Logs are located under TMPDIR/log
[YOCTO #7567]
(From OE-Core rev: 7d1c3470bb06e43245ccb7f067121de606506430)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the distro check functions for the change of nativesdk
being a suffix to a prefix. Also added crosssdk as another
case for converting to PN for matching in the distro_tracking
(From OE-Core rev: ae9dbd0e1e26ba2b35cbd08ec731aee62adedc23)
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace os.system with subprocess.call since the older function would
fail (more or less) silently if the executed program cannot be found
More info:
http://docs.python.org/library/subprocess.html#subprocess-replacements
[YOCTO #2454]
(From OE-Core rev: a07d03cc6f67c88feb9813ae7deb6e4a93552dfe)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Complete the bb.data.getVar/setVar replacements with accesses
directly to the data store object.
(From OE-Core rev: 2864ff6a4b3c3f9b3bbb6d2597243cc5d3715939)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes a problem with package names from inherits showing up
(From OE-Core rev: edb2a11994ac21e790155ea519bded4b37ef9307)
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
distro_check.py: Create a new function called create_log_file to reduce a lot of repeat code in distrodata.bbclass.
We needn't to create log file in function save_distro_check_result, because the log file has been generated in check_eventhandler.
Another bug is that we maybe access the /tmp/Meego-1.0 before we create this file.
Add a judge statement to decide whether we need to create this file firstly.
distrodata.bbclass: Add a new task checklicense to collect missing text license information.
This can help package-report system to know how many recipes are missing license text.
(From OE-Core rev: b41148cda9f0cc292b662a8473f26bc1ee0148f3)
Signed-off-by: Mei Lei <lei.mei@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recipe meta-ide-support's alias is Poky, but not defined in distro_exceptions, this will lead an error.
The compare_in_distro_packages_list function omit a split process, this will lead to a form issue.
Change the log file generate way, use symbolic links link to the latest log file, instead of appending every log file in old log.
(From OE-Core rev: 123a7f49753aef4d5ccb5f5a5590c3c88775c6d0)
Signed-off-by: Mei Lei <lei.mei@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
distro_check.py: Add DISTRO_PN_ALIAS check for Poky, OpenedHand,
Intel and Upstream, to mark packages that are
not in known upstreams
distro_tracking_fields.inc:
Merged private files into 1 master file
Added additional Maintainer Info
Added Distro Alias information for Poky and OpenedHand
Signed-off-by: Saul Wold <Saul.Wold@intel.com>
datetime checking is changed to be once per day
Signed-off-by: Saul Wold <Saul.Wold@intel.com>
distro_check: fix for natives, cross, and initial recipe types
Signed-off-by: Saul Wold <Saul.Wold@intel.com>
This adds a new task (distro_check) for each recipe.
The task generates the source package list for Fedora OpenSuSE
Ubuntu Debian & Mandriva Linux distros.
As one recipe or source package can generate multiple target packages
the recipe name is compared with the source package name list of LInux
distributions.
Thread locking is used to avoid multiple threads racing for the
package list update.
Then the recipe name (PN) is checked if it exists in the package
list of distros. And if the DISTRO_PN_ALIAS then it is used to copmare
pacakge_name instead of the PN variable. Just for example the
DISTRO_PN_ALIAS can be defined in the recipe (.bb) files like this
In the file xset_1.0.4.bb:
DISTRO_PN_ALIAS = "Fedora=xorg-x11-server-utils;\
Ubuntu=x11-xserver-utils; Debian=x11-xserver-utils;Opensuse=xorg-x11"
The final results are stored in the tmp/log/distro_check-${DATETIME}.result
file.
FYI this command will generate the results for all recipies:
bitbake world -f -c distro_check
Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com>
Richard Purdie
Ross Burton
Leonardo Sandoval
Anders Roxell
Saul Wold
Robert Yang
Mei Lei
Saul Wold
Kevin Tian
Nitin A Kamble