Commit Graph

30 Commits

Author SHA1 Message Date
Lianhao Lu aca161f8a0 bitbake/persist_data: Reconnect when DB is locked
[YOCTO #1761]
Reconnect to the backend Sqlite DB in 'database is locked' exception so
the timeout can be leveraged in each time retry.

(Bitbake rev: b310382764367b573c84f33d847c6eb821266f9e)

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2012-05-31 13:16:41 -07:00
Richard Purdie f1c0c97f4d bitbake/persist_data: Sync file with upstream bitbake
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 18:07:23 +01:00
Chris Larson 754d1c6983 persist_data: implement comparison, same as dict
(Bitbake rev: 1190406c526c7bb7cf415867be83e0403812a7dd)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 18:05:11 +01:00
Chris Larson 824acff967 persist_data: don't allow non-string keys/values
(Bitbake rev: 28958cd55e592853c68f5f2ba79381d1b8dcfb8f)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 15:50:29 +01:00
Chris Larson d43e7a91f2 persist_data: add has_key
(Bitbake rev: 7942833ca0685cf4f3b243dde6203499ef97420c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 15:50:14 +01:00
Chris Larson 7fc020aa15 persist_data: raise KeyError on missing elements
(Bitbake rev: a4f62433845c29f98c6a9746d5d2847bf9506ea5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 15:49:17 +01:00
Chris Larson 88dbb0523c Initial work on getting bitbake working under pypy
- use os.chmod, not os.fchmod, as the latter is missing under pypy
- rearrange our imports a bit
- don't die if sqlite3 is missing shared cache support

(Bitbake rev: f229824dc9c453adf6067500e2bf6761536e4f2f)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 15:42:03 +01:00
Chris Larson c68c5f6e31 persist_data: kill unreachable break line
(Bitbake rev: 7486b38603f2766adaf976a9f95e9276c83abe31)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-10 22:35:12 +00:00
Chris Larson 1385162cd2 persist_data: loop on database lock for table creation
(Bitbake rev: d93fcbd64ab5d806288424170f55323b4297e7d6)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-10 22:32:12 +00:00
Chris Larson 478677fb62 persist_data: resurrect the lock wait for selects
Think this got inadvertantly dropped when switching to the new API.

(Bitbake rev: 628c5159d1151b89f2b7210c8819489e8dc9a84d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-05 00:58:23 +00:00
Chris Larson 2671bb4197 Rework the persist_data API
Rather than having to run .addDomain() and then .getValue(domain, key),
.setValue(domain, key), etc, now it just works as mappings.

As an example:

    setValue(domain, key) -> persist[domain][key] = value

It also arranges things so we can construct objects of this type using any
arbitrary filename/path for the sqlite3 database, rather than being so
tightly bound to the metadata.

(Bitbake rev: d9e8b8af308ae871efdc8ef0782be30af8c1f894)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-05 00:58:23 +00:00
Chris Larson 3069c0b258 Revert "persist_data: cache connection and use cursor"
Caching the database connection can cause serious issues if it results in
multiple processes (e.g. multiple tasks) simultaneously using the same
connection.

This reverts commit 8a6876752b90efd81d92f0947bfc9527d8260969.

(Bitbake rev: 60b9b18eafad5ac46c7cf1048d749d673c2ee0ad)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-05 00:58:23 +00:00
Chris Larson 3a2176fe73 persist_data: handle locked db for SELECT
Parallel processes interacting with the persist_data db can quite easily
explode without this.

(Bitbake rev: b3d5432cff0ff28f4c8a5bcf10efa3e383b4fd4d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Chris Larson ecc68fa4fb Switch bitbake internals to use logging directly rather than bb.msg
We use a custom Logger subclass for our loggers

This logger provides:
- 'debug' method which accepts a debug level
- 'plain' method which bypasses log formatting
- 'verbose' method which is more detail than info, but less than debug

(Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Richard Purdie fac06aaf2a bitbake/persist_data: Attempt to fix locking issues
It appears the timeout sometimes has no effect and we see database access failures. Combat
this by wrapping the execute function in all cases and retrying manually ourselves.

Thanks to Kevin Tian for help debugging this.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-20 11:24:02 +01:00
Bernhard Reutner-Fischer 726802b980 persist_data: cache connection and use cursor
Store database connection to persistent database in fetcher.

(Bitbake rev: 8a6876752b90efd81d92f0947bfc9527d8260969)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:35 +01:00
Chris Larson 1180bab54e Apply some 2to3 transforms that don't cause issues in 2.6
(Bitbake rev: d39ab776e7ceaefc8361150151cf0892dcb70d9c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson bcd9c57a96 Switch some references to moved functions
(Bitbake rev: ddea2978cb969cf1381cfee5c055de0c9e56a4e3)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson 7acc132cac Formatting cleanups
(Bitbake rev: 2caf134b43a44dad30af4fbe33033b3c58deee57)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Richard Purdie 433c2d2318 bitbake: Remove persist_data domain renaming code as it appears to expose sqlite bugs and data consistency issues
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-30 21:24:31 +01:00
Richard Purdie ef9c095334 bitbake: Make sure tables exist when renaming
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-30 18:03:51 +01:00
Richard Purdie 231b5f6784 bitbake: Add renameDomain and getKeyValues functions to the persistent data code
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-29 14:10:11 +01:00
Richard Purdie e14d7dcbee bitbake: Sync with 1.8 branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@4463 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-05-13 07:53:18 +00:00
Richard Purdie 05c49eaac2 bitbake: Update to 1.8.10 release
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3555 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-01-20 14:58:00 +00:00
Richard Purdie 6883189d5f bitbake persist_data: Fix for python 2.4
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2366 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-04 23:58:56 +00:00
Richard Purdie ce4ba3c2b7 bitbake persist_data: Fix for python 2.4
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2365 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-04 23:54:31 +00:00
Richard Purdie e33eaf13d5 bitbake persist_data: Fix module reference
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2364 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-04 22:45:37 +00:00
Richard Purdie 71068e7cee bitbake persist_data: Add own retry logic
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2363 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-04 22:12:42 +00:00
Richard Purdie 9858ddb9c5 bitbake persist_data: Use immediate connection mode
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2362 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-04 21:15:47 +00:00
Richard Purdie bfc70eb24e bitbake: Update along 1.8 branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2345 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-03 13:40:52 +00:00