Although it is very rare and timing dependent, the potential exists for the
call to 'daemon' to cause what appears to be a deadlock in Asterisk during
startup. This can occur when a recursive mutex is obtained prior to the
daemon call executing. Since daemon uses fork to send the process into the
background, any threading primitives are unsafe to re-use after the call.
Implementations of pthread recursive mutexes are highly likely to store the
thread identifier of the thread that previously obtained the mutex. If
the mutex was locked prior to the fork, a subsequent unlock operation will
potentially fail as the thread identifier is no longer valid. Since the
mutex is still locked, all subsequent attempts to grab the mutex by other
threads will block.
This behavior exhibited itself most often when DEBUG_THREADS was enabled, as
this compile time option surrounds the mutexes in Asterisk with another
recursive mutex that protects the storage of thread related information. This
made it much more likely that a recursive mutex would be obtained prior to
daemon and unlocked after the call.
This patch does the following:
a) It backports a patch from Asterisk 11 that prevents the spawning of the
localtime monitoring thread. This thread is now spawned after Asterisk has
fully booted.
b) It re-orders the startup sequence to call daemon earlier during Asterisk
startup. This limits the potential of threading primitives being accessed
by initialization calls before daemon is called.
c) It removes calls to ast_verbose/ast_log/etc. prior to daemon being called.
Developers should send error messages directly to stderr prior to daemon,
as calls to ast_log may access recursive mutexes that store thread related
information.
d) It reorganizes when thread local storage is created for storing lock
information during the creation of threads. Prior to this patch, the
read/write lock protecting the list of threads in ast_register_thread would
utilize the lock in the thread local storage prior to it being initialized;
this patch prevents that.
On a very related note, this patch will *greatly* improve the stability of the
Asterisk Test Suite.
Review: https://reviewboard.asterisk.org/r/2197
(closes issue ASTERISK-19463)
Reported by: mjordan
Tested by: mjordan
........
Merged revisions 376428 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........
Merged revisions 376431 from http://svn.asterisk.org/svn/asterisk/branches/10
........
Merged revisions 376441 from http://svn.asterisk.org/svn/asterisk/branches/11
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@376447 65c4cc65-6c06-0410-ace0-fbb531ad65f3
530ce21b5d