In the AO2_ALLOC_OPT_LOCK_NOLOCK case the referenced obj
structure is freed, but is then referenced later if ref_log is
enabled. The change is to store the obj->priv_data.options value
locally and reference it instead of the value from the freed obj
ASTERISK-29730
Change-Id: I60cc5dc1f5a4330e7ad56976fc38a42de0ab6072
Previous to this patch passing a NULL tag to ao2_alloc or ao2_ref based
functions would result in the reference not being logged under
REF_DEBUG. This could sometimes cause inaccurate logging if NULL was
accidentally passed to a reference action. Now reference logging is
only disabled by option passed to the allocation method.
Change-Id: I3c17d867d901d53f9fcd512bef4d52e342637b54
astobj2.c declares DEBUG_THREADS_LOOSE_ABI to avoid overhead of debug
threads tracking information in the internal structures of astobj2.
Unfortunately this means that ao2_global_obj contains the statically
allocated debug threads tracking fields which are used by initialization
and cleanup but main/astobj2.c believed those fields and associated
space did not exist.
Change-Id: Icef41ad97d88a8c1d1515e034ec8133cab3b1527
This allows us to process AO2 statistics for total objects, memory
usage, memory overhead and lock usage.
* Install refstats.py and reflocks.py into the Asterisk scripts folder.
* Enable support for reflocks.py without DEBUG_THREADS.
Steal a bit from the ao2 magic to flag when an object lock is used.
Remove 'lockobj' from reflocks.py since we can now record 'used' or
'unused' for those objects.
Add comments to explain thread safety of the 'struct __priv_data'
bitfields.
Change-Id: I84e9d679cc86d772cc97c888d9d856a17e0d3a4a
When DEBUG_THREADS is enabled we can know if the astobj2 mutex / rwlock
was ever used, so it can be recorded in the REF_DEBUG destructor entry.
Create contrib/scripts/reflocks.py to process locking used by
allocator. This can be used to identify places where
AO2_ALLOC_OPT_LOCK_NOLOCK should be used to reduce memory usage.
Change-Id: I2e3cd23336a97df2692b545f548fd79b14b53bf4
Add a volatile flag to lock tracking structures so we only need to use
the global lock when first initializing tracking.
Additionally add support for DEBUG_THREADS_LOOSE_ABI. This is used by
astobj2.c to eliminate storage for tracking fields when DEBUG_THREADS is
not defined.
Change-Id: Iabd650908901843e9fff47ef1c539f0e1b8cb13b
Reduce options to 2-bit field, magic to 30 bit field. Move ref_counter
next to options and explicitly use int32_t so the fields will pack.
This reduces memory overhead for every ao2 object by 8 bytes on x86_64.
Change-Id: Idc1baabb35ec3b3d8de463c4fa3011eaf7fcafb5
When REF_DEBUG and AO2_DEBUG are both enabled we closed the refs log
before we shutdown astobj2_container. This caused the AO2_DEBUG
container registration container to be reported as a leak.
Change-Id: If9111c4c21c68064b22c546d5d7a41fac430430e
Fix instances of:
* Retreive
* Recieve
* other then
* different then
* Repeated words ("the the", "an an", "and and", etc).
* othterwise, teh
ASTERISK-24198 #close
Change-Id: I3809a9c113b92fd9d0d9f9bac98e9c66dc8b2d31
Copy the list of weakproxy callbacks to temporary memory so they can be
run without holding the weakproxy lock.
Change-Id: Ib167622a8a0f873fd73938f7611b2a5914308047
The ao2_global_obj_release() function holds an exclusive lock on the
global object while it is being dereferenced. Any destructors that
run during this time that call ao2_global_obj_ref() will deadlock
because a read lock is required.
Instead, we make the global object inaccessible inside of the write
lock and only dereference it once we have released the lock. This
allows the affected destructors to fail gracefully.
While this doesn't completely solve the referenced issue (the error
message about not being able to create an IQ continues to be shown)
it does solve the backtrace spew that accompanied it.
ASTERISK-21009 #close
Reported by: Marcello Ceschia
Change-Id: Idf40ae136b5070dba22cb576ea8414fbc9939385
Every ao2 object contains storage for a private variable data_size,
though the value is never read if AO2_DEBUG is disabled. This change
makes the variable conditional, reducing memory usage.
ASTERISK-26524 #close
Change-Id: If859929e507676ebc58b0f84247a4231e11da07f
ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes
all traces of it.
Previously exported symbols removed:
* __ast_register_file
* __ast_unregister_file
* ast_complete_source_filename
This also removes the mtx_prof static variable that was declared when
MTX_PROFILE was enabled. This variable was only used in lock.c so it
is now initialized in that file only.
ASTERISK-26480 #close
Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966
* Compile __ast_assert_failed unconditionally.
* Use __ast_assert_failed to log messages from log_bad_ao2
* Remove calls to ast_assert(0) that happen after log_bad_ao2 was run.
Change-Id: I48f1af44b2718ad74a421ff75cb6397b924a9751
Allocator functions that take file/line/func parameters are prefixed
with single-underscore when MALLOC_DEBUG is not defined,
double-underscore when it is defined. This change updates all
allocators that accept file/line/func to have the same prototype in
either ABI mode. The parameter order of __ast_vasprintf and
__ast_asprintf in utils.h have been changed to match that of astmm.h.
End-use allocator macro's have been removed from astmm.h and moved to an
unconditional part of utils.h.
Change-Id: I823bb6ce2b5675b3a4735948f10a3b420e9a023a
Modify ao2_weakproxy_subscribe so each new subscription is added
to the head of the list. This ensures that when other objects
are allocated and use a subscription to the weakproxy for cleanup,
cleanup will occur in the correct order.
ASTERISK-25120 #close
Change-Id: Ie0476f08ec21330de1b3f5a2dd3d9eb683df3d3d
This ensures that refdebug is initialized before AO2_DEBUG if
both are enabled, since AO2_DEBUG allocates a container.
This change also makes AO2_DEBUG initialization critical, a
failure will abort Asterisk startup. This is needed since
the failure would be caused by reg_containers allocation
failure, and that would result in a segmentation fault by
ao2_container_register later in startup.
ASTERISK-25048 #close
Reported by: Corey Farrell
Change-Id: I9a243ea3fc5653b48b931ba6d61971cb2e530244
This function allows code to run ao2_ref against the real
object associated with a weakproxy. It is useful when
all of the following conditions are true:
* You have a pointer to weakproxy.
* You do not have or need a pointer to the real object.
* You need to ensure the real object exists and is not
destroyed during a process.
In this case it's wasteful to store a pointer to the real
object just for the sake of releasing it later.
Change-Id: I38a319b83314de75be74207a8771aab269bcca46
* The REF_DEBUG compiler flag no longer has any effect on code that uses
Astobj2. It is used to determine if reference debugging is enabled by
default. Reference debugging can be enabled or disabled in asterisk.conf.
* Caller information is provided in logger errors for ao2 bad magic numbers.
* Optimizes AO2 by merging internal functions with the public counterpart.
This was possible now that we no longer require a dual ABI.
ASTERISK-24974 #close
Reported by: Corey Farrell
Change-Id: Icf3552721fe999365ba8a8cf00a965aa6b897cc1
This implements "weak" references. The weakproxy object is a real ao2 with
normal reference counting of its own. When a weakproxy is pointed to a normal
object they hold references to each other. The normal object is automatically
freed when a single reference remains (the weakproxy). The weakproxy also
supports subscriptions that will notify callbacks when it does not point
to any real object.
ASTERISK-24936 #close
Reported by: Corey Farrell
Change-Id: Ib9f73c02262488d314d9d9d62f58165b9ec43c67
Git does not support the ability to replace a token with a version
string during check-in. While it does have support for replacing a
token on clone, this is somewhat sub-optimal: the token is replaced
with the object hash, which is not particularly easy for human
consumption. What's more, in practice, the source file version was often
not terribly useful. Generally, when triaging bugs, the overall version
of Asterisk is far more useful than an individual SVN version of a file. As a
result, this patch removes Asterisk's support for showing source file
versions.
Specifically, it does the following:
* Rename ASTERISK_FILE_VERSION macro to ASTERISK_REGISTER_FILE, and
remove passing the version in with the macro. Other facilities
than 'core show file version' make use of the file names, such as
setting a debug level only on a specific file. As such, the act of
registering source files with the Asterisk core still has use. The
macro rename now reflects the new macro purpose.
* main/asterisk:
- Refactor the file_version structure to reflect that it no longer
tracks a version field.
- Remove the "core show file version" CLI command. Without the file
version, it is no longer useful.
- Remove the ast_file_version_find function. The file version is no
longer tracked.
- Rename ast_register_file_version/ast_unregister_file_version to
ast_register_file/ast_unregister_file, respectively.
* main/manager: Remove value from the Version key of the ModuleCheck
Action. The actual key itself has not been removed, as doing so would
absolutely constitute a backwards incompatible change. However, since
the file version is no longer tracked, there is no need to attempt to
include it in the Version key.
* UPGRADE: Add notes for:
- Modification to the ModuleCheck AMI Action
- Removal of the "core show file version" CLI command
Change-Id: I6cf0ff280e1668bf4957dc21f32a5ff43444a40e
This introduces a new logger routine ast_log_safe. This routine should be
used for all error messages in code that can be run as a result of ast_log.
ast_log_safe does nothing if run recursively. All error logging in
astobj2.c, strings.c and utils.h have been switched to ast_log_safe.
This required adding support for raw threadstorage. This provides direct
access to the void* pointer in threadstorage. In ast_log_safe, NULL is used
to signify that this thread is not already running ast_log_safe, (void*)1 when
it is already running. This was done since it's critical that ast_log_safe
do nothing that could log during recursion checking.
ASTERISK-24155 #close
Reported by: Timo Teräs
Review: https://reviewboard.asterisk.org/r/4502/
........
Merged revisions 433522 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 433523 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433524 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Since 'core stop now' and 'core restart now' do not stop modules,
it is unsafe for most of the core to run cleanups. Originally all
cleanups used ast_register_atexit, and were only changed when it
was shown to be unsafe. ast_register_atexit is now used only when
absolutely required to prevent corruption and close child processes.
Exceptions that need to use ast_register_atexit:
* CDR: Flush records.
* res_musiconhold: Kill external applications.
* AstDB: Close the DB.
* canary_exit: Kill canary process.
ASTERISK-24142 #close
Reported by: David Brillert
ASTERISK-24683 #close
Reported by: Peter Katzmann
ASTERISK-24805 #close
Reported by: Badalian Vyacheslav
ASTERISK-24881 #close
Reported by: Corey Farrell
Review: https://reviewboard.asterisk.org/r/4500/
Review: https://reviewboard.asterisk.org/r/4501/
........
Merged revisions 433495 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 433497 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433498 65c4cc65-6c06-0410-ace0-fbb531ad65f3
If a reference count goes negative, instead of
just logging that fact, be more helpful with a
backtrace and an assert that will DO_CRASH.
This patch also removes the duplicate ao2_bt()
function and cleans up extraneous usage of the
ast_log_backtrace() call.
Review: https://reviewboard.asterisk.org/r/3765/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@418963 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This patch does the following:
(1) It makes REF_DEBUG a meneselect item. Enabling REF_DEBUG now enables
REF_DEBUG globally throughout Asterisk.
(2) The ref debug log file is now created in the AST_LOG_DIR directory.
Every run will now blow away the previous run (as large ref files
sometimes caused issues). We now also no longer open/close the file
on each write, instead relying on fflush to make sure data gets written
to the file (in case the ao2 call being performed is about to cause a
crash)
(3) It goes with a comma delineated format for the ref debug file. This
makes parsing much easier. This also now includes the thread ID of the
thread that caused ref change.
(4) A new python script instead for refcounting has been added in the
contrib/scripts folder.
(5) The old refcounter implementation in utils/ has been removed.
Review: https://reviewboard.asterisk.org/r/3377/
........
Merged revisions 412114 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........
Merged revisions 412115 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 412153 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@412154 65c4cc65-6c06-0410-ace0-fbb531ad65f3
........
r399887 | dlee | 2013-09-26 10:41:47 -0500 (Thu, 26 Sep 2013) | 1 line
Minor performance bump by not allocate manager variable struct if we don't need it
........
r400138 | dlee | 2013-09-30 10:24:00 -0500 (Mon, 30 Sep 2013) | 23 lines
Stasis performance improvements
This patch addresses several performance problems that were found in
the initial performance testing of Asterisk 12.
The Stasis dispatch object was allocated as an AO2 object, even though
it has a very confined lifecycle. This was replaced with a straight
ast_malloc().
The Stasis message router was spending an inordinate amount of time
searching hash tables. In this case, most of our routers had 6 or
fewer routes in them to begin with. This was replaced with an array
that's searched linearly for the route.
We more heavily rely on AO2 objects in Asterisk 12, and the memset()
in ao2_ref() actually became noticeable on the profile. This was
#ifdef'ed to only run when AO2_DEBUG was enabled.
After being misled by an erroneous comment in taskprocessor.c during
profiling, the wrong comment was removed.
Review: https://reviewboard.asterisk.org/r/2873/
........
r400178 | dlee | 2013-09-30 13:26:27 -0500 (Mon, 30 Sep 2013) | 24 lines
Taskprocessor optimization; switch Stasis to use taskprocessors
This patch optimizes taskprocessor to use a semaphore for signaling,
which the OS can do a better job at managing contention and waiting
that we can with a mutex and condition.
The taskprocessor execution was also slightly optimized to reduce the
number of locks taken.
The only observable difference in the taskprocessor implementation is
that when the final reference to the taskprocessor goes away, it will
execute all tasks to completion instead of discarding the unexecuted
tasks.
For systems where unnamed semaphores are not supported, a really
simple semaphore implementation is provided. (Which gives identical
performance as the original taskprocessor implementation).
The way we ended up implementing Stasis caused the threadpool to be a
burden instead of a boost to performance. This was switched to just
use taskprocessors directly for subscriptions.
Review: https://reviewboard.asterisk.org/r/2881/
........
r400180 | dlee | 2013-09-30 13:39:34 -0500 (Mon, 30 Sep 2013) | 28 lines
Optimize how Stasis forwards are dispatched
This patch optimizes how forwards are dispatched in Stasis.
Originally, forwards were dispatched as subscriptions that are invoked
on the publishing thread. This did not account for the vast number of
forwards we would end up having in the system, and the amount of work it
would take to walk though the forward subscriptions.
This patch modifies Stasis so that rather than walking the tree of
forwards on every dispatch, when forwards and subscriptions are changed,
the subscriber list for every topic in the tree is changed.
This has a couple of benefits. First, this reduces the workload of
dispatching messages. It also reduces contention when dispatching to
different topics that happen to forward to the same aggregation topic
(as happens with all of the channel, bridge and endpoint topics).
Since forwards are no longer subscriptions, the bulk of this patch is
simply changing stasis_subscription objects to stasis_forward objects
(which, admittedly, I should have done in the first place.)
Since this required me to yet again put in a growing array, I finally
abstracted that out into a set of ast_vector macros in
asterisk/vector.h.
Review: https://reviewboard.asterisk.org/r/2883/
........
r400181 | dlee | 2013-09-30 13:48:57 -0500 (Mon, 30 Sep 2013) | 28 lines
Remove dispatch object allocation from Stasis publishing
While looking for areas for performance improvement, I realized that an
unused feature in Stasis was negatively impacting performance.
When a message is sent to a subscriber, a dispatch object is allocated
for the dispatch, containing the topic the message was published to, the
subscriber the message is being sent to, and the message itself.
The topic is actually unused by any subscriber in Asterisk today. And
the subscriber is associated with the taskprocessor the message is being
dispatched to.
First, this patch removes the unused topic parameter from Stasis
subscription callbacks.
Second, this patch introduces the concept of taskprocessor local data,
data that may be set on a taskprocessor and provided along with the data
pointer when a task is pushed using the ast_taskprocessor_push_local()
call. This allows the task to have both data specific to that
taskprocessor, in addition to data specific to that invocation.
With those two changes, the dispatch object can be removed completely,
and the message is simply refcounted and sent directly to the
taskprocessor.
Review: https://reviewboard.asterisk.org/r/2884/
........
Merged revisions 399887,400138,400178,400180-400181 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@400186 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Review https://reviewboard.asterisk.org/r/2580/ tried to fix the mismatch
in memory pools but had a math error determining the buffer size and
didn't address other similar memory pool mismatches.
* Effectively reverted the previous patch to go in the same direction as
trunk for the returned memory pool of ast_bt_get_symbols().
* Fixed memory leak in ast_bt_get_symbols() when BETTER_BACKTRACES is
defined.
* Fixed some formatting in ast_bt_get_symbols().
* Fixed sig_pri.c freeing memory allocated by libpri when MALLOC_DEBUG is
enabled.
* Fixed __dump_backtrace() freeing memory from ast_bt_get_symbols() when
MALLOC_DEBUG is enabled.
* Moved __dump_backtrace() because of compile issues with the utils
directory.
(closes issue ASTERISK-22221)
Reported by: Matt Jordan
Review: https://reviewboard.asterisk.org/r/2778/
........
Merged revisions 397525 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........
Merged revisions 397528 from http://svn.asterisk.org/svn/asterisk/branches/11
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397570 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This moves ast_str_container_alloc, ast_str_container_add,
ast_str_container_remove, and related private functions into
strings.c/h since they really don't belong in astobj2.c/h.
As a result of this move, utils also had to be updated.
Review: https://reviewboard.asterisk.org/r/2719/
(closes issue ASTERISK-22041)
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396105 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This patch allows astmm to access the backtrace generation code in Asterisk.
When memory is allocated, a backtrace is created and stored with the memory
region that tracks the allocation. If a memory corruption is detected, the
backtrace is printed to the astmm log. The backtrace will make use of the
BETTER_BACKTRACES build option if available.
As a result, this patch moves the backtrace generation code into its own file
and uses the non-wrapped versions of the C library memory allocation routines.
This allows the memory allocation code to safely use the backtrace generation
routines without infinitely recursing.
Review: https://reviewboard.asterisk.org/r/2567
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@391012 65c4cc65-6c06-0410-ace0-fbb531ad65f3
The debug versions of ao2_ref() should only be used if REF_DEBUG is
enabled so nothing is written to /tmp/refs unexpectedly.
(closes issue ASTERISK-21785)
Reported by: abelbeck
Patches:
jira_asterisk_21785_v11.patch (license #5621) patch uploaded by rmudgett
Tested by: abelbeck
........
Merged revisions 388700 from http://svn.asterisk.org/svn/asterisk/branches/11
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@388701 65c4cc65-6c06-0410-ace0-fbb531ad65f3
An endpoint is an external device/system that may offer/accept
channels to/from Asterisk. While this is a very useful concept for end
users, it is surprisingly not a core concept within Asterisk itself.
This patch defines ast_endpoint as a separate object, which channel
drivers may use to expose their concept of an endpoint. As the channel
driver creates channels, it can use ast_endpoint_add_channel() to
associate channels to the endpoint. This updated the endpoint
appropriately, and forwards all of the channel's events to the
endpoint's topic.
In order to avoid excessive locking on the endpoint object itself, the
mutable state is not accessible via getters. Instead, you can create a
snapshot using ast_endpoint_snapshot_create() to get a consistent
snapshot of the internal state.
This patch also includes a set of topics and messages associated with
endpoints, and implementations of the endpoint-related RESTful
API. chan_sip was updated to create endpoints with SIP peers, but the
state of the endpoints is not updated with the state of the peer.
Along for the ride in this patch is a Stasis test API. This is a
stasis_message_sink object, which can be subscribed to a Stasis
topic. It has functions for blocking while waiting for conditions in
the message sink to be fulfilled.
(closes issue ASTERISK-21421)
Review: https://reviewboard.asterisk.org/r/2492/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@387932 65c4cc65-6c06-0410-ace0-fbb531ad65f3
The pimp_my_sip branch is being merged at this point because
it offers basic functionality, and from an API standpoint, things
are complete.
SIP work is *not* feature-complete; however, with the completion
of the SUBSCRIBE/NOTIFY API, all APIs (except a PUBLISH API) have
been created, and thus it is possible for developers to attempt
to create new SIP work.
API documentation can be found in the doxygen in the code, but
usability documentation is still lacking.
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386540 65c4cc65-6c06-0410-ace0-fbb531ad65f3
OBJ_PARTIAL_KEY searching a rbtree did not find all possible matches if
the container did not accept duplicates.
Added matching node bias to indicate which matching node is being searched
for: first, last, any.
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@384616 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Alexander Traud
Mike Bradeen
George Joseph
Corey Farrell
Jenkins2
Sean Bright
Richard Mudgett
Matt Jordan
Scott Griepentrog
George Joseph
Kinsey Moore
Jonathan Rose
David M. Lee
Mark Michelson