When a topic is created for an object, its name is only
<object>:<uniqueid>
For example:
bridge:cb68b3a8-fce7-4738-8a17-d7847562f020
When a topic is added to a pool, its name has the pool's topic
name prepended. For example:
bridge:all/bridge:cb68b3a8-fce7-4738-8a17-d7847562f020
The topic_pool_entry's name however, is only what was passed
in to stasis_topic_pool_get_topic which is
bridge:cb68b3a8-fce7-4738-8a17-d7847562f020
That's actually correct because the entry is qualified by the
pool that's in.
When you're ready to delete the entry from the pool, you retrieve
the tropic name from the object but since it now has the pool's
topic name prepended, it won't be found in the pool container.
Fix:
* Modified stasis_topic_pool_delete_topic() to skip past the
pool topic's name, if it was prepended to the topic name,
before searching the container for a pool entry.
ASTERISK-28633
Reported by: Joeran Vinzens
Change-Id: I4396aa69dd83e4ab84c5b91b39293cfdbcf483e6
This patch fixes several issues reported by the lgtm code analysis tool:
https://lgtm.com/projects/g/asterisk/asterisk
Not all reported issues were addressed in this patch. This patch mostly fixes
confirmed reported errors, potential problematic code points, and a few other
"low hanging" warnings or recommendations found in core supported modules.
These include, but are not limited to the following:
* innapropriate stack allocation in loops
* buffer overflows
* variable declaration "hiding" another variable declaration
* comparisons results that are always the same
* ambiguously signed bit-field members
* missing header guards
Change-Id: Id4a881686605d26c94ab5409bc70fcc21efacc25
This avoids use of the global variable and ensures topic_all remains
active until all topics are freed.
ASTERISK-28553
patches:
ASTERISK-28553.patch by coreyfarrell (license 5909)
Change-Id: I9a8cd8977f3c3a6aa00783f8336d2cfb9c2820f1
It is possible for topic->name to be NULL, this causes the allocation
reference to not be logged. Use the name variable instead which has
been verified to be a non-empty.
Change-Id: I3d0031d03c8356e4808f00cdf2d5428712575883
When compiling in dev mode, stasis statistics are enabled and can cause
a crash at shutdown due to the following:
- Containers are freed
- Topics and subscriptions remain
- When those topics and subscriptions are deallocated, they go to do
things with the container
This changes the containers to global ao2 objects, and whenever needed
in the code, a reference must be obtained and checked before any
operations can be done.
ASTERISK-28353 #close
Change-Id: Ie7d5e907fcfcb4d65bd36d5e4eb923126fde8d33
Currently, the "stasis show app" cli doesn't give detail
of subscription/subscriber information.
Added more printings to show details.
ASTERISK-28378
Change-Id: If25a6f14fe4f622bfb37462e891333da1fdf875f
Added topic_all container for centralizing the topic. This makes more
easier to managing the topics.
Added cli commands.
stasis show topics : It shows all registered topics.
stasis show topic <name> : It shows speicifed topic's detail info.
ASTERISK-28264
Change-Id: Ie86d125d2966f93de74ee00f47ae6fbc8c081c5f
As part of an earlier voicemail refactor, ast_delete_mwi_state_full
was modified to remove the pool topic for a mailbox when the state
was deleted. This was an attempt to prevent stale topics from
accumulating when app_voicemail was reloaded and a mailbox went
away. Unfortunately because of the fact that when app_voicemail
reloads, ALL mailboxes are deleted then only current ones recreated,
topics were being removed from the pool that still had subscribers
on them, then recreated as new topics of the same name. So now
modules like res_pjsip_mwi are listening on a topic that will
never receive any messages because app_voicemail is publishing on
a different topic that happens to have the same name. The solutiuon
to this is not easy and given that accumulating topics for
deleted mailboxes is less evil that not sending NOTIFYs...
* Removed the call to stasis_topic_pool_delete_topic in
ast_delete_mwi_state_full.
Also:
* Fixed a topic reference leak in res_pjsip_mwi
mwi_stasis_subscription_alloc.
* Added some debugging to mwi_stasis_subscription_alloc,
stasis_topic_create, and topic_dtor.
* Fixed a topic reference leak in an error path in
internal_stasis_subscribe.
ASTERISK-28306
Reported-by: Jared Hull
Change-Id: Id7da0990b3ac4be4b58491536b35f41291247b27
Topic names now follow: <subsystem>:<functionality>[/<object>]
This ensures that they are all unique, and also provides better
insight in to what each topic is for.
Subscriber ids now also use the main topic name they are
subscribed to and an incrementing integer as their identifier to
make it easier to understand what the subscription is primarily
responsible for.
Both the CLI commands for listing topic and subscription statistics
now sort to make it a bit easier to see what is going on.
Subscriptions will now show all topics that they are receiving messages
from, not just the main topic they were subscribed to.
ASTERISK-28335
Change-Id: I484e971a38c3640f2bd156282e532eed84bf220d
This change provides an easier mechanism to determine which
subscribers are subscribed to a topic. Using this you can
inspect the specific subscribers for further details.
Change-Id: I8deea21703cd5c5357b85593b46c3eaf24e18c0c
To prevent one subsystem's taskprocessors from causing others
to stall, new capabilities have been added to taskprocessors.
* Any taskprocessor name that has a '/' will have the part
before the '/' saved as its "subsystem".
Examples:
"sorcery/acl-0000006a" and "sorcery/aor-00000019"
will be grouped to subsystem "sorcery".
"pjsip/distributor-00000025" and "pjsip/distributor-00000026"
will bn grouped to subsystem "pjsip".
Taskprocessors with no '/' have an empty subsystem.
* When a taskprocessor enters high-water alert status and it
has a non-empty subsystem, the subsystem alert count will
be incremented.
* When a taskprocessor leaves high-water alert status and it
has a non-empty subsystem, the subsystem alert count will be
decremented.
* A new api ast_taskprocessor_get_subsystem_alert() has been
added that returns the number of taskprocessors in alert for
the subsystem.
* A new CLI command "core show taskprocessor alerted subsystems"
has been added.
* A new unit test was addded.
REMINDER: The taskprocessor code itself doesn't take any action
based on high-water alerts or overloading. It's up to taskprocessor
users to check and take action themselves. Currently only the pjsip
distributor does this.
* A new pjsip/global option "taskprocessor_overload_trigger"
has been added that allows the user to select the trigger
mechanism the distributor uses to pause accepting new requests.
"none": Don't pause on any overload condition.
"global": Pause on ANY taskprocessor overload (the default and
current behavior)
"pjsip_only": Pause only on pjsip taskprocessor overloads.
* The core pjsip pool was renamed from "SIP" to "pjsip" so it can
be properly grouped into the "pjsip" subsystem.
* stasis taskprocessor names were changed to "stasis" as the
subsystem.
* Sorcery core taskprocessor names were changed to "sorcery" to
match the object taskprocessors.
Change-Id: I8c19068bb2fc26610a9f0b8624bdf577a04fcd56
An int64_t is not likely the same size as a long.
* Changed the int64_t values in the statistics structs to longs so casting
is not necessary when generating the formatted CLI output. The offending
members did not need to be int64_t anyway as they were only set by an int
type variable which was already truncating bits.
* Reordered the statistics structs to reduce potential padding bytes.
Change-Id: Ic090a070e9dc4ca650ebdb9c01ed50a581289962
This change adds statistics gathering to Stasis topics,
subscriptions, and message types. These can be viewed using
CLI commands and provide insight into how Stasis is used
and how long certain operations take to execute.
These are only available when Asterisk is compiled in
developer mode and do not have any impact under normal
operation.
ASTERISK-28117
Change-Id: I94411b53767f89ee01714daaecf0c2f1666e863f
A subscriber can now indicate that it only wants messages
that have formatters of a specific type. For instance,
manager can indicate that it only wants messages that have a
"to_ami" formatter. You can combine this with the existing
filter for message type to get only messages with specific
formatters or messages of specific types.
ASTERISK-28186
Change-Id: Ifdb7a222a73b6b56c6bb9e4ee93dc8a394a5494c
Replace usage of ao2_container_alloc with ao2_container_alloc_hash or
ao2_container_alloc_list. Remove ao2_container_alloc macro.
Change-Id: I0907d78bc66efc775672df37c8faad00f2f6c088
When a subscribe or unsubscribe occurs a message is published
containing this information. This change makes it so that the
message no longer uses stringfields or a lock, as both are not
really needed for the message.
Change-Id: I3f4831931d79f94fd979baf48048738df5dc1632
This change adds the ability for subscriptions to indicate
which message types they are interested in accepting. By
doing so the filtering is done before being dispatched
to the subscriber, reducing the amount of work that has
to be done.
This is optional and if a subscriber does not add
message types they wish to accept and set the subscription
to selective filtering the previous behavior is preserved
and they receive all messages.
There is also the ability to explicitly force the reception
of all messages for cases such as AMI or ARI where a large
number of messages are expected that are then generically
converted into a different format.
ASTERISK-28103
Change-Id: I99bee23895baa0a117985d51683f7963b77aa190
Add attribute_warn_unused_result to ast_taskprocessor_push,
ast_taskprocessor_push_local and ast_threadpool_push. This will help
ensure we perform the necessary cleanup upon failure.
Change-Id: I7e4079bd7b21cfe52fb431ea79e41314520c3f6d
* Use "o*" format specifier for optional fields in ast_json_party_id.
* Stop using ast_json_deep_copy on immutable objects, it is now thread
safe to just use ast_json_ref.
Additional changes to ast_json_pack calls in the vicinity:
* Use "O" when an object needs to be bumped. This was previously
avoided as it was not thread safe.
* Use "o?" and "O?" to replace NULL with ast_json_null(). The
"?" is a new feature of ast_json_pack starting with Asterisk 16.
Change-Id: I8382d28d7d83ee0ce13334e51ae45dbc0bdaef48
There's been a long standing leak when using topic pools. The
topics in the pool get cleaned up when the last pool reference is
released but you can't remove a topic specifically. If you reloaded
app_voicemail for instance, and mailboxes went away, their topics
were left in the pool.
* Added stasis_topic_pool_delete_topic() so modules can clean up
topics from pools.
* Registered the topic pool containers so it can be examined from
the CLI when AO2_DEBUG is enabled. They'll be named
"<topic_pool_name>-pool".
Change-Id: Ib7957951ee5c9b9b4482af7b9b4349112d62bc25
When publishing a device state the change can be marked as being
cachable or not. If it is not cached the change is just published
to all interested and not stored away for later query. This was not
fully taken into account when publishing in stasis. The act of
publishing would create a topic for the device even if it may be
ephemeral.
This change makes it so messages which are not cached won't create
a topic for the device. If a topic does already exist it will be
published to but otherwise the change will only be published to
the device state all topic.
ASTERISK-27591
Change-Id: I18da0e8cbb18e79602e731020c46ba4101e59f0a
Fixes issue where error msg
"Use of before/init after destruction"
was being printed on disabled messages
in dev mode. With this
fix if message is disabled
a warning will print.
ASTERISK-25548
Change-Id: Ie0d866d1cbc60c16dbef08bc65e99505c3c1adfa
Remove nearly all use of regex from ACO users. Still remaining:
* app_confbridge has a legitamate use of option name regex.
* ast_sorcery_object_fields_register is implemented with regex, all
callers use simple prefix based regex. I haven't decided the best
way to fix this in both 13/15 and master.
Change-Id: Ib5ed478218d8a661ace4d2eaaea98b59a897974b
When (v)asprintf() fails, the state of the allocated buffer is undefined.
The library had better not leave an allocated buffer as a result or no one
will know to free it. The most likely way it can return failure is for an
allocation failure. If the printf conversion fails then you actually have
a threading problem which is much worse because another thread modified
the parameter values.
* Made __ast_asprintf()/__ast_vasprintf() set the returned buffer to NULL
on failure. That is much more useful than either an uninitialized pointer
or a pointer that has already been freed. Many uses won't have to check
for failure to ensure that the buffer won't be double freed or prevent an
attempt to free an uninitialized pointer.
* stasis.c: Fixed memory leak in multi_object_blob_to_ami() allocated by
ast_asprintf().
* ari/resource_bridges.c:ari_bridges_play_helper(): Remove assignment to
the wrong thing which is now not needed even if assigning to the right
thing.
Change-Id: Ib5252fb8850ecf0f78ed0ee2ca0796bda7e91c23
ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes
all traces of it.
Previously exported symbols removed:
* __ast_register_file
* __ast_unregister_file
* ast_complete_source_filename
This also removes the mtx_prof static variable that was declared when
MTX_PROFILE was enabled. This variable was only used in lock.c so it
is now initialized in that file only.
ASTERISK-26480 #close
Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966
Stasis subscriptions and message routers create taskprocessors to process
the event messages. API calls are needed to be able to set the congestion
levels of these taskprocessors for selected subscriptions and message
routers.
* Updated CDR, CEL, and manager's stasis subscription congestion levels
based upon stress testing. Increased the congestion levels to reduce the
potential for bursty call setup/teardown activity from triggering the
taskprocessor overload alert. CDRs in particular need an extra high
congestion level because they can take awhile to process the stasis
messages.
ASTERISK-26088
Reported by: Richard Mudgett
Change-Id: Id0a716394b4eee746dd158acc63d703902450244
The JSON library Asterisk uses, jansson, is not thread
safe for us in a few ways. To help with this wrappers for JSON
object reference count increasing and decreasing were added
which use a global lock to ensure they don't clobber over
each other. This does not extend to reference count manipulation
within the jansson library itself. This means you can't safely
use the object borrowing specifier (O) in ast_json_pack and
you can't share JSON instances between objects.
This change removes uses of the O specifier and replaces them
with the o specifier and an explicit ast_json_ref. Some cases
of instance sharing have also been removed.
ASTERISK-25601 #close
Change-Id: I06550d8b0cc1bfeb56cab580a4e608ae4f1ec7d1
Many uses of stasis_unsubscribe in modules can be reached through unload.
These have been switched to stasis_unsubscribe_and_join.
Some subscription callbacks do nothing, for these I've created a noop
callback function in stasis.c. This is used by some modules that monitor
MWI topics in order to enable cache, since the callback does not become
invalid after dlclose it is safe to use stasis_unsubscribe on these, even
during module unload.
ASTERISK-25121 #close
Change-Id: Ifc2549fbd8eef7d703c222978e8f452e2972189c
Git does not support the ability to replace a token with a version
string during check-in. While it does have support for replacing a
token on clone, this is somewhat sub-optimal: the token is replaced
with the object hash, which is not particularly easy for human
consumption. What's more, in practice, the source file version was often
not terribly useful. Generally, when triaging bugs, the overall version
of Asterisk is far more useful than an individual SVN version of a file. As a
result, this patch removes Asterisk's support for showing source file
versions.
Specifically, it does the following:
* Rename ASTERISK_FILE_VERSION macro to ASTERISK_REGISTER_FILE, and
remove passing the version in with the macro. Other facilities
than 'core show file version' make use of the file names, such as
setting a debug level only on a specific file. As such, the act of
registering source files with the Asterisk core still has use. The
macro rename now reflects the new macro purpose.
* main/asterisk:
- Refactor the file_version structure to reflect that it no longer
tracks a version field.
- Remove the "core show file version" CLI command. Without the file
version, it is no longer useful.
- Remove the ast_file_version_find function. The file version is no
longer tracked.
- Rename ast_register_file_version/ast_unregister_file_version to
ast_register_file/ast_unregister_file, respectively.
* main/manager: Remove value from the Version key of the ModuleCheck
Action. The actual key itself has not been removed, as doing so would
absolutely constitute a backwards incompatible change. However, since
the file version is no longer tracked, there is no need to attempt to
include it in the Version key.
* UPGRADE: Add notes for:
- Modification to the ModuleCheck AMI Action
- Removal of the "core show file version" CLI command
Change-Id: I6cf0ff280e1668bf4957dc21f32a5ff43444a40e
Since 'core stop now' and 'core restart now' do not stop modules,
it is unsafe for most of the core to run cleanups. Originally all
cleanups used ast_register_atexit, and were only changed when it
was shown to be unsafe. ast_register_atexit is now used only when
absolutely required to prevent corruption and close child processes.
Exceptions that need to use ast_register_atexit:
* CDR: Flush records.
* res_musiconhold: Kill external applications.
* AstDB: Close the DB.
* canary_exit: Kill canary process.
ASTERISK-24142 #close
Reported by: David Brillert
ASTERISK-24683 #close
Reported by: Peter Katzmann
ASTERISK-24805 #close
Reported by: Badalian Vyacheslav
ASTERISK-24881 #close
Reported by: Corey Farrell
Review: https://reviewboard.asterisk.org/r/4500/
Review: https://reviewboard.asterisk.org/r/4501/
........
Merged revisions 433495 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 433497 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433498 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Prior to this patch, all Stasis subscriptions would receive a dedicated
thread for servicing published messages. In contrast, prior to r400178
(see review https://reviewboard.asterisk.org/r/2881/), the subscriptions
shared a thread pool. It was discovered during some initial work on Stasis
that, for a low subscription count with high message throughput, the
threadpool was not as performant as simply having a dedicated thread per
subscriber.
For situations where a subscriber receives a substantial number of messages
and is always present, the model of having a dedicated thread per subscriber
makes sense. While we still have plenty of subscriptions that would follow
this model, e.g., AMI, CDRs, CEL, etc., there are plenty that also fall into
the following two categories:
* Large number of subscriptions, specifically those tied to endpoints/peers.
* Low number of messages. Some subscriptions exist specifically to coordinate
a single message - the subscription is created, a message is published, the
delivery is synchronized, and the subscription is destroyed.
In both of the latter two cases, creating a dedicated thread is wasteful (and
in the case of a large number of peers/endpoints, harmful). In those cases,
having shared delivery threads is far more performant.
This patch adds the ability of a subscriber to Stasis to choose whether or not
their messages are dispatched on a dedicated thread or on a threadpool. The
threadpool is configurable through stasis.conf.
Review: https://reviewboard.asterisk.org/r/4193
ASTERISK-24533 #close
Reported by: xrobau
Tested by: xrobau
........
Merged revisions 428681 from http://svn.asterisk.org/svn/asterisk/branches/12
........
Merged revisions 428687 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@428688 65c4cc65-6c06-0410-ace0-fbb531ad65f3
When message type creation is declined via stasis.conf, certain
operations log errors assuming that the declined type is being used
before initialization or after destruction. These error messages get
quite spammy for oft used message types and should not be logged in the
first place since the message type is validly NULL.
Reported by: Matt DiMeo
........
Merged revisions 424769 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@424770 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This introduces stasis.conf and a mechanism to prevent certain message
types from being published. Internally, this works by preventing the
chosen message types from being created which ensures that those
message types can never be published. This patch also adjusts message
publishers such that message payloads are not created if the related
message type is not available.
ASTERISK-23943 #close
Review: https://reviewboard.asterisk.org/r/3823/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@420124 65c4cc65-6c06-0410-ace0-fbb531ad65f3
User events can now be generated from ARI. Events can be signalled with
arbitrary json variables, and include one or more of channel, bridge, or
endpoint snapshots. An application must be specified which will receive
the event message (other applications can subscribe to it). The message
will also be delivered via AMI provided a channel is attached. Dialplan
generated user event messages are still transmitted via the channel, and
will only be received by a stasis application they are attached to or if
the channel is subscribed to.
This change also introduces the multi object blob mechanism used to send
multiple snapshot types in a single message. The dialplan app UserEvent
was also changed to use multi object blob, and a new stasis message type
created to handle them.
ASTERISK-22697 #close
Review: https://reviewboard.asterisk.org/r/3494/
........
Merged revisions 414405 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@414406 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This patch fixes res_corosync such that it works with Asterisk 12. This
restores the functionality that was present in previous versions of
Asterisk, and ensures compatibility with those versions by restoring the
binary message format needed to pass information from/to them.
The following changes were made in the core to support this:
* The event system has been partially restored. All event definition and
event types in this patch were pulled from Asterisk 11. Previously, we had
hoped that this information would live in res_corosync; however, the
approach in this patch seems to be better for a few reasons:
(1) Theoretically, ast_events can be used by any module as a binary
representation of a Stasis message. Given the structure of an ast_event
object, that information has to live in the core to be used universally.
For example, defining the payload of a device state ast_event in
res_corosync could result in an incompatible device state representation
in another module.
(2) Much of this representation already lived in the core, and was not
easily extensible.
(3) The code already existed. :-)
* Stasis message types now have a message formatter that converts their
payload to an ast_event object.
* Stasis message forwarders now handle forwarding to themselves. Previously
this would result in an infinite recursive call. Now, this simply creates a
new forwarding object with no forwards set up (as it is the thing it is
forwarding to). This is advantageous for res_corosync, as returning NULL
would also imply an unrecoverable error. Returning a subscription in this
case allows for easier handling of message types that are published directly
to an aggregate topic that has forwarders.
Review: https://reviewboard.asterisk.org/r/3486/
ASTERISK-22912 #close
ASTERISK-22372 #close
........
Merged revisions 414330 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@414331 65c4cc65-6c06-0410-ace0-fbb531ad65f3
* Remove some unnecessary RAII_VAR() usage.
* Made the struct stasis_subscription ao2 object use the ao2 lock instead
of a redundant join_lock in the struct for ast_cond_wait().
* Removed locks on some ao2 objects that don't need the lock.
* Made the topic pool entries container use the ao2 template functions.
* Add some missing allocation failure checks.
* Add missing cleanup in off nominal path of dispatch_message().
........
Merged revisions 409270 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@409271 65c4cc65-6c06-0410-ace0-fbb531ad65f3
George Joseph
Kevin Harwell
Joshua Colp
Corey Farrell
Ben Ford
sungtae kim
Friendly Automation
Richard Mudgett
Chris-Savinovich
Sean Bright
Matt Jordan
Kinsey Moore
Scott Griepentrog