There have been crashes and general instability seen in the pubsub code,
so this patch introduces three changes to increase the stability.
First, the ownership model for subscriptions has been modified. Due to
RLS, subscriptions are stored in memory as a tree structure. Prior to my
patch, the PJSIP subscription was the owner of the subscription tree.
When the PJSIP subscription told us that it was terminating, we started
destroying the subscription tree along with all of the individual leaf
subscriptions that belong to the tree. The problem with this model is
that the two actors in play here, the PJSIP subscription and the
individual leaf subscriptions, need to have joint ownership of the
subscription tree. So now, the PJSIP subscription and the individual
leaf subscriptions each have a reference to the subscription tree. This
way, we will not actually free memory until no players are left that
care. The PJSIP subscription is a bigger stakeholder, in that if the
PJSIP subscription's reference to the subscription tree is removed, the
subscription tree instructs the leaf subscriptions to shut down and drop
their references to the subscription tree when possible. The individual
leaf subscriptions, upon being told to shut down, can drop their stasis
subscriptions or whatever they use to learn of new state, and then drop
their reference to the subscription tree once they are ready to die.
Second, the lifetime of a PJSIP subscription's reference to our
subscription tree has been altered. As I learned from doing a deep dive,
the PJSIP evsub code can tell Asterisk multiple times that the
subscription has been terminated, and not all of these times
are especially helpful. I have altered the message flow that we use for
SIP subscriptions such that we will always drop the PJSIP subscription's
reference to the subscription tree when we send the NOTIFY that
terminates a SIP subscription. This also means that we will now queue
NOTIFY requests to be sent after responding to incoming SUBSCRIBEs so
that we can have predictable state changes from the PJSIP evsub code.
Third, the synchronization of operations has been improved. PJSIP can
call into our code from a serializer thread (e.g. upon receiving an
incoming request) or from the monitor thread (e.g. when a subscription
times out). Because of this, there is the possibility of competing
threads stepping on each other. PJSIP attempts to do some
synchronization on its own by always keeping the dialog lock held when
it calls into us. However, since we end up pushing tasks into the
serializer, the result was that serialized operations were not grabbing
the dialog lock and could, as a result, step on something that was being
attempted by a different thread. Now we ensure that serialized
operations grab the dialog lock, then check for extenuating
circumstances, then proceed with their operation if they can.
Change-Id: Iff2990c40178dad9cc5f6a5c7f76932ec644b2e5
MWI subscriptions can crash or corrupt memory when using the subscription
datastore to access the MWI subscription object because the datastore is
not holding a reference to the object.
* Give the subscription datastore a ref to the MWI subscription object.
It is unfortunate that the ref causes a circular ref chain that must be
explicitly broken to allow the memory to get released. The loop is broken
when the subscription is shutdown and if the subscription setup fails.
ASTERISK-25168 #close
Reported by: Carl Fortin
Change-Id: Ice4fa823f138ff10a6c74d280699c41a82836d4f
The res_pjsip_mwi previously required a reload to set up the proper
subscriptions to allow unsolicited MWI to work. This change
makes it so the act of registering will also cause this to occur.
This is particularly useful if realtime is involved as no reload
needs to occur within Asterisk to cause the MWI information
to get sent.
ASTERISK-25180 #close
Change-Id: Id847b47de4b8b3ab8858455ccc2f07b0f915f252
Many uses of stasis_unsubscribe in modules can be reached through unload.
These have been switched to stasis_unsubscribe_and_join.
Some subscription callbacks do nothing, for these I've created a noop
callback function in stasis.c. This is used by some modules that monitor
MWI topics in order to enable cache, since the callback does not become
invalid after dlclose it is safe to use stasis_unsubscribe on these, even
during module unload.
ASTERISK-25121 #close
Change-Id: Ifc2549fbd8eef7d703c222978e8f452e2972189c
Currently the res_pjsip_mwi module only sends an unsolicited MWI NOTIFY upon
a mailbox state change (such as a new message being left, or one being deleted).
In practice this is not sufficient to keep clients aware of the current MWI status.
This change makes the module send unsolicited MWI NOTIFY on startup so that
clients are guaranteed to have the most up to date MWI information. It also makes
clients receive an unsolicited MWI NOTIFY upon registration so if they are unaware
of the current MWI status they receive it.
ASTERISK-24982 #close
Reported by: Joshua Colp
Change-Id: I043f20230227e91218f18a82c7d5bb2aa62b1d58
When res_pjsip loads and an endpoint auto-subscribes a mailbox for mwi,
if a contact hasn't registered yet, res_pjsip_mwi spits out a warning.
This is a perfectly normal situation though and doesn't require something
as serious as a warning. It's also self correcting. The device will start
getting mwi as soon as it registers.
This patch changes the warning to a notice.
Tested-by: George Joseph
Review: https://reviewboard.asterisk.org/r/4314/
........
Merged revisions 430227 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@430228 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Prior to this patch, all Stasis subscriptions would receive a dedicated
thread for servicing published messages. In contrast, prior to r400178
(see review https://reviewboard.asterisk.org/r/2881/), the subscriptions
shared a thread pool. It was discovered during some initial work on Stasis
that, for a low subscription count with high message throughput, the
threadpool was not as performant as simply having a dedicated thread per
subscriber.
For situations where a subscriber receives a substantial number of messages
and is always present, the model of having a dedicated thread per subscriber
makes sense. While we still have plenty of subscriptions that would follow
this model, e.g., AMI, CDRs, CEL, etc., there are plenty that also fall into
the following two categories:
* Large number of subscriptions, specifically those tied to endpoints/peers.
* Low number of messages. Some subscriptions exist specifically to coordinate
a single message - the subscription is created, a message is published, the
delivery is synchronized, and the subscription is destroyed.
In both of the latter two cases, creating a dedicated thread is wasteful (and
in the case of a large number of peers/endpoints, harmful). In those cases,
having shared delivery threads is far more performant.
This patch adds the ability of a subscriber to Stasis to choose whether or not
their messages are dispatched on a dedicated thread or on a threadpool. The
threadpool is configurable through stasis.conf.
Review: https://reviewboard.asterisk.org/r/4193
ASTERISK-24533 #close
Reported by: xrobau
Tested by: xrobau
........
Merged revisions 428681 from http://svn.asterisk.org/svn/asterisk/branches/12
........
Merged revisions 428687 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@428688 65c4cc65-6c06-0410-ace0-fbb531ad65f3
res_pjsip_pubsub has two separate checks that it makes when a SUBSCRIBE
arrives.
* It checks that there is a subscription handler for the Event
* It checks that there are body generators for the types in the Accept header
The problem is, there's nothing that ensures that these two things will
actually mesh with each other. For instance, Asterisk will accept a subscription
to MWI that accepts pidf+xml bodies. That doesn't make sense.
With this commit, we add some type information to the mix. Subscription
handlers state they generate data of type X, and body generators state
that they consume data of type X. This way, Asterisk doesn't end up in
some hilariously mismatched situation like the one in the previous paragraph.
ASTERISK-24136 #close
Reported by Mark Michelson
Review: https://reviewboard.asterisk.org/r/3877
Review: https://reviewboard.asterisk.org/r/3878
........
Merged revisions 423344 from http://svn.asterisk.org/svn/asterisk/branches/12
........
Merged revisions 423348 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@423350 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This commit adds the ability for a user to configure
a resource list in pjsip.conf. Subscribing to this
list simultaneously subscribes the subscriber to all
resources listed. This has the potential to reduce
the amount of SIP traffic when loads of subscribers
on a system attempt to subscribe to each others' states.
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@420384 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This helps to pave the way for RLS work that is to come.
Since this is a self-contained change and subscription
tests still pass, this work is being committed directly
to trunk instead of a working branch.
ASTERISK-23865 #close
Review: https://reviewboard.asterisk.org/r/3628
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@417233 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This change makes res_pjsip_pubsub persist inbound subscriptions in sorcery. By default
this uses the local astdb but it can also be configured to store within an outside
database. When Asterisk is started these subscriptions are recreated if they have not
expired. Notifications are sent to the devices which have subscribed and they are none
the wiser that the system has restarted.
Review: https://reviewboard.asterisk.org/r/3598/
........
Merged revisions 415766 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@415767 65c4cc65-6c06-0410-ace0-fbb531ad65f3
When the PJSIP pubsub framework was created, subscription handlers were required
to state what event they handled along with what body types they knew how to
generate. While this serves well when implementing a base RFC, it has problems
when trying to extend the body to support non-standard or proprietary body
elements. The code also was NOTIFY-specific, meaning that when the time comes
that we start writing code to send out PUBLISH requests with MWI or presence
bodies, we would likely find ourselves duplicating code that had previously been
written.
This changeset introduces the concept of body generators and body supplements. A
body generator is responsible for allocating a native structure for a given body
type, providing the primary body content, converting the native structure to a
string, and deallocating resources. A body supplement takes the primary body
content (the native structure, not a string) generated by the body generator and
adds nonstandard elements to the body. With these elements living in their own
module, it becomes easy to extend our support for body types and to re-use
resources when sending a PUBLISH request.
Body generators and body supplements register themselves with the pubsub core,
similar to how subscription and publish handlers had done. Now, subscription
handlers do not need to know what type of body content they generate, but they
still need to inform the pubsub core about what the default body type for a
given event package is. The pubsub core keeps track of what body generators and
body supplements have been registered. When a SUBSCRIBE arrives, the pubsub core
will check that there is a subscription handler for the event in the SUBSCRIBE,
then it will check that there is a body generator that can provide the content
specified in the Accept header(s).
Because of the nature of body generators and supplements, it means
res_pjsip_exten_state and res_pjsip_mwi have been completely gutted. They no
longer worry about body types, instead calling
ast_sip_pubsub_generate_body_content() when they need to generate a NOTIFY body.
Review: https://reviewboard.asterisk.org/r/3150
........
Merged revisions 407016 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@407030 65c4cc65-6c06-0410-ace0-fbb531ad65f3
When subscribing to MWI (res_pjsip_mwi) and the sip uri did not contain a name
(ex: sip:<ip address>) then the subscription would fail since it would be unable
to locate an associated aor. This patch makes it so that when a subscribe comes
with no aor name then it will subscribe to all aors on the located endpoint.
(closes issue ASTERISK-23072)
Reported by: Bob M
Review: https://reviewboard.asterisk.org/r/3164/
........
Merged revisions 407014 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@407015 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This adds Path support to chan_pjsip in res_pjsip_path.c with minimal
additions in res_pjsip_registrar.c to store the path and additions in
res_pjsip_outbound_registration.c to enable advertisement of path
support to registrars and intervening proxies.
Path information is stored on contacts and is enabled via Address of
Record (AoRs) and Registration configuration sections.
While adding path support, it became necessary to be able to add SIP
supplements that handled messages outside of sessions, so a framework
for handling these types of hooks was added in parallel to the
already-existing session supplements and several senders of
out-of-dialog requests were refactored as a result.
(closes issue ASTERISK-21084)
Review: https://reviewboard.asterisk.org/r/3050/
........
Merged revisions 405565 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@405566 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Created the following AMI commands and corresponding events for res_pjsip:
PJSIPShowEndpoints - Provides a listing of all pjsip endpoints and a few
select attributes on each.
Events:
EndpointList - for each endpoint a few attributes.
EndpointlistComplete - after all endpoints have been listed.
PJSIPShowEndpoint - Provides a detail list of attributes for a specified
endpoint.
Events:
EndpointDetail - attributes on an endpoint.
AorDetail - raised for each AOR on an endpoint.
AuthDetail - raised for each associated inbound and outbound auth
TransportDetail - transport attributes.
IdentifyDetail - attributes for the identify object associated with
the endpoint.
EndpointDetailComplete - last event raised after all detail events.
PJSIPShowRegistrationsInbound - Provides a detail listing of all inbound
registrations.
Events:
InboundRegistrationDetail - inbound registration attributes for each
registration.
InboundRegistrationDetailComplete - raised after all detail records have
been listed.
PJSIPShowRegistrationsOutbound - Provides a detail listing of all outbound
registrations.
Events:
OutboundRegistrationDetail - outbound registration attributes for each
registration.
OutboundRegistrationDetailComplete - raised after all detail records
have been listed.
PJSIPShowSubscriptionsInbound - A detail listing of all inbound subscriptions
and their attributes.
Events:
SubscriptionDetail - on each subscription detailed attributes
SubscriptionDetailComplete - raised after all detail records have
been listed.
PJSIPShowSubscriptionsOutbound - A detail listing of all outboundbound
subscriptions and their attributes.
Events:
SubscriptionDetail - on each subscription detailed attributes
SubscriptionDetailComplete - raised after all detail records have
been listed.
(issue ASTERISK-22609)
Reported by: Matt Jordan
Review: https://reviewboard.asterisk.org/r/2959/
........
Merged revisions 403131 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@403133 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This patch fixes a reference counting memory leak on the ao2_container
created as part of create_mwi_subscriptions. When we create the container
in this routine, the intent is to hand lifetime ownership over to the global
container unsolicited_mwi. When ao2_global_obj_replace_unref is called, the
reference count on mwi_subscriptions (the container) will be bumped by 1;
however, the function does not decrement the reference count on
mwi_subscriptions when this occurs. This will prevent the container from being
fully disposed of when Asterisk exits (or on any subsequent call to this
operation, such as during a reload).
........
Merged revisions 402940 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@402942 65c4cc65-6c06-0410-ace0-fbb531ad65f3
........
r399887 | dlee | 2013-09-26 10:41:47 -0500 (Thu, 26 Sep 2013) | 1 line
Minor performance bump by not allocate manager variable struct if we don't need it
........
r400138 | dlee | 2013-09-30 10:24:00 -0500 (Mon, 30 Sep 2013) | 23 lines
Stasis performance improvements
This patch addresses several performance problems that were found in
the initial performance testing of Asterisk 12.
The Stasis dispatch object was allocated as an AO2 object, even though
it has a very confined lifecycle. This was replaced with a straight
ast_malloc().
The Stasis message router was spending an inordinate amount of time
searching hash tables. In this case, most of our routers had 6 or
fewer routes in them to begin with. This was replaced with an array
that's searched linearly for the route.
We more heavily rely on AO2 objects in Asterisk 12, and the memset()
in ao2_ref() actually became noticeable on the profile. This was
#ifdef'ed to only run when AO2_DEBUG was enabled.
After being misled by an erroneous comment in taskprocessor.c during
profiling, the wrong comment was removed.
Review: https://reviewboard.asterisk.org/r/2873/
........
r400178 | dlee | 2013-09-30 13:26:27 -0500 (Mon, 30 Sep 2013) | 24 lines
Taskprocessor optimization; switch Stasis to use taskprocessors
This patch optimizes taskprocessor to use a semaphore for signaling,
which the OS can do a better job at managing contention and waiting
that we can with a mutex and condition.
The taskprocessor execution was also slightly optimized to reduce the
number of locks taken.
The only observable difference in the taskprocessor implementation is
that when the final reference to the taskprocessor goes away, it will
execute all tasks to completion instead of discarding the unexecuted
tasks.
For systems where unnamed semaphores are not supported, a really
simple semaphore implementation is provided. (Which gives identical
performance as the original taskprocessor implementation).
The way we ended up implementing Stasis caused the threadpool to be a
burden instead of a boost to performance. This was switched to just
use taskprocessors directly for subscriptions.
Review: https://reviewboard.asterisk.org/r/2881/
........
r400180 | dlee | 2013-09-30 13:39:34 -0500 (Mon, 30 Sep 2013) | 28 lines
Optimize how Stasis forwards are dispatched
This patch optimizes how forwards are dispatched in Stasis.
Originally, forwards were dispatched as subscriptions that are invoked
on the publishing thread. This did not account for the vast number of
forwards we would end up having in the system, and the amount of work it
would take to walk though the forward subscriptions.
This patch modifies Stasis so that rather than walking the tree of
forwards on every dispatch, when forwards and subscriptions are changed,
the subscriber list for every topic in the tree is changed.
This has a couple of benefits. First, this reduces the workload of
dispatching messages. It also reduces contention when dispatching to
different topics that happen to forward to the same aggregation topic
(as happens with all of the channel, bridge and endpoint topics).
Since forwards are no longer subscriptions, the bulk of this patch is
simply changing stasis_subscription objects to stasis_forward objects
(which, admittedly, I should have done in the first place.)
Since this required me to yet again put in a growing array, I finally
abstracted that out into a set of ast_vector macros in
asterisk/vector.h.
Review: https://reviewboard.asterisk.org/r/2883/
........
r400181 | dlee | 2013-09-30 13:48:57 -0500 (Mon, 30 Sep 2013) | 28 lines
Remove dispatch object allocation from Stasis publishing
While looking for areas for performance improvement, I realized that an
unused feature in Stasis was negatively impacting performance.
When a message is sent to a subscriber, a dispatch object is allocated
for the dispatch, containing the topic the message was published to, the
subscriber the message is being sent to, and the message itself.
The topic is actually unused by any subscriber in Asterisk today. And
the subscriber is associated with the taskprocessor the message is being
dispatched to.
First, this patch removes the unused topic parameter from Stasis
subscription callbacks.
Second, this patch introduces the concept of taskprocessor local data,
data that may be set on a taskprocessor and provided along with the data
pointer when a task is pushed using the ast_taskprocessor_push_local()
call. This allows the task to have both data specific to that
taskprocessor, in addition to data specific to that invocation.
With those two changes, the dispatch object can be removed completely,
and the message is simply refcounted and sent directly to the
taskprocessor.
Review: https://reviewboard.asterisk.org/r/2884/
........
Merged revisions 399887,400138,400178,400180-400181 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@400186 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Prior to this change, we would reject SUBSCRIBE requests that had no Accept
headers. Now event package handlers that handle the default type for the
event package indicate that they do so. Therefore, if we have a handler that
can handle the default type, we can allow SUBSCRIBEs for the handler's event
package that have no Accept headers.
(closes issue ASTERISK-22067)
reported by Mark Michelson
Review: https://reviewboard.asterisk.org/r/2774
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397441 65c4cc65-6c06-0410-ace0-fbb531ad65f3
The general gist is to have a clear boundary between old SIP stuff
and new SIP stuff by having the word "SIP" for old stuff and "PJSIP"
for new stuff. Here's a brief rundown of the changes:
* The word "Gulp" in dialstrings, functions, and CLI commands is now
"PJSIP"
* chan_gulp.c is now chan_pjsip.c
* Function names in chan_gulp.c that were "gulp_*" are now "chan_pjsip_*"
* All files that were "res_sip*" are now "res_pjsip*"
* The "res_sip" directory is now "res_pjsip"
* Files in the "res_pjsip" directory that began with "sip_*" are now "pjsip_*"
* The configuration file is now "pjsip.conf" instead of "res_sip.conf"
* The module info for all PJSIP-related files now uses "PJSIP" instead of "SIP"
* CLI and AMI commands created by Asterisk's PJSIP modules now have "pjsip" as
the starting word instead of "sip"
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395764 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Mark Michelson
Matt Jordan
Richard Mudgett
Joshua Colp
Corey Farrell
Rodrigo Ramírez Norambuena
Kevin Harwell
George Joseph
Kinsey Moore
David M. Lee