There were a number of places in the res_pjsip stack that were getting
all endpoints or all aors, and then filtering them locally.
A good example is pjsip_options which, on startup, retrieves all
endpoints, then the aors for those endpoints, then tests the aors to see
if the qualify_frequency is > 0. One issue was that it never did
anything with the endpoints other than retrieve the aors so we probably
could have skipped a step and just retrieved all aors. But nevermind.
This worked reasonably well with local config files but with a realtime
backend and thousands of objects, this was a nightmare. The issue
really boiled down to the fact that while realtime supports predicates
that are passed to the database engine, the non-realtime sorcery
backends didn't.
They do now.
The realtime engines have a scheme for doing simple comparisons. They
take in an ast_variable (or list) for matching, and the name of each
variable can contain an operator. For instance, a name of
"qualify_frequency >" and a value of "0" would create a SQL predicate
that looks like "where qualify_frequency > '0'". If there's no operator
after the name, the engines add an '=' so a simple name of
"qualify_frequency" and a value of "10" would return exact matches.
The non-realtime backends decide whether to include an object in a
result set by calling ast_sorcery_changeset_create on every object in
the internal container. However, ast_sorcery_changeset_create only does
exact string matches though so a name of "qualify_frequency >" and a
value of "0" returns nothing because the literal "qualify_frequency >"
doesn't match any name in the objset set.
So, the real task was to create a generic string matcher that can take a
left value, operator and a right value and perform the match. To that
end, strings.c has a new ast_strings_match(left, operator, right)
function. Left and right are the strings to operate on and the operator
can be a string containing any of the following: = (or NULL or ""), !=,
>, >=, <, <=, like or regex. If the operator is like or regex, the
right string should be a %-pattern or a regex expression. If both left
and right can be converted to float, then a numeric comparison is
performed, otherwise a string comparison is performed.
To use this new function on ast_variables, 2 new functions were added to
config.c. One that compares 2 ast_variables, and one that compares 2
ast_variable lists. The former is useful when you want to compare 2
ast_variables that happen to be in a list but don't want to traverse the
list. The latter will traverse the right list and return true if all
the variables in it match the left list.
Now, the backends' fields_cmp functions call ast_variable_lists_match
instead of ast_sorcery_changeset_create and they can now process the
same syntax as the realtime engines. The realtime backend just passes
the variable list unaltered to the engine. The only gotcha is that
there's no common realtime engine support for regex so that's been noted
in the api docs for ast_sorcery_retrieve_by_fields.
Only one more change to sorcery was done... A new config flag
"allow_unqualified_fetch" was added to reg_sorcery_realtime.
"no": ignore fetches if no predicate fields were supplied.
"error": same as no but emit an error. (good for testing)
"yes": allow (the default);
"warn": allow but emit a warning. (good for testing)
Now on to res_pjsip...
pjsip_options was modified to retrieve aors with qualify_frequency > 0
rather than all endpoints then all aors. Not only was this a big
improvement in realtime retrieval but even for config files there's an
improvement because we're not going through endpoints anymore.
res_pjsip_mwi was modified to retieve only endpoints with something in
the mailboxes field instead of all endpoints then testing mailboxes.
res_pjsip_registrar_expire was completely refactored. It was retrieving
all contacts then setting up scheduler entries to check for expiration.
Now, it's a single thread (like keepalive) that periodically retrieves
only contacts whose expiration time is < now and deletes them. A new
contact_expiration_check_interval was added to global with a default of
30 seconds.
Ross Beer reports that with this patch, his Asterisk startup time dropped
from around an hour to under 30 seconds.
There are still objects that can't be filtered at the database like
identifies, transports, and registrations. These are not going to be
anywhere near as numerous as endpoints, aors, auths, contacts however.
Back to allow_unqualified_fetch. If this is set to yes and you have a
very large number of objects in the database, the pjsip CLI commands
will attempt to retrive ALL of them if not qualified with a LIKE.
Worse, if you type "pjsip show endpoint <tab>" guess what's going to
happen? :) Having a cache helps but all the objects will have to be
retrieved at least once to fill the cache. Setting
allow_unqualified_fetch=no prevents the mass retrieve and should be used
on endpoints, auths, aors, and contacts. It should NOT be used for
identifies, registrations and transports since these MUST be
retrieved in bulk.
Example sorcery.conf:
[res_pjsip]
endpoint=config,pjsip.conf,criteria=type=endpoint
endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error
ASTERISK-25826 #close
Reported-by: Ross Beer
Tested-by: Ross Beer
Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
This work-in-progress is the first step to being able to reliably
build Asterisk containers from the Asterisk source. I'm submitting
this based on feedback gained at AstriDevCon 2015.
Information about how to use this is provided in contrib/docker/README.md
and will result in a local Asterisk container being built right from
your source. I believe this can eventually be automated via
hub.docker.com.
Change-Id: Ifa070706d40e56755797097b6ed72c1e243bd0d1
RedHat/CentOS needs python-devel
Debian/Ubuntu needs automake, libsrtp-dev and python-dev
Ubuntu also needed libncurses5-dev for cmenuselect so while not
needed for pjproject, I adedd it anyway.
Change-Id: Idf5fa16e2d87c687439621507e122cb9461d7089
Downgrade had a few issues. First there was an errant 'update' statement in
add_auto_dtmf_mode that looks like it was a copy/paste error. Second, we
weren't cleaning up the ENUMs so subsequent upgrades on postgres failed
because the types already existed.
For sqlite... sqlite doesn't support ALTER or DROP COLUMN directly.
Fortunately alembic batch_operations takes care of this for us if we
use it so the alter and drops were converted to use batch operations.
Here's an example downgrade:
with op.batch_alter_table('ps_endpoints') as batch_op:
batch_op.drop_column('tos_audio')
batch_op.drop_column('tos_video')
batch_op.add_column(sa.Column('tos_audio', yesno_values))
batch_op.add_column(sa.Column('tos_video', yesno_values))
batch_op.drop_column('cos_audio')
batch_op.drop_column('cos_video')
batch_op.add_column(sa.Column('cos_audio', yesno_values))
batch_op.add_column(sa.Column('cos_video', yesno_values))
with op.batch_alter_table('ps_transports') as batch_op:
batch_op.drop_column('tos')
batch_op.add_column(sa.Column('tos', yesno_values))
# Can't cast integers to YESNO_VALUES, so dropping and adding is required
batch_op.drop_column('cos')
batch_op.add_column(sa.Column('cos', yesno_values))
Upgrades from base to head and downgrades from head to base were tested
repeatedly for postgresql, mysql/mariadb, and sqlite3.
Change-Id: I862b0739eb3fd45ec3412dcc13c2340e1b7baef8
The 'reload' mechanism actually involves closing the underlying
socket and calling the appropriate udp, tcp or tls start functions
again. Only outbound_registration, pubsub and session needed work
to reset the transport before sending requests to insure that the
pjsip transport didn't get pulled out from under them.
In my testing, no calls were dropped when a transport was changed
for any of the 3 transport types even if ip addresses or ports were
changed. To be on the safe side however, a new transport option was
added (allow_reload) which defaults to 'no'. Unless it's explicitly
set to 'yes' for a transport, changes to that transport will be ignored
on a reload of res_pjsip. This should preserve the current behavior.
Change-Id: I5e759850e25958117d4c02f62ceb7244d7ec9edf
A recent commit set qualify_timeout to Decimal which isn't supported.
This path corrects it to Float.
Change-Id: I038f5274ba8cb60f8518a5845ce448d49306aadf
Corrects the qualify_timeout column type from Integer to Decimal
ASTERISK-25686 #close
Reported-by: Marcelo Terres
Change-Id: I757d0e3c011ee9be6cd5abd48bc92441a405d3c8
Added new global option (regcontext) to pjsip. When set, Asterisk will
dynamically create and destroy a NoOp priority 1 extension
for a given endpoint who registers or unregisters with us.
ASTERISK-25670 #close
Reported-by: Daniel Journo
Change-Id: Ib1530c5b45340625805c057f8ff1fb240a43ea62
PJPROJECT has a function available to dump the compile time
options used when building the library.
* Add CLI "pjsip show buildopts" command.
* Update contrib/scripts/autosupport to get pjproject information.
Change-Id: Id93a6a916d765b2a2e5a1aeb54caaf83206be748
On a system with multiple ip addresses in the same subnet, if a
transport is bound to a specific ip address and endpoint/media_address
is set, the SIP/SDP will have the correct address in all fields but
the rtp stream MAY still originate from one of the other ip addresses,
most probably the "primary" ip address. This happens because
res_pjsip_sdp_rtp/create_rtp always calls ast_instance_new with
the "all" ip address (0.0.0.0 or ::).
The new option causes res_pjsip_sdp_rtp/create_rtp to call
ast_rtp_instance_new with the endpoint's media_address (if specified)
instead of the "all" address. This causes the packets to originate from
the specified address.
ASTERISK-25632
ASTERISK-25637
Reported-by: Olivier Krief
Reported-by: Dan Journo
Change-Id: I3dfaa079e54ba7fb7c4fd1f5f7bd9509bbf8bd88
The keep_alive_interval option was added about a year ago, but no
alembic revision was created to add the appropriate column to the
database.
This commit fixes the problem and adds the column. This was discovered
by running the testsuite with automatic conversion to realtime enabled.
Change-Id: If3ef92a7c4f4844d08f8aae170d2178aec5c4c1a
The spandspflow2pcap.py creates pcap files from fax.log files, generated
through 'fax set debug on' when receiving a fax. An example fax.log is
included as spandspflow2pcap.log.
The sipp-sendfax.xml SIPp scenario can be used to replay that fax with a
recent version of SIPp.
ASTERISK-25660 #close
Change-Id: I4de8f28b084055b482ab8a5b28d28b605b0ed526
When running the PJSIP AMI "show_endpoint" test with automatic
conversion to realtime, the test would fail. This was because the AOR
"contact" column was sized at 40, and the configured contact was larger
than that.
This commit increases the size of the contact column to 255 characters.
Change-Id: Ia65bc7fd37699b7c0eaef9629a1a31eab9a24ba1
This increases the maximum length of account code's to match
extensions. This ensures it is always possible to set an
accountcode to ${EXTEN} without truncation.
ASTERISK-23904
Reported by: Ben Merrills
Change-Id: If122602304ce03362722eb213a3111b32da5eeb9
When to install packages the indexed local is more old of the
version of software on the repository they have been upgraded by security
update then get the package will give 404 not found.
The patch prevent by update local index to repository for aptitude before
install.
ASTERISK-25495 #close
Reporte by: Rodrigo Ramírez Norambuena
Change-Id: I645959e553aac542805ced394cac2dca964051fa
During outbound registration it is possible to receive a fatal (any permanent/
non-temporary 4xx, 5xx, 6xx) response from the registrar that is simply due
to a problem with the registrar itself. Upon receiving the failure response
Asterisk terminates outbound registration for the given endpoint.
This patch adds an option, 'fatal_retry_interval', that when set continues
outbound registration at the given interval up to 'max_retries' upon receiving
a fatal response.
ASTERISK-25485 #close
Change-Id: Ibc2c7b47164ac89cc803433c0bbe7063bfa143a2
This patch adds some minor tweaks for autosupport to update it for Asterisk 13.
This includes:
* Finally removing most references to Zaptel
* Adding support for some additional 'core' commands, and fixing nomenclature
that generally hasn't been used for some time
* Adding some PJSIP/SIP commands to gather endpoints/peers and active channels
Change-Id: Ic997b418cbd9313588b6608e50f47b0ce6f4f1f1
(cherry picked from commit 9fc9777fa3)
To help in diagnosing mismatched modules and libraries, this
script scans for version, repository, and source information
and reports what is found.
ASTERISK-25376 #close
Reported by: Ashley Sanders
Change-Id: Ib0642d0fb96712476f59760d6d137a24633fe2d6
When Asterisk sends an outbound SIP request, if there is no direct
reason to place a specific value for the username in the From header,
Asterisk would generate a UUID. For example, this would happen when
sending outbound OPTIONS requests when qualifying or when sending
outbound INVITE requests when originating (if no explicit caller ID were
provided). The issue is that some SIP providers reject these sorts of
requests with a "Name too long" error response.
This patch aims to fix this by changing the default outbound username in
From headers to "asterisk". This value can be overridden by changing the
default_from_user option in the global options if desired.
ASTERISK-25377 #close
Reported by Mark Michelson
Change-Id: I6a4d34a56ff73ff4f661b0075aeba5461b7f3190
Asterisk needs the sqlite 3 library, which is package
sqlite-devel in CentOS. By adding this package to the
script, a problem with configure failing is resolved.
ASTERISK-25331 #close
Reported by: Kevin Harwell
Change-Id: I90efaf6a01914fea03f21e5cdbd91c348f44b0ec
This change adds support for the 'rtp_timeout' and 'rtp_timeout_hold'
endpoint options. These allow the channel to be hung up if RTP
is not received from the remote endpoint for a specified number of
seconds.
ASTERISK-25259 #close
Change-Id: I3f39daaa7da2596b5022737b77799d16204175b9
This adds an "rtp_keepalive" option for PJSIP endpoints. Similar to the
chan_sip option, this specifies an interval, in seconds, at which we
will send RTP comfort noise frames. This can be useful for keeping RTP
sessions alive as well as keeping NAT associations alive during lulls.
ASTERISK-25242 #close
Reported by Mark Michelson
Change-Id: I3b9903d99e35fe5d0b53ecc46df82c750776bc8d
Some phones send g.726 audio packed for AAL2, which differs from what is
recommended by RFC 3351. If Asterisk receives audio formatted as such when
negotiating g.726 then it sounds a bit distorted. Added an option to
res_pjsip_endpoint that allows g.726 negotiated audio to be treated as g.726
AAL2 packed.
ASTERISK-25158 #close
Reported by: Steve Pitts
Change-Id: Ie7e21f75493d7fe53e75e12c971e72f5afa33615
If in Debian or system based, dont have aptitude installed the script do
nothing. This patch checked if aptitude installed, if not installed.
Also, if execute script with all packages installed yet, the script not show
nothing and return exit 1 because the command 'grep' get nothing from pipe from
'awk'.
ASTERISK-25113 #close
Reported By: Rodrigo Ramírez Norambuena <decipher.hk@gmail.com>
Change-Id: Iebdff55805d3917166e5e08e0a1e2176f36ff27f
There are 3 ways that calls directly to standard allocator functions can
be dealt with:
1. Block their use, cause them to generate an error. This is the default.
2. Replace them with the Asterisk equivalent function calls.
3. Leave them alone.
This change allows one of these 3 options to be selected by any source.
The source just needs to define ASTMM_LIBC to ASTMM_BLOCK, ASTMM_REDIRECT,
or ASTMM_IGNORE to use option 1, 2 or 3 respectively. Normally ASTMM_BLOCK
is the correct option, so it is default when ASTMM_LIBC is not defined.
In some cases when building 3rd party code it is desirable to have it use
Asterisk functions, without changing the whole source - ASTMM_REDIRECT
accomplishes this. When using 3rd party libraries sometimes a static
inline function will make use of malloc or free. In these cases it may
be unsafe to replace the allocator in the header, as it's possible the
memory could be freed by the library using standard allocators. For
those cases ASTMM_IGNORE is needed.
Change-Id: I8afef4bc7f3b93914263ae27d3a5858b69663fc7
Moved contrib/asterisk-ng-doxygen to doc/asterisk-ng-doxygen.in
Changed /Makefile to copy asterisk-ng-doxygen.in to
asterisk-ng-doxygen then modify it with version instead of
modifying asterisk-ng-doxygen directly. Updated clean
targets as well.
Updated /.gitignore and doc/.gitignore.
Change-Id: I38712d3e334fa4baec19d30d05de8c6f28137622
* Added a lookbehind to one-line comment matcher to skip escaped
semicolons.
* Added support for block comments.
Change-Id: Id17dfaeda8ed4be572e8107a0c010066584aaee7
The upgrade script for auto DTMF mode (31cd4f4891ec) added in 88b0fa7755
failed to add ENUM support for Postgres databases. This requires a
specific import from the sqlalchemy.dialects.postgresql package. This
patch corrects this error, which allows for Postgres update scripts to
be generated.
ASTERISK-24706
Change-Id: I4742ac8efa533cd6f18e0bdd907b339a9aedf015
* The REF_DEBUG compiler flag no longer has any effect on code that uses
Astobj2. It is used to determine if reference debugging is enabled by
default. Reference debugging can be enabled or disabled in asterisk.conf.
* Caller information is provided in logger errors for ao2 bad magic numbers.
* Optimizes AO2 by merging internal functions with the public counterpart.
This was possible now that we no longer require a dual ABI.
ASTERISK-24974 #close
Reported by: Corey Farrell
Change-Id: Icf3552721fe999365ba8a8cf00a965aa6b897cc1
Currently when Asterisk starts initial qualifies of contacts are spread out
randomly between 0 and qualify_timeout to prevent network and system overload.
If a contact's qualify_frequency is 5 minutes however, that contact may be
unavailable to accept calls for the entire 5 minutes after startup. So while
staggering the initial qualifies is a good idea, basing the time on
qualify_timeout could leave contacts unavailable for too long.
This patch adds a new global parameter "max_initial_qualify_time" that sets the
maximum time for the initial qualifies. This way you could make sure that all
your contacts are initialy, randomly qualified within say 30 seconds but still
have the contact's ongoing qualifies at a 5 minute interval.
If max_initial_qualify_time is > 0, the formula is initial_interval =
min(max_initial_interval, qualify_timeout * random(). If not set,
qualify_timeout is used.
The default is "0" (disabled).
ASTERISK-24863 #close
Change-Id: Ib80498aa1ea9923277bef51d6a9015c9c79740f4
Tested-by: George Joseph <george.joseph@fairview5.com>
This is the second follow-on to https://reviewboard.asterisk.org/r/4572/ and the
discussion at
http://lists.digium.com/pipermail/asterisk-dev/2015-March/073921.html
The basic issues are that changes in contact status don't cause events to be
emitted for the associated endpoint. Only dynamic contact add/delete actions
update the endpoint. Also, the qualify timeout is fixed by pjsip at 32 seconds
which is a long time.
This patch makes use of the new transaction timeout feature in r4585 and
provides the following capabilities...
1. A new aor/contact variable 'qualify_timeout' has been added that allows the
user to specify the maximum time in milliseconds to wait for a response to an
OPTIONS message. The default is 3000ms. When the timer expires, the contact is
marked unavailable.
2. Contact status changes are now propagated up to the endpoint as follows...
When any contact is 'Available', the endpoint is marked as 'Reachable'. When
all contacts are 'Unavailable', the endpoint is marked as 'Unreachable'. The
existing endpoint events are generated appropriately.
ASTERISK-24863 #close
Change-Id: Id0ce0528e58014da1324856ea537e7765466044a
Tested-by: Dmitriy Serov
Tested-by: George Joseph <george.joseph@fairview5.com>
This patch adds support for automatically detecting the type of DTMF that a
PJSIP endpoint supports. When the 'dtmf_mode' endpoint option is set to 'auto',
the channel created for an endpoint will attempt to determine if RFC 4733
DTMF is supported. If so, it will use that DTMF type. If not, the DTMF type
for the channel will be set to inband.
Review: https://reviewboard.asterisk.org/r/4438
ASTERISK-24706 #close
Reported by: yaron nahum
patches:
yaron_patch_3_Feb.diff submitted by yaron nahum (License 6676)
........
Merged revisions 434637 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@434638 65c4cc65-6c06-0410-ace0-fbb531ad65f3
The script was added in 13, but when committed to trunk it caused a branch to
occur due to some trunk only alemebic changes. This fixes it so that the new
'add_pjsip_endpoint_identifier_order script points to the correct down revision.
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433152 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This patch fixes previously reverted code that caused binary incompatibility
problems with some modules. And like the original patch it makes sure that
no matter what order the endpoint identifier modules were loaded, priority is
given based on the ones specified in the new global 'endpoint_identifier_order'
option.
ASTERISK-24840
Reported by: Mark Michelson
Review: https://reviewboard.asterisk.org/r/4489/
........
Merged revisions 433028 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433029 65c4cc65-6c06-0410-ace0-fbb531ad65f3
It's possible to have a scenario that will create a conflict between endpoint
identifiers. For instance an incoming call could be identified by two different
endpoint identifiers and the one chosen depended upon which identifier module
loaded first. This of course causes problems when, for example, the incoming
call is expected to be identified by username, but instead is identified by ip.
This patch adds a new 'global' option to res_pjsip called
'endpoint_identifier_order'. It is a comma separated list of endpoint
identifier names that specifies the order by which identifiers are processed
and checked.
ASTERISK-24840 #close
Reported by: Mark Michelson
Review: https://reviewboard.asterisk.org/r/4455/
........
Merged revisions 432638 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@432639 65c4cc65-6c06-0410-ace0-fbb531ad65f3
On Debian based systems, the install_prereq tool uses a search command on
Debian that results in selecting both 64-bit and 32-bit packages. Besides the
waste of disk space, this can actually cause aptitude use 100% of memory on a
VM with 1GB of RAM as it tried to work out all of the 32-bit package
dependencies.
This patch filters out the 32-bit packages on a 64-bit machine, and leaves
32-bit machines alone.
ASTERISK-24048 #close
Reported by: Ben Klang
Tested by: Ben Klang, Matt Jordan
patches:
install_prereq_64-bit_compat.patch uploaded by Ben Klang (License 5876)
........
Merged revisions 430798 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 430799 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@430800 65c4cc65-6c06-0410-ace0-fbb531ad65f3
General improvements to SIP to PJSIP conversion utility:
1) track default section of input file to allow parsing
an include file that doesn't specify a [section]
2) informatively handle case of assignment without [section]
3) correctly handle getting sections from included files
- [section]'s are inherited by included file
4) provide null string as default transport bind ip
5) gracefully handle missing portions of registration string
6) denote steps of operation during conversion and confirm
top level files as a convenience
ASTERISK-24474 #close
Review: https://reviewboard.asterisk.org/r/4280/
Reported by: John Kiniston
........
Merged revisions 430469 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@430470 65c4cc65-6c06-0410-ace0-fbb531ad65f3
