Commit Graph

31890 Commits

Author SHA1 Message Date
George Joseph 945451af90 Merge "bridges: Remove reliance on stasis caching" 2018-11-29 15:05:33 -06:00
George Joseph 4f0bf0270e Revert "app_voicemail: Remove need to subscribe to stasis"
This reverts commit 29115e2384.

That commit closed a long standing hole which allowed subscriptions
to mailboxes that weren't configured in voicemail.conf.  This
caused an issue with FreePBX which depdended on that behavior.
The commit is being reverted until FreePBX can handle the new
behavior.

ASTERISK-28151
Reported by: Ronald Raikes

Change-Id: I57b7b85e75d7dd97c742b5c69d718a0f61260c15
2018-11-29 12:29:34 -07:00
Kevin Harwell 7d37967e7e Merge "jansson: Upgrade to 2.12." 2018-11-29 12:57:32 -06:00
George Joseph f4924d40db test_cel: Plug a few ref leaks
These are only a few of the leaks.  The large number of macros
and return paths in this file would make a weeks worth of work
to plug them all.

Change-Id: Ie2369fa944023d44767871c5c30974cb077ffb56
2018-11-26 15:18:00 -07:00
George Joseph 3667c5e1d2 bridges: Remove reliance on stasis caching
* The bridging core no longer uses the stasis cache for bridge
  snapshots.  The latest bridge snapshot is now stored on the
  ast_bridge structure itself.

* The following APIs are no longer available since the stasis cache
  is no longer used:
    ast_bridge_topic_cached()
    ast_bridge_topic_all_cached()

* A topic pool is now used for individual bridge topics.

* The ast_bridge_cache() function was removed since there's no
  longer a separate container of snapshots.

* A new function "ast_bridges()" was created to retrieve the
  container of all bridges.  Users formerly calling
  ast_bridge_cache() can use the new function to iterate over
  bridges and retrieve the latest snapshot directly from the
  bridge.

* The ast_bridge_snapshot_get_latest() function was renamed to
  ast_bridge_get_snapshot_by_uniqueid().

* A new function "ast_bridge_get_snapshot()" was created to retrieve
  the bridge snapshot directly from the bridge structure.

* The ast_bridge_topic_all() function now returns a normal topic
  not a cached one so you can't use stasis cache functions on it
  either.

* The ast_bridge_snapshot_type() stasis message now has the
  ast_bridge_snapshot_update structure as it's data.  It contains
  the last snapshot and the new one.

* cdr, cel, manager and ari have been updated to use the new
  arrangement.

Change-Id: I7049b80efa88676ce5c4666f818fa18ad1985369
2018-11-26 14:30:02 -07:00
Jenkins2 4ca709768d Merge "stasis: Segment channel snapshot to reduce creation cost." 2018-11-26 14:07:47 -06:00
Joshua Colp a6b37e5c43 Merge "astobj2: Create function to copy weak proxied objects from container." 2018-11-26 13:48:00 -06:00
Joshua Colp b80c9071e3 Merge "RTP: need to reset DTMF last seqno/timestamp on voice packet with marker bit" 2018-11-26 13:47:32 -06:00
Joshua Colp 50ac85cb40 stasis: Segment channel snapshot to reduce creation cost.
When a channel snapshot was created it used to be done
from scratch, copying all data (many strings). This incurs
a cost when doing so.

This change segments the channel snapshot into different
components which can be reused if unchanged from the
previous snapshot creation, reducing the cost. In normal
cases this results in some pointers being copied with
reference count being bumped, some integers being set,
and a string or two copied. The other benefit is that it
is now possible to determine if a channel snapshot update
is redundant and thus stop it before a message is published
to stasis.

The specific segments in the channel snapshot were split up
based on whether they are changed together, how often they
are changed, and their general grouping. In practice only
1 (or 0) of the segments actually get changed in normal
operation.

Invalidation is done by setting a flag on the channel when
the segment source is changed, forcing creation of a new
segment when the channel snapshot is created.

ASTERISK-28119

Change-Id: I5d7ef3df963a88ac47bc187d73c5225c315f8423
2018-11-26 12:56:24 -06:00
Joshua Colp d0ccbb3377 stasis: Use an implementation specific channel snapshot cache.
Channels no longer use the Stasis cache for channel snapshots. Instead
they are stored in a hash table in stasis_channels which reduces the
number of Stasis messages created and allows better storage.

As a result the following APIs are no longer available since the stasis
cache is no longer used:
ast_channel_topic_cached()
ast_channel_topic_all_cached()

The ast_channel_cache_all() and ast_channel_cache_by_name() functions
now return an ao2_container of ast_channel_snapshots rather than
a container of stasis_messages therefore you can't (and don't need
to) call stasis_cache functions on it.

The ast_channel_topic_all() function now returns a normal topic not
a cached one so you can't use stasis cache functions on it either.

The ast_channel_snapshot_type() stasis message now has the
ast_channel_snapshot_update structure as it's data. It contains the
last snapshot and the new one.

ast_channel_snapshot_get_latest() still returns the latest snapshot.

The latest snapshot is now stored on the channel itself to eliminate
cache hits when Stasis messages that have the snapshot as a payload
are created.

ASTERISK-28102

Change-Id: I9334febff60a82d7c39703e49059fa3a68825786
2018-11-26 18:43:53 +00:00
Joshua Colp 0a60bc1a68 Merge "func_strings: HASHKEY - negative array index can cause corruption" 2018-11-26 07:44:11 -06:00
Corey Farrell 8e1ab4f11c
jansson: Upgrade to 2.12.
This brings in jansson-2.12, removes all patches that were merged
upstream.  README is created in third-party/jansson/patches to explain
how to add patches but also because the patches folder must exist for
the build process to succeed.

Change-Id: If0f2d541c50997690660c21fb7b03d625a5cdadd
2018-11-26 07:17:54 -05:00
Alexei Gradinari 3f53041267 RTP: need to reset DTMF last seqno/timestamp on voice packet with marker bit
The marker bit set on the voice packet indicates the start
of a new stream and a new time stamp.
Need to reset the DTMF last sequence number and the timestamp
of the last END packet.

If the new time stamp is lower then the timestamp of the last DTMF END packet
the asterisk drops all DTMF frames as out of order.

This bug was caught using Cisco ip-phone SPA50X and codec g722.
On SIP session update the SPA50X resets stream indicating it with market bit
and a new timestamp is twice smaller then the previous.

ASTERISK-28162 #close

Change-Id: If9c5742158fa836ad549713a9814d46a5d2b1620
2018-11-23 10:41:52 -05:00
Corey Farrell 021ce938ca
astobj2: Remove legacy ao2_container_alloc routine.
Replace usage of ao2_container_alloc with ao2_container_alloc_hash or
ao2_container_alloc_list.  Remove ao2_container_alloc macro.

Change-Id: I0907d78bc66efc775672df37c8faad00f2f6c088
2018-11-21 09:56:16 -05:00
Corey Farrell bc7f4f4db3
astobj2: Create function to copy weak proxied objects from container.
Create ao2_container_dup_weakproxy_objs to perform a similar function to
ao2_container_dup.  This function expects the source container to have
weakproxy objects, inserts the associated non-weak objects into the
destination container.  Orphaned weakproxy objects are ignored.

Create test for this new function and for ao2_weakproxy_find.

Change-Id: I898387f058057e08696fe9070f8cd94ef3a27482
2018-11-21 08:47:27 -05:00
Joshua Colp b7af9c8b19 Merge "stasis: Remove stringfields and lock from change message." 2018-11-20 05:06:01 -06:00
Joshua Colp 9b0808e2d0 Merge "app_queue: Cleanup queue_ref / queue_unref routines." 2018-11-20 05:05:32 -06:00
Kevin Harwell 4b5d11ec17 func_strings: HASHKEY - negative array index can cause corruption
This patch makes it so only matching non-empty key names, and keys created by
the HASH function are eligible for inclusion in the comma separated string. It
also fixes a bug where it was possible to write to a negative index if the
result buffer was empty.

ASTERISK-28159
patches:
  ASTERISK-28159.diff submitted by Michael Walton (license 6502)

Change-Id: I6e57fe7307dfd856271753aed5ba64c59b511487
2018-11-19 14:41:57 -05:00
George Joseph bcdfb90362 CI: Get job timeouts from environment
The job timeouts were hard coded in the jenkinsfiles which
means changes had to go through gerrit.  Now they are taken
from the following environment variables (and their defaults) that
can be set in Jenkins configuration...

TIMEOUT_GATES =      "60 MINUTES"
TIMEOUT_DAILIES =    "3 HOURS"
TIMEOUT_REF_DEBUG =  "24 HOURS"
TIMEOUT_UNITTESTS =  "30 MINUTES"

Change-Id: I673a551c1780bf665a3bc160b245da574aa4bbab
2018-11-19 13:05:11 -05:00
Joshua Colp 9df772530f Merge "test_res_pjsip_scheduler: Fix possible write after free in scheduler_policy." 2018-11-19 09:38:50 -06:00
Joshua Colp 9f996ca1f0 Merge "res/res_ari: Fix null endpoint handle" 2018-11-19 09:37:22 -06:00
Joshua Colp e0816637a4 Merge "bridge_native_rtp.c: Fail native bridge if no framing match." 2018-11-19 09:36:17 -06:00
Joshua Colp b285333673 Merge "res_pjsip_caller_id: Use static pj_str_t for fromto header names." 2018-11-19 08:40:05 -06:00
Joshua Colp 614c8a60d7 Merge "stasis: Add internal filtering of messages." 2018-11-19 08:36:50 -06:00
Corey Farrell 64e21c9ea9 app_queue: Cleanup queue_ref / queue_unref routines.
This replaces the inline functions with macros.  This removes the need
to directly use __ao2_ref, opts instead for standard ao2_bump and
ao2_cleanup macros.

Change-Id: If4e04e9bab2e3c883188437cb9f487b3e498a21b
2018-11-19 08:04:01 -05:00
George Joseph ece5f8015f backtrace: Refactor ast_bt_get_symbols so it doesn't crash
We've been seeing crashes in libbfd when we attempt to generate
a stack trace from multiple threads.  It turns out that libbfd
is NOT thread-safe.  It can cache the bfd structure and give it to
multiple threads without protecting itself.  To get around this,
we've added a global mutex around the bfd functions and also have
refactored the use of those functions to be more efficient and
to provide more information about inlined functions.

Also added a few more tests to test_pbx.c.  One just calls
ast_assert() and the other calls ast_log_backtrace().  Neither are
run by default.

WARNING:  This change necessitated changing the return value of
ast_bt_get_symbols() from an array of strings to a VECTOR of
strings.  However, the use of this function outside Asterisk is not
likely.

ASTERISK-28140

Change-Id: I79d02862ddaa2423a0809caa4b3b85c128131621
2018-11-19 05:49:39 -07:00
Joshua C. Colp 56eb18f395 stasis: Remove stringfields and lock from change message.
When a subscribe or unsubscribe occurs a message is published
containing this information. This change makes it so that the
message no longer uses stringfields or a lock, as both are not
really needed for the message.

Change-Id: I3f4831931d79f94fd979baf48048738df5dc1632
2018-11-18 19:53:14 -04:00
Alexei Gradinari fa048183aa pjsip: New function PJSIP_PARSE_URI to parse URI and return part of URI
New dialplan function PJSIP_PARSE_URI added to parse an URI and return
a specified part of the URI.

This is useful when need to get part of the URI instead of cutting it
using a CUT function.

For example to get 'user' part of Remote URI
${PJSIP_PARSE_URI(${CHANNEL(pjsip,remote_uri)},user)}

ASTERISK-28144 #close

Change-Id: I5d828fb87f6803b6c1152bb7b44835f027bb9d5a
2018-11-18 15:27:48 -05:00
George Joseph d81a43893d Merge "pjproject-bundled: Use AST_DEVMODE for conditional compilation." 2018-11-18 14:11:58 -06:00
Joshua Colp 3077ad0c24 stasis: Add internal filtering of messages.
This change adds the ability for subscriptions to indicate
which message types they are interested in accepting. By
doing so the filtering is done before being dispatched
to the subscriber, reducing the amount of work that has
to be done.

This is optional and if a subscriber does not add
message types they wish to accept and set the subscription
to selective filtering the previous behavior is preserved
and they receive all messages.

There is also the ability to explicitly force the reception
of all messages for cases such as AMI or ARI where a large
number of messages are expected that are then generically
converted into a different format.

ASTERISK-28103

Change-Id: I99bee23895baa0a117985d51683f7963b77aa190
2018-11-18 15:08:16 -05:00
George Joseph 915b80709d CI: Add tmpfs to all jenkinsfiles
Change-Id: Ida29d70d48d5f39aabf0b25c66b51f79324a8cba
2018-11-18 11:39:21 -05:00
George Joseph ccc981e91a Merge "res/res_pjsip_nat: Fix logic for REINVITES" 2018-11-18 08:47:10 -06:00
George Joseph fe64ca0be8 Merge "taskprocessor: Prevent race creating new taskprocessor." 2018-11-17 17:28:41 -06:00
George Joseph f5e3832dff CI: Mount a tmpfs on /tmp for testsuite docker containers
Change-Id: I0566d81b0852f22066cd76d58eae5f1fda5602aa
(cherry picked from commit 73efe86436)
2018-11-17 17:07:31 -05:00
George Joseph be87185f6d CI: Pass work directory to runTestsuite
The testsuite can now use a user-specified work directory for
all it's temp files.  This allows the docker containers to use
a tmpfs backed directory for the temp files instead of it's
own write-layer image.

* runTestsuite.sh now accepts a --work-dir command line argument
  that gets exported as AST_WORK_DIR before running the testsuite.

* gates.jenkinsfile now specifies --work-dir to be
  <testsuite_dir>/astroot.

Since the Asterisk CI docker hosts now mount /srv/jenkins/workspace
on a tmpfs, asterisk should be compiled and the testsuite run all in
memory.

Change-Id: If5ee905a15821296c355bb84cda38950ad8edc45
(cherry picked from commit a335f4c9ad)
2018-11-17 17:05:39 -05:00
George Joseph 8399c8dae6 Merge "CI: Allow runUnittests to use 'expect' to run the tests" 2018-11-17 11:30:30 -06:00
Sungtae Kim 1dea497454 res/res_ari: Fix null endpoint handle
The res_ari(POST /channels/create handler) deos not check the endpoint
parameter length. And it causes core
dump.
Fixed it to check the parameter length. Also fixed memory leak.

ASTERISK-28169

Change-Id: Ibf10a9eb8a2e3a9ee1e13fbe748b2ecf955c3993
2018-11-17 04:05:39 +01:00
George Joseph d2f784cc3a Merge "core: Fix handling of restart from remote console." 2018-11-16 09:22:26 -06:00
George Joseph 8ff3435c8a CI: Allow runUnittests to use 'expect' to run the tests
There seems to be a race condition between starting the asterisk
daemon and attempting to use 'asterisk -r' that can cause the
control socket file to not be created.  Since all of the Jenkins
slaves have 'expect' installed, the runUnittests script can use
it to start asterisk in the forground and issue the commands
interactively.  This is much more reliable and it can also make
startup errors more visible since they'll be in the Jenkins console
output.

If 'expect' isn't installed, the original daemon/asterisk -r
process is used.

Also added a "core show settings" before running the tests
and added "notice,warning,error" to the console log.

Change-Id: Idd656085f854afede813ac241b9e312b31358160
2018-11-16 10:03:51 -05:00
Corey Farrell 9abd5e1004
taskprocessor: Prevent race creating new taskprocessor.
Task processors are retrieved using a 'get or create' pattern.  The
singleton container was unlocked between the get and create steps so
it's possible that two threads could create task processors with the
same name at the same time.

Change-Id: Id64fae94a6a1e940ddf38fde622dcd4391635382
2018-11-16 08:55:31 -05:00
Corey Farrell 752fd06d12
pjproject-bundled: Use AST_DEVMODE for conditional compilation.
We previously allowed resample and g711 codecs to be built when
TEST_FRAMEWORK was enabled.  This could cause errors if the testsuite
was run without this option enabled.  Switch the build system to allow
those codecs to be built when --enable-dev-mode is used.  This removes a
chance for strange testsuite errors from use of an inadequate pjsua
binary.

Change-Id: Iee8a3613cdb711fa7e7d217c5a775a575907ae22
2018-11-16 08:26:32 -05:00
Corey Farrell 02c7a061ea res_pjsip_caller_id: Use static pj_str_t for fromto header names.
PJSIP assumes that these header names are not allocated, does not clone
the name strings when reusing headers.

Block unload of res_pjsip_caller_id until shutdown to ensure static
memory stays valid.  It was previously unsafe to unload while any
sessions are active.

Change-Id: I190854dea943d6e441cf03733f8a0da661aea27f
2018-11-15 15:49:12 -05:00
George Joseph fc82312aab Merge "pbx_config: Only the first [globals] section is seen." 2018-11-15 07:48:20 -06:00
Torrey Searle d0554783e2 res/res_pjsip_nat: Fix logic for REINVITES
The presence of Record-Route in re-invites is optional, thus it is
important to make sure the dialog doesn't have a routset before
rewriting the contact header.

ASTERISK-28129 #close

Change-Id: Ic8ceb54ccfc93f7e315e476c514a2c777f2da7dc
2018-11-15 06:35:24 -05:00
Corey Farrell c3d7b19cdd core: Fix handling of restart from remote console.
We cannot use need_el_end and SIGURG when restarting.  Instead we need
to run el_end within the SIGHUP restartnow handler.

ASTERISK-28158

Change-Id: Ia852276363c81bdcf1aa29eb4558c5c2fa1218a0
2018-11-15 06:34:19 -05:00
George Joseph eb5b83b8ea AST-2018-010: Fix length of buffer needed for SRV and NAPTR results
When dn_expand was being called on SRV and NAPTR results, the
return value was being used to calculate the size of the buffer
needed to store the host names.  Since dn_expand returns the
length of the COMPRESSED name the buffer could be too short
to hold the EXPANDED name.  The expanded name is NULL terminated
so using strlen() is the correct way to determine the length
actually needed for the buffer.

ASTERISK-28127
Reported by: Jan Hoffmann

patches:
  patch.diff submitted by janhoffmann (license 6986)

Change-Id: I4d35d6c431c6c6836cb61d37b1378cc47f0b414d
2018-11-14 09:25:32 -05:00
Joshua Colp 9e151413db Merge "core: Ensure that el_end is always run when needed." 2018-11-14 07:06:38 -06:00
Joshua Colp dbfc5ae020 Merge "taskprocessor: Do not use separate allocation for stats or name." 2018-11-14 07:05:10 -06:00
George Joseph 39cfc56dd8 Merge "jansson-bundled: Patch for off-nominal crash." 2018-11-13 14:39:37 -06:00
Corey Farrell 4b24731640 test_res_pjsip_scheduler: Fix possible write after free in scheduler_policy.
It's possible for a 4th task to be spawned before we cancel.  This
results in a write to the already freed test_data1.  Wait long enough to
verify success of the cancelation before freeing test_data1.

Change-Id: I057e2fcbe97f8a175e50890be89c28c20490a20f
2018-11-13 13:26:06 -05:00