Currently in app_confbridge if someone mutes a channel while that channel
is talking, the talk detection code is suspended while the channel is
muted. As far an an external observer is concerned, the muted channel's
talk status is still "talking" even though the channel is not contributing
audio to the conference bridge. When the channel is later unmuted, it
takes the usual 'dsp_silence_threshold' option time to clear the talking
status even though the channel may have stopped talking while the channel
was muted.
* In bridge_softmix.c, clear the talking status and report talking stopped
if the channel was talking when the channel is muted. When the channel is
unmuted and the channel is still talking then report the channel as
talking since it is contributing audio to the bridge again.
ASTERISK-27647
Change-Id: Ie4fdbc05a0bc7343c2972bab012e2567917b3d4e
The dsp_talking_threshold does not represent time in milliseconds. It
represents the average magnitude per sample in the audio packets. This is
what the DSP uses to determine if a packet is silence or talking/noise.
Change-Id: If6f939c100eb92a5ac6c21236559018eeaf58443
I've audited all modules that include any header which includes
asterisk/optional_api.h. All modules which use OPTIONAL_API now declare
those dependencies in AST_MODULE_INFO using requires or optional_modules
as appropriate.
In addition ARI dependency declarations have been reworked. Instead of
declaring additional required modules in res/ari/resource_*.c we now add
them to an optional array "requiresModules" in api-docs for each module.
This allows the AST_MODULE_INFO dependencies to include those missing
modules.
Change-Id: Ia0c70571f5566784f63605e78e1ceccb4f79c606
* validate_stream: zero result from ast_format_cap_identical indicates
they are not identical, rather than non-zero indicating an error.
* validate_original_streams: use num_streams instead of
ARRAY_LEN(params).
* Fix declaration of alice_dest_stream and bob_dest_stream.
Change-Id: I6b1dd8bed10439d3c7406f033eb1896b6c419147
If two channels enter different native rtp bridges at the same time it is
possible that the framehook interface data pointer can be corrupted
because the struct variable was declared static.
* Fixed the reentrancy corruption by changing the framehook interface
struct static variable to a stack local variable.
* Moved the hook.data assignment outside of the channel lock. It did not
need the lock's protection. It probably was giving a false sense of
security.
The testsuite
channels/pjsip/basic_calls/two_parties/nominal/alice_initiated/bob_hangs_up
test caught this with MALLOC_DEBUG and DO_CRASH enabled.
Change-Id: If9e35b97d19209b0f984941c1d8eb5f7c55eea91
The return value of remove_destination_streams() now means we removed a
stream from the topology by making it a dead stream. Now we won't try to
request a topology change if we didn't remove any streams.
Change-Id: Icd91571d856a1d04299a24c411e325c1d9d5c61d
* Made is_video_source() and is_video_dest() not match dead streams.
* Optimized is_video_dest() to reduce duplicated code.
Change-Id: I4e7ab762c7ee98395e78e6516399f57a2609b9a1
This appeared in my audit of ast_stream_topology_set_stream callers
not checking for errors but in this situation the call cannot fail.
Add comment so this can be ignored in the future.
Change-Id: I91d25704859efbe50b8b82cfe1cd3c40ba177c9f
When (v)asprintf() fails, the state of the allocated buffer is undefined.
The library had better not leave an allocated buffer as a result or no one
will know to free it. The most likely way it can return failure is for an
allocation failure. If the printf conversion fails then you actually have
a threading problem which is much worse because another thread modified
the parameter values.
* Made __ast_asprintf()/__ast_vasprintf() set the returned buffer to NULL
on failure. That is much more useful than either an uninitialized pointer
or a pointer that has already been freed. Many uses won't have to check
for failure to ensure that the buffer won't be double freed or prevent an
attempt to free an uninitialized pointer.
* stasis.c: Fixed memory leak in multi_object_blob_to_ami() allocated by
ast_asprintf().
* ari/resource_bridges.c:ari_bridges_play_helper(): Remove assignment to
the wrong thing which is now not needed even if assigning to the right
thing.
Change-Id: Ib5252fb8850ecf0f78ed0ee2ca0796bda7e91c23
As channels join and leave an SFU the bridge_softmix module
needs to renegotiate to add and remove their streams from
the other participants. Previously this was done by constructing
the ideal stream topology every time but in the case of leave
this was incomplete.
This change makes it so bridge_softmix keeps an ideal stream
topology for each channel and uses it when making changes. This
ensures that when we request a renegotiation we are always
certain that we are aiming for the best stream topology
possible. In the case of a channel leaving this ensures that
we try to have an existing participant fill their place if
a participant has a fixed limit on the maximum number of video
streams they allow.
ASTERISK-27354
Change-Id: I58070f421ddeadd2844a33b869b052630cf2e514
When making channels compatible the bridge_simple module
will renegotiate one to better match the other. Some
endpoints incorrectly terminate the call if this process
fails.
To better handle this scenario the audio streams present
on the new requested topology will include any existing
negotiated formats that happen to exist on the first
valid audio stream. This ensures formats are persent that
are known to be acceptable to the remote endpoint.
ASTERISK-27259
Change-Id: I8fc0cc03e8bcfd0be8302f13b9f32d8268977f43
Some endpoints do not like a stream being reused for a new
media stream. The frame/jitterbuffer can rely on underlying
attributes of the media stream in order to order the packets.
When a new stream takes its place without any notice the
buffer can get confused and the media ends up getting dropped.
This change uses the SSRC change to determine that a new source
is reusing an existing stream and then bridge_softmix renegotiates
each participant such that they see a new media stream. This
causes the frame/jitterbuffer to start fresh and work as expected.
ASTERISK-27277
Change-Id: I30ccbdba16ca073d7f31e0e59ab778c153afae07
When two channels were early bridged in a native_rtp bridge, the RTP description
on one side was not updated when the other side answered.
This patch forbids non-answered channels to enter a native_rtp bridge, and
triggers a bridge reconfiguration when an ANSWER frame is received.
ASTERISK-27257
Change-Id: If1aaee1b4ed9658a1aa91ab715ee0a6413b878df
* Fix framehook to test frame type for control frame.
* Made framehook exit early if frame type is not a control frame.
* Eliminated RAII_VAR in framehook.
* Use switch instead of else-if ladder for control frame handling.
Change-Id: Ia555fc3600bd85470e3c0141147dbe3ad07c1d18
* Fix deadlock in
bridge_softmix.c:softmix_bridge_stream_topology_changed() between
bridge_channel and channel locks.
* The new bridge technology topology change callbacks must be called with
the bridge locked. The callback references the bridge channel list, the
bridge technology could change, and the bridge stream mapping is updated.
ASTERISK-27212
Change-Id: Ide4360ab853607e738ad471721af3f561ddd83be
This change fixes a few locking issues and some video misrouting.
1. When accessing the stream topology of a channel the channel lock
must be held to guarantee the topology remains valid.
2. When a channel was joined to a bridge the bridge specific
implementation for stream mapping was not invoked, causing video
to be misrouted for a brief period of time.
ASTERISK-27182
Change-Id: I5d2f779248b84d41c5bb3896bf22ba324b336b03
This change does a few things to improve packet loss and renegotiation:
1. On outgoing RTP streams we will now properly reflect out of order
packets and packet loss in the sequence number. This allows the
remote jitterbuffer to better reorder things.
2. Video updates can now be discarded for a period of time
after one has been sent to prevent flooding of clients.
3. For declined and removed streams we will now release any
media session resources associated with them. This was not
previously done and caused an issue where old state was being
used for a new stream.
4. RTP bundling was not actually removing bundled RTP instances
from the parent. This has been resolved by removing based on
the RTP instance itself and not the SSRC.
5. The code did not properly handle explicitly unbundling an
RTP instance from its parent. This now works as expected.
ASTERISK-27143
Change-Id: Ibd91362f0e4990b6129638e712bc8adf0899fd45
When a participant leaves a bridge while operating in SFU mode
their respective stream on every other participant needs to be
removed. Leaving the stream out of the new topology results in
every stream after it being moved and reordered. This causes
problems with clients. Instead simply mark the stream as removed
which leaves it in place in the SDP and doesn't reorder or touch
any other streams.
ASTERISK-27136
Change-Id: I4b3f840adcdf69b83842b0d8a737665ba0ef9cb1
Streams are never truly removed in SDP, they still occupy
a location within the SDP. This location can be reused by
another stream if it so chooses.
This change takes advantage of this such that if a new stream
is needing to be added for a new participant any removed streams
are instead replaced first. This reduces the size of the SDP
and the number of streams.
ASTERISK-27134
Change-Id: I95cdcfd55cf47e02ea52abb5d94008db3fb68b1d
This change fixes a few things uncovered during SFU testing.
1. Unreal channels incorrectly forwarded video frames when
no video stream was present on them. This caused a crash when
they were read as the core requires a stream to exist for the
underlying media type. The Unreal channel will now ensure a
stream exists for the media type before forwarding the frame
and if no stream exists then the frame is dropped.
2. Mapping of frames during bridging from the stream number of
the underlying channel to the stream number of the bridge was
done in the wrong location. This resulted in the frame getting
dropped. This mapping now occurs on reading of the frame from
the channel.
3. Bridging was using the wrong ast_read function resulting in
it living in a non-multistream world.
4. In bridge_softmix when adding new streams to existing channels
the wrong stream topology was copied resulting in no streams
being added.
Change-Id: Ib7445722c3219951d6740802a0feddf2908c18c8
The video stream was using the audio stream RTP instance addresses to
check if the video RTP gets directed to an allowed direct media Access
Control List (ACL) address. There is no guarantee that the video RTP
instance uses the same addresses as the audio RTP instance.
This looks like it has been a bug since v11 when direct media ACL was
first added to chan_sip and then faithfully reproduced through a couple
code refactorings into the new bridging architecture.
Change-Id: I8ddd56320e0eea769f3ceed3fa5b6bdfb51d681a
There have been reports of deadlocks caused by an attempt to send a frame
to a channel's rtp instance after the channel has left the native bridge
and been destroyed. This patch effectively causes the bridge channel to
keep a reference to the glue and both the audio and video rtp instances
so what gets started will get stopped.
ASTERISK-26978 #close
Reported-by: Ross Beer
Change-Id: I9e1ac49fa4af68d64826ccccd152593cf8cdb21a
This change adds a deferred queue to bridging. If a bridge
technology determines that a frame can not be written and
should be deferred it can indicate back to bridging to do so.
Bridging will then requeue any deferred frames upon a new
channel joining the bridge.
This change has been leveraged for T.38 request negotiate
control frames. Without the deferred queue there is a race
condition between the bridge receiving the T.38 request
negotiate and the second channel joining and being in the
bridge. If the channel is not yet in the bridge then the T.38
negotiation fails.
A unit test has also been added that confirms that a T.38
request negotiate control frame is deferred when no other
channel is in the bridge and that it is requeued when a new
channel joins the bridge.
ASTERISK-26923
Change-Id: Ie05b08523f399eae579130f4a5f562a344d2e415
This sets up the "plumbing" in bridge_softmix to
be able to accommodate Asterisk asking as an SFU
(selective forwarding unit) for conferences.
The way this works is that whenever a channel enters or leaves a
conference, all participants in the bridge get sent a stream topology
change request. The topologies consist of the channels' original
topology, along with video destination streams corresponding to each
participants' source video streams. So for instance, if Alice, Bob, and
Carol are in the conference, and each supplies one video stream, then
the topologies for each would look like so:
Alice:
Audio,
Source video(Alice),
Destination Video(Bob),
Destination video (Carol)
Bob:
Audio,
Source video(Bob)
Destination Video(Alice),
Destination video (Carol)
Carol:
Audio,
Source video(Carol)
Destination Video(Alice),
Destination video (Bob)
This way, video that arrives from a source video stream can then be
copied out to the destination video streams on the other participants'
channels.
Once the bridge gets told that a topology on a channel has changed, the
bridge constructs a map in order to get the video frames routed to the
proper destination streams. This is done using the bridge channel's
stream_map.
This change is bare-bones with regards to SFU support. Some key features
are missing at this point:
* Stream limits. This commit makes no effort to limit the number of
streams on a specific channel. This means that if there were 50 video
callers in a conference, bridge_softmix will happily send out topology
change requests to every channel in the bridge, requesting 50+
streams.
* Configuration. The plumbing has been added to bridge_softmix, but
there has been nothing added as of yet to app_confbridge to enable SFU
video mode.
* Testing. Some functions included here have unit tests.
However, the functionality as a whole has only been verified by
hand-tracing the code.
* Selectivenss. For a "selective" forwarding unit, this does not
currently have any means of being selective.
* Features. Presumably, someone might wish to only receive video from
specific sources. There are no external-facing functions at the moment
that allow for users to select who they receive video from.
* Efficiency. The current scheme treats all video streams as being
unidirectional. We could be re-using a source video stream as a
desetnation, too. But to simplify things on this first round, I did it
this way.
Change-Id: I7c44a829cc63acf8b596a337b2dc3c13898a6c4d
This patch is the first cut at adding stream support to the bridging framework.
Changes were made to the framework that allows mapping of stream topologies to
a bridge's supported media types.
The first channel to enter a bridge initially defines the media types for a
bridge (i.e. a one to one mapping is created between the bridge and the first
channel). Subsequently added channels merge their media types into the bridge's
adding to it when necessary. This allows channels with different sized
topologies to map correctly to each other according to media type. The bridge
drops any frame that does not have a matching index into a given write stream.
For now though, bridge_simple will align its two channels according to size or
first to join. Once both channels join the bridge the one with the most streams
will indicate to the other channel to update its streams to be the same as that
of the other. If both channels have the same number of streams then the first
channel to join is chosen as the stream base.
A topology change source was also added to a channel when a stream toplogy
change request is made. This allows subsystems to know whether or not they
initiated a change request. Thus avoiding potential recursive situations.
ASTERISK-26966 #close
Change-Id: I1eb5987921dd80c3cdcf52accc136393ca2d4163
When a T.38 happens immediatly after call establishment, the control
frame can be lost because the other leg is not yet in the bridge.
This patch detects this case an makes sure T.38 negotation happens
when the 2nd leg is being made compatible with the negotating
first leg
ASTERISK-26923 #close
Change-Id: If334125ee61ed63550d242fc9efe7987e37e1d94
Some codecs - codec_speex specifically - take voice frames and return
other types of frames, like CNG. If we subsequently treat those as
voice frames, we'll run into trouble when destroying the frame because
of the requirement that each voice frame have an associated format.
ASTERISK-26880 #close
Reported by: Kirsty Tyerman
Change-Id: I43f8450c48fb276ad8b99db8512be82949c1ca7c
The bridge_native_rtp module did not properly handle the case where
a smart bridge operation occurs while a channel is suspended. In this
scenario the module would incorrectly set up local or remote RTP
bridging despite the media having to flow through Asterisk. The remote
endpoint would see two media streams and experience wonky audio.
The module has been changed so that it ensures both channels are
not suspended when performing the native RTP bridging and this
requirement has been documented in the bridge technology.
ASTERISK-26781
Change-Id: Id4022d73ace837d4a293106445e3ade10dbc7c7c
Adds binaural synthesis to bridge_softmix (via convolution using libfftw3).
Binaural synthesis is conducted at 48kHz.
For a conference, only one spatial representation is rendered.
The default rendering is applied for mono-capable channels.
ASTERISK-26292
Change-Id: Iecdb381b6adc17c961049658678f6219adae1ddf
In native_rtp_bridge_compatible_check()
* Made one variable declaration per line.
* Extracted if test assignment to make the test easier to see.
* Made long if tests easier to see the combinatorial logic.
* Added bridge id to a couple debug messages.
Change-Id: I65bc5732aa7c9a2537f062f106fbea711cf2daad
native_rtp_bridge_compatible() didn't lock the bridge channels before
checking the channels for native bridging ability. As a result, one of
the channel's native format capabilities structure got replaced out from
under the native bridge check. Use of a stale pointer to freed memory
causes bad things to happen.
MALLOC_DEBUG, DO_CRASH, and the
tests/channels/pjsip/transfers/blind_transfer/caller_direct_media
testsuite test caught this.
* Add missing channel locking in native_rtp_bridge_compatible().
Change-Id: If25fdb3ac8e85563c4857fb8216b3d9dc3d0fa53
This is a regression over Asterisk 11, introduced by
2dc8a06006. Previously, recordings started via
the automon DTMF code would automatically be mixed together using sox because
app_monitor would be called with the m option. This commit restores this
behavior.
Change-Id: Ibaf58684285c3f1b6ca3714524e6d638ae3b3759
WebRTC clients really, really want to know the SSRC of the media they're
getting. Changing the SSRC is generally not a good thing.
bridge_softmix, starting in Asterisk 12, started changing the SSRC of
parties as they joined or left the bridge. With most phones, this isn't
a problem: phones just play back the stream they're getting. With WebRTC
clients, however, the SSRC is tied to a media stream that may be
negotiated. When a new SSRC just shows up, the media can be dropped.
As it turns out, the SSRC change shouldn't even be necessary. From the
perspective of the client, it's still talking to Asterisk with the same
media stream: why indicate that the far party has suddenly changed to a
different source of media?
This patch opts to just remove the SSRC changes. With this patch, video
clients that join/leave a softmix bridge actually get the video stream
instead of freaking out.
ASTERISK-26555
Change-Id: I27fec098b32e7c8718b4b65f3fd5fa73527968bf
Joshua Colp
Richard Mudgett
Corey Farrell
Alexander Traud
Sean Bright
Jenkins2
Jean Aunis
George Joseph
Mark Michelson
Kevin Harwell
Torrey Searle
frahaase
Michael Kuron
Matt Jordan