The stream topology has no lock of its own resulting in
another lock protecting it in some way (for example the
channel lock). If multiple channels are being juggled at
the same time this can be problematic. This change makes
the topology a reference counted object instead which
guarantees it will remain valid even without the channel
lock being held.
Change-Id: I4f4d3dd856a033ed55fe218c3a4fab364afedb03
When an externally initiated renegotiation occurred it was
possible for video streams to be incorrectly remapped,
resulting in no video flowing to some receivers.
This change ensures that only the video source sets up
mappings and also that removed streams do not have mappings
set up.
Change-Id: Iab05f2254df3606670774844bb0935f833d3a9b0
This change fixes a bug where a REMB collector may be
freed twice, and also tweaks REMB combining such that if
there is no bitrate from anyone (or there are no sources)
we report 0 instead of using an old bitrate.
ASTERISK-27804
Change-Id: Ia9dc9c150043890ee7ff85e9cdec007f1a77fcfd
This change adds the ability for multiple REMB reports in
bridge_softmix to be combined according to a configured
behavior into a single report. This single report is sent
back to the sender of video, which adjusts the encoding bitrate
to be at or below the bitrate of the report. The available
behaviors are: lowest, highest, and average. Lowest uses the
lowest received bitrate. Highest uses the highest received
bitrate. Average goes through the received bitrates adding
them to the previous average and creates a new average.
Other behaviors can be added in the future and the existing
average one may be adjusted, but this provides the foundation
to do so.
Support for configuring which behavior to use has been
added to app_confbridge.
ASTERISK-27804
Change-Id: I9eafe4e7c1f72d67074a8d6acb26bfcf19322b66
Core bridging and, more specifically, bridge_softmix have been
enhanced to relay received frames of type TEXT or TEXT_DATA to all
participants in a softmix bridge. res_pjsip_messaging and
chan_pjsip have been enhanced to take advantage of this so when
res_pjsip_messaging receives an in-dialog MESSAGE message from a
user in a conference call, it's relayed to all other participants
in the call.
res_pjsip_messaging already queues TEXT frames to the channel when
it receives an in-dialog MESSAGE from an endpoint and chan_pjsip
will send an MESSAGE when it gets a TEXT frame. On a normal
point-to-point call, the frames are forwarded between the two
correctly. bridge_softmix was not though so messages weren't
getting forwarded to conference bridge participants. Even if they
were, the bridging code had no way to tell the participants who
sent the message so it would look like it came from the bridge
itself.
* The TEXT frame type doesn't allow storage of any meta data, such
as sender, on the frame so a new TEXT_DATA frame type was added that
uses the new ast_msg_data structure as its payload. A channel
driver can queue a frame of that type when it receives a message
from outside. A channel driver can use it for sending messages
by implementing the new send_text_data channel tech callback and
setting the new AST_CHAN_TP_SEND_TEXT_DATA flag in its tech
properties. If set, the bridging/channel core will use it instead
of the original send_text callback and it will get the ast_msg_data
structure. Channel drivers aren't required to implement this. Even
if a TEXT_DATA enabled driver uses it for incoming messages, an
outgoing channel driver that doesn't will still have it's send_text
callback called with only the message text just as before.
* res_pjsip_messaging now creates a TEXT_DATA frame for incoming
in-dialog messages and sets the "from" to the display name in the
"From" header, or if that's empty, the caller id name from the
channel. This allows the chat client user to set a friendly name
for the chat.
* bridge_softmix now forwards TEXT and TEXT_DATA frames to all
participants (except the sender).
* A new function "ast_sendtext_data" was added to channel which
takes an ast_msg_data structure and calls a channel's
send_text_data callback, or if that's not defined, the original
send_text callback.
* bridge_channel now calls ast_sendtext_data for TEXT_DATA frame
types and ast_sendtext for TEXT frame types.
* chan_pjsip now uses the "from" name in the ast_msg_data structure
(if it exists) to set the "From" header display name on outgoing text
messages.
Change-Id: Idacf5900bfd5f22ab8cd235aa56dfad090d18489
This patch clears the talking flag from the channel (if already set), and
notifies listeners when that channel is put on hold. Note however, if the
endpoint continues to send audio frames and these are received by the bridge
then that channel will be put back into a "talking" state even though they
are on hold.
ASTERISK-27755 #close
Change-Id: I930e16c4662810f9f02043d69062f88173c5e2ef
Currently in app_confbridge if someone mutes a channel while that channel
is talking, the talk detection code is suspended while the channel is
muted. As far an an external observer is concerned, the muted channel's
talk status is still "talking" even though the channel is not contributing
audio to the conference bridge. When the channel is later unmuted, it
takes the usual 'dsp_silence_threshold' option time to clear the talking
status even though the channel may have stopped talking while the channel
was muted.
* In bridge_softmix.c, clear the talking status and report talking stopped
if the channel was talking when the channel is muted. When the channel is
unmuted and the channel is still talking then report the channel as
talking since it is contributing audio to the bridge again.
ASTERISK-27647
Change-Id: Ie4fdbc05a0bc7343c2972bab012e2567917b3d4e
The dsp_talking_threshold does not represent time in milliseconds. It
represents the average magnitude per sample in the audio packets. This is
what the DSP uses to determine if a packet is silence or talking/noise.
Change-Id: If6f939c100eb92a5ac6c21236559018eeaf58443
* validate_stream: zero result from ast_format_cap_identical indicates
they are not identical, rather than non-zero indicating an error.
* validate_original_streams: use num_streams instead of
ARRAY_LEN(params).
* Fix declaration of alice_dest_stream and bob_dest_stream.
Change-Id: I6b1dd8bed10439d3c7406f033eb1896b6c419147
The return value of remove_destination_streams() now means we removed a
stream from the topology by making it a dead stream. Now we won't try to
request a topology change if we didn't remove any streams.
Change-Id: Icd91571d856a1d04299a24c411e325c1d9d5c61d
* Made is_video_source() and is_video_dest() not match dead streams.
* Optimized is_video_dest() to reduce duplicated code.
Change-Id: I4e7ab762c7ee98395e78e6516399f57a2609b9a1
This appeared in my audit of ast_stream_topology_set_stream callers
not checking for errors but in this situation the call cannot fail.
Add comment so this can be ignored in the future.
Change-Id: I91d25704859efbe50b8b82cfe1cd3c40ba177c9f
When (v)asprintf() fails, the state of the allocated buffer is undefined.
The library had better not leave an allocated buffer as a result or no one
will know to free it. The most likely way it can return failure is for an
allocation failure. If the printf conversion fails then you actually have
a threading problem which is much worse because another thread modified
the parameter values.
* Made __ast_asprintf()/__ast_vasprintf() set the returned buffer to NULL
on failure. That is much more useful than either an uninitialized pointer
or a pointer that has already been freed. Many uses won't have to check
for failure to ensure that the buffer won't be double freed or prevent an
attempt to free an uninitialized pointer.
* stasis.c: Fixed memory leak in multi_object_blob_to_ami() allocated by
ast_asprintf().
* ari/resource_bridges.c:ari_bridges_play_helper(): Remove assignment to
the wrong thing which is now not needed even if assigning to the right
thing.
Change-Id: Ib5252fb8850ecf0f78ed0ee2ca0796bda7e91c23
As channels join and leave an SFU the bridge_softmix module
needs to renegotiate to add and remove their streams from
the other participants. Previously this was done by constructing
the ideal stream topology every time but in the case of leave
this was incomplete.
This change makes it so bridge_softmix keeps an ideal stream
topology for each channel and uses it when making changes. This
ensures that when we request a renegotiation we are always
certain that we are aiming for the best stream topology
possible. In the case of a channel leaving this ensures that
we try to have an existing participant fill their place if
a participant has a fixed limit on the maximum number of video
streams they allow.
ASTERISK-27354
Change-Id: I58070f421ddeadd2844a33b869b052630cf2e514
Some endpoints do not like a stream being reused for a new
media stream. The frame/jitterbuffer can rely on underlying
attributes of the media stream in order to order the packets.
When a new stream takes its place without any notice the
buffer can get confused and the media ends up getting dropped.
This change uses the SSRC change to determine that a new source
is reusing an existing stream and then bridge_softmix renegotiates
each participant such that they see a new media stream. This
causes the frame/jitterbuffer to start fresh and work as expected.
ASTERISK-27277
Change-Id: I30ccbdba16ca073d7f31e0e59ab778c153afae07
* Fix deadlock in
bridge_softmix.c:softmix_bridge_stream_topology_changed() between
bridge_channel and channel locks.
* The new bridge technology topology change callbacks must be called with
the bridge locked. The callback references the bridge channel list, the
bridge technology could change, and the bridge stream mapping is updated.
ASTERISK-27212
Change-Id: Ide4360ab853607e738ad471721af3f561ddd83be
This change fixes a few locking issues and some video misrouting.
1. When accessing the stream topology of a channel the channel lock
must be held to guarantee the topology remains valid.
2. When a channel was joined to a bridge the bridge specific
implementation for stream mapping was not invoked, causing video
to be misrouted for a brief period of time.
ASTERISK-27182
Change-Id: I5d2f779248b84d41c5bb3896bf22ba324b336b03
This change does a few things to improve packet loss and renegotiation:
1. On outgoing RTP streams we will now properly reflect out of order
packets and packet loss in the sequence number. This allows the
remote jitterbuffer to better reorder things.
2. Video updates can now be discarded for a period of time
after one has been sent to prevent flooding of clients.
3. For declined and removed streams we will now release any
media session resources associated with them. This was not
previously done and caused an issue where old state was being
used for a new stream.
4. RTP bundling was not actually removing bundled RTP instances
from the parent. This has been resolved by removing based on
the RTP instance itself and not the SSRC.
5. The code did not properly handle explicitly unbundling an
RTP instance from its parent. This now works as expected.
ASTERISK-27143
Change-Id: Ibd91362f0e4990b6129638e712bc8adf0899fd45
When a participant leaves a bridge while operating in SFU mode
their respective stream on every other participant needs to be
removed. Leaving the stream out of the new topology results in
every stream after it being moved and reordered. This causes
problems with clients. Instead simply mark the stream as removed
which leaves it in place in the SDP and doesn't reorder or touch
any other streams.
ASTERISK-27136
Change-Id: I4b3f840adcdf69b83842b0d8a737665ba0ef9cb1
Streams are never truly removed in SDP, they still occupy
a location within the SDP. This location can be reused by
another stream if it so chooses.
This change takes advantage of this such that if a new stream
is needing to be added for a new participant any removed streams
are instead replaced first. This reduces the size of the SDP
and the number of streams.
ASTERISK-27134
Change-Id: I95cdcfd55cf47e02ea52abb5d94008db3fb68b1d
This change fixes a few things uncovered during SFU testing.
1. Unreal channels incorrectly forwarded video frames when
no video stream was present on them. This caused a crash when
they were read as the core requires a stream to exist for the
underlying media type. The Unreal channel will now ensure a
stream exists for the media type before forwarding the frame
and if no stream exists then the frame is dropped.
2. Mapping of frames during bridging from the stream number of
the underlying channel to the stream number of the bridge was
done in the wrong location. This resulted in the frame getting
dropped. This mapping now occurs on reading of the frame from
the channel.
3. Bridging was using the wrong ast_read function resulting in
it living in a non-multistream world.
4. In bridge_softmix when adding new streams to existing channels
the wrong stream topology was copied resulting in no streams
being added.
Change-Id: Ib7445722c3219951d6740802a0feddf2908c18c8
This sets up the "plumbing" in bridge_softmix to
be able to accommodate Asterisk asking as an SFU
(selective forwarding unit) for conferences.
The way this works is that whenever a channel enters or leaves a
conference, all participants in the bridge get sent a stream topology
change request. The topologies consist of the channels' original
topology, along with video destination streams corresponding to each
participants' source video streams. So for instance, if Alice, Bob, and
Carol are in the conference, and each supplies one video stream, then
the topologies for each would look like so:
Alice:
Audio,
Source video(Alice),
Destination Video(Bob),
Destination video (Carol)
Bob:
Audio,
Source video(Bob)
Destination Video(Alice),
Destination video (Carol)
Carol:
Audio,
Source video(Carol)
Destination Video(Alice),
Destination video (Bob)
This way, video that arrives from a source video stream can then be
copied out to the destination video streams on the other participants'
channels.
Once the bridge gets told that a topology on a channel has changed, the
bridge constructs a map in order to get the video frames routed to the
proper destination streams. This is done using the bridge channel's
stream_map.
This change is bare-bones with regards to SFU support. Some key features
are missing at this point:
* Stream limits. This commit makes no effort to limit the number of
streams on a specific channel. This means that if there were 50 video
callers in a conference, bridge_softmix will happily send out topology
change requests to every channel in the bridge, requesting 50+
streams.
* Configuration. The plumbing has been added to bridge_softmix, but
there has been nothing added as of yet to app_confbridge to enable SFU
video mode.
* Testing. Some functions included here have unit tests.
However, the functionality as a whole has only been verified by
hand-tracing the code.
* Selectivenss. For a "selective" forwarding unit, this does not
currently have any means of being selective.
* Features. Presumably, someone might wish to only receive video from
specific sources. There are no external-facing functions at the moment
that allow for users to select who they receive video from.
* Efficiency. The current scheme treats all video streams as being
unidirectional. We could be re-using a source video stream as a
desetnation, too. But to simplify things on this first round, I did it
this way.
Change-Id: I7c44a829cc63acf8b596a337b2dc3c13898a6c4d
Some codecs - codec_speex specifically - take voice frames and return
other types of frames, like CNG. If we subsequently treat those as
voice frames, we'll run into trouble when destroying the frame because
of the requirement that each voice frame have an associated format.
ASTERISK-26880 #close
Reported by: Kirsty Tyerman
Change-Id: I43f8450c48fb276ad8b99db8512be82949c1ca7c
Adds binaural synthesis to bridge_softmix (via convolution using libfftw3).
Binaural synthesis is conducted at 48kHz.
For a conference, only one spatial representation is rendered.
The default rendering is applied for mono-capable channels.
ASTERISK-26292
Change-Id: Iecdb381b6adc17c961049658678f6219adae1ddf
WebRTC clients really, really want to know the SSRC of the media they're
getting. Changing the SSRC is generally not a good thing.
bridge_softmix, starting in Asterisk 12, started changing the SSRC of
parties as they joined or left the bridge. With most phones, this isn't
a problem: phones just play back the stream they're getting. With WebRTC
clients, however, the SSRC is tied to a media stream that may be
negotiated. When a new SSRC just shows up, the media can be dropped.
As it turns out, the SSRC change shouldn't even be necessary. From the
perspective of the client, it's still talking to Asterisk with the same
media stream: why indicate that the far party has suddenly changed to a
different source of media?
This patch opts to just remove the SSRC changes. With this patch, video
clients that join/leave a softmix bridge actually get the video stream
instead of freaking out.
ASTERISK-26555
Change-Id: I27fec098b32e7c8718b4b65f3fd5fa73527968bf
ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes
all traces of it.
Previously exported symbols removed:
* __ast_register_file
* __ast_unregister_file
* ast_complete_source_filename
This also removes the mtx_prof static variable that was declared when
MTX_PROFILE was enabled. This variable was only used in lock.c so it
is now initialized in that file only.
ASTERISK-26480 #close
Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966
softmix_bridge_join() failed because of an allocation failure. To address
this, the softmix bridge technology now checks if the channel failed to
join softmix successfully. In addition, the bridge now begins the process
of kicking the channel out of the bridge so we don't have channels
partially in the bridge for very long.
* Fix the test_channel_feature_hooks.c unit tests. The test channel must
have a valid codec to join the simple_bridge technology. This patch makes
joining a bridge more strict by not allowing partially joined channels to
remain in the bridge.
Change-Id: I97e2ade6a2bcd1214f24fb839fda948825b61a2b
Fix off nominal crash where we could not setup the channel to process
frames for the softmix bridge technology because of allocation failure.
Change-Id: Ic307a8386e46bf551e48fcd1eb97276714d56372
Git does not support the ability to replace a token with a version
string during check-in. While it does have support for replacing a
token on clone, this is somewhat sub-optimal: the token is replaced
with the object hash, which is not particularly easy for human
consumption. What's more, in practice, the source file version was often
not terribly useful. Generally, when triaging bugs, the overall version
of Asterisk is far more useful than an individual SVN version of a file. As a
result, this patch removes Asterisk's support for showing source file
versions.
Specifically, it does the following:
* Rename ASTERISK_FILE_VERSION macro to ASTERISK_REGISTER_FILE, and
remove passing the version in with the macro. Other facilities
than 'core show file version' make use of the file names, such as
setting a debug level only on a specific file. As such, the act of
registering source files with the Asterisk core still has use. The
macro rename now reflects the new macro purpose.
* main/asterisk:
- Refactor the file_version structure to reflect that it no longer
tracks a version field.
- Remove the "core show file version" CLI command. Without the file
version, it is no longer useful.
- Remove the ast_file_version_find function. The file version is no
longer tracked.
- Rename ast_register_file_version/ast_unregister_file_version to
ast_register_file/ast_unregister_file, respectively.
* main/manager: Remove value from the Version key of the ModuleCheck
Action. The actual key itself has not been removed, as doing so would
absolutely constitute a backwards incompatible change. However, since
the file version is no longer tracked, there is no need to attempt to
include it in the Version key.
* UPGRADE: Add notes for:
- Modification to the ModuleCheck AMI Action
- Removal of the "core show file version" CLI command
Change-Id: I6cf0ff280e1668bf4957dc21f32a5ff43444a40e
With this patch, chan_pjsip/res_pjsip now sets the native formats to the
codecs negotiated by a call.
* The changes in chan_pjsip.c and res_pjsip_sdp_rtp.c set the native
formats to include all the negotiated audio codecs instead of only the
initial preferred audio codec and later the currently received audio
codec.
* The audio frame handling in channel.c:ast_read() is more streamlined and
will automatically adjust to changes in received frame formats. The new
policy is to remove translation and pass the new frame format to the
receiver except if the translation was to a signed linear format. A more
long winded version is commented in ast_read() along with some caveats.
* The audio frame handling in channel.c:ast_write() is more streamlined
and will automatically adjust any needed translation to changes in the
frame formats sent. Frame formats sent can change for many reasons such
as a recording is being played back or the bridged peer changed the format
it sends. Since it is a normal expectation that sent formats can change,
the codec mismatch warning message is demoted to a debug message.
* Removed the short circuit check in
channel.c:ast_channel_make_compatible_helper(). Two party bridges need to
make channels compatible with each other. However, transfers and moving
channels among bridges can result in otherwise compatible channels having
sub-optimal translation paths if the make compatible check is short
circuited. A result of forcing the reevaluation of channel compatibility
is that the asterisk.conf:transcode_via_slin and codecs.conf:genericplc
options take effect consistently now. It is unfortunate that these two
options are enabled by default and negate some of the benefits to the
changes in channel.c:ast_read() by forcing translation through signed
linear on a two party bridge.
* Improved the softmix bridge technology to better control the translation
of frames to the bridge. All of the incoming translation is now normally
handled by ast_read() instead of splitting any translation steps between
ast_read() and the slin factory. If any frame comes in with an unexpected
format then the translation path in ast_read() is updated for the next
frame and the slin factory handles the current frame translation.
This is the final patch in a series of patches aimed at improving
translation path choices. The other patches are on the following reviews:
https://reviewboard.asterisk.org/r/4600/https://reviewboard.asterisk.org/r/4605/
ASTERISK-24841 #close
Reported by: Matt Jordan
Review: https://reviewboard.asterisk.org/r/4609/
........
Merged revisions 434671 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@434672 65c4cc65-6c06-0410-ace0-fbb531ad65f3
When a channel enters the bridging system it is first made compatible with
the bridge and then the bridge technology makes the channel compatible
with the technology. For all but the DAHDI native and softmix bridge
technologies the make channel compatible with the bridge step is an
effective noop because the other technologies allow all audio formats.
For the DAHDI native bridge technology it doesn't matter because it is not
an initial bridge technology and chan_dahdi allows only one native format
per channel. For the softmix bridge technology, it is a noop at best and
harmful at worst because the wrong translation path could be setup if the
channel's native formats allow more than one audio format.
This is an intermediate patch for a series of patches aimed at improving
translation path choices.
* Removed code dealing with the unnecessary step of making the channel
compatible with the bridge.
ASTERISK-24841
Reported by: Matt Jordan
Review: https://reviewboard.asterisk.org/r/4600/
........
Merged revisions 434424 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@434430 65c4cc65-6c06-0410-ace0-fbb531ad65f3
When more than one call using the same codec type enters into a softmix bridge
and no audio is present for a channel the bridge optimizes the out frame by
using the same one for all channels with the same codec type. Unfortunately,
when that number (channels with same codec type) dropped to <= 1 the codec
was not dereferenced. At least not until all parties left the bridge. Thus in
the case of G.729 the license was not released. This patch ensures that the
codec is dereferenced immediately when the optimization no longer applies.
ASTERISK-24797 #close
Reported by: Luke Hulsey
Review: https://reviewboard.asterisk.org/r/4429/
........
Merged revisions 432174 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 432175 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@432176 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Joshua Colp
Jenkins2
George Joseph
Kevin Harwell
Richard Mudgett
Corey Farrell
Alexander Traud
Mark Michelson
Sean Bright
frahaase
Matt Jordan
Kinsey Moore