This removes references that are no longer needed due to automatic
references created by module dependencies.
In addition this removes most calls to ast_module_check as they were
checking modules which are listed as dependencies.
Change-Id: I332a6e8383d4c72c8e89d988a184ab8320c4872e
There are many places in the code base where we ignore the return value
of fcntl() when getting/setting file descriptior flags. This patch
introduces a convenience function that allows setting or clearing file
descriptor flags and will also log an error on failure for later
analysis.
Change-Id: I8b81901e1b1bd537ca632567cdb408931c6eded7
Previously for PJSIP the local address of WebSocket connections
was set to the remote address. For logging purposes this is
not particularly useful.
The WebSocket API has been extended to allow the local
address to be queried and this is used in PJSIP to set the
local address to the correct value.
The PJSIP HEP support has also been tweaked so that reliable
transports always use the local address on the transport
and do not try to (wrongly) guess. As they are connection
based it is impossible for the source to be anything else.
ASTERISK-26758
ASTERISK-27363
Change-Id: Icd305fd038ad755e2682ab2786e381f6bf29e8ca
Once an Optional API module is loaded it should stay loaded. Unloading
an optional API module runs the risk of a crash if something else is
using it. This patch causes all optional API providers to tell the
module loader not to unload except at shutdown.
ASTERISK-27389
Change-Id: Ia07786fe655681aec49cc8d3d96e06483b11f5e6
In all non-pbx modules, AST_MODULE_LOAD_FAILURE has been changed
to AST_MODULE_LOAD_DECLINE. This prevents asterisk from exiting
if a module can't be loaded. If the user wishes to retain the
FAILURE behavior for a specific module, they can use the "require"
or "preload-require" keyword in modules.conf.
A new API was added to logger: ast_is_logger_initialized(). This
allows asterisk.c/check_init() to print to the error log once the
logger subsystem is ready instead of just to stdout. If something
does fail before the logger is initialized, we now print to stderr
instead of stdout.
Change-Id: I5f4b50623d9b5a6cb7c5624a8c5c1274c13b2b25
When doing some WebRTC testing, I found that the websocket would
disconnect whenever I attempted to place a call into Asterisk. After
looking into it, I pinpointed the problem to be due to the iostreams
change being merged in.
Under certain circumstances, a call to ast_iostream_read() can return a
negative value. However, in this circumstance, the websocket code was
treating this negative return as if it were a partial read from the
websocket. The expected length would get adjusted by this negative
value, resulting in the expected length being too large.
This patch simply adds an if check to be sure that we are only updating
the expected length of a read when the return from a read is positive.
ASTERISK-26842 #close
Reported by Mark Michelson
Change-Id: Ib4423239828a013d27d7bc477d317d2f02db61ab
fopencookie/funclose is a non-standard API and should not be used
in portable software. Additionally, the way FILE's fd is used in
non-blocking mode is undefined behaviour and cannot be relied on.
This introduces internal abstraction for io streams, that allows
implementing the desired virtualization of read/write operations
with necessary timeout handling.
ASTERISK-24515 #close
ASTERISK-24517 #close
Change-Id: Id916aef418b665ced6a7489aef74908b6e376e85
Not surprisingly, using Respoke (and possibly other systems) it is
possible to blow past the 16k limit for a WebSocket packet size. This
patch bumps it up to 32k, which, at least for Respoke, is sufficient.
For now.
Because 32k is laughable on a LOW_MEMORY system (as is 16k, for that
matter), this patch adds a LOW_MEMORY directive that sets the buffer to
8k for systems who have asked for their reduced memory availability to
be considered.
Change-Id: Id235902537091b58608196844dc4b045e383cd2e
ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes
all traces of it.
Previously exported symbols removed:
* __ast_register_file
* __ast_unregister_file
* ast_complete_source_filename
This also removes the mtx_prof static variable that was declared when
MTX_PROFILE was enabled. This variable was only used in lock.c so it
is now initialized in that file only.
ASTERISK-26480 #close
Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966
Updated ast_websocket_write to encode the entire frame in to one
write operation, to ensure that we don't end up with a situation
where the websocket header has been sent, while the body can not
be written.
Previous to August's patch in commit b9bd3c14, certain network
conditions could cause the header to be written, and then the
sub-sequent body to fail - which would cause the next successful
write to contain a new header, and a new body (resulting in
the peer receiving two headers - the second of which would be
read as part of the body for the first header).
This was patched to have both write operations individually fail
by closing the websocket.
In a case available to the submitter of this patch, the same
body which would consistently fail to write, would succeed
if written at the same time as the header.
This update merges the two operations in to one, adds debug messages
indicating the reason for a websocket connection being closed during
a write operation, and clarifies some variable names for code legibility.
Change-Id: I4db7a586af1c7a57184c31d3d55bf146f1a40598
Due to the use of ast_websocket_close in session termination it is
possible for the underlying socket to already be closed when the
session is terminated. This occurs when the close frame is attempted
to be written out but fails.
Change-Id: I7572583529a42a7dc911ea77a974d8307d5c0c8b
The res_http_websocket module will currently attempt to close
the WebSocket connection if fatal cases occur, such as when
attempting to write out data and being unable to. When the
fatal cases occur the code attempts to write a WebSocket close
frame out to have the remote side close the connection. If
writing this fails then the connection is not terminated.
This change forcefully terminates the connection if the
WebSocket is to be closed but is unable to send the close frame.
ASTERISK-25312 #close
Change-Id: I10973086671cc192a76424060d9ec8e688602845
We don't have a compatability function to fill in a missing htobe64; but
we already have one for the identical htonll.
Change-Id: Ic0a95db1c5b0041e14e6b127432fb533b97e4cac
Commit 39cc28f6ea attempted to fix a
test failure observed on 32 bit test agents by ensuring that a cast from
a 32 bit unsigned integer to a 64 bit unsigned integer was happening in
a predictable place. As it turns out, this did not cause test runs to
succeed.
This commit adds several redundant debug messages that print the payload
lengths of websocket frames. The idea here is that this commit will not
cause tests to succeed for the faulty test agent, but we might deduce
where the fault lies more easily this way by observing at what point the
expected value (537) changes to some ungangly huge number.
If you are wondering why something like this is being committed to the
branch, keep in mind that in commit
39cc28f6ea I noted that the observed test
failures only happen when automated tests are run. Attempts to run the
tests by hand manually on the test agent result in the tests passing.
Change-Id: I14a65c19d8af40dadcdbd52348de3b0016e1ae8d
We have seen a rash of test failures on a 32-bit build agent. Commit
48698a5e21 solved an obvious problem where
we were not encoding a 64-bit value correctly over the wire. This
commit, however, did not solve the test failures.
In the failing tests, ARI is attempting to send a 537 byte text frame
over a websocket. When sending a frame this small, 16 bits are all that
is required in order to encode the payload length on the websocket
frame. However, ast_websocket_write() thinks that the payload length is
greater than 65535 and therefore writes out a 64 bit payload length.
Inspecting this payload length, the lower 32 bits are exactly what we
would expect it to be, 537 in hex. The upper 32 bits, are junk values
that are not expected to be there.
In the failure, we are passing the result of strlen() to a function that
expects a uint64_t parameter to be passed in. strlen() returns a size_t,
which on this 32-bit machine is 32 bits wide. Normally, passing a 32-bit
unsigned value to somewhere where a 64-bit unsigned value is expected
would cause no problems. In fact, in manual runs of failing tests, this
works just fine. However, ast_websocket_write() uses the Asterisk
optional API, which means that rather than a simple function call, there
are a series of macros that are used for its declaration and
implementation. These macros may be causing some sort of error to occur
when converting from a 32 bit quantity to a 64 bit quantity.
This commit changes the logic by making existing ast_websocket_write()
calls use ast_websocket_write_string() instead. Within
ast_websocket_write_string(), the 64-bit converted strlen is saved in a
local variable, and that variable is passed to ast_websocket_write()
instead.
Note that this commit message is full of speculation rather than
certainty. This is because the observed test failures, while always
present in automated test runs, never occur when tests are manually
attempted on the same test agent. The idea behind this commit is to fix
a theoretical issue by performing changes that should, at the least,
cause no harm. If it turns out that this change does not fix the failing
tests, then this commit should be reverted.
Change-Id: I4458dd87d785ca322b89c152b223a540a3d23e67
Prior to ASTERISK-24988, the WebSocket handshake was resolved before Stasis
applications were registered. This was done such that the WebSocket would be
ready when an application is registered. However, by creating the WebSocket
first, the client had the ability to make requests for the Stasis application
it thought had been created with the initial handshake request. The inevitable
conclusion of this scenario was the cart being put before the horse.
ASTERISK-24988 resolved half of the problem by ensuring that the applications
were created and registered with Stasis prior to completing the handshake
with the client. While this meant that Stasis was ready when the client
received the green-light from Asterisk, it also meant that the WebSocket was
not yet ready for Stasis to dispatch messages.
This patch introduces a message queuing mechanism for delaying messages from
Stasis applications while the WebSocket is being constructed. When the ARI
event processor receives the message from the WebSocket that it is being
created, the event processor instantiates an event session which contains a
message queue. It then tries to create and register the requested applications
with Stasis. Messages that are dispatched from Stasis between this point and
the point at which the event processor is notified the WebSocket is ready, are
stashed in the queue. Once the WebSocket has been built, the queue's messages
are dispatched in the order in which they were originally received and the
queue is concurrently cleared.
ASTERISK-25181 #close
Reported By: Matt Jordan
Change-Id: Iafef7b85a2e0bf78c114db4c87ffc3d16d671a17
A test agent was continuously failing all ARI tests when run against
Asterisk 13. As it turns out, the reason for this is that on those test
runs, for some reason we decided to use the super extended 64 bit
payload length for websocket text frames instead of the extended 16 bit
payload length. For 64-bit payloads, the expected byte order over the
network is
7, 6, 5, 4, 3, 2, 1, 0
However, we were sending the payload as
3, 2, 1, 0, 7, 6, 5, 4
This meant that we were saying to expect an absolutely MASSIVE payload
to arrive. Since we did not follow through on this expected payload
size, the client would sit patiently waiting for the rest of the payload
to arrive until the test would time out.
With this change, we use the htobe64() function instead of htonl() so
that a 64-bit byte-swap is performed instead of a 32 bit byte-swap.
Change-Id: Ibcd8552392845fbcdd017a8c8c1043b7fe35964a
This change makes it so that when accepting a WebSocket
connection the HTTP response is sent as one packet instead of
fragmented. Browsers don't like it when you send it fragmented.
ASTERISK-25103
Change-Id: I9b82c4ec2949b0bce692ad0bf6f7cea9709e7f69
This patch updates http_websocket and its corresponding implementation
with a pre-session established callback. This callback allows for
WebSocket server consumers to be notified when a WebSocket connection is
attempted, but before we accept it. Consumers can choose to reject the
connection, if their application specific logic allows for it.
As a result, this patch pulls out the previously private
websocket_protocol struct and makes it public, as
ast_websocket_protocol. In order to preserve backwards compatibility
with existing modules, the existing APIs were left as-is, and new APIs
were added for the creation of the ast_websocket_protocol as well as for
adding a sub-protocol to a WebSocket server.
In particular, the following new API calls were added:
* ast_websocket_add_protocol2 - add a protocol to the core WebSocket
server
* ast_websocket_server_add_protocol2 - add a protocol to a specific
WebSocket server
* ast_websocket_sub_protocol_alloc - allocate a sub-protocol object.
Consumers can populate this with whatever callbacks they wish to
support, then add it to the core server or a specified server.
ASTERISK-24988
Reported by: Joshua Colp
Change-Id: Ibe0bbb30c17eec6b578071bdbd197c911b620ab2
Git does not support the ability to replace a token with a version
string during check-in. While it does have support for replacing a
token on clone, this is somewhat sub-optimal: the token is replaced
with the object hash, which is not particularly easy for human
consumption. What's more, in practice, the source file version was often
not terribly useful. Generally, when triaging bugs, the overall version
of Asterisk is far more useful than an individual SVN version of a file. As a
result, this patch removes Asterisk's support for showing source file
versions.
Specifically, it does the following:
* Rename ASTERISK_FILE_VERSION macro to ASTERISK_REGISTER_FILE, and
remove passing the version in with the macro. Other facilities
than 'core show file version' make use of the file names, such as
setting a debug level only on a specific file. As such, the act of
registering source files with the Asterisk core still has use. The
macro rename now reflects the new macro purpose.
* main/asterisk:
- Refactor the file_version structure to reflect that it no longer
tracks a version field.
- Remove the "core show file version" CLI command. Without the file
version, it is no longer useful.
- Remove the ast_file_version_find function. The file version is no
longer tracked.
- Rename ast_register_file_version/ast_unregister_file_version to
ast_register_file/ast_unregister_file, respectively.
* main/manager: Remove value from the Version key of the ModuleCheck
Action. The actual key itself has not been removed, as doing so would
absolutely constitute a backwards incompatible change. However, since
the file version is no longer tracked, there is no need to attempt to
include it in the Version key.
* UPGRADE: Add notes for:
- Modification to the ModuleCheck AMI Action
- Removal of the "core show file version" CLI command
Change-Id: I6cf0ff280e1668bf4957dc21f32a5ff43444a40e
Some WebSocket applications, like [chan_respoke][], require a larger
frame size than the default 8k; this patch bumps the default to 16k.
This patch also fixes some problems exacerbated by large frames.
The sanity counter was decremented on every fread attempt in
ws_safe_read(), regardless of whether data was read from the socket or
not. For large frames, this could result in loss of sanity prior to
reading the entire frame. (16k frame / 1448 bytes per segment = 12
segments).
This patch changes the sanity counter so that it only decrements when
fread() doesn't read any bytes. This more closely matches the original
intention of ws_safe_read(), given that the error message is
"Websocket seems unresponsive".
This patch also properly logs EOF conditions, so disconnects are no
longer confused with unresponsive connections.
[chan_respoke]: https://github.com/respoke/chan_respoke
Review: https://reviewboard.asterisk.org/r/4431/
........
Merged revisions 432236 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 432237 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@432238 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Persistent HTTP connection support is needed due to the increased usage of
the Asterisk core HTTP transport and the frequency at which REST API calls
are going to be issued.
* Add http.conf session_keep_alive option to enable persistent
connections.
* Parse and discard optional chunked body extension information and
trailing request headers.
* Increased the maximum application/json and
application/x-www-form-urlencoded body size allowed to 4k. The previous
1k was kind of small.
* Removed a couple inlined versions of ast_http_manid_from_vars() by
calling the function. manager.c:generic_http_callback() and
res_http_post.c:http_post_callback()
* Add missing va_end() in ast_ari_response_error().
* Eliminated unnecessary RAII_VAR() use in http.c:auth_create().
ASTERISK-23552 #close
Reported by: Scott Griepentrog
Review: https://reviewboard.asterisk.org/r/3691/
........
Merged revisions 417880 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@417901 65c4cc65-6c06-0410-ace0-fbb531ad65f3
When a client takes a long time to process information received from Asterisk,
a write operation using fwrite may fail to write all information. This causes
the underlying file stream to be in an unknown state, such that the socket
must be disconnected. Unfortunately, there are two problems with this in
Asterisk's existing websocket code:
1. Periodically, during the read loop, Asterisk must write to the connected
websocket to respond to pings. As such, Asterisk maintains a reference to
the session during the loop. When ast_http_websocket_write fails, it may
cause the session to decrement its ref count, but this in and of itself
does not break the read loop. The read loop's write, on the other hand,
does not break the loop if it fails. This causes the socket to get in a
'stuck' state, preventing the client from reconnecting to the server.
2. More importantly, however, is that the fwrite in ast_http_websocket_write
fails with a large volume of data when the client takes awhile to process
the information. When it does fail, it fails writing only a portion of
the bytes. With some debugging, it was shown that this was failing in a
similar fashion to ASTERISK-12767. Switching this over to ast_careful_fwrite
with a long enough timeout solved the problem.
Note that this version of the patch, unlike r417310 in Asterisk 11, exposes
configuration options beyond just chan_sip's sip.conf. Configuration options
to configure the write timeout have also been added to pjsip.conf and ari.conf.
#ASTERISK-23917 #close
Reported by: Matt Jordan
Review: https://reviewboard.asterisk.org/r/3624/
........
Merged revisions 417310 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 417311 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@417317 65c4cc65-6c06-0410-ace0-fbb531ad65f3
There was a problem when reading a string from the websocket. It assumed the
received data had a null terminator and tried to write the data to an ast_str.
This of course could/would read past the end of the given buffer while
writing the data to the internal buffer of ast_str. Modified the the code to
correctly place a null terminator on the result string.
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@416394 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Simply establishing a TCP connection and never sending anything to the
configured HTTP port in http.conf will tie up a HTTP connection. Since
there is a maximum number of open HTTP sessions allowed at a time you can
block legitimate connections.
A similar problem exists if a HTTP request is started but never finished.
* Added http.conf session_inactivity timer option to close HTTP
connections that aren't doing anything. Defaults to 30000 ms.
* Removed the undocumented manager.conf block-sockets option. It
interferes with TCP/TLS inactivity timeouts.
* AMI and SIP TLS connections now have better authentication timeout
protection. Though I didn't remove the bizzare TLS timeout polling code
from chan_sip.
* chan_sip can now handle SSL certificate renegotiations in the middle of
a session. It couldn't do that before because the socket was non-blocking
and the SSL calls were not restarted as documented by the OpenSSL
documentation.
* Fixed an off nominal leak of the ssl struct in
handle_tcptls_connection() if the FILE stream failed to open and the SSL
certificate negotiations failed.
The patch creates a custom FILE stream handler to give the created FILE
streams inactivity timeout and timeout after a specific moment in time
capability. This approach eliminates the need for code using the FILE
stream to be redesigned to deal with the timeouts.
This patch indirectly fixes most of ASTERISK-18345 by fixing the usage of
the SSL_read/SSL_write operations.
ASTERISK-23673 #close
Reported by: Richard Mudgett
........
Merged revisions 415841 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........
Merged revisions 415854 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 415896 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@415907 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Added a websocket server client in Asterisk. Asterisk has a websocket server,
but not a client. The ability to have Asterisk be able to connect to a websocket
server can potentially be useful for future work (for instance this could allow
ARI to connect back to some external system, although more work would be needed
in order to incorporate that).
Also a couple of things to note - proxy connection support has not been
implemented and there is limited http response code handling (basically, it is
connect or not).
Also added an initial new URI handling mechanism to core. Internet type URI's
are parsed into a data structure that contains pointers to the various parts of
the URI.
(closes issue ASTERISK-23742)
Reported by: Kevin Harwell
Review: https://reviewboard.asterisk.org/r/3541/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@415223 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Several fixes for the WebSockets implementation in res/res_http_websocket.c
* Flush the websocket session FILE* as fwrite() may not actually guarantee sending
the data to the network. If we do not flush, it seems that buffering on the SSL
socket for outbound messages causes issues
* Refactored ast_websocket_read to take into account that SSL file descriptors
may be ready to read via fread() but poll() will not actually say so because
the data was already read from the network buffers and is now in the libc buffers
(closes issue ASTERISK-23099)
(closes issue ASTERISK-21930)
Review: https://reviewboard.asterisk.org/r/3248/
........
Merged revisions 409681 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 409697 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@409831 65c4cc65-6c06-0410-ace0-fbb531ad65f3
When I moved the ARI WebSocket from /ws to /ari/events, I added code to
allow a WebSocket to connect without specifying the subprotocol if
there's only one subprotocol handler registered for the WebSocket.
Naively, I coded it to always respond with the subprotocol in use.
Unfortunately, according to RFC 6455, if the server's response includes
a subprotocol header field that "indicates the use of a subprotocol that
was not present in the client's handshake [...], the client MUST _Fail
the WebSocket Connection_.", emphasis theirs.
This patch correctly omits the Sec-WebSocket-Protocol if one is not
specified by the client.
(closes issue ASTERISK-22441)
Review: https://reviewboard.asterisk.org/r/2828/
........
Merged revisions 399039 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@399042 65c4cc65-6c06-0410-ace0-fbb531ad65f3
With the new work in Asterisk 12, there are some uses of the
optional_api that are prone to failure. The details are rather involved,
and captured on [the wiki][1].
This patch addresses the issue by removing almost all of the magic from
the optional API implementation. Instead of relying on weak symbol
resolution, a new optional_api.c module was added to Asterisk core.
For modules providing an optional API, the pointer to the implementation
function is registered with the core. For modules that use an optional
API, a pointer to a stub function, along with a optional_ref function
pointer are registered with the core. The optional_ref function pointers
is set to the implementation function when it's provided, or the stub
function when it's now.
Since the implementation no longer relies on magic, it is now supported
on all platforms. In the spirit of choice, an OPTIONAL_API flag was
added, so we can disable the optional_api if needed (maybe it's buggy on
some bizarre platform I haven't tested on)
The AST_OPTIONAL_API*() macros themselves remained unchanged, so
existing code could remain unchanged. But to help with debugging the
optional_api, the patch limits the #include of optional API's to just
the modules using the API. This also reduces resource waste maintaining
optional_ref pointers that aren't used.
Other changes made as a part of this patch:
* The stubs for http_websocket that wrap system calls set errno to
ENOSYS.
* res_http_websocket now properly increments module use count.
* In loader.c, the while() wrappers around dlclose() were removed. The
while(!dlclose()) is actually an anti-pattern, which can lead to
infinite loops if the module you're attempting to unload exports a
symbol that was directly linked to.
* The special handling of nonoptreq on systems without weak symbol
support was removed, since we no longer rely on weak symbols for
optional_api.
[1]: https://wiki.asterisk.org/wiki/x/wACUAQ
(closes issue ASTERISK-22296)
Reported by: Matt Jordan
Review: https://reviewboard.asterisk.org/r/2797/
........
Merged revisions 397989 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397990 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This patch moves the RESTful URL's around to more appropriate
locations for release.
The /stasis URL's are moved to /ari, since Asterisk REST Interface was
a more appropriate name than Stasis-HTTP. (Most of the code still has
stasis_http references, but they will be cleaned up after there are no
more outstanding branches that would have merge conflicts with such a
change).
A larger change was moving the ARI events WebSocket off of the shared
/ws URL to its permanent home on /ari/events. The Swagger code
generator was extended to handle "upgrade: websocket" and
"websocketProtocol:" attributes on an operation.
The WebSocket module was modified to better handle WebSocket servers
that have a single registered protocol handler. If a client
connections does not specify the Sec-WebSocket-Protocol header, and
the server has a single protocol handler registered, the WebSocket
server will go ahead and accept the client for that subprotocol.
(closes issue ASTERISK-21857)
Review: https://reviewboard.asterisk.org/r/2621/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@393528 65c4cc65-6c06-0410-ace0-fbb531ad65f3