Add an AMI events Load and Unload for notify when the
module has been loaded and unloaded.
ASTERISK-27661
Change-Id: Ib916c41eddd63651952998f2f49c57c42ef87a64
This addresses all performance issues with 'module load' completion. In
addition to using ast_cli_completion_add we stop using libedit's
filename_completion_function, instead using ast_file_read_dir. This
ensures all results are produced from a single call to opendir.
Change-Id: I8bf51ffaa7ef1606f3bd1b5bb13f1905d72c6134
The code which handled loading modules had too many situations which
would result in halting Asterisk startup. Treat most errors as declines
instead of failures. The exception is when the module load function
returns AST_MODULE_LOAD_FAILURE or an invalid code.
Clear the missingdeps vector when appropriate to ensure the next loop
starts clean.
ASTERISK-27620
Change-Id: I45547d9641fd45bd86d80250224417625631ad84
* Add SRC_EMBEDDED variable to main/Makefile. Built-in module sources
must be listed in this variable to ensure they get the correct CFLAGS.
Change-Id: I920852bc17513a9c2627061a4ad40511e3a20499
Use a single loop in a loop to scan the resource list attempting to
dlopen each module. The inner loop is repeated until it doesn't do any
work, then it is run one more time to allow printing of error messages.
Change-Id: I60c15cd57ff9680b62e2a94c7519401fa4a38e45
Dependency loader is now in place so we no longer need a separate loader
phase for global symbols only. This simplifies the loader and allows us
to minimize calls to dlopen.
Change-Id: I33e3174d67f3b4552d3d536326dcaf0ebabb097d
* Add string vectors for requires, optional_apis and enhances.
* Add reffed_deps module vector for holding references to dependencies.
* Initialize string vectors after final dlopen of each module.
* Free string vectors and clear references from reffed_deps in
module_destroy.
* Create functions necessary to process module dependencies and enforce
load order.
Module dependencies result in automatic references being managed by the
module loader. This enforces unload order.
Change-Id: I9be08d1dd331aceadc1dcba00b804d71360b2fbb
* Remove comment about lazy load.
* Improve message about module already being loaded and running.
* Handle allocation error in add_to_load_order.
* Dead code elimination from modules_shutdown.
Change-Id: I22261599c46d0f416e568910ec9502f45143197f
This function returns NULL if the module in question is not running. I
did not change ast_module_ref as most callers do not check the result
and they always call ast_module_unref.
Make use of this function when running registered items from:
* app_stack API's
* bridge technologies
* CLI commands
* File formats
* Manager Actions
* RTP engines
* Sorcery Wizards
* Timing Interfaces
* Translators
* AGI Commands
* Fax Technologies
ASTERISK-20346 #close
Change-Id: Ia16fd28e188b2fc0b9d18b8a5d9cacc31df73fcc
Some compiler optimizers seem to assume that dlopen will not use
__attribute__((constructor)) functions to call back to the program.
This was causing resource_being_loaded to be optimized away completely.
ASTERISK-27531 #close
Tested By: abelbeck
Change-Id: If17a3b889e06811a0e7119f0539d052494d6ece9
This is needed for future changes which will require being able to
process the load priority out of order.
Change-Id: Ia23421197f09789940510b03ebbbf3bf24d51bea
* Split off load_dlopen to perform actual dlopen, check results and log
warnings when needed.
* Always use RTLD_NOW.
* Use flags which minimize number of calls to dlopen required. First
attempt always uses RTLD_GLOBAL when global_symbols_only is enabled,
RTLD_LOCAL when it is not.
This patch significantly reduces the number of dlopen's performed. With
299 modules my system ran dlopen 857 times before this patch, 655 times
after this patch.
Change-Id: Ib2c9903cfddcc01aed3e01c1e7fe4a3fb9af0f8b
This protects the module loader itself against crashing if dlopen is
called on a module from outside loader.c.
* Expand scope of lock inside ast_module_register to include reading of
resource_being_loaded.
* NULL check resource_being_loaded.
* Set resource_being_loaded NULL as soon as dlopen returns. This fixes
some error paths where it was not NULL'ed.
* Create module_destroy function to deduplicate code from
ast_module_unregister and modules_shutdown.
* Resolve leak that occured if a module did not successfully register.
* Simplify checking for successful registration.
Change-Id: I40f07a315e55b92df4fc7faf525ed6d4f396e7d2
Optimize resource_name_match. This change eliminates use of
ast_strdupa, instead verifying that both basename's are the same length,
then using strncasecmp.
Change-Id: I477275c0e954c99d74be5abfc8bb6545b04e5a3d
Make the comments follow doxygen format, move comments to the line
before each field they describe.
Change-Id: Ic445468398b5e88f13910f7c2f70bd15aad33a27
Replace 'needsreload' argument with a 'type' argument to specify which
type of modules you want completion. This provides more accurate CLI
completion for load and unload commands.
* 'module unload' now excludes modules that have active references or are
not running.
* 'module load' now excludes modules that are already running.
* 'core set debug [atleast] <level> [module]' shows running modules only.
ASTERISK-27378
Change-Id: Iea3e00054461484196c46f688f02635cc886bad1
* Stop using ast_module_helper to check if a module is loaded, use
ast_module_check instead (app_confbridge and app_meetme).
* Stop ast_module_helper from listing reload classes when needsreload
was not requested.
ASTERISK-27378
Change-Id: Iaed8c1e4fcbeb242921dbac7929a0fe75ff4b239
In all non-pbx modules, AST_MODULE_LOAD_FAILURE has been changed
to AST_MODULE_LOAD_DECLINE. This prevents asterisk from exiting
if a module can't be loaded. If the user wishes to retain the
FAILURE behavior for a specific module, they can use the "require"
or "preload-require" keyword in modules.conf.
A new API was added to logger: ast_is_logger_initialized(). This
allows asterisk.c/check_init() to print to the error log once the
logger subsystem is ready instead of just to stdout. If something
does fail before the logger is initialized, we now print to stderr
instead of stdout.
Change-Id: I5f4b50623d9b5a6cb7c5624a8c5c1274c13b2b25
It is only safe to run ast_register_cleanup callbacks when all modules
have been unloaded. Previously these callbacks were run during graceful
shutdown, making it possible to crash during shutdown.
ASTERISK-26513 #close
Change-Id: Ibfa635bb688d1227ec54aa211d90d6bd45052e21
ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes
all traces of it.
Previously exported symbols removed:
* __ast_register_file
* __ast_unregister_file
* ast_complete_source_filename
This also removes the mtx_prof static variable that was declared when
MTX_PROFILE was enabled. This variable was only used in lock.c so it
is now initialized in that file only.
ASTERISK-26480 #close
Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966
sd_notify() is used to notify systemd of changes to the status of the
process. This allows the systemd daemon to know when the process
finished loading (and thus only start another program after Asterisk has
finished loading).
To use this, use a systemd unit with 'Type=notify' for Asterisk.
This commit also adds the function ast_sd_notify(), a wrapper around
sd_notify that does nothing if not built with systemd support.
Also adds support for libsystemd detection in the configure script.
Change-Id: Ied6a59dafd5ef331c5c7ae8f3ccd2dfc94be7811
Although we use the RTLD_LAZY flag when calling dlopen
the first time on a module, this only defers resolution
for function calls. Pointer references to functions are
determined at link time so dlopen expects them to be there.
Since we don't cross-module link, pointers to functions
in other modules won't be available and dlopen will fail.
Doing a "hardened" build also causes problems because it
typically sets "-z now" on the ld command line which
overrides RTLD_LAZY at run time.
If the failing module isn't a GLOBAL_SYMBOLS module, then
dlopen will be called again after all the GLOBAL_SYMBOLS
modules have been loaded and they'll eventually resolve.
If the calling module IS a GLOBAL_SYMBOLS module itself
and a third module depends on it, then there's an issue
because the second time through the dlopen loop,
GLOBAL_SYMBOLS modules aren't given any special treatment
and since the order in which dlopen is called isn't
deterministic, the dependent may again be tried before the
module it needs is loaded.
Simple solution: Save modules that fail load_resource
because of a dlopen error in a list and retry them
immediately after the first pass. Keep retrying until
the failed list is empty or we reach a #defined max
retries. Error messages are suppressed until the final
pass which also gets rid of those confusing error messages
about module failures that are later corrected.
Change-Id: Iddae1d97cd2f00b94e61662447432765755f64bb
The JSON library Asterisk uses, jansson, is not thread
safe for us in a few ways. To help with this wrappers for JSON
object reference count increasing and decreasing were added
which use a global lock to ensure they don't clobber over
each other. This does not extend to reference count manipulation
within the jansson library itself. This means you can't safely
use the object borrowing specifier (O) in ast_json_pack and
you can't share JSON instances between objects.
This change removes uses of the O specifier and replaces them
with the o specifier and an explicit ast_json_ref. Some cases
of instance sharing have also been removed.
ASTERISK-25601 #close
Change-Id: I06550d8b0cc1bfeb56cab580a4e608ae4f1ec7d1
An http request can be sent to retrieve information on a single
module, including the resource name, description, use count, status,
and support level.
The command "curl -v -u user:pass -X GET 'http://localhost:8088/ari
/asterisk/modules/{moduleName}'" (or something similar, depending on
configuration) can be run in the terminal to access this new
functionality.
For more information, see:
https://wiki.asterisk.org/wiki.display/~bford/Asterisk+ARI+Resource
* Added new ARI functionality
* Information on a single module can now be retrieved
ASTERISK-25173
Change-Id: Ibce5a94e70ecdf4e90329cf0ba66c33a62d37463
An http request can be sent to retrieve a list of all existing modules,
including the resource name, description, use count, status, and
support level.
The command "curl -v -u user:pass -X GET 'http://localhost:8088/ari/
asterisk/modules" (or something similar, depending on configuration)
can be run in the terminal to access this new functionality.
For more information, see:
https://wiki.asterisk.org/wiki.display/~bford/Asterisk+ARI+Resource
* Added new ARI functionality
* Information on modules can now be retrieved
Change-Id: I63cbbf0ec0c3544cc45ed2a588dceabe91c5e0b0
* The REF_DEBUG compiler flag no longer has any effect on code that uses
Astobj2. It is used to determine if reference debugging is enabled by
default. Reference debugging can be enabled or disabled in asterisk.conf.
* Caller information is provided in logger errors for ao2 bad magic numbers.
* Optimizes AO2 by merging internal functions with the public counterpart.
This was possible now that we no longer require a dual ABI.
ASTERISK-24974 #close
Reported by: Corey Farrell
Change-Id: Icf3552721fe999365ba8a8cf00a965aa6b897cc1
Git does not support the ability to replace a token with a version
string during check-in. While it does have support for replacing a
token on clone, this is somewhat sub-optimal: the token is replaced
with the object hash, which is not particularly easy for human
consumption. What's more, in practice, the source file version was often
not terribly useful. Generally, when triaging bugs, the overall version
of Asterisk is far more useful than an individual SVN version of a file. As a
result, this patch removes Asterisk's support for showing source file
versions.
Specifically, it does the following:
* Rename ASTERISK_FILE_VERSION macro to ASTERISK_REGISTER_FILE, and
remove passing the version in with the macro. Other facilities
than 'core show file version' make use of the file names, such as
setting a debug level only on a specific file. As such, the act of
registering source files with the Asterisk core still has use. The
macro rename now reflects the new macro purpose.
* main/asterisk:
- Refactor the file_version structure to reflect that it no longer
tracks a version field.
- Remove the "core show file version" CLI command. Without the file
version, it is no longer useful.
- Remove the ast_file_version_find function. The file version is no
longer tracked.
- Rename ast_register_file_version/ast_unregister_file_version to
ast_register_file/ast_unregister_file, respectively.
* main/manager: Remove value from the Version key of the ModuleCheck
Action. The actual key itself has not been removed, as doing so would
absolutely constitute a backwards incompatible change. However, since
the file version is no longer tracked, there is no need to attempt to
include it in the Version key.
* UPGRADE: Add notes for:
- Modification to the ModuleCheck AMI Action
- Removal of the "core show file version" CLI command
Change-Id: I6cf0ff280e1668bf4957dc21f32a5ff43444a40e
Until we have a true module management facility it's sometimes necessary for one
module to force a reload on another before its own load is complete. If
Asterisk isn't fully booted yet, these reloads are deferred. The problem is
that asterisk reports fully booted before processing the deferred reloads which
means Asterisk really isn't quite ready when it says it is.
This patch moves the report of fully booted after the processing of the deferred
reloads is complete.
Since the pjsip stack has the most number of related modules, I ran the
channels/pjsip testsuite to make sure there aren't any issues. All tests
passed.
Tested-by: George Joseph
Review: https://reviewboard.asterisk.org/r/4604/
........
Merged revisions 434544 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@434546 65c4cc65-6c06-0410-ace0-fbb531ad65f3
In r413586 (1.8) various casts were added to silence gcc 4.10 warnings.
Those fixes included things like:
-out += sprintf(out, "%%%02X", (unsigned char) *ptr);
+out += sprintf(out, "%%%02X", (unsigned) *ptr);
That works for low ascii characters, but for the high range that yields
e.g. FFFFFFC3 when C3 is expected.
This changeset:
- fixes those casts to use the 'hh' unsigned char modifier instead
- consistently uses %02x instead of %2.2x (or other non-standard usage)
- adds a few 'h' modifiers in various places
- fixes a 'replcaes' typo
- dev/urandon typo (in 13+ patch)
Review: https://reviewboard.asterisk.org/r/4263/
ASTERISK-24619 #close
Reported by: Stefan27 (on IRC)
........
Merged revisions 429673 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 429674 from http://svn.asterisk.org/svn/asterisk/branches/12
........
Merged revisions 429675 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@429683 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This gets rid of most old libc free/malloc/realloc and replaces them
with ast_free and friends. When compiling with MALLOC_DEBUG you'll
notice it when you're mistakenly using one of the libc variants. For
the legacy cases you can define WRAP_LIBC_MALLOC before including
asterisk.h.
Even better would be if the errors were also enabled when compiling
without MALLOC_DEBUG, but that's a slightly more invasive header
file change.
Those compiling addons/format_mp3 will need to rerun
./contrib/scripts/get_mp3_source.sh.
ASTERISK-24348 #related
Review: https://reviewboard.asterisk.org/r/4015/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@423978 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This introduces stasis.conf and a mechanism to prevent certain message
types from being published. Internally, this works by preventing the
chosen message types from being created which ensures that those
message types can never be published. This patch also adjusts message
publishers such that message payloads are not created if the related
message type is not available.
ASTERISK-23943 #close
Review: https://reviewboard.asterisk.org/r/3823/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@420124 65c4cc65-6c06-0410-ace0-fbb531ad65f3
When creating the alphabetical sorted list each module is added to a list
temporarily. On the second iteration each module already has a pointer to
another module, causing stuff to go into a loop.
ASTERISK-24123 #close
Reported by: Malcolm Davenport
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@419612 65c4cc65-6c06-0410-ace0-fbb531ad65f3
When Asterisk starts a module (calling its load_module function), it re-orders
the module list, sorting it alphabetically. Ostensibly, this was done so that
the output of 'module show' listed modules in alphabetic order. This had the
unfortunate side effect of making modules with complex usage patterns
unloadable. A module that has a large number of modules that depend on it is
typically abandoned during the unloading process. This results in its memory
not being reclaimed during exit.
Generally, this isn't harmful - when the process is destroyed, the operating
system will reclaim all memory allocated by the process. Prior to Asterisk 12,
we also didn't have many modules with complex dependencies. However, with
the advent of ARI and PJSIP, this can make make unloading those modules
successfully nearly impossible, and thus tracking memory leaks or ref debug
leaks a real pain.
While this patch is not a complete overhaul of the module loader - such an
effort would be beyond the scope of what could be done for Asterisk 13 -
this does make some marginal improvements to the loader such that modules
like res_pjsip or res_stasis *may* be made properly un-loadable in the future.
1. The linked list of modules has been replaced with a doubly linked list. This
allows traversal of the module list to occur backwards. The module shutdown
routine now walks the global list backwards when it attempts to unload
modules.
2. The alphabetic reorganization of the module list on startup has been
removed. Instead, a started module is placed at the end of the module list.
3. The ast_update_module_list function - which is used by the CLI to display
the modules - now does the sorting alphabetically itself. It creates its own
linked list and inserts the modules into it in alphabetic order. This allows
for the intent of the previous code to be maintained.
This patch also contains a fix for res_calendar. Without calendar.conf, the
calendar modules were improperly bumping the use count of res_calendar, then
failing to load themselves. This patch makes it so that we detect whether or
not calendaring is enabled before altering the use count.
Review: https://reviewboard.asterisk.org/r/3777/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@419563 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This patch addresses some aesthetic issues in Asterisk. These are all just
minor tweaks to improve the look of the CLI when used in a variety of
settings. Specifically:
* A number of chatty verbose messages were removed or demoted to DEBUG
messages. Verbose messages with a verbosity level of 5 or higher were -
if kept as verbose messages - demoted to level 4. Several messages
that were emitted at verbose level 3 were demoted to 4, as announcement
of dialplan applications being executed occur at level 3 (and so the
effects of those applications should generally be less).
* Some verbose messages that only appear when their respective 'debug'
options are enabled were bumped up to always be displayed.
* Prefix/timestamping of verbose messages were moved to the verboser
handlers. This was done to prevent duplication of prefixes when the
timestamp option (-T) is used with the CLI.
* Verbose magic is removed from messages before being emitted to
non-verboser handlers. This prevents the magic in multi-line verbose
messages (such as SIP debug traces or the output of DumpChan) from
being written to files.
* _Slightly_ better support for the "light background" option (-W) was
added. This includes using ast_term_quit in the output of XML
documentation help, as well as changing the "Asterisk Ready" prompt to
bright green on the default background (which stands a better chance of
being displayed properly than bright white).
Review: https://reviewboard.asterisk.org/r/3547/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@414798 65c4cc65-6c06-0410-ace0-fbb531ad65f3
For awhile now, we've noticed continuous integration builds hanging on CentOS 6
64-bit build agents. After resolving a number of problems with symbols, strange
locks, and other shenanigans, the problem has persisted. In all cases, gdb
shows the Asterisk process stuck in loader.c on one of the infinite while loops
that calls dlclose repeatedly until success.
The documentation of dlclose states that it returns 0 on success; any other
value on error. It does not state that repeatedly calling it will eventually
clear those errors. Most likely, the repeated calls to dlclose was to force a
close by exhausting the references on the library; however, that will never
succeed if:
(a) There is some fundamental error at work in the loaded library that
precludes unloading it
(b) Some other loaded module is referencing a symbol in the currently loaded
module
This results in Asterisk sitting forever.
Since we have matching pairs of dlopen/dlclose, this patch opts to only call
dlclose once, and log out as an ERROR if dlclose fails to return success. If
nothing else, this might help to determine why on the CentOS 6 64-bit build agent
things are not closing successfully.
Review: https://reviewboard.asterisk.org/r/2970
........
Merged revisions 402287 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........
Merged revisions 402288 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 402289 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@402290 65c4cc65-6c06-0410-ace0-fbb531ad65f3
With the new work in Asterisk 12, there are some uses of the
optional_api that are prone to failure. The details are rather involved,
and captured on [the wiki][1].
This patch addresses the issue by removing almost all of the magic from
the optional API implementation. Instead of relying on weak symbol
resolution, a new optional_api.c module was added to Asterisk core.
For modules providing an optional API, the pointer to the implementation
function is registered with the core. For modules that use an optional
API, a pointer to a stub function, along with a optional_ref function
pointer are registered with the core. The optional_ref function pointers
is set to the implementation function when it's provided, or the stub
function when it's now.
Since the implementation no longer relies on magic, it is now supported
on all platforms. In the spirit of choice, an OPTIONAL_API flag was
added, so we can disable the optional_api if needed (maybe it's buggy on
some bizarre platform I haven't tested on)
The AST_OPTIONAL_API*() macros themselves remained unchanged, so
existing code could remain unchanged. But to help with debugging the
optional_api, the patch limits the #include of optional API's to just
the modules using the API. This also reduces resource waste maintaining
optional_ref pointers that aren't used.
Other changes made as a part of this patch:
* The stubs for http_websocket that wrap system calls set errno to
ENOSYS.
* res_http_websocket now properly increments module use count.
* In loader.c, the while() wrappers around dlclose() were removed. The
while(!dlclose()) is actually an anti-pattern, which can lead to
infinite loops if the module you're attempting to unload exports a
symbol that was directly linked to.
* The special handling of nonoptreq on systems without weak symbol
support was removed, since we no longer rely on weak symbols for
optional_api.
[1]: https://wiki.asterisk.org/wiki/x/wACUAQ
(closes issue ASTERISK-22296)
Reported by: Matt Jordan
Review: https://reviewboard.asterisk.org/r/2797/
........
Merged revisions 397989 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397990 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This reworks the CLI commands used to access sounds information from
"sounds show[ soundid]" to "core show sounds" and
"core show sound <soundid>". This also reworks the "sounds reload" CLI
command to fall under normal module reloading ("module reload sounds").
Also, make trunk build when DEBUG_MALLOC is not enabled.
Review: https://reviewboard.asterisk.org/r/2745/
(closes issue ASTERISK-22141)
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396829 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Sungtae Kim
Jenkins2
Corey Farrell
Joshua Colp
George Joseph
Sean Bright
Tzafrir Cohen
George Joseph
Benjamin Ford
Matt Jordan
Walter Doekes
Kinsey Moore
Mark Michelson
David M. Lee