Commit Graph

15 Commits

Author SHA1 Message Date
Matthew Jordan 3ea0d38396 media cache: Add a core API and facade for a backend agnostic media cache
This patch adds a new API to the Asterisk core that acts as a media
cache. The core API itself is mostly a thin wrapper around some bucket
API provided implementation that itself acts as the mechanism of
retrieval for media. The media cache API in the core provides the
following:
 * A very thin in-memory cache of the active bucket_file items. Unlike a
   more traditional cache, it provides no expiration mechanisms. Most
   queries that hit the in-memory cache will also call into the bucket
   implementations as well. The bucket implementations are responsible
   for determining whether or not the active record is active and valid.
   This makes sense for the most likely implementation of a media cache
   backend, i.e., HTTP. The HTTP layer itself is the actual arbiter of
   whether or not a record is truly active; as such, the in-memory cache
   in the core has to defer to it.
 * The ability to create new items in the media cache from local
   resources. This allows for re-creation of items in the cache on
   restart.
 * Synchronization of items in the media cache to the AstDB. This
   also includes various pieces of important metadata.

The API provides sufficient access that higher level APIs, such as the
file or app APIs, do not have to worry about the semantics of the bucket
APIs when needing to playback a resource.

In addition, this patch provides unit tests for the media cache API. The
unit tests use a fake bucket backend to verify correctness.

Change-Id: I11227abbf14d8929eeb140ddd101dd5c3820391e
2015-07-12 20:44:16 -05:00
Matt Jordan 887945d410 main/bucket: Add a callback function for ast_bucket_file objects
This patch adds a new function to the bucket API for ast_bucket_file
objects, ast_bucket_file_metadata_callback. It will call ao2_callback on
the ast_bucket_file's ao2_container of metadata, calling the provided
ao2_callback_fn callback on each piece of metadata associated with the
file.

This is particularly useful when a bucket backend has added metadata,
and a higher level API wants to be aware of/access said metadata,
without knowing for sure what the key is.

Change-Id: I96f6757717f47b650df91a437f7df16406227466
2015-07-12 20:44:16 -05:00
Matt Jordan ef8d3f6506 bucket: Add clone/staleness operations for ast_bucket/ast_bucket_file
This patch enhances the bucket API in two ways.

First, since ast_bucket and ast_bucket_file instances are immutable, a 'clone'
operation has been added that provides a 'clone' of an existing
ast_bucket/ast_bucket_file object. Note that this makes use of the
ast_sorcery_copy operation, along with the copy callback handler on the
"bucket" and "file" object types for the bucket sorcery instance.

Second, there is a need for the bucket API to ask a wizard if an object
is stale. This is particularly useful with the upcoming media cache
enhancements, where we want to ask the backing data storage if the
object we are currently operating on has known updates. This patch adds
API calls for ast_bucket and ast_bucket_file objects, which callback
into their respective sorcery wizards via the sorcery API.

Unit tests have also been added to cover the respective
ast_bucket/ast_bucket_file clone and staleness operations.

Change-Id: Ib0240ba915ece313f1678a085a716021d75d6b4a
2015-07-04 20:32:09 -05:00
Matt Jordan 4a58261694 git migration: Refactor the ASTERISK_FILE_VERSION macro
Git does not support the ability to replace a token with a version
string during check-in. While it does have support for replacing a
token on clone, this is somewhat sub-optimal: the token is replaced
with the object hash, which is not particularly easy for human
consumption. What's more, in practice, the source file version was often
not terribly useful. Generally, when triaging bugs, the overall version
of Asterisk is far more useful than an individual SVN version of a file. As a
result, this patch removes Asterisk's support for showing source file
versions.

Specifically, it does the following:

* Rename ASTERISK_FILE_VERSION macro to ASTERISK_REGISTER_FILE, and
  remove passing the version in with the macro. Other facilities
  than 'core show file version' make use of the file names, such as
  setting a debug level only on a specific file. As such, the act of
  registering source files with the Asterisk core still has use. The
  macro rename now reflects the new macro purpose.

* main/asterisk:
  - Refactor the file_version structure to reflect that it no longer
    tracks a version field.
  - Remove the "core show file version" CLI command. Without the file
    version, it is no longer useful.
  - Remove the ast_file_version_find function. The file version is no
    longer tracked.
  - Rename ast_register_file_version/ast_unregister_file_version to
    ast_register_file/ast_unregister_file, respectively.

* main/manager: Remove value from the Version key of the ModuleCheck
  Action. The actual key itself has not been removed, as doing so would
  absolutely constitute a backwards incompatible change. However, since
  the file version is no longer tracked, there is no need to attempt to
  include it in the Version key.

* UPGRADE: Add notes for:
  - Modification to the ModuleCheck AMI Action
  - Removal of the "core show file version" CLI command

Change-Id: I6cf0ff280e1668bf4957dc21f32a5ff43444a40e
2015-04-13 03:48:57 -04:00
Richard Mudgett e953d15223 A couple minor cleanup tweaks.
* In res/res_sorcery_realtime.c: Broke long line.

* In main/bucket.c: Eliminated unnecessary NULL check as
ast_sorcery_unref() is NULL tolerant and set the global object to NULL
after unref in the system shutdown bucket_cleanup().
........

Merged revisions 433420 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433421 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2015-03-25 18:37:51 +00:00
Corey Farrell e66b874f5d Allow shutdown to unload modules that register bucket scheme's or codec's.
* Change __ast_module_shutdown_ref to be NULL safe (11+).
* Allow modules that call ast_bucket_scheme_register or ast_codec_register
  to be unloaded during graceful shutdown only (13+ only).

ASTERISK-24796 #close
Reported by: Corey Farrell
Review: https://reviewboard.asterisk.org/r/4428/
........

Merged revisions 432058 from http://svn.asterisk.org/svn/asterisk/branches/11
........

Merged revisions 432059 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@432060 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2015-02-21 02:58:19 +00:00
Kinsey Moore abd3e4040b Allow Asterisk to compile under GCC 4.10
This resolves a large number of compiler warnings from GCC 4.10 which
cause the build to fail under dev mode. The vast majority are
signed/unsigned mismatches in printf-style format strings.
........

Merged revisions 413586 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........

Merged revisions 413587 from http://svn.asterisk.org/svn/asterisk/branches/11
........

Merged revisions 413588 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@413589 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-05-09 22:49:26 +00:00
Mark Michelson eefcb79bfb Prevent duplicate sorcery wizards from being applied to sorcery object types.
This commit contains several changes to sorcery:

1) Application of sorcery configuration based on module name is automatically performed
when sorcery is opened for a module.
2) Sorcery will not attempt to apply the same wizard to an object type more than once.
3) Sorcery gives more exact results when attempting to apply a wizard, whether as the
default or based on configuration.

Sorcery unit tests still pass for me after making these changes.

Review: https://reviewboard.asterisk.org/r/3326
........

Merged revisions 411159 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@411656 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-04-02 18:57:29 +00:00
George Joseph a4906e9f86 sorcery: Create AST_SORCERY dialplan function.
This patch creates the AST_SORCERY dialplan function which allows someone to
retrieve any value from a sorcery-based config file.  It's similar to 
AST_CONFIG.

The creation of the function itself was fairly straightforward but it required
changes to the underlying sorcery infrastructure that rippled into individual
sorcery objects.  The changes stemmed from inconsistencies in how sorcery
created ast_variable objectsets from sorcery objects and the inconsistency
in how individual objects used that feature especially when it came to
parameters that can be specified multiple times like contact in aor and match
in identify.  You can read more here...
http://lists.digium.com/pipermail/asterisk-dev/2014-February/065202.html

So, what this patch does, besides actually creating the AST_SORCERY function,
is the following...

* Creates ast_variable_list_append which is a helper to append one ast_variable
  list to another.
* Modifies the ast_sorcery_object_field_register functions to accept the
  already-defined sorcery_fields_handler callback.
* Modifies ast_sorcery_objectset_create to accept a parameter indicating return
  type preference...a single ast_variable with all values concatenated or an
  ast_variable list with multiple entries.  Also fixed a few bugs.
* Modifies individual sorcery object implementations to use the new function
  definition of the ast_sorcery_object_field_register functions.
* Modifies location.c and res_pjsip_endpoint_identifier_ip.c to implement
  sorcery_fields_handler handlers so they return multiple occurrences as an
  ast_variable_list.
* Added a whole bunch of tests to test_sorcery.

(closes issue ASTERISK-22537)
Review: http://reviewboard.asterisk.org/r/3254/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410042 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-03-06 22:39:54 +00:00
Richard Mudgett 3357c494cb sorcery, bucket: Change observer remove calls to take const callbacks struct.
* Make ast_sorcery_observer_remove() accept a const callbacks struct.

* Make ast_sorcery_observer_remove() tolerant of the sorcery parameter
being NULL.  Now it can be called within a module unload routine if the
sorcery initialization fails.

* Fix ast_sorcery_observer_add() to fail if the container link fails.
........

Merged revisions 403324 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@403327 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-12-03 17:35:54 +00:00
Richard Mudgett f62373b7a3 bucket: Fix scheme ref leak in __ast_bucket_scheme_register().
........

Merged revisions 402944 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@402945 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-11-21 18:11:04 +00:00
Mark Michelson c30170d9a2 Resolve some memory leaks due to incorrect for loop / ao2 ref usage.
A common idiom in Asterisk is to due something like:

for (ao2_obj = list_beginning; ao2_obj = next_item; ao2_ref(ao2_obj, -1)) {
    ...do stuff...
}

This is nice because it automatically takes care of the object references
for you. However, there is a pitfall here. If a break statement is in the
for loop, then the current reference is not cleaned up. In some cases, this
is on purpose, but in others there is a leak. This commit fixes the leak
cases.
........

Merged revisions 401248 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@401249 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-10-18 18:44:21 +00:00
David M. Lee 6c410d00d1 Fixed bucket.c for systems where tv_usec is not an unsigned long.
........

Merged revisions 397673 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397674 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-08-26 21:32:13 +00:00
Joshua Colp 9c713d12da Fix building of trunk.
Note: This is why I commit on the weekend.


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397613 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-08-23 22:20:39 +00:00
Joshua Colp dd33217762 Add the bucket API.
Bucket is a URI based API for the creation, retrieval, updating, and deletion
of "buckets" and files contained within them.

Review: https://reviewboard.asterisk.org/r/2715/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397600 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-08-23 21:49:47 +00:00