sysmocom
/
asterisk
11
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
asterisk/tests/test_sorcery_realtime.c

938 lines
30 KiB
C
Raw Normal View History

Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
/*
* Asterisk -- An open source telephony toolkit.
*
* Copyright (C) 2013, Digium, Inc.
*
* Joshua Colp <jcolp@digium.com>
*
* See http://www.asterisk.org for more information about
* the Asterisk project. Please do not directly contact
* any of the maintainers of this project for assistance;
* the project provides a web site, mailing lists and IRC
* channels for your use.
*
* This program is free software, distributed under the terms of
* the GNU General Public License Version 2. See the LICENSE file
* at the top of the source tree.
*/
/*!
* \file
* \brief Sorcery Unit Tests
*
* \author Joshua Colp <jcolp@digium.com>
*
*/
/*** MODULEINFO
<depend>TEST_FRAMEWORK</depend>
<support_level>core</support_level>
***/
#include "asterisk.h"
#include "asterisk/test.h"
#include "asterisk/module.h"
#include "asterisk/sorcery.h"
#include "asterisk/astdb.h"
#include "asterisk/logger.h"
/*! \brief Configuration structure which contains all stored objects */
static struct ast_config *realtime_objects;
static struct ast_variable *realtime_sorcery(const char *database, const char *table, const struct ast_variable *fields)
{
char *object_id = NULL;
while ((object_id = ast_category_browse(realtime_objects, object_id))) {
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if (!ast_variable_lists_match(ast_category_root(realtime_objects, object_id), fields, 0)) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
continue;
}
return ast_variables_dup(ast_category_root(realtime_objects, object_id));
}
return NULL;
}
static struct ast_config *realtime_sorcery_multi(const char *database, const char *table, const struct ast_variable *fields)
{
struct ast_config *objects;
char *object_id = NULL;
if (!(objects = ast_config_new())) {
return NULL;
}
while ((object_id = ast_category_browse(realtime_objects, object_id))) {
struct ast_category *object;
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
const struct ast_variable *object_fields = ast_category_root(realtime_objects, object_id);
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if (!ast_variable_lists_match(object_fields, fields, 0)) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
continue;
}
if (!(object = ast_category_new("", "", 0))) {
ast_config_destroy(objects);
return NULL;
}
ast_variable_append(object, ast_variables_dup(ast_category_root(realtime_objects, object_id)));
ast_category_append(objects, object);
}
return objects;
}
static int realtime_sorcery_update(const char *database, const char *table, const char *keyfield, const char *entity, const struct ast_variable *fields)
{
manager/config: Support templates and non-unique category names via AMI This patch provides the capability to manipulate templates and categories with non-unique names via AMI. Summary of changes: GetConfig and GetConfigJSON: Added "Filter" parameter: A comma separated list of name_regex=value_regex expressions which will cause only categories whose variables match all expressions to be considered. The special variable name TEMPLATES can be used to control whether templates are included. Passing 'include' as the value will include templates along with normal categories. Passing 'restrict' as the value will restrict the operation to ONLY templates. Not specifying a TEMPLATES expression results in the current default behavior which is to not include templates. UpdateConfig: NewCat now includes options for allowing duplicate category names, indicating if the category should be created as a template, and specifying templates the category should inherit from. The rest of the actions now accept a filter string as defined above. If there are non-unique category names, you can now update specific ones based on variable values. To facilitate the new capabilities in manager, corresponding changes had to be made to config, most notably the addition of filter criteria to many of the APIs. In some cases it was easy to change the references to use the new prototype but others would have required touching too many files for this patch so a wrapper with the original prototype was created. Macros couldn't be used in this case because it would break binary compatibility with modules such as res_digium_phone that are linked to real symbols. Tested-by: George Joseph Review: https://reviewboard.asterisk.org/r/4033/ ........ Merged revisions 425383 from http://svn.asterisk.org/svn/asterisk/branches/12 ........ Merged revisions 425384 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@425385 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-10-13 16:12:17 +00:00
struct ast_category *object, *found;
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
manager/config: Support templates and non-unique category names via AMI This patch provides the capability to manipulate templates and categories with non-unique names via AMI. Summary of changes: GetConfig and GetConfigJSON: Added "Filter" parameter: A comma separated list of name_regex=value_regex expressions which will cause only categories whose variables match all expressions to be considered. The special variable name TEMPLATES can be used to control whether templates are included. Passing 'include' as the value will include templates along with normal categories. Passing 'restrict' as the value will restrict the operation to ONLY templates. Not specifying a TEMPLATES expression results in the current default behavior which is to not include templates. UpdateConfig: NewCat now includes options for allowing duplicate category names, indicating if the category should be created as a template, and specifying templates the category should inherit from. The rest of the actions now accept a filter string as defined above. If there are non-unique category names, you can now update specific ones based on variable values. To facilitate the new capabilities in manager, corresponding changes had to be made to config, most notably the addition of filter criteria to many of the APIs. In some cases it was easy to change the references to use the new prototype but others would have required touching too many files for this patch so a wrapper with the original prototype was created. Macros couldn't be used in this case because it would break binary compatibility with modules such as res_digium_phone that are linked to real symbols. Tested-by: George Joseph Review: https://reviewboard.asterisk.org/r/4033/ ........ Merged revisions 425383 from http://svn.asterisk.org/svn/asterisk/branches/12 ........ Merged revisions 425384 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@425385 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-10-13 16:12:17 +00:00
if (!(found = ast_category_get(realtime_objects, entity, NULL))) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
return 0;
} else if (!(object = ast_category_new(entity, "", 0))) {
return -1;
}
manager/config: Support templates and non-unique category names via AMI This patch provides the capability to manipulate templates and categories with non-unique names via AMI. Summary of changes: GetConfig and GetConfigJSON: Added "Filter" parameter: A comma separated list of name_regex=value_regex expressions which will cause only categories whose variables match all expressions to be considered. The special variable name TEMPLATES can be used to control whether templates are included. Passing 'include' as the value will include templates along with normal categories. Passing 'restrict' as the value will restrict the operation to ONLY templates. Not specifying a TEMPLATES expression results in the current default behavior which is to not include templates. UpdateConfig: NewCat now includes options for allowing duplicate category names, indicating if the category should be created as a template, and specifying templates the category should inherit from. The rest of the actions now accept a filter string as defined above. If there are non-unique category names, you can now update specific ones based on variable values. To facilitate the new capabilities in manager, corresponding changes had to be made to config, most notably the addition of filter criteria to many of the APIs. In some cases it was easy to change the references to use the new prototype but others would have required touching too many files for this patch so a wrapper with the original prototype was created. Macros couldn't be used in this case because it would break binary compatibility with modules such as res_digium_phone that are linked to real symbols. Tested-by: George Joseph Review: https://reviewboard.asterisk.org/r/4033/ ........ Merged revisions 425383 from http://svn.asterisk.org/svn/asterisk/branches/12 ........ Merged revisions 425384 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@425385 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-10-13 16:12:17 +00:00
ast_category_delete(realtime_objects, found);
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
ast_variable_append(object, ast_variables_dup((struct ast_variable*)fields));
ast_variable_append(object, ast_variable_new(keyfield, entity, ""));
ast_category_append(realtime_objects, object);
return 1;
}
static int realtime_sorcery_store(const char *database, const char *table, const struct ast_variable *fields)
{
/* The key field is explicit within res_sorcery_realtime */
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
const struct ast_variable *keyfield = ast_variable_find_variable_in_list(fields, "id");
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
struct ast_category *object;
manager/config: Support templates and non-unique category names via AMI This patch provides the capability to manipulate templates and categories with non-unique names via AMI. Summary of changes: GetConfig and GetConfigJSON: Added "Filter" parameter: A comma separated list of name_regex=value_regex expressions which will cause only categories whose variables match all expressions to be considered. The special variable name TEMPLATES can be used to control whether templates are included. Passing 'include' as the value will include templates along with normal categories. Passing 'restrict' as the value will restrict the operation to ONLY templates. Not specifying a TEMPLATES expression results in the current default behavior which is to not include templates. UpdateConfig: NewCat now includes options for allowing duplicate category names, indicating if the category should be created as a template, and specifying templates the category should inherit from. The rest of the actions now accept a filter string as defined above. If there are non-unique category names, you can now update specific ones based on variable values. To facilitate the new capabilities in manager, corresponding changes had to be made to config, most notably the addition of filter criteria to many of the APIs. In some cases it was easy to change the references to use the new prototype but others would have required touching too many files for this patch so a wrapper with the original prototype was created. Macros couldn't be used in this case because it would break binary compatibility with modules such as res_digium_phone that are linked to real symbols. Tested-by: George Joseph Review: https://reviewboard.asterisk.org/r/4033/ ........ Merged revisions 425383 from http://svn.asterisk.org/svn/asterisk/branches/12 ........ Merged revisions 425384 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@425385 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-10-13 16:12:17 +00:00
if (!keyfield || ast_category_exist(realtime_objects, keyfield->value, NULL) || !(object = ast_category_new(keyfield->value, "", 0))) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
return -1;
}
ast_variable_append(object, ast_variables_dup((struct ast_variable*)fields));
ast_category_append(realtime_objects, object);
Fix failing realtime sorcery tests. The store realtime callback needs to return a positive value for sorcery to treat the store as a success. ........ Merged revisions 410625 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410626 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-03-14 21:56:21 +00:00
return 1;
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
}
static int realtime_sorcery_destroy(const char *database, const char *table, const char *keyfield, const char *entity, const struct ast_variable *fields)
{
manager/config: Support templates and non-unique category names via AMI This patch provides the capability to manipulate templates and categories with non-unique names via AMI. Summary of changes: GetConfig and GetConfigJSON: Added "Filter" parameter: A comma separated list of name_regex=value_regex expressions which will cause only categories whose variables match all expressions to be considered. The special variable name TEMPLATES can be used to control whether templates are included. Passing 'include' as the value will include templates along with normal categories. Passing 'restrict' as the value will restrict the operation to ONLY templates. Not specifying a TEMPLATES expression results in the current default behavior which is to not include templates. UpdateConfig: NewCat now includes options for allowing duplicate category names, indicating if the category should be created as a template, and specifying templates the category should inherit from. The rest of the actions now accept a filter string as defined above. If there are non-unique category names, you can now update specific ones based on variable values. To facilitate the new capabilities in manager, corresponding changes had to be made to config, most notably the addition of filter criteria to many of the APIs. In some cases it was easy to change the references to use the new prototype but others would have required touching too many files for this patch so a wrapper with the original prototype was created. Macros couldn't be used in this case because it would break binary compatibility with modules such as res_digium_phone that are linked to real symbols. Tested-by: George Joseph Review: https://reviewboard.asterisk.org/r/4033/ ........ Merged revisions 425383 from http://svn.asterisk.org/svn/asterisk/branches/12 ........ Merged revisions 425384 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@425385 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-10-13 16:12:17 +00:00
struct ast_category *found;
if (!(found = ast_category_get(realtime_objects, entity, NULL))) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
return 0;
}
manager/config: Support templates and non-unique category names via AMI This patch provides the capability to manipulate templates and categories with non-unique names via AMI. Summary of changes: GetConfig and GetConfigJSON: Added "Filter" parameter: A comma separated list of name_regex=value_regex expressions which will cause only categories whose variables match all expressions to be considered. The special variable name TEMPLATES can be used to control whether templates are included. Passing 'include' as the value will include templates along with normal categories. Passing 'restrict' as the value will restrict the operation to ONLY templates. Not specifying a TEMPLATES expression results in the current default behavior which is to not include templates. UpdateConfig: NewCat now includes options for allowing duplicate category names, indicating if the category should be created as a template, and specifying templates the category should inherit from. The rest of the actions now accept a filter string as defined above. If there are non-unique category names, you can now update specific ones based on variable values. To facilitate the new capabilities in manager, corresponding changes had to be made to config, most notably the addition of filter criteria to many of the APIs. In some cases it was easy to change the references to use the new prototype but others would have required touching too many files for this patch so a wrapper with the original prototype was created. Macros couldn't be used in this case because it would break binary compatibility with modules such as res_digium_phone that are linked to real symbols. Tested-by: George Joseph Review: https://reviewboard.asterisk.org/r/4033/ ........ Merged revisions 425383 from http://svn.asterisk.org/svn/asterisk/branches/12 ........ Merged revisions 425384 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@425385 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-10-13 16:12:17 +00:00
ast_category_delete(realtime_objects, found);
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
return 1;
}
struct ast_config_engine sorcery_config_engine = {
.name = "sorcery_realtime_test",
.realtime_func = realtime_sorcery,
.realtime_multi_func = realtime_sorcery_multi,
.update_func = realtime_sorcery_update,
.store_func = realtime_sorcery_store,
.destroy_func = realtime_sorcery_destroy,
};
/*! \brief Dummy sorcery object */
struct test_sorcery_object {
SORCERY_OBJECT(details);
unsigned int bob;
unsigned int joe;
};
/*! \brief Internal function to allocate a test object */
static void *test_sorcery_object_alloc(const char *id)
{
Add missing ast_sorcery_generic_alloc conversions. git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@392647 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-06-23 13:24:18 +00:00
return ast_sorcery_generic_alloc(sizeof(struct test_sorcery_object), NULL);
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
static struct ast_sorcery *alloc_and_initialize_sorcery(char *table)
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
{
struct ast_sorcery *sorcery;
if (!(sorcery = ast_sorcery_open())) {
return NULL;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if ((ast_sorcery_apply_default(sorcery, "test", "realtime", table) != AST_SORCERY_APPLY_SUCCESS) ||
Fix sorcery unit tests When strict XML documentation checking was re-enabled, the test objects used in sorcery would fail to register as the types were not marked internal and the nodoc option wasn't used for the options. This fixes that problem, such that, as one would hope, they once again pass. git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397571 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-08-23 18:10:11 +00:00
ast_sorcery_internal_object_register(sorcery, "test", test_sorcery_object_alloc, NULL, NULL) ||
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
!(realtime_objects = ast_config_new())) {
ast_sorcery_unref(sorcery);
return NULL;
}
Fix sorcery unit tests When strict XML documentation checking was re-enabled, the test objects used in sorcery would fail to register as the types were not marked internal and the nodoc option wasn't used for the options. This fixes that problem, such that, as one would hope, they once again pass. git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397571 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-08-23 18:10:11 +00:00
ast_sorcery_object_field_register_nodoc(sorcery, "test", "bob", "5", OPT_UINT_T, 0, FLDSET(struct test_sorcery_object, bob));
ast_sorcery_object_field_register_nodoc(sorcery, "test", "joe", "10", OPT_UINT_T, 0, FLDSET(struct test_sorcery_object, joe));
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
return sorcery;
}
static void deinitialize_sorcery(struct ast_sorcery *sorcery)
{
ast_config_destroy(realtime_objects);
realtime_objects = NULL;
ast_sorcery_unref(sorcery);
}
AST_TEST_DEFINE(object_create)
{
RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
switch (cmd) {
case TEST_INIT:
info->name = "object_create";
info->category = "/res/sorcery_realtime/";
info->summary = "sorcery realtime object creation unit test";
info->description =
"Test object creation in sorcery using realtime wizard";
return AST_TEST_NOT_RUN;
case TEST_EXECUTE:
break;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
ast_test_status_update(test, "Failed to open sorcery structure\n");
return AST_TEST_FAIL;
}
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Failed to allocate a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create object using realtime wizard\n");
return AST_TEST_FAIL;
}
return AST_TEST_PASS;
}
AST_TEST_DEFINE(object_retrieve_id)
{
RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
switch (cmd) {
case TEST_INIT:
info->name = "object_retrieve_id";
info->category = "/res/sorcery_realtime/";
info->summary = "sorcery object retrieval using id unit test";
info->description =
"Test object retrieval using id in sorcery with realtime wizard";
return AST_TEST_NOT_RUN;
case TEST_EXECUTE:
break;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
ast_test_status_update(test, "Failed to open sorcery structure\n");
return AST_TEST_FAIL;
}
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Failed to allocate a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create object using astdb wizard\n");
return AST_TEST_FAIL;
}
ao2_cleanup(obj);
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah2"))) {
ast_test_status_update(test, "Failed to allocate second instance of a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create second object using astdb wizard\n");
return AST_TEST_FAIL;
}
ao2_cleanup(obj);
if (!(obj = ast_sorcery_retrieve_by_id(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Failed to retrieve properly created object using id of 'blah'\n");
return AST_TEST_FAIL;
} else if (strcmp(ast_sorcery_object_get_id(obj), "blah")) {
ast_test_status_update(test, "Retrieved object does not have correct id\n");
return AST_TEST_FAIL;
}
return AST_TEST_PASS;
}
AST_TEST_DEFINE(object_retrieve_field)
{
RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
RAII_VAR(struct ast_variable *, fields, ast_variable_new("joe", "42", ""), ast_variables_destroy);
switch (cmd) {
case TEST_INIT:
info->name = "object_retrieve_field";
info->category = "/res/sorcery_realtime/";
info->summary = "sorcery object retrieval using a specific field unit test";
info->description =
"Test object retrieval using a specific field in sorcery with realtime wizard";
return AST_TEST_NOT_RUN;
case TEST_EXECUTE:
break;
}
if (!fields) {
ast_test_status_update(test, "Failed to create fields for object retrieval attempt\n");
return AST_TEST_FAIL;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
ast_test_status_update(test, "Failed to open sorcery structure\n");
return AST_TEST_FAIL;
}
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Failed to allocate a known object type\n");
return AST_TEST_FAIL;
}
obj->joe = 42;
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create object using realtime wizard\n");
return AST_TEST_FAIL;
}
ao2_cleanup(obj);
if (!(obj = ast_sorcery_retrieve_by_fields(sorcery, "test", AST_RETRIEVE_FLAG_DEFAULT, fields))) {
ast_test_status_update(test, "Failed to retrieve properly created object using 'joe' field\n");
return AST_TEST_FAIL;
}
ao2_cleanup(obj);
ast_variables_destroy(fields);
if (!(fields = ast_variable_new("joe", "49", ""))) {
ast_test_status_update(test, "Failed to create fields for object retrieval attempt\n");
return AST_TEST_FAIL;
}
if ((obj = ast_sorcery_retrieve_by_fields(sorcery, "test", AST_RETRIEVE_FLAG_DEFAULT, fields))) {
ast_test_status_update(test, "Retrieved an object using a field with an in-correct value... that should not happen\n");
return AST_TEST_FAIL;
}
return AST_TEST_PASS;
}
AST_TEST_DEFINE(object_retrieve_multiple_all)
{
RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
RAII_VAR(struct ao2_container *, objects, NULL, ao2_cleanup);
switch (cmd) {
case TEST_INIT:
info->name = "object_retrieve_multiple_all";
info->category = "/res/sorcery_realtime/";
info->summary = "sorcery multiple object retrieval unit test";
info->description =
"Test multiple object retrieval in sorcery using realtime wizard";
return AST_TEST_NOT_RUN;
case TEST_EXECUTE:
break;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
ast_test_status_update(test, "Failed to open sorcery structure\n");
return AST_TEST_FAIL;
}
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Failed to allocate a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create object using realtime wizard\n");
return AST_TEST_FAIL;
}
ao2_cleanup(obj);
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah2"))) {
ast_test_status_update(test, "Failed to allocate second instance of a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create second object using realtime wizard\n");
return AST_TEST_FAIL;
}
if (!(objects = ast_sorcery_retrieve_by_fields(sorcery, "test", AST_RETRIEVE_FLAG_MULTIPLE | AST_RETRIEVE_FLAG_ALL, NULL))) {
ast_test_status_update(test, "Failed to retrieve a container of all objects\n");
return AST_TEST_FAIL;
} else if (ao2_container_count(objects) != 2) {
ast_test_status_update(test, "Received a container with no objects in it when there should be some\n");
return AST_TEST_FAIL;
}
return AST_TEST_PASS;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
AST_TEST_DEFINE(object_retrieve_multiple_all_nofetch)
{
RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
RAII_VAR(struct ao2_container *, objects, NULL, ao2_cleanup);
switch (cmd) {
case TEST_INIT:
info->name = "object_retrieve_multiple_all_nofetch";
info->category = "/res/sorcery_realtime/";
info->summary = "sorcery multiple object retrieval unit test";
info->description =
"Test multiple object retrieval in sorcery using realtime wizard";
return AST_TEST_NOT_RUN;
case TEST_EXECUTE:
break;
}
if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test,allow_unqualified_fetch=no"))) {
ast_test_status_update(test, "Failed to open sorcery structure\n");
return AST_TEST_FAIL;
}
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Failed to allocate a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create object using realtime wizard\n");
return AST_TEST_FAIL;
}
ao2_cleanup(obj);
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah2"))) {
ast_test_status_update(test, "Failed to allocate second instance of a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create second object using realtime wizard\n");
return AST_TEST_FAIL;
}
if (!(objects = ast_sorcery_retrieve_by_fields(sorcery, "test", AST_RETRIEVE_FLAG_MULTIPLE | AST_RETRIEVE_FLAG_ALL, NULL))) {
ast_test_status_update(test, "Failed to retrieve a container of all objects\n");
return AST_TEST_FAIL;
} else if (ao2_container_count(objects) != 0) {
ast_test_status_update(test, "Received a container with objects in it when there should be none\n");
return AST_TEST_FAIL;
}
return AST_TEST_PASS;
}
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
AST_TEST_DEFINE(object_retrieve_multiple_field)
{
RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
RAII_VAR(struct ao2_container *, objects, NULL, ao2_cleanup);
RAII_VAR(struct ast_variable *, fields, ast_variable_new("joe", "6", ""), ast_variables_destroy);
switch (cmd) {
case TEST_INIT:
info->name = "object_retrieve_multiple_field";
info->category = "/res/sorcery_realtime/";
info->summary = "sorcery multiple object retrieval unit test";
info->description =
"Test multiple object retrieval in sorcery using fields using realtime wizard";
return AST_TEST_NOT_RUN;
case TEST_EXECUTE:
break;
}
if (!fields) {
ast_test_status_update(test, "Failed to create fields for multiple retrieve\n");
return AST_TEST_FAIL;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
ast_test_status_update(test, "Failed to open sorcery structure\n");
return AST_TEST_FAIL;
}
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Failed to allocate a known object type\n");
return AST_TEST_FAIL;
}
obj->joe = 6;
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create object using realtime wizard\n");
return AST_TEST_FAIL;
}
if (!(objects = ast_sorcery_retrieve_by_fields(sorcery, "test", AST_RETRIEVE_FLAG_MULTIPLE, fields))) {
ast_test_status_update(test, "Failed to retrieve a container of all objects\n");
return AST_TEST_FAIL;
} else if (ao2_container_count(objects) != 1) {
ast_test_status_update(test, "Received a container with no objects in it when there should be some\n");
return AST_TEST_FAIL;
}
ao2_cleanup(objects);
ast_variables_destroy(fields);
if (!(fields = ast_variable_new("joe", "7", ""))) {
ast_test_status_update(test, "Failed to create fields for multiple retrieval\n");
return AST_TEST_FAIL;
} else if (!(objects = ast_sorcery_retrieve_by_fields(sorcery, "test", AST_RETRIEVE_FLAG_MULTIPLE, fields))) {
ast_test_status_update(test, "Failed to retrieve an empty container when retrieving multiple\n");
return AST_TEST_FAIL;
} else if (ao2_container_count(objects)) {
ast_test_status_update(test, "Received a container with objects when there should be none in it\n");
return AST_TEST_FAIL;
}
return AST_TEST_PASS;
}
AST_TEST_DEFINE(object_retrieve_regex)
{
RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
RAII_VAR(struct ao2_container *, objects, NULL, ao2_cleanup);
switch (cmd) {
case TEST_INIT:
info->name = "object_retrieve_regex";
info->category = "/res/sorcery_realtime/";
info->summary = "sorcery multiple object retrieval using regex unit test";
info->description =
"Test multiple object retrieval in sorcery using regular expression for matching using realtime wizard";
return AST_TEST_NOT_RUN;
case TEST_EXECUTE:
break;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
ast_test_status_update(test, "Failed to open sorcery structure\n");
return AST_TEST_FAIL;
}
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah-98joe"))) {
ast_test_status_update(test, "Failed to allocate a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create object using realtime wizard\n");
return AST_TEST_FAIL;
}
ao2_cleanup(obj);
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah-93joe"))) {
ast_test_status_update(test, "Failed to allocate second instance of a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create second object using astdb wizard\n");
return AST_TEST_FAIL;
}
ao2_cleanup(obj);
if (!(obj = ast_sorcery_alloc(sorcery, "test", "neener-93joe"))) {
ast_test_status_update(test, "Failed to allocate third instance of a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create third object using astdb wizard\n");
return AST_TEST_FAIL;
}
res_sorcery_realtime: Remove leading ^ requirement. res_sorcery_realtime's search-by-regex callback performed a check to ensure that the passed-in regex began with a caret (^). If it did not, then no results would be returned. This callback only started to become used when "like" support was added to PJSIP CLI commands. The CLI command for listing objects would pass an empty regex ("") to the sorcery backend if no "like" statement was present. For most sorcery backends, this resulted in returning all objects. However, for realtime, this resulted in returning no objects. This commit seeks to fix the regression by removing the requirement from res_sorcery_realtime for the passed-in-regex to begin with a caret. ASTERISK-25689 #close Reported by Marcelo Terres Change-Id: I22b4dc5d7f3f11bb29ac2e42ef94682e9bab3b20
2016-01-12 16:36:15 +00:00
if (!(objects = ast_sorcery_retrieve_by_regex(sorcery, "test", "blah-"))) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
ast_test_status_update(test, "Failed to retrieve a container of objects\n");
return AST_TEST_FAIL;
} else if (ao2_container_count(objects) != 2) {
manager/config: Support templates and non-unique category names via AMI This patch provides the capability to manipulate templates and categories with non-unique names via AMI. Summary of changes: GetConfig and GetConfigJSON: Added "Filter" parameter: A comma separated list of name_regex=value_regex expressions which will cause only categories whose variables match all expressions to be considered. The special variable name TEMPLATES can be used to control whether templates are included. Passing 'include' as the value will include templates along with normal categories. Passing 'restrict' as the value will restrict the operation to ONLY templates. Not specifying a TEMPLATES expression results in the current default behavior which is to not include templates. UpdateConfig: NewCat now includes options for allowing duplicate category names, indicating if the category should be created as a template, and specifying templates the category should inherit from. The rest of the actions now accept a filter string as defined above. If there are non-unique category names, you can now update specific ones based on variable values. To facilitate the new capabilities in manager, corresponding changes had to be made to config, most notably the addition of filter criteria to many of the APIs. In some cases it was easy to change the references to use the new prototype but others would have required touching too many files for this patch so a wrapper with the original prototype was created. Macros couldn't be used in this case because it would break binary compatibility with modules such as res_digium_phone that are linked to real symbols. Tested-by: George Joseph Review: https://reviewboard.asterisk.org/r/4033/ ........ Merged revisions 425383 from http://svn.asterisk.org/svn/asterisk/branches/12 ........ Merged revisions 425384 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@425385 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-10-13 16:12:17 +00:00
ast_test_status_update(test, "Received a container with incorrect number of objects in it: %d instead of 2\n", ao2_container_count(objects));
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
return AST_TEST_FAIL;
}
return AST_TEST_PASS;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
AST_TEST_DEFINE(object_retrieve_regex_nofetch)
{
RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
RAII_VAR(struct ao2_container *, objects, NULL, ao2_cleanup);
switch (cmd) {
case TEST_INIT:
info->name = "object_retrieve_regex_nofetch";
info->category = "/res/sorcery_realtime/";
info->summary = "sorcery multiple object retrieval using regex unit test";
info->description =
"Test multiple object retrieval in sorcery using regular expression for matching using realtime wizard";
return AST_TEST_NOT_RUN;
case TEST_EXECUTE:
break;
}
if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test,allow_unqualified_fetch=no"))) {
ast_test_status_update(test, "Failed to open sorcery structure\n");
return AST_TEST_FAIL;
}
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah-98joe"))) {
ast_test_status_update(test, "Failed to allocate a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create object using realtime wizard\n");
return AST_TEST_FAIL;
}
ao2_cleanup(obj);
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah-93joe"))) {
ast_test_status_update(test, "Failed to allocate second instance of a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create second object using astdb wizard\n");
return AST_TEST_FAIL;
}
ao2_cleanup(obj);
if (!(obj = ast_sorcery_alloc(sorcery, "test", "neener-93joe"))) {
ast_test_status_update(test, "Failed to allocate third instance of a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create third object using astdb wizard\n");
return AST_TEST_FAIL;
}
if (!(objects = ast_sorcery_retrieve_by_regex(sorcery, "test", ""))) {
ast_test_status_update(test, "Failed to retrieve a container of objects\n");
return AST_TEST_FAIL;
} else if (ao2_container_count(objects) != 0) {
ast_test_status_update(test, "Received a container with incorrect number of objects in it: %d instead of 0\n", ao2_container_count(objects));
return AST_TEST_FAIL;
}
return AST_TEST_PASS;
}
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
AST_TEST_DEFINE(object_update)
{
RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
RAII_VAR(struct test_sorcery_object *, obj2, NULL, ao2_cleanup);
switch (cmd) {
case TEST_INIT:
info->name = "object_update";
info->category = "/res/sorcery_realtime/";
info->summary = "sorcery object update unit test";
info->description =
"Test object updating in sorcery using realtime wizard";
return AST_TEST_NOT_RUN;
case TEST_EXECUTE:
break;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
ast_test_status_update(test, "Failed to open sorcery structure\n");
return AST_TEST_FAIL;
}
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Failed to allocate a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create object using realtime wizard\n");
return AST_TEST_FAIL;
}
if (!(obj2 = ast_sorcery_copy(sorcery, obj))) {
ast_test_status_update(test, "Failed to allocate a known object type for updating\n");
return AST_TEST_FAIL;
}
ao2_cleanup(obj);
obj2->bob = 1000;
obj2->joe = 2000;
if (ast_sorcery_update(sorcery, obj2)) {
ast_test_status_update(test, "Failed to update sorcery with new object\n");
return AST_TEST_FAIL;
}
if (!(obj = ast_sorcery_retrieve_by_id(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Failed to retrieve properly updated object\n");
return AST_TEST_FAIL;
} else if ((obj->bob != obj2->bob) || (obj->joe != obj2->joe)) {
ast_test_status_update(test, "Object retrieved is not the updated object\n");
return AST_TEST_FAIL;
}
return AST_TEST_PASS;
}
AST_TEST_DEFINE(object_delete)
{
RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
switch (cmd) {
case TEST_INIT:
info->name = "object_delete";
info->category = "/res/sorcery_realtime/";
info->summary = "sorcery object deletion unit test";
info->description =
"Test object deletion in sorcery using realtime wizard";
return AST_TEST_NOT_RUN;
case TEST_EXECUTE:
break;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
ast_test_status_update(test, "Failed to open sorcery structure\n");
return AST_TEST_FAIL;
}
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Failed to allocate a known object type\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_create(sorcery, obj)) {
ast_test_status_update(test, "Failed to create object using realtime wizard\n");
return AST_TEST_FAIL;
}
if (ast_sorcery_delete(sorcery, obj)) {
ast_test_status_update(test, "Failed to delete object using realtime wizard\n");
return AST_TEST_FAIL;
}
ao2_cleanup(obj);
if ((obj = ast_sorcery_retrieve_by_id(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Retrieved deleted object that should not be there\n");
return AST_TEST_FAIL;
}
return AST_TEST_PASS;
}
AST_TEST_DEFINE(object_delete_uncreated)
{
RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
switch (cmd) {
case TEST_INIT:
info->name = "object_delete_uncreated";
info->category = "/res/sorcery_realtime/";
info->summary = "sorcery object deletion unit test";
info->description =
"Test object deletion of an uncreated object in sorcery using realtime wizard";
return AST_TEST_NOT_RUN;
case TEST_EXECUTE:
break;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
ast_test_status_update(test, "Failed to open sorcery structure\n");
return AST_TEST_FAIL;
}
if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Failed to allocate a known object type\n");
return AST_TEST_FAIL;
}
if (!ast_sorcery_delete(sorcery, obj)) {
ast_test_status_update(test, "Successfully deleted an object which was never created\n");
return AST_TEST_FAIL;
}
return AST_TEST_PASS;
}
Make res_sorcery_realtime filter unknown retrieved results. When retrieving data from a database or other realtime backend, it's quite possible to retrieve variables that Asterisk does not care about but that are legitimate to exist. Asterisk does not need to throw a hissy fit when these variables are encountered but rather just filter them out. Review: https://reviewboard.asterisk.org/r/3305 ........ Merged revisions 410187 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410207 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-03-07 21:23:39 +00:00
AST_TEST_DEFINE(object_allocate_on_retrieval)
{
RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
struct ast_category *cat;
switch (cmd) {
case TEST_INIT:
info->name = "object_allocate_on_retrieval";
info->category = "/res/sorcery_realtime/";
info->summary = "sorcery object allocation upon retrieval unit test";
info->description =
"This test creates data in a realtime backend, not through sorcery. Sorcery is then\n"
"instructed to retrieve an object with the id of the object that was created in the\n"
"realtime backend. Sorcery should be able to allocate the object appropriately";
return AST_TEST_NOT_RUN;
case TEST_EXECUTE:
break;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
Make res_sorcery_realtime filter unknown retrieved results. When retrieving data from a database or other realtime backend, it's quite possible to retrieve variables that Asterisk does not care about but that are legitimate to exist. Asterisk does not need to throw a hissy fit when these variables are encountered but rather just filter them out. Review: https://reviewboard.asterisk.org/r/3305 ........ Merged revisions 410187 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410207 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-03-07 21:23:39 +00:00
ast_test_status_update(test, "Failed to open sorcery structure\n");
return AST_TEST_FAIL;
}
cat = ast_category_new("blah", "", 0);
ast_variable_append(cat, ast_variable_new("id", "blah", ""));
ast_variable_append(cat, ast_variable_new("bob", "42", ""));
ast_variable_append(cat, ast_variable_new("joe", "93", ""));
ast_category_append(realtime_objects, cat);
if (!(obj = ast_sorcery_retrieve_by_id(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Failed to allocate object 'blah' base on realtime data\n");
return AST_TEST_FAIL;
}
if (obj->bob != 42) {
Fix more dev-mode build issues ........ Merged revisions 419129 from http://svn.asterisk.org/svn/asterisk/branches/1.8 ........ Merged revisions 419162 from http://svn.asterisk.org/svn/asterisk/branches/11 ........ Merged revisions 419163 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@419175 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-07-22 14:22:00 +00:00
ast_test_status_update(test, "Object's 'bob' field does not have expected value: %u != 42\n",
Make res_sorcery_realtime filter unknown retrieved results. When retrieving data from a database or other realtime backend, it's quite possible to retrieve variables that Asterisk does not care about but that are legitimate to exist. Asterisk does not need to throw a hissy fit when these variables are encountered but rather just filter them out. Review: https://reviewboard.asterisk.org/r/3305 ........ Merged revisions 410187 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410207 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-03-07 21:23:39 +00:00
obj->bob);
return AST_TEST_FAIL;
} else if (obj->joe != 93) {
Fix more dev-mode build issues ........ Merged revisions 419129 from http://svn.asterisk.org/svn/asterisk/branches/1.8 ........ Merged revisions 419162 from http://svn.asterisk.org/svn/asterisk/branches/11 ........ Merged revisions 419163 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@419175 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-07-22 14:22:00 +00:00
ast_test_status_update(test, "Object's 'joe' field does not have expected value: %u != 93\n",
Make res_sorcery_realtime filter unknown retrieved results. When retrieving data from a database or other realtime backend, it's quite possible to retrieve variables that Asterisk does not care about but that are legitimate to exist. Asterisk does not need to throw a hissy fit when these variables are encountered but rather just filter them out. Review: https://reviewboard.asterisk.org/r/3305 ........ Merged revisions 410187 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410207 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-03-07 21:23:39 +00:00
obj->joe);
return AST_TEST_FAIL;
}
return AST_TEST_PASS;
}
AST_TEST_DEFINE(object_filter)
{
RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
struct ast_category *cat;
switch (cmd) {
case TEST_INIT:
info->name = "object_filter";
info->category = "/res/sorcery_realtime/";
info->summary = "sorcery object field filter unit test";
info->description =
"This test creates data in a realtime backend, not through sorcery. In addition to\n"
"the object fields that have been registered with sorcery, there is data in the\n"
"realtime backend that is unknown to sorcery. When sorcery attempts to retrieve\n"
"the object from the realtime backend, the data unknown to sorcery should be\n"
"filtered out of the returned objectset, and the object should be successfully\n"
Unit tests: Fix unit test description strings. Analyzing the code shows that the unit test summary and description strings should not end with a new-line character. Where these strings are used in the code a new-line is provided for output. Change-Id: I129284f5e7ca93d82532334076da4c462d3d9fba
2015-06-24 19:39:01 +00:00
"allocated by sorcery";
Make res_sorcery_realtime filter unknown retrieved results. When retrieving data from a database or other realtime backend, it's quite possible to retrieve variables that Asterisk does not care about but that are legitimate to exist. Asterisk does not need to throw a hissy fit when these variables are encountered but rather just filter them out. Review: https://reviewboard.asterisk.org/r/3305 ........ Merged revisions 410187 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410207 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-03-07 21:23:39 +00:00
return AST_TEST_NOT_RUN;
case TEST_EXECUTE:
break;
}
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
Make res_sorcery_realtime filter unknown retrieved results. When retrieving data from a database or other realtime backend, it's quite possible to retrieve variables that Asterisk does not care about but that are legitimate to exist. Asterisk does not need to throw a hissy fit when these variables are encountered but rather just filter them out. Review: https://reviewboard.asterisk.org/r/3305 ........ Merged revisions 410187 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410207 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-03-07 21:23:39 +00:00
ast_test_status_update(test, "Failed to open sorcery structure\n");
return AST_TEST_FAIL;
}
cat = ast_category_new("blah", "", 0);
ast_variable_append(cat, ast_variable_new("id", "blah", ""));
ast_variable_append(cat, ast_variable_new("bob", "42", ""));
ast_variable_append(cat, ast_variable_new("joe", "93", ""));
ast_variable_append(cat, ast_variable_new("fred", "50", ""));
ast_category_append(realtime_objects, cat);
if (!(obj = ast_sorcery_retrieve_by_id(sorcery, "test", "blah"))) {
ast_test_status_update(test, "Failed to retrieve properly created object using id of 'blah'\n");
return AST_TEST_FAIL;
}
if (obj->bob != 42) {
Fix more dev-mode build issues ........ Merged revisions 419129 from http://svn.asterisk.org/svn/asterisk/branches/1.8 ........ Merged revisions 419162 from http://svn.asterisk.org/svn/asterisk/branches/11 ........ Merged revisions 419163 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@419175 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-07-22 14:22:00 +00:00
ast_test_status_update(test, "Object's 'bob' field does not have expected value: %u != 42\n",
Make res_sorcery_realtime filter unknown retrieved results. When retrieving data from a database or other realtime backend, it's quite possible to retrieve variables that Asterisk does not care about but that are legitimate to exist. Asterisk does not need to throw a hissy fit when these variables are encountered but rather just filter them out. Review: https://reviewboard.asterisk.org/r/3305 ........ Merged revisions 410187 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410207 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-03-07 21:23:39 +00:00
obj->bob);
return AST_TEST_FAIL;
} else if (obj->joe != 93) {
Fix more dev-mode build issues ........ Merged revisions 419129 from http://svn.asterisk.org/svn/asterisk/branches/1.8 ........ Merged revisions 419162 from http://svn.asterisk.org/svn/asterisk/branches/11 ........ Merged revisions 419163 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@419175 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-07-22 14:22:00 +00:00
ast_test_status_update(test, "Object's 'joe' field does not have expected value: %u != 93\n",
Make res_sorcery_realtime filter unknown retrieved results. When retrieving data from a database or other realtime backend, it's quite possible to retrieve variables that Asterisk does not care about but that are legitimate to exist. Asterisk does not need to throw a hissy fit when these variables are encountered but rather just filter them out. Review: https://reviewboard.asterisk.org/r/3305 ........ Merged revisions 410187 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410207 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-03-07 21:23:39 +00:00
obj->joe);
return AST_TEST_FAIL;
}
return AST_TEST_PASS;
}
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
static int unload_module(void)
{
ast_config_engine_deregister(&sorcery_config_engine);
AST_TEST_UNREGISTER(object_create);
AST_TEST_UNREGISTER(object_retrieve_id);
AST_TEST_UNREGISTER(object_retrieve_field);
AST_TEST_UNREGISTER(object_retrieve_multiple_all);
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
AST_TEST_UNREGISTER(object_retrieve_multiple_all_nofetch);
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
AST_TEST_UNREGISTER(object_retrieve_multiple_field);
AST_TEST_UNREGISTER(object_retrieve_regex);
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
AST_TEST_UNREGISTER(object_retrieve_regex_nofetch);
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
AST_TEST_UNREGISTER(object_update);
AST_TEST_UNREGISTER(object_delete);
AST_TEST_UNREGISTER(object_delete_uncreated);
Make res_sorcery_realtime filter unknown retrieved results. When retrieving data from a database or other realtime backend, it's quite possible to retrieve variables that Asterisk does not care about but that are legitimate to exist. Asterisk does not need to throw a hissy fit when these variables are encountered but rather just filter them out. Review: https://reviewboard.asterisk.org/r/3305 ........ Merged revisions 410187 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410207 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-03-07 21:23:39 +00:00
AST_TEST_UNREGISTER(object_allocate_on_retrieval);
AST_TEST_UNREGISTER(object_filter);
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
return 0;
}
static int load_module(void)
{
ast_config_engine_register(&sorcery_config_engine);
ast_realtime_append_mapping("sorcery_realtime_test", "sorcery_realtime_test", "test", "test", 1);
AST_TEST_REGISTER(object_create);
AST_TEST_REGISTER(object_retrieve_id);
AST_TEST_REGISTER(object_retrieve_field);
AST_TEST_REGISTER(object_retrieve_multiple_all);
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
AST_TEST_REGISTER(object_retrieve_multiple_all_nofetch);
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
AST_TEST_REGISTER(object_retrieve_multiple_field);
AST_TEST_REGISTER(object_retrieve_regex);
sorcery/res_pjsip: Refactor for realtime performance There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-08 21:55:30 +00:00
AST_TEST_REGISTER(object_retrieve_regex_nofetch);
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
AST_TEST_REGISTER(object_update);
AST_TEST_REGISTER(object_delete);
AST_TEST_REGISTER(object_delete_uncreated);
Make res_sorcery_realtime filter unknown retrieved results. When retrieving data from a database or other realtime backend, it's quite possible to retrieve variables that Asterisk does not care about but that are legitimate to exist. Asterisk does not need to throw a hissy fit when these variables are encountered but rather just filter them out. Review: https://reviewboard.asterisk.org/r/3305 ........ Merged revisions 410187 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410207 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-03-07 21:23:39 +00:00
AST_TEST_REGISTER(object_allocate_on_retrieval);
AST_TEST_REGISTER(object_filter);
Add support for a realtime sorcery module. This change does the following: 1. Adds the sorcery realtime module 2. Adds unit tests for the sorcery realtime module 3. Changes the realtime core to use an ast_variable list instead of variadic arguments 4. Changes all realtime drivers to accept an ast_variable list Review: https://reviewboard.asterisk.org/r/2424/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-04-27 12:01:29 +00:00
return AST_MODULE_LOAD_SUCCESS;
}
AST_MODULE_INFO_STANDARD(ASTERISK_GPL_KEY, "Sorcery Realtime Wizard test module");