The change to allow easier hacking on bundled pjproject created
a few issues:
* The new Makefile was trying to run the bundled make even if
PJPROJECT_BUNDLED=no. third-party/Makefile now checks for
PJPROJECT_BUNDLED and JANSSON_BUNDLED and skips them if they
are "no".
* When building with bundled, config_site.h was being copied
only if a full make or a "make main" was done. A "make res"
would fail all the pjsip modules because they couldn't find
config_site.h. The Makefile now copies config_site.h and
asterisk_malloc_debug.h into the pjproject source tree
when it's "configure" is performed. This is how it used
to be before the big change.
ASTERISK-29858
Change-Id: I9427264fa3cb8b3f59a95e5f9693eac236a6f76d
There are times when you need to troubleshoot issues with bundled
pjproject or add new features that need to be pushed upstream
but...
* The source directory created by extracting the pjproject tarball
is not scanned for code changes so you have to keep forcing
rebuilds.
* The source directory isn't a git repo so you can't easily create
patches, do git bisects, etc.
* Accidentally doing a make distclean will ruin your day by wiping
out the source directory, and your changes.
* etc.
This commit makes that easier.
See third-party/pjproject/README-hacking.md for the details.
ASTERISK-29824
Change-Id: Idb1251040affdab31d27cd272dda68676da9b268
OpenSSL is one of those packages that often have alternatives
with later versions. For instance, CentOS/EL 7 has an
openssl package at version 1.0.2 but there's an openssl11
package from the epel repository that has 1.1.1. This gets
installed to /usr/include/openssl11 and /usr/lib64/openssl11.
Unfortunately, the existing --with-ssl and --with-crypto
./configure options expect to point to a source tree and
don't work in this situation. Also unfortunately, the
checks in ./configure don't use pkg-config.
In order to make this work with the existing situation, you'd
have to run...
./configure --with-ssl=/usr/lib64/openssl11 \
--with-crypto=/usr/lib64/openssl11 \
CFLAGS=-I/usr/include/openssl11
BUT... those options don't get passed down to bundled pjproject
so when you run make, you have to include the CFLAGS again
which is a big pain.
Oh... To make matters worse, although you can specify
PJPROJECT_CONFIGURE_OPTS on the ./configure command line,
they don't get saved so if you do a make clean, which will
force a re-configure of bundled pjproject, those options
don't get used.
So...
* In configure.ac... Since pkg-config is installed by install_prereq
anyway, we now use it to check for the system openssl >= 1.1.0.
If that works, great. If not, we check for the openssl11
package. If that works, great. If not, we fall back to just
checking for any openssl. If pkg-config isn't installed for some
reason, or --with-ssl=<dir> or --with-crypto=<dir> were specified
on the ./configure command line, we fall back to the existing
logic that uses AST_EXT_LIB_CHECK().
* The whole OpenSSL check process has been moved up before
THIRD_PARTY_CONFIGURE(), which does the initial pjproject
bundled configure, is run. This way the results of the above
checks, which may result in new include or library directories,
is included.
* Although not strictly needed for openssl, We now save the value of
PJPROJECT_CONFIGURE_OPTS in the makeopts file so it can be used
again if a re-configure is triggered.
ASTERISK-29693
Change-Id: I341ab7603e6b156aa15a66f43675ac5029d5fbde
This patch makes the usual necessary changes when upgrading to a new
version pjproject. For instance, version number bump, patches removed
from third-party, new *.md5 file added, etc..
This patch also includes a change to the Asterisk pjproject Makefile to
explicitly create the 'source/pjsip-apps/lib' directory. This directory
is no longer there by default so needs to be added so the Asterisk
malloc debug can be built.
This patch also includes some minor changes to Asterisk that were a result
of the upgrade. Specifically, there was a backward incompatibility change
made in 2.10 that modified the "expires header" variable field from a
signed to an unsigned value. This potentially effects comparison. Namely,
those check for a value less than zero. This patch modified a few locations
in the Asterisk code that may have been affected.
Lastly, this patch adds a new macro PJSIP_MINVERSION that can be used to
check a minimum version of pjproject at compile time.
ASTERISK-28899 #close
Change-Id: Iec8821c6cbbc08c369d0e3cd2f14e691b41d0c81
This patch fixes three compatibility issues for Darwin compatible builds:
(1) Use BSD compatible command line option for sed
For some versions of BSD sed, the -r command line option is unknown.
Both GNU and BSD sed support the -E command line option for enabling
extended regular expressions; as such, this patch replaces the -r
option with -E.
(2) Look for '_' in pjproject generated symbols
In Darwin comaptible systems, the symbols generated for pjproject may be
prefixed with an '_'. When exporting these to a symbol file, the invocation
to sed has to optionally look for a prefix of said '_' character.
(3) Use -all_load/-noall_load when linking
The flags -whole-archive/-no-whole-archive are not supported by the
linker, and must instead be replaced with -all_load/-noall_load.
Change-Id: I58121756de6a0560a6e49ca9d6bf9566a333cde3
When the gcc version is >= 8.2.1, we were already setting the
--fno-partial-inlining flag for Asterisk source files to get around
a gcc bug but we weren't passing the flag down to the bundled
builds of pjproject and jansson.
ASTERISK-28392
Change-Id: I99ede9bc35408ecd096f7d5369e8192d3dc75704
We previously allowed resample and g711 codecs to be built when
TEST_FRAMEWORK was enabled. This could cause errors if the testsuite
was run without this option enabled. Switch the build system to allow
those codecs to be built when --enable-dev-mode is used. This removes a
chance for strange testsuite errors from use of an inadequate pjsua
binary.
Change-Id: Iee8a3613cdb711fa7e7d217c5a775a575907ae22
Changing any Menuselect option in the `Compiler Flags` section causes a
full rebuild of the Asterisk source tree. Every enabled option causes
a #define to be added to buildopts.h, thus breaking ccache caching for
every source file that includes "asterisk.h". In most cases each option
only applies to one or two files. Now we only define those options for
the specific sources which use them, this causes much better cache
matching when working with multiple builds. For example testing code
with an without MALLOC_DEBUG will now use just over half the ccache
size, only main/astmm.o will have two builds cached instead of every
file.
Reorder main/Makefile so _ASTCFLAGS set on specific object files are all
together, sorted by filename. Stop adding -DMALLOC_DEBUG to CFLAGS of
bundled pjproject, this define is no longer used by any header so only
serves to break cache.
The only code change is a slight adjustment to how main/astmm.c is
initialized. Initialization functions always exist so main/asterisk.c
can call them unconditionally. Additionally rename the astmm
initialization functions so they are not exported.
Change-Id: Ie2085237a964f6e1e6fff55ed046e2afff83c027
Asterisk modules that use PJPROJECT services have their compiler
optimization and possibly their symbolic debug options overridden by the
PJPROJECT configure script selected settings.
* We need to filter-out any -O and -g options in PJ_CFLAGS before echoing
out the result so the PJPROJECT_INCLUDE variable does not override the
Asterisk module settings when using bundled PJPROJECT.
NOTE: This patch only has an effect when using bundled PJPROJECT.
ASTERISK-27563
Change-Id: If124169735ecf572ad1535cd43bff94cb44d5b30
This allows asterisk to be compiled with MALLOC_DEBUG to load modules
built without MALLOC_DEBUG. Now pre-compiled third-party modules will
still work regardless of MALLOC_DEBUG being enabled or not.
Change-Id: Ic07ad80b2c2df894db984cf27b16a69383ce0e10
OSX does not support 'readlink -f' or 'sed -r'. Replace readlink with
the GNU make macro 'realpath'. Replace sed with grep in one place, cut
in the other.
ASTERISK-27332
Change-Id: I5d34ecca905384decb22ead45c913ae5e8aff748
On a fresh Asterisk source directory, the bundled pjproject tarball is
unconditionally downloaded even if the tarball is already in a specified
cache directory.
* Made check if the pjproject tarball is valid in the cache directory
before downloading the tarball on a fresh source directory.
Change-Id: Ic7ec842d3c97ecd8dafbad6f056b7fdbce41cae5
We now mirror the pjproject tarball and md5 at
https://github.com/asterisk/third-party/tree/master/pjproject
To improve download reliability, we now get the tarball from
our mirror instead of from pjsip.org.
ASTERISK-27052 #close
Reported-by: 'alex'
Change-Id: I60236587a8935bfa71fcc391f4e2ecb31918c08a
After configuring Asterisk with '--with-pjproject-bundled' the configure/build
process attempts to download pjproject from its download site. Currently, a
timeout of 10 seconds is used that will stop the download process if pjproject
has not been fully downloaded in that time. For some systems this was not enough
time and the process was timing out too early.
This patch raises the download timeout value to '60'. Also, this patch fixes
another bug where the DOWNLOAD_TIMEOUT variable was not being properly exported
due to a naming error. DOWNLOAD_MAX_TIMEOUT is now properly renamed to
DOWNLOAD_TIMEOUT.
ASTERISK-26814 #close
Change-Id: Ia56e4e8a3d39db76bc8a1852b2cf07ec10b39842
Bundled pjproject should now only rebuild if one of the menuselect
"Compiler Flags" options changes.
Change-Id: If114a2e16b9e77af371a600d6a5e197bbf28fe43
An earlier attempt to prevent pjsua from spitting out an extra 6795
lines of debug output every time the testsuite called it was also
turning off the ability for asterisk to output debug info when it
needed to. This patch reverts the earlier fix and instead adds
a pjproject patch that sets the startup log level to 1 for pjsua
pjsystest and the pjsua python binding. This is an asterisk-only
patch that does not affect pjproject functionality and will not be
submitted upstream.
Change-Id: I347a8b58b2626f2906ccfc1d339e907627a0c9e8
When MALLOC_DEBUG was specified, make was failing. Immediately
remaking would work. The issues was in the ordering of the make
dependencies.
Change-Id: If6030b54fc693f3179f32bfd20c6b5d5f1b3f7cd
A while back, we changed config_site.h to set PJ_LOG_MAX_LEVEL = 6.
This allowed us to control the log level better from inside Asterisk.
An unfortunate side effect of this was that the pjsua binary and
python bindings were also compiled with log level set to 6 so whenever
a testsuite test that uses pjsua runs, it spits out 6795 lines of
debug in an instant even before the test starts. I believe this
overruns the Jenkins capture buffer and prevents the test from
properly terminating. In turn, this results in the testsuite just
hanging until the job is killed. It's more frequent on the higher
end agents because they can spit out the messages faster.
Unfortunately, the messages are all spit out before we have control
of the python pj.Lib instance where we can set logging levels so the
only alternative was to actually compile pjsua and _pjsua.so with an
overridden PJ_LOG_MAX_LEVEL. Although defining a lower max level was
done in the Makefile, the define in config_site.h had to be wrapped
with "#ifndef" so the change would take effect.
Change-Id: I2af9e7d48dde1927279c586c9c725d868fe6f3ff
There were just too many issues in various environments with
multi threaded building of pjproject. It doesn't really speed
things up anyway since asterisk is already being compiled in
parallel.
Change-Id: Ie5648fb91bb89b4224b6bf43a0daa1af793c4ce1
If a tarball is corrupted during download, the makefile will attempt to
download it again. If the tarball somehow gets corrupted after it's
downloaded however, the makefile was just failing. We now
retry the download.
ASTERISK-26653 #close
Change-Id: I1b24d454852d80186f60c5a65dc4624ea8a1c359
Added back in a -g3, and an -O3 when DONT_OPTIMIZE is not set, to
the CFLAGS. Not sure how they went missing.
Also fixed an uninstall problem where we weren't removing the
symlink from libasteriskpj.so.2 to libasteriskpj.so. While I was
there, I fixed it for libasteriskssl as well.
Change-Id: I9e00873b1e9082d05b5549d974534b48a2142556
The download process now has a timeout which will cause wget to retry
if it stops retrieving data for 5 seconds and fetch and curl to timeout
if the whole retrieval take smore than 30 seconds.
If the tarball retrieval works, the MD5SUM file is retrieved from
the downloads site and the md5 checksum is verified.
If either the tarball retrieval or MD5SUM retrieval fails, or the
checksums don't match, the entire process is retried once. If it
fails again, any incomplete tarball is deleted.
.DELETE_ON_ERROR: was also added to the Makefile. Not only does
this delete the tarball on failure, it till also delete corrupted
library files from the pjproject source directory should they
fail to build correctly.
Tested all the way back to FreeBSD 9, CentOS 6, Debian 6 and
Ubuntu 14.
Change-Id: Iea7d33b96a31622ab1b6e54baebaf271959514e1
OpenBSD's 'find' doesn't take the -delete argument so you have to pipe
through 'xargs rm -rf'.
'echo -e' doesn't like \t starting a line. It just prints 't' which
causes the libasteriskpj.exports file to be garbage. They were just
cosmetic so they were removed.
librt doesn't exist so the link of libasteriskpj.so fails. It's not
actually needed for linux anyway so -lrt was removed from the link.
res_rtp_asterisk was failing to load because of an undefined
DTLS_method. '|| defined(LIBRESSL_VERSION_NUMBER)' was added to the #if
so DTLSv1_method is used instead.
ASTERISK-26608
Change-Id: I926ec95b0b69633231e3ad1d6e803b977272c49c
libresample is only needed by pjproject if we're building pjsua, which
we only do if TEST_FRAMEWORK is selected. It's required by pjsua to
process audio which is needed by some testsuite tests. Unfortunately,
pjproject relies on a newer version of libresample than the version
that ships by most distros so we need to compile the version that's
bundled with pjproject. Since we only need it for pjsua, we DON'T want
it's symbols exposed when we actually build asterisk.
There was a problem however... TEST_FRAMEWORK is only known AFTER we've
already run ./configure on both asterisk and pjproject but pjproject's
./configure needs to test it to know whether to set up to build
libresample or not. The previous way of figuring this out was to
always tell ./configure "yes" but not actually build the library. This
caused an issue where building libasteriskpj was being told to include
libresample but it wasn't actually there.
The solution is to still do a default pjproject configure during an
asterisk ./configure but if makeopts or menuselect.makeopts changes
subsequently, we now reconfigure pjproject, taking into account the
current state of TEST_FRAMEWORK. Previously, if makeopts or
menuselect.makeopts changed, only a recompile of pjproject was done.
Change-Id: I9b5d84c61384a3ae07fe30e85c49698378cc4685
In order for pjsua and its python binding to actually negotiate
audio for the testsuite tests, it needs g711 and resample. The
pj* libraries themselves do not. Unfortunately, pjproject relies
on a brand new libresample that most distros don't ship so we need
to use the libresample already bundled with pjproject. Only the pjsua
executable and the _pjsua.so python library are linked with it so it
shouldn't interfere with asterisk itself.
Also it was pointed out that apply_patches couldn't handle multiple
patches that depended on each other during the dry-run, so the
dry-run was removed.
Change-Id: I24f397462b486dcdde0dcafe40e6c55a6593f098
Older versions of tar don't support the --strip-components option so
instead of doing 'tar --strip-components=1 -C source', we now just
untar to the tarball's root directory (pjproject-<version>) and
rename that directory to 'source'.
Also fixed an issue where the pjproject source directory is a hard
coded absolute pathname.
ASTERISK-26510 #close
ASTERISK-22480 #close
Change-Id: I9ec92952507a91ff4e4d01e0149e09fd8e8f32b0
* CFLAGS is now properly set when using older gcc.
* All third-party pjproject targets have been removed. This fixes
an issue with older libsrtp in some distros.
* Manually removing the source directory now causes a rebuild.
* EXTERNALS_CACHE_DIR is now properly checked.
* Whitespace fixes.
Change-Id: I98fec6847efc5602a9f41cb95096fd660a49fa60
Previously when testing I had a preexisting makeopts in ASTTOPDIR. The
ordering of configure.ac causes --with-externals-cache to be processed
after third-party configure. In cases where the Asterisk clone is
cleaned it would cause pjproject to be downloaded to /tmp. This
moves processing of the externals cache and sounds cache to happen
before third-party configure.
This also addresses a possible issue with the third-party Makefile. If
TMPDIR is set by the environment it would override the path given to
--with-externals-cache.
ASTERISK-26416
Change-Id: Ifab7f35bfcd5a31a31a3a4353cc26a68c8c6592d
Added tests for bzip2, tar, patch, sed and nm to configure.ac.
Set DOWNLOAD_TO_STDOUT to a working command line regardless of
whether the download program is wget, curl or fetch.
Added a 'configure.m4' file to the third-party directory which takes
care of calling any third-party project setup. Had to move some
pjproject_bundled stuff up in configure.ac so it was called before
the third-party configure macro.
The pjproject tarball is now downloaded to the externals_cache_dir if
it was specified on the ./configure command line
Removed regeneration of the pjproject aconfigure file. It was only
needed for an old patch that no longer applies.
Converted the tests for symbols to explicit tests since we know that
they're now available in the bundled version. Saves a little time
during configure.
ASTERISK-26416 #close
Reported-by: Corey Farrell
Change-Id: Id1d94251c0155f8dd41b7de7067f35cfbaafbb9b
(cherry picked from commit e6b0053d75)
(cherry picked from commit a0d02f3832)
pjproject_bundled will now use the asterisk memory debugging APIs
if MALLOC_DEBUG is turned on in menuselect.
Because this required stubs for the executable programs and the python
bindings, some Makefile reorganization was needed to properly handle
the dependencies. As a result, the makefile now individually makes
each of the pjproject libraries separately instead of making them all
in 1 shot. The only visible change is that there are separate status
lines printed for each library instead oif 1 for all libs. Also, the
making of the pjproject dependency files was eliminated. They're not
needed for building unless you're actively modifying pjproject source
files and it makes the build process faster. Finally, any issues with
parallel builds should be resolved again making the build faster.
Change-Id: Icc5e3d658fbfb00e0a46b44c66dcc2522d5171b0
The pjsua and pjsystest apps are now built only if TEST_FRAMEWORK is set.
The python bindings are now built only if TEST_FRAMEWORK is set and a
python development package is installed.
libresample was also disabled.
ASTERISK-25993 #close
Reported-by: Joshua Colp
Change-Id: If4e91c503a02f113d5b71bc8b972081fa3ff6f03
For all OSes:
* Disabled third-party codecs in pjproject and added
'--disable-speex-codec --disable-speex-aec --disable-gsm-codec' to the
configure options since we don't use the pjsip codec capability.
FreeBSD:
* Added FreeBSD support to install_prereq.
* Changed pjproject/configure.m4 to use $GNU_MAKE instead of hardcoding "make".
* Added __progname and environ to asterisk.exports.in.
* Reverted the use of ldconfig to create shared library symlinks to ln.
* Only enable epoll in pjproject if `uname -s` is Linux.
* Added a patch to pjproject to take the name of the 'make' command from
an environment variable if supplied. This is needed for the python bindings.
(merged by Teluu into pjproject trunk 5/3/2016)
FreeBSD support isn't complete. Still some general issues regarding
make/gmake having nothing to do with pjproject. With some handholding it DOES
build successfully.
CentOS:
Added 'patch' and 'bzip2' to install_prereq PACKAGES_RH.
CentOS 6/7 32/64 build and run the pjsip testsuite successfully.
Ubuntu:
No changes required.
Ubuntu 15/16 32/64 build and run the pjsip testsuite successfully.
Debian:
No changes required.
Debian 6/7/8 32/64 build and run the pjsip testsuite successfully.
There will utimately be a follow-up patch to create an install_prereq for
the testsuite as I've discovered a few missing requirements.
ASTERISK-25968 #close
Change-Id: I5756a07facfc63798115a5e73a8709382fe9259c
The pjproject Makefile now uses the Asterisk optimization flags which
are determined by the setting of the DONT_OPTMIZE menuselect flag.
The Makefile was also restructured so a change to the top level
menuselect.makeopts will result in a rebuild of pjproject.
Also, "--disable-resample" was removed from the pjproject configure
options. Without resample, pjsua (which is used by the testsuite)
can't make audio calls. When it can't, it segfaults.
Change-Id: I24b0a4d0872acef00ed89b3c527a713ee4c2ccd4
Background here:
http://lists.digium.com/pipermail/asterisk-dev/2016-January/075266.html
From CHANGES:
* To help insure that Asterisk is compiled and run with the same known
version of pjproject, a new option (--with-pjproject-bundled) has been
added to ./configure. When specified, the version of pjproject specified
in third-party/versions.mak will be downloaded and configured. When you
make Asterisk, the build process will also automatically build pjproject
and Asterisk will be statically linked to it. Once a particular version
of pjproject is configured and built, it won't be configured or built
again unless you run a 'make distclean'.
To facilitate testing, when 'make install' is run, the pjsua and pjsystest
utilities and the pjproject python bindings will be installed in
ASTDATADIR/third-party/pjproject.
The default behavior remains building with the shared pjproject
installation, if any.
Building:
All you have to do is include the --with-pjproject-bundled option on
the ./configure command line (and remove any existing --with-pjproject
option if specified). Everything else is automatic.
Behind the scenes:
The top-level Makefile was modified to include 'third-party' in the
list of MOD_SUBDIRS.
The third-party directory was created to contain any third party
packages that may be needed in the future. Its Makefile automatically
iterates over any subdirectories passing on targets.
The third-party/pjproject directory was created to house the pjproject
source distribution. Its Makefile contains targets to download, patch
configure, generate dependencies, compile libs, apps and python bindings,
sanitized build.mak and generate a symbols list.
When bootstrap.sh is run, it automatically includes the configure.m4
file in third-party/pjproject. This file has a macro to download and
conifgure pjproject and get and set PJPROJECT_INCLUDE, PJPROJECT_DIR
and PJPROJECT_BUNDLED. It also tests for the capabilities like
PJ_TRANSACTION_GRP_LOCK by parsing preprocessor output as opposed to
trying to compile. Of course, bootstrap.sh is only run once and the
configure file is incldued in the patch.
When configure is run with the new options, the macro in configure.m4
triggers the download, patch, conifgure and tests. No compilation is
performed at this time. The downloaded tarball is cached in /tmp so
it doesn't get downloaded again on a distclean.
When make is run in the top-level Asterisk source directory, it will
automatically descend all the subdirectories in third_party just as it
does for addons, apps, etc. The top-level Makefile makes sure that
the 'third-party' is built before 'main' so that dependencies from the
other directories are built first.
When main does build, a new shared library (libasteriskpj) is created that
links statically to the pjproject .a files and exports all their symbols.
The asterisk binary links to that, just as it does with libasteriskssl.
When Asterisk is installed, the pjsua and pjsystest apps, and the pjproject
python bindings are installed in ASTDATADIR/third-party/pjproject. This
will facilitate testing, including running the testsuite which will be
updated to check that directory for the pjsua module ahead of the system
python library.
Modules should continue to depend on pjproject if they use pjproject APIs
directly. They should not care about the implementation. No changes to any
res_pjsip modules were made.
Change-Id: Ia7a60c28c2e9ba9537c5570f933c1ebcb20a3103
George Joseph
Kevin Harwell
Alexander Traud
Matt Jordan
Corey Farrell
Richard Mudgett
zuul
George Joseph