Mbuni: Open Source MMS Gateway

User Guide

This document describes the installation and usage of the MMS Gateway.

Table of Contents


Chapter 1: Introduction

This chapter provides an overview of MMS (short for Multimedia Messaging Service) and the Mbuni MMS Gateway.

MMS offers mobile users enhanced messaging capabilities like the ability to send pictures and sound from a cell phone. It is generally considered the natural successor to the very popular SMS service.

MMS usage has continued to grow since introduction, and it is expected that projects such as Mbuni should further boost the adoption of MMS and its explosion.

The Mbuni Project attempts to provide that little bit of boost to MMS adoption/growth, by providing a crucial part of the MMS infrastructure in the form of a fully-fledged Open Source MMS Gateway (more commonly called a MMS Centre).

Mbuni aims to support all the major MMS interfaces, including phone-to-phone (so-called MM1 interface), phone-to-email (MM3), inter-MMSC (MM4) and MMS VAS (MM7). The current release fully supports the MM1, MM3 and MM7 interfaces, and provides rudimentary support for the MM4 interface. This version also supports network-side MMBox storage and transactions as specified in the OMA MMS v1.2 specification.

Mbuni is inspired, in part by the Kannel project, and utilises Kannel's GWLIB and WAP libraries. Kannel provides well-designed, simple interfaces for management of octet strings, lists, threads, servers, etc, and a certified WAP implementation. This made it a natural choice for Mbuni, rather than re-inventing the wheel.

Overview of MMS

The Multimedia Messaging Service (MMS), is intended to provide a rich set of content to subscribers (pictures, audio, games, etc). It supports both sending and receiving of rich content by properly enabled client devices. MMS is a non-real-time delivery service, much like SMS or email. The service utilises a store-and-forward usage model.

MMS is designed to be transported largely over IP rather than traditional GSM (SS7) networks. It is also designed to interoperate with other IP services such as email and WAP. In fact, MMS messages are typically transported over WAP, and are encoded using WAP MIME formats.

The MMS Gateway acts as the message-switching device in the MMS architecture. The general architecture is shown below.

The elements shown in the figure can be summarised as follows:

Typically, the message cycle begins with a user sending a multimedia message (MM) via the MMS client. The client must be configured for MMS, which includes bearer settings (i.e. GPRS or GSM/CSD settings), WAP gateway address and MMS Gateway address (a URL).

An MM is typically a multi-part message with pictures, sound, text and other media. Each part of the message is identified by media (MIME) type, name and/or Content ID. When submitting a message, the MMS client indicates the intended recipient list, but usually not the sender address, which the MMS Gateway retrieves from the WAP gateway. Like Email, a single MMS can specify multiple recipients (MSISDNs and Email addresses), and it is up to the Gateway to ensure correct delivery to each of the recipients.

When the gateway receives a message destined for an email address, it typically re-codes the message as standard MIME and passes it on to an SMTP server for delivery. Email messages received are similarly re-coded as MMS and forwarded to the relevant MMS Client.

When the gateway receives a message destined to MMS Clients in the area served by the gateway, the message is stored and an MMS notification sent to the recipient via WAP Push. On receipt of the notification, the client typically fetches the message via a URL provided in the notification.

When a recipient requests an incoming MM from the server, it indicates to the server its capabilities for a User Agent Profile URL. The profile data includes such things as supported media types, screen size, supported character sets, etc. Typically, the gateway will re-code the MM to suit the client's capabilities before returning the message. Messages destined to email may also be re-coded to make them more suitable for email readers.

 

The gateway may also interface with a subscriber database, which controls message delivery and billing. The subscriber database will provide such information as which subscribers are provisioned for MMS, tariffs, etc.

Features

Mbuni MMS gateway is a modular software system, designed to be full-featured, efficient and simple, supporting current generation two-way multimedia messaging. Feature highlights include:

The Gateway is designed and tested to conform to Open Mobile Alliance (OMA), WAP and 3rd Generation Partnership Project (3GPP) MMS standards including:

Requirements

Mbuni is being developed on MacOS X and Linux systems using the C programming language. It should compile and run on any similar system.

Mbuni utilises some libraries that are part of the Kannel source, specifically GWLIB and WAP libraries. In order to install Mbuni you will need to install (a patched) Kannel (and therefore fulfil those dependencies Mbuni shares with it).

The following additional components are required: Hardware requirements will depend on amount of traffic, required response times, etc. Keep in mind however that the gateway performs a lot of heavy media re-coding tasks, particularly during content adaptation, so you should always err on the side of more rather than less power.

Chapter 2: Installing The Gateway

This section explains the steps required to install the gateway. Currently only installation from source is provided (binary option coming soon).

In brief, to install Mbuni, you need to:

The source code for Mbuni is available for download from the download area of the website

Patching and Installing Kannel

In order to compile the software, you will first need to download, patch, compile and install Kannel v1.4.0 from kannel.org:

Unpack the kannel source files using a command like:

bzip2 -cd gateway-1.4.0.tar.bz2 | tar xf -

The kannel sources need to be patched for Mbuni using one of the supplied patch files from the Mbuni downloads section given above. You can use either one of the patch files:

Which patch you choose is a matter of taste. Apply the patch like this

cd gateway-1.4.0
patch -p1 < ../mbuni-kannel-patch-full

Then proceed to compile and install Kannel normally:

./configure
make install

Installing Mbuni MMS Gateway

Download and unzip/tar Mbuni sources in a directory of your choice:

tar xzf mbuni-version.tgz

Where version is the verion of Mbuni (e.g. 0.9.8). Compile and install mbuni as follows:

cd mbuni-version
./configure
make insall

If you installed Kannel in a non-standard location, you will need to supply the location to configure using --with-kannel=kannel_directory

Installing from CVS

If you want to try out the development version of Mbuni, you can download it from the CVS on sourceforge.net:

cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/mbuni login
followed by
cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/mbuni co -P mbuni
you can then cd mbuni and use the ./configure and make install as above.

Mbuni consists of 3 programs:

which are by default installed in /usr/local/bin

mmsrelay is the main relay server. It routes incoming messages to email, other MMS gateways, MMS clients/handsets, etc. It also manages routing messages to MMS clients using WAP Push, sending of delivery reports, etc.

mmsproxy provides the HTTP interface via which messages are sent and received by MMS clients.

mmsfromemail Is the email2MMS gateway module. All programs must be configured, and the first three running for the gateway to function smoothly.

Installing Required Components

Be sure to install all other required components as detailed above, otherwise parts of the MMS gateway may not function correctly.

Mbuni expects that an AMR decoder is installed and can be invoked as:
amrdecoder infile outfile
to decode an AMR file to header-less (raw), 16-bit signed, mono, 8kHz audio samples in the output file. (Input and output files may be '-' for standard input or output respectively.)

Similarly, it is expected that an AMR encoder called amrencoder exists and can be executed as follows:
amrencoder mode infile outfile
and convert raw, 16-bit signed, 8kHz mono audio samples in the input file to AMR using the supplied encoding mode.

For the AMR encoder/decoder, we have adapted the sample provided on the 3GPP website. Follow the instructions below to install it:

This AMR encoder is not very efficient but it works (which is important!)

Chapter 3: Running the Gateway

To run the gateway, you must run the two programs listed above (mmsrelay and mmsproxy). mmsfromemail should be called from your MTA (SMTP Mailer) to convert and deliver an MMS from an email sender. The order in which they are started is unimportant. They expect the configuration file to be passed as the last argument on the command line (default is mmsc.conf). The configuration file controls most aspects of the operation of the gateway.

General Configuration File

The configuration file format is the same as that used by Kannel. The configuration file consists of groups of configuration variables. Groups are separated by empty lines, each variable is defined on its own line. Each group begins with the group name. Comments are lines that begin with a hash (#) and are ignored.

A variable definition line has the name of the variable, an equals sign (=) and the value of the variable. The name of the variable can contain any characters except white space and equals. The value of the variable is a string, with or without quotation marks (") around it. Quotation marks are needed if the value begins or ends with white space or contains special characters. Normal C escape character syntax works inside quotation marks.

The variable group marks the beginning of a new group with the given name.

The core group is core and defines the log file location, log level (amount of debugging information – the lower the number the more debugging information), the location of the access log and the HTTP proxy host/port if any. (HTTP proxy host/port is specified using the exact same parameters as used by Kannel.)

group = core
log-file = log/mmsgw.log
log-level = 0
access-log = log/access.log

This should be followed by the main MMS gateway configuration group (mmsbox).

group = mmsbox
name = "My MMSC"
hostname = ds.co.ug
host-alias = mmsc
local-mmsc-domains= mbuni.org,service.com
local-prefixes = 075;+25675;25675
directory-store = spool
max-send-threads = 5
send-mail-prog = /usr/sbin/sendmail -f '%f' '%t'
...

The table below lists all the configuration directives

Variable Name     Type     Description    
group     mmsbox     Mandatory variable    
name     string     User-friendly name for the Gateway, used in notices, etc    
hostname     string     Local hostname. This is added as a qualifier to the sender address when MMS is forwarded to Email or to a foreign MMSC via SMTP. Defaults to localhost    
host-alias     string     Short form of hostname. This is used in generating message IDs. It is also used to generate the message retrieval URL (sent as part of the MMS notification): For instance if you have this as mmsc then the retrieval URL will have the form http://mmsc/msgtoken (no port is added). Be sure to keep this value short as some handsets do not like long URLs in MMS notifications. If you do not supply a host alias, the gateway will create a long form URL (http://hostname:port/msgtoken) when it sends notifications    
local-mmsc-domains     List of Internet domains (comma separated)     A list of internet domains that should be considered local to this MMS gateway. Email or MMS messages received destined to these domains should be treated as local    
local-prefixes     Number prefix list     Number prefixes that should be considered local. Messages to numbers that match these prefixes will be delivered locally (via mmsrelay)    
storage-directory     Directory name (string)     Directory where Mbuni creates message queues, MMBoxes, and the User Agent profiles cache. Mbuni creates a set of sub-directories in here for each function    
max-send-threads     Number     How many queue processing threads to start. A higher value means messages get delivered faster.    
send-mail-prog     String     Command to use for sending email messages (MMS-to-email or to foreign MMS gateways via SMTP). This command can include variables: %f – replaced with the message from address, %t – replaced with the recipient address (RFC 822 compliant), %s – the message subject, %m – the message ID    
unified-prefix     Number list     A string to unify received phone numbers, so that routing works correctly. Format is that first comes the unified prefix, then all prefixes which are replaced by the unified prefix, separated with comma (','). For example "+256,000256,0;+,000" should ensure correct UG prefixes. If there are several unified prefixes, separate their rules with semicolon (';')    
maximum-send-attempts     integer     Maximum number of attempts gateway must make to deliver message before giving up (e.g. mobile phone is off, email domain is unavailable)    
default-message-expiry     Integer     Default number of seconds in which message expires and is purged from queue (if not yet delivered). This figure is overridden by whatever is in the message.    
queue-run-interval     Real     How many seconds between each queue run    
send-attempt-back-off     Integer     mmsrelay uses a form of exponential back-off when sending notifications to MMS clients. This figure provides the starting back-off value (in seconds).    
sendsms-url     String     URL of the service through which SMS can be sent to a mobile subscriber (e.g. for WAP Push). It is expected that this url expects/supports Kannel-style send-sms parameters (udh, from, to, text, etc.)    
sendsms-username     String     Username to pass (for authentication) to send-sms URL    
sendsms-password     String     Password to pass (for authentication) to send-sms URL    
sendsms-global-sender     String     Optional sender (to field) to use in send sms url    
mms-port     Integer     Port on which mmsproxy listens for MMS messages from MMS clients (Default is 8191).    
mm7-port     Integer     Port on which mmsproxy listens for MM7 requests from Value Added Services providers. If this port is not supplied, the MM7 sub-system is not started.    
allow-ip     List of IP addresses     List of IP addresses of hosts allowed to use/access the MMS Port (above). You can use this for instance to insist that only connections coming via a known/trusted WAP gateway are serviced. Leave out to allow all machines to connect.    
mms-client-msisdn-header     String     Name of HTTP Header sent/inserted by WAP gateway as part of MMS request to indicate MSISDN of sender. Note that typically the MMS client does not indicate its MSISDN in the MMS message, it is up to the gateway to discover this and insert it. We rely on the WAP gateway to provide the MSISDN as an HTTP request header (default header name is X-WAP-Network-Client-MSISDN)    
mms-client-ip-header     String     Name of HTTP Header sent/inserted by WAP gateway as part of MMS request to indicate IP Address of sender. Similar to the above, if the MSISDN is not set, then we assume that the client is identified by IP address, which we extract from the request headers (using this header). Default header name is X-WAP-Network-Client-IP. If the header is not found, we assume the IP address as seen by Mbuni's MM1 interface.    
allow-ip-type     Boolean     Set this to false to prevent Mbuni accepting and processing messages from senders identified by IP address (i.e. not by MSISDN). Default: True.    
optimize-notification-size     Boolean     Set this to true make Mbuni attempt to squeeze MMS notifications in one WAP Push SMS, by leaving out subject and sender fields. Default: false    
content-adaptation     Boolean     Set this to false to turn off content adaptation in Mbuni. This will cause the MMSC to ignore client capabilities when sending messages, and could cause problems so beware! Default: true    
email2mms-relay-prefixes     Number list     When MMS is received via SMTP, the gateway needs to determine whether it is for a local or a foreign recipient. To determine if the recipient is local recipient, we use the local-prefixes setting. If the recipient is not local, the message should be forwarded on to the relevant foreign MMS gateway, only if the recipient number matches one of the prefixes in this comma-separated list.    
billing-library     String     Optional library containing billing and CDR functions. This library is loaded at runtime and should contain functions to be called to effect billing and CDR generation. See mms_billing.h for details.    
billing-module-parameters     String     Parameters to pass to the billing module specified above when it is loaded. This is a generic string whose interpretation is entirely up to the module.    
resolver-library     String     Optional library containing functions for resolving recipient MSISDN to hostname of Proxy-Relay that should handle the message. Supplying this libary over-rides the local-prefixes setting given above. If the Proxy-Relay hostname returned by the module is the hostname of the local MMSC, then the recipient is considered local. See mms_resolve.h for details.    
resolver-module-parameters     String     Parameters to pass to the Resolver module specified above when it is loaded. This is a generic string whose interpretation is entirely up to the module.    
detokenizer-library     String     Optional library containing functions for finding MSISDN from request URL sent by client. The last part of URL is treated as a string that is interpreted by the library and transformed into an MSISDN. This libary is only a fall-back in case the default sender address resolution fails. See mms_detokenize.h for details.    
detokenizer-module-parameters     String     Parameters to pass to the De-tokenizer module specified above when it is loaded. This is a generic string whose interpretation is entirely up to the module.    
prov-server-notify-script     String     Subscriber database interface script 1: This script will be called by the gateway to notify the subscriber database of per-subscriber events such as when a subscriber sends a message, successfully fetches a message, etc. This script is called with 2-3 arguments. Argument 1 is one of fetched, sent, failedfetch; argument 2 is the subscriber MSISDN; argument 3, in case of a failed fetch provides a description of the error (e.g. message expired).    
prov-server-sub-status-script     string     Subscriber database interface script 2: This script is called by mmsrelay to determine whether the recipient's device supports MMS. The script should exit with a value of 0 to indicate that the device does not support receipt of MMS notifications; 1 to indicate that the device supports MMS; -1 if the subscriber is not known or not provisioned for MMS. The return value determines how mmsrelay will deliver the message (see below).    
notify-unprovisioned     Boolean     Whether subscribers who are not provisioned for MMS should receive any notifications (e.g. SMS) when an MMS message is received for them.    
mms-notify-text     String     Message to send to device that does not support MMS, when a message is received for the user. This message is sent as plain SMS via the Send SMS URL specified above.    
mms-notify-unprovisioned-text     String     Message to send to devices that are not provisioned for MMS (only if notify-unprovisioned is true).    
mms-message-too-large-txt     String     If a device tries to fetch a message, which during content adaptation is determined to be too large for the target device (based on capabilities data supplied by the device), the message is discarded, this text is sent to the device instead as part of an MMS message.    
mms-to-email-html     string     When an MM is destined for email, we must format it to make it more suitable for email readers. (For instance, the SMIL part of the MM will make no sense to most email readers.) The gateway formats the message as follows: It generates a multi-part MIME message with the main part being an HTML entity in which MM parts are embedded. The text given here is tagged at the bottom of the HTML.    
mms-to-email-txt     String     This string is placed in the MMS converted to email as an alternative to the HTML part, for email clients that do not support HTML.    

MM7 Configuration

MMS Value-Added Service Providers (VASPs) are configured using one or more mms-vasp groups:


group = mms-vasp
vasp-id = newcorp
type = soap
short-code = 100
vasp-username = newscorp
vasp-password = news123
vasp-url = http://example.vasp.com:8080/mm7
mmsc-username = mymmsc
mmsc-password = mypass


Variable Type Description
group String Mandatory: mms-vasp
vasp-id String User friendly name
type String This should be one of: soap, eaif
short-code Number Short number for this VASP: Messages received by Mbuni to this number are routed to the VASP via MM7.
vasp-url String Outgoing messages to the VASP are sent via this URL (using HTTP POST)
mmsc-username String Outgoing HTTP authentication: The username Mbuni must use when sending data to the VASP
mmsc-password String Outgoing HTTP authentication: password Mbuni must use when sending data to the VASP
vasp-username String Incoming HTTP authentication: The username used by the VASP to authenticate itself to Mbuni when sending data
vasp-password String Incoming HTTP authentication: The password used by the VASP to authenticate itself to Mbuni when sending data to the VASP
Note that currently only HTTP Basic Authentication Scheme is supported by Mbuni (for both incoming and out-going requests).

MM4 Configuration

Foreign MMS Gateways are configured using one or more mmsproxy groups:


group = mmsproxy
name = "A test mms proxy"
host = test.com
allowed-prefix = "075"
denied-prefix = "077"


Variable Type Description
group String Mandatory: mmsproxy
name String User friendly name
host String Fully qualified domain name
allowed-prefix Number list List of recipient number prefixes that can be delivered via this Proxy
denied-prefix Number list List of recipient number prefixes that cannot be delivered via this proxy

When an MM destined to an MSISDN cannot be delivered locally, the gateway searches the list of Proxies to see if one of them can handle the message. If one is found, the message is formatted as MIME and sent via SMTP to the proxy.

Chapter 4: Gateway Architecture

In this section we provide an overview of the gateway architecture.

As indicated, there are thre components to the gateway: The Relay (mmsrelay), the Proxy (mmsproxy) and SMTP/Email Interface (mmsfromemail). We describe the function of each of these in turn.

MMS Proxy

This component (mmsproxy) is the main point of interaction between the gateway and MMS clients and VASPs. It provides an HTTP interface through which clients can send MMS messages. From clients, message types expected on this interface are typically:

All the above messages are sent to the proxy as the body of an HTTP POST request. Messages are retrieved by supplying the message URL in an HTTP GET request. When such a request is received, the proxy:
  1. Locates the message: From the URL, the proxy can tell if this is a message in the MMbox or in the in the queue for client-destined messages.
  2. Extracts the User Agent Profile URL from the (HTTP) request headers. If that is missing, the profile information is built out of the HTTP Accept headers. The profile URL is passed to the content adaptation module, which performs various modifications to the MM such as: note that profile data is cached (in storage-directory/UserAgent_Profiles) so as not to have to fetch it each time.
  3. The message is then packed and returned to the client as the result of the HTTP request.

Currently, no MMbox quotas are imposed.

From VASPs mmsproxy expects and processes:
Note that only SOAP MM7 requests are supported at this stage.
This component must always be running.

MMS Relay

The Relay manages routing of all messages (to phone, email, VASP).

The Relay watches the global queue for incoming messages (from VASP, external MMSCs or clients). For each message that arrives in the queue, the relay:

  1. Determines if the message is due for delivery attempt. An attempt is made to deliver the message as soon as it is received (deffered delivery requests are honoured however), with exponential back-off in case of failure.
  2. At the first delivery attempt, a call is made to the billing module to effect billing and CDR generation. If the billing module indicates that the sender does not have sufficient credit, the message is discarded and the sender notified via delivery report.
  3. If the message is due for delivery attempt, the global sender determines, for each recipient, how to deliver the message:
    1. If the message is destined for a local MMS client, the message is transferred to the mobile/local queue. A copy of the message is sent (as MIME) to the MMBox host (if one is configured)
    2. If the message is destined for an email user, the message is re-formatted as MIME, sender and recipient addresses normalised as RFC 822 addresses, and the message passed to the mailer.
    3. If the message is destined for a foreign gateway, it is coded as MIME and passed to the mailer for delivery via SMTP
    4. If the the message is destined for a VASP (identified by short code), then it is sent using MM7 protocols to the relevant VASP.
    5. If the message cannot be delivered, the sender is notified.

For messages placed in the mobile/local queue (i.e. those destined to MSISDNs in the area served by this MMSC or IP-based clients), the relay performs the following functions:

  1. Notification is sent to the recipient client via WAP Push
  2. Delivery of other notifications such as delivery and read reports to clients via WAP Push
SMS is used as the transport for WAP Push messages, if the recipient is an MSISDN, otherwise UDP is used.
The Relay maintains a separate queue for messages pending delivery. At set intervals (see configuration section), it sends notifications to the recipient. It keeps sending notifications until the message is fetched or the client indicates that it wishes to defer message retrieval. A back-off mechanism is utilised to prevent flooding of notifications. A message will be removed from the queue if:


A word about queue management: A simple queue management scheme is used. Each queue entry consists of two files: The 'q' file (which is plain text) contains the entry control data (list of recipients, next delivery attempt time, etc), the 'd' file contains the message data. This scheme is similar to that used by popular MTAs. Queue processors mostly operate on the 'q' file, and use file locking to guard against duplicate delivery, file corruption, etc.
See mms_queue.h for details.

SMTP/Mail Interface

The SMTP/Mail interface receives MMS from the MTA and routes them depending on recipient or sending proxy. Specifically:
  1. If the message is a send request, it is queued to the global queue for delivery as long as the recipient is permitted via the interface (see configs)
  2. If the message is a notification (e.g. delivery report), the interface carries out the necessary action (e.g. forwarding of receipt or deletion of message from local queue)

This interface should be invoked from your MTA as follows:

mmsfromemail -f from_address -t recipient_address -p sender_mmsc_hostname conf_file

Note that no IP-based security is provided at this interface. It is expected that security measures (e.g. firewalls, etc) will have been setup to ensure that messages can only reach the MTA and be handed over to this interface if they are legitimate.

Utilities

We plan to add a number of utilities to the gateway. The first of these is mmssend.

mmsssend can be used to submit (inject) a message into the global queue. It should be invoked as follows:

mmssend -f from_address -t recipient_list -m mmsfile [-b] conf_file

Notes: The message is placed in he global queue with expiry set to the system maximum, and the queue entry ID is printed to standard output.

Chapter 5: Tips & Tricks

This section is a compilation of tips and tricks on making Mbuni work better for you

Passing MSISDNs to Mbuni

As indicated earlier, Mbuni expects the MSISDN to be sent to it as a special HTTP request header. There are however times when it is either not possible, or not practical to insert the header into the MMS request. For such cases, Mbuni provides another way to specify the sender MSISDN: The last part of the URL passed in the HTTP transaction is passed to the De-tokenizer module (if specified), which should return a valid sender address. So for instance you can configure a clients to use a URL like http://mmsc/xYz12R2 as the MMSC address, and Mbuni will pass xYz12R2 to the de-tokenizer module, which must return the sender address. Mbuni will only do this it has failed to find the address request header.
If no sender address (MSISDN) is found, Mbuni assumes that the MMS client is identified by IP address, and attempts to look up the IP address of the sender (see config section) and use that as the sender address. You can block this by specifying allow-ip-type = false

Note that because of the above feature, you need to configure your WAP gateway and Mbuni IP security to ensure the system is not easily spoofed.

Sample Kannel WAP configuration

We provide a sample Kannel wapbox config below, with some explanation
group = wapbox
bearerbox-host = localhost
log-file = "/tmp/wapbox.log"
syslog-level = none
access-log = "/tmp/wapaccess.log"
timer-freq = 10
map-url = "http://mmsc/* http://localhost:1981/*"

This is a live example that was used in tests. In the example we use:

Chapter 6: Log Files

The gateway writes a log file of important actions to the log file configured. Message traffic is written to the access log in a standard format.