January 2000

Taming Media Stream Processing With M.100

BY MIKE COFFEE

The MSP Consortium’s M.100 recommendation is a media-processing API that separates media-processing software from the underlying hardware, fostering independent competition and development in these two new computer telephony
value-adding layers.

M.100 allows vendor-independent development of M.100-compliant media-processing software and hardware resources. What MVIP did at the PC level in 1990, M.100 does at the board or host level today.

OpenMedia is Commetrex’s implementation of the M.100 recommendation. OpenMedia is an open multi-stream, media-processing software environment that supports IP media-stream processing for voice and fax over IP (VoIP and FoIP), as well as traditional computer telephony circuit-switched applications. OpenMedia reduces the cost of developing and porting media-processing products and supports straightforward media software deployment on Commetrex’s Media Stream Gateway boards, host PCs, and other hardware platforms. OpenMedia is designed to work efficiently with Open Telecommunications Framework (OTF), Commetrex’s implementation of the ECTF S.100 host-level environment recommendation, as well as with other vendors’ software environments.

OpenMedia can exist on any stream-processing computing resource or resources, making it just as applicable to proprietary embedded systems as to general-purpose DSP-resource boards, or even to a host PC. Commetrex is licensing OpenMedia for:

• Commetrex Media Stream Gateway products.
• Windows NT PC host implementations.
• Proprietary-platform implementations.

FLEXIBLE MEDIA PROCESSING
OpenMedia is the only software environment that promotes the interoperability and portability of media-processing system resources provided by various vendors. With Open Media, developers can easily “mix and match” M.100-compliant media-processing resources, create proprietary integrated multiple-media products, avoid years of software development, and improve system performance. Today, with M.100, the industry has the opportunity to integrate voice, fax, data, VoIP, FoIP, text-to-speech, speech recognition, and even video, on open low-cost media-processing resources.

Design Objectives
M.100 specifies an API; it only implies an implementation. The implementation goals of OpenMedia were:

• System resource efficiency.
• Development efficiency.
• Skill-set separation.
• Hardware independence and multi-processor support.

In a signal-processing system, resource efficiency means that DSP MIPS and on-chip RAM are conserved. The easiest way to improve MIPS efficiency is to limit the system to one media-processing task. Multiple media, with widely varying resource requirements, require OS-like task management on the DSP. With media-diversity as an overriding requirement for OpenMedia, an efficient resource manager is absolutely necessary.

Development efficiency is promoted when functions common to most applications are factored out of the application and placed in the supporting environment, an underlying principal of M.100. In OpenMedia this concept extends across all processors in the environment: the host PC, the MSP media gateway’s coprocessor, and its two DSPs.

Skill-set separation also promotes development efficiency. Most algorithm developers are not well-trained computer scientists. An effective environment will separate the DSP software from the rest of the system so the algorithm developers need not have system-level concerns.

Even with recent advances in DSP compiler design, most DSP algorithms are written in assembly code, which by definition is hardware-specific. But aside from this, there is no reason a multi-stream environment cannot be hardware independent. This means any stream-transform software in the system can run on any processor in the system transparent to the control program.

THE MSP ENVIRONMENT
The software environment specified in M.100 is a logical aggregation of software components that control the flow of media streams and the operation of Media Stream Transforms (MSTs) on those streams. A media stream is a unidirectional flow of data between processing components with real-time constraints that usually involve the pacing requirements of isochronous (“equal-time”) data. MSTs are software components that process media stream data in some way: change the format, extract information, or add information.

The environment’s components comprise a processing system that may include single or multiple processors. One of the environment’s key software components is the MSP application, responsible for requesting stream interconnections and bringing MSTs into execution to deliver a specified function to an external client entity.

Streams enter and leave the MSP environment through stream servers, which may be of various types: file stream servers that source or sync isochronous media (“time-varying media” in S.100 speak), image files for fax (spatial media), and text-based vocabularies for text-to-speech to and from persistent (file) storage. PCM stream servers interface PCM highways, such as H.100, to the MSP environment. Packet servers source and sync packets in IP applications.

Stream-Paced Execution
An important M.100 concept is that of stream-paced execution. “Pacing streams” have real-time constraints. Resource efficiency can be improved if the natural processing rhythm of the MST is matched to that of the availability of pacing stream data. MSTs are required to specify to a packaging utility an optimum amount of data to process. This information is then used by the environment (Execution Controller in OpenMedia) to start an MST’s execution when an integral number of the specified buffers are available. Once the MST has processed the specified amount of data or an integral multiple of it, the MST relinquishes the processor without incurring preemption overhead.

MSP applications are responsible for implementing and exposing the environment’s functionality to client processes. In OpenMedia each MSP application implements related functions, such as voice play/record or fax send/receive. Other components route streams, while others (the MSTs) implement stream-processing algorithms. In OpenMedia, these components can span multiple processors.

OpenMedia includes a stream router, which transports streams between board- and host-level execution environments. A fax send/receive facility could, therefore, be implemented with the fax modems (V.21, V.17, etc.) and image conversions on DSP-based execution environments and the fax protocol (T.30, T.37, T.38, etc.) on the host processor. Low-port-count applications may be easily (and inexpensively) implemented on the host. At Commetrex, all MSTs are first coded and implemented in C, and validated in the host’s execution environment, so host signal processing, which is extremely price-competitive in small systems, is a “free” fallout of this development approach.

Packaging Utility
M.100 specifies an MSP Packaging Utility (MPU) that processes a set of directives to combine the executable elements of the MSP environment with descriptive parameters to produce files suitable for loading and execution. The MPU will typically define the entire environment, but functions are available to dynamically modify the environment’s resources.

Session Manager
The OpenMedia Session Manager (MSES) handles all service requests from an MSP application. The MSES is responsible for assigning these service requests to an execution environment. The MSES tracks the resources available in each execution environment and assigns them to “load-balance” the system. Then the MSES learns of the resources it can control from the packaging utility or by registration commands from MSTs and stream servers.

All functions of an MSP process — MSP applications, MSTs, and stream servers — are performed in the context of a stream session. An MSP application can establish more than one session (define more than one stream graph). A session context maintained by the MSES describes the graph with descriptions of system resources, streams, and MSTs that are a part of that processing session. Once a stream session/graph is defined, the MSP application uses start/stop graph commands to control the actual stream processing.

Stream Interface
OpenMedia is a stream-processing environment. All data, media, commands, and events are conveyed between entities via MSP streams, where a stream is a unidirectional flow of data with one writer and one or more readers. All streams have a fixed data type, and, for pacing streams, a fixed data rate. A pacing stream is a stream that drives the rate of execution of an MST and the stream servers attached to it. A PCM stream is an example of a pacing stream. Ancillary streams are streams that are not pacing streams, such as command or event streams.

The M.100 interface has two components: the Stream API and the Stream Processing Interface. The Stream API provides functions to manage stream sessions, describe streams, and write/read ancillary stream data, usually by MSP applications. The Stream Processing Interface is specifically designed to implement an M.100 environment’s pacing stream functionality, and serves as the interface between the Execution Controller (MEC) and stream graph elements that transfer pacing stream data. A stream process entry point is published by a stream server, stream router, or MST for each pacing stream interfaced. It is called repeatedly by the MEC to move buffers of pacing stream data between stream graph entities.

Stream Servers
OpenMedia supports several stream servers: file stream servers source and sync streams from and to disk storage. PCM stream servers interface with H.100/110 PCM highways and the MSP-H8 line-interface card. Packet stream servers source and sync packet-based media data. Finally, conference bridge stream servers implement the special requirements of both PCM- and packet-based conference bridges.

Stream Router
A stream router’s job is to isolate the environment’s inter-execution environment communication mechanism. In OpenMedia a stream router exists on the NT host and, if an MSP media gateway board exists in the system, on each execution environment on the board. Since more latency exists between the host and board-level environments an extra “slip-joint” is provided by the stream router in the form of flow-control tokens. This allows the stream router to fetch data ahead of the pacing requirements of the next element in the chain, usually an MST, up to the limits imposed by the board’s data buffering capabilities.

Execution Environment
The work of media processing is accomplished in an MSP execution environment, the dispatchable unit managed by the MSES. An OpenMedia execution environment is comprised of the resident MSP Supervisor (MSPV), an MEC, a stream server and router, and MSTs. The MSES determines which environment should execute a function and sends the command to that environment’s MSPV. The MSPV manages the local resources, including MSTs, stream servers, and a stream router, if present. Each media processor on an MSP media gateway has an MEC that manages the execution of MSTs and stream servers that are located on the processor. The MEC also implements memory management and stream buffering. On the MSP-320 the TMS320C6201 has a high-performance PCM serial port that interfaces directly to the board’s H.100 PCM interface, requiring that a PCM stream server execute on the DSP.

Supervisor (MSPV)
Each of the two MSPVs located on the MSP-320’s co-processor (and the MSPV on the NT host) manages a single execution environment. They are designed to offload processing from the DSP-based MEC by managing the interface with the MSES. By managing the interface and thus hiding the specifics of the execution environment, the MSES can operate on different environments transparently. Some commands result in a MEC’s dispatch queue being modified by the MSPV via the DSP’s host-port interface.

Execution Controller (MEC)
The MEC manages the execution of the MSTs, stream routers, and stream servers. It also implements memory management and stream buffering. The MEC is controlled by commands from the MSPV or from API calls made by the MSTs and the stream server and router under its control.

Media Stream Transforms (MSTs)
The M.100 recommendation defines the MST as follows: “An MST is a software function operating on a set of media streams and presenting a Stream Processing API and MST API to the MSP Environment.” During MSP environment initialization, the available MSTs are defined to the MSP environment by their MST prototypes.

An MST prototype is a software construct that identifies executable code and data, and provides a description of the processing requirements and media streams processed by the MST. This description is used by the MSP environment to bring an MST into execution on a processor resource. The MSP environment also uses the description to connect the MST to streams in the session and to control the execution of the MST.

The MST prototype description includes the processor type, and the CPU cycle and memory requirements of the MST. It also defines all of the streams the transform takes as inputs and outputs. The MSP environment uses the resource requirements and stream blocking information to schedule the execution of the MST.

MST Portability And Interoperability
One of the key objectives of M.100 is media-processing software portability. The M.100 specification makes it possible for developers to independently develop and market M.100-conforming MSTs that will operate cooperatively on MSP streams. This means software media-processing components can be even more portable than hardware components.

All communications between an MST and the environment are through streams. Since streams are precisely defined in the M.100 specification, the environment can “hook up” MSTs obtained from different vendors, and, as long as the output stream of one matches the stream definition of the input of the downstream MST, they will work cooperatively. This means MSTs can be developed to the M.100 standard and then packaged for a specific environment by that environment’s packaging utility.

Message Logger
The message logger accepts activity reports and error messages from the various components of the OpenMedia environment. The message logger is an independent task connected to the transport used by OpenMedia, usually the OTF transport. The MSES and MSPV open connections directly to the message logger. Messages, which are time-stamped at the source, are posted by the message logger to a log file and optionally to a screen display. There can be one or more message loggers on a system, allowing all environments to log to the same file, or for each environment to have a separate log file.

Adding Media Resources
The OpenMedia SDK allows you to easily add a media resource to your OpenMedia-based system by developing the MST, its MSP application control program, and a client API. If a stream server is required that is not part of the supplied environment, you will develop it using the stream server template supplied in the SDK. The SDK includes source-code shells for MSTs, MSP applications, and stream servers, as well as working examples.

Here’s how to add a media resource:
Develop the MST: First, set the system’s requirements as seen by client applications. This determines the client API and ultimately the lowest level of function to be implemented in the MST. The MST implements the sub-system’s atomic-level functions that are to be managed by the MSP application.

The MST will implement your algorithm — a proprietary modem, for example. So you will probably have a pretty good idea how it should be implemented. But resist the urge to code it, and instead develop your MST from the outside in. Define the externals, such as the streams (input and output pacing streams, command and event ancillary streams), define all the MST’s commands and events, and settle on an execution strategy.

Execution strategy defines how much data will be processed for each invocation of the MST by the MEC. For example, a VoIP vocoder might want to process 30 milliseconds of data while a DTMF detector should have 10 milliseconds. For the modem in our example, you might decide to divide the task into two MSTs, one executing an integral number of samples, and the other on an integral number of bauds. The sample-rate MST might execute several times for each execution of the baud-rate MST.

You are now ready to code the MST. But wait. Don’t code it in assembler; code it C. The OpenMedia SDK includes an MST unit-test environment, provided in C source, that will allow you to fully unit test your MST in C on your workstation. Maintaining a C-coded baseline for your design improves maintainablity, and with the power of today’s DSPs and increasingly capable compilers, you may not have to use assembly language to meet your density requirements. You then specify the test-input file and verify the output file. Once you’ve validated your design, you are ready to move towards the target environment, unless it’s the host, in which case you’re almost done.

If the MST’s destination is an embedded DSP environment, you will want to run it through the processor’s simulator and count cycles so you can complete the MST’s specifications to the target processor’s packaging utility.

Develop the MSP Application: After your MST has passed unit test, your task is to develop the MSP application. Of course, it could be done in parallel, and probably will, as the skill sets are quite different. Your MSP application will establish a session with the environment’s MSES and “mount” the necessary streams. Once the streams are mounted the application will load MSTs to complete the stream graph. When the graph is complete, the MSP applications initiates processing with an MSPStart_Graph command. The application will send events to the client to apprise it of execution progress and completion.

One of the critical benefits of OpenMedia is development productivity, and one of the best examples is how easy it is to test your software. The SDK includes shells for the components you will need to develop. The environment makes the location of the MST transparent to the MSP application, so you can test the application and the MSTs it is controlling on the host processor, speeding development.

SUMMARY
OpenMedia gives the developer of media stream processing system resources a standards-based software environment that provides:

• Multi-vendor integration of media-processing software.
• Software portability.
• Common multi-vendor environment.
• Developer skill-set separation.
• Hardware independence.
• Cooperative multi-vendor stream processing.

Mike Coffee is the president of Commetrex Corp. Commetrex develops, produces, and markets software products for computer telephony integration original equipment manufacturers (OEMs), system developers, and integrators. For more information, visit their Web site at www.commetrex.com