Multimedia Platform Project

Click here for Japanese version.

Overview

There are increasing needs in developing distributed multimedia applications such as video mail, video conference programs and others which are capable of handling various media in a distributed computing environment. However, these applications have been built using specially dedicated hardwares and systems which are not interoperable with workstations nor the current network environment.

The goal of our project is to develop the software to realize the distributed multimedia environment which can integrate various multimedia equipment connected to commonly available workstations. The followings are important in this R&D:

  1. to develop the software on off the shelf hardware platform with good price/performance,
  2. to support high performance multimedia processing, and
  3. to make it easy to develop multimedia applications.
There already are several multimedia systems for workstations in the market, but these systems only offer a partial solution to the distributed multimedia environment. In this project, we vertically integrate the system and aim to solve all the aspects of the distributed multimedia environment.
Software Architecture
Distributed Real Time Micro Kernel
Real Time Network Protocol
Media Server
Toolkit
Media Base Server
Contact

Software Architecture

The software architecture of our system consists of a micro kernel, middle-wares and a toolkit. As for the micro kernel, we developed RMK95 which is an extension of Mach 3.0 micro kernel developed by Carnegie Mellon University (U.S.A.). We enhanced RMK95 to be able to handle continuous media. It operates on i386/486 or Pentium base PC/AT compatible machines as well as some machines having R3000/4000 or SPARC CPU. As for the network, it currently supports Ethernet and FDDI. ATM is in the scope of our future support.

On top of RMK95 micro kernel, we may have more than one OS personality module. At the moment, we have UX Server which is under the restriction of UNIX license and 4.4BSD-Lite Server (4.4BSD-Lites) which is not. Users can choose one which is suited for their need. In addition, modules for supporting multimedia protocols are also separated from the micro kernel. There are a module for ST-II protocol which supports QOS control and an MPC module for QOS controlled multicast communication.

In order to make it easy to develop distributed multimedia applications, we implemented Media Servers based on the conductor-performer model. They are implemented as servers on top of the micro kernel. A conductor controls flows of continuous media data, and performers actually process various media data. Multimedia applications do not need to communicate with performers, but only with the conductor. If we need to support a new media type, we only need to prepare a performer for the new media data.

In addition, a toolkit based on Tcl/Tk is available. Users can easily develop distributed multimedia applications with friendly GUI.

Distributed Real Time Micro Kernel

RMK95 is a distributed real time micro kernel which is an extension of Mach 3.0 micro kernel developed by Carnegie Mellon University (U.S.A.). It is enhanced to handle continuous media data.

In order to handle continuous media data which have timing constraints, the traditional UNIX time sharing process scheduling mechanism is not suited, but real time scheduling mechanisms are necessary. For handling continuous media data, soft real time control is important where processes and threads are dynamically scheduled based on their QOS (Quality of Service). In RMK95, the dynamic control is done by observing deadline misses of periodic threads

RMK95 micro kernel has the following features:
resource management for continuous media:
The concept of QOS tickets is introduced and mechanisms for resource reservation, resource utilization adaptation and resource arbitration among sessions are available.
real time thread package:
A user level real time thread package is available. Threads are lightweight and more precise thread scheduling is possible.
various device drivers:
Several device drivers are added for a distributed multimedia system: e.g. a driver for capturing video and a driver for FDDI.
public domain:
Instead of UX Server which is under the restriction of UNIX license, 4.4BSD Lite Server based on 4.4BSD Lite is supported. It is compatible with FreeBSD, can share the same file system, and most of the binaries are compatible.
new hardware platforms:
Not only i386/486 or Pentium base PC/AT compatible machines but also some machines having R3000/4000 or SPARC CPU are supported.

Real Time Protocol

In order to handle distributed multimedia data, a high speed real time network protocol which is capable of reserving bandwidth and controlling QOS is in need. We developed a high speed protocol processing architecture on top of RMK95 micro kernel. At the moment, ST-II protocol is available over Ethernet and FDDI. We also developed a multicast communication protocol MPC (Multimedia Protocol for Continuous Media) for continuous media data. Using MPC, dynamic and relative QOS control is now possible over mixed network environments of ATM, FDDI, Ethernet and Wireless.

Media Servers

Based on the conductor-performer model, media Servers are implemented as multiple servers on top of RMK95 micro kernel. Each host has a single conductor and several performers for various media data. Each conductor accepts requests from applications and controls performers of the local host as well as performers of remote hosts. Applications need not communicate with performers. A performer of a remote host is controlled via the conductor of the remote host.

Media Servers have the following features:

to simplify the construction of distributed multimedia applications:
Applications only need to communicate with local hosts and request to create media streams. Actual continuous media data handling is done by Media Servers.
to easily adapt to new media data:
In order to support a new media data or a new multimedia hardware, only a performer is need to be added.
to absorb differences of hardwares:
Since differences of hardwares are absorbed by performers and conductors, applications do not need to be aware.
to support synchronous/asynchronous streams:
Because continuous media data have timing constraints, media streams must be synchronized with time. However, if media streams are synchronized at every points, there is no place to adjust the variation of processing time. Therefore, Media Servers combine both synchronous and asynchronous streams so that media data are processed asynchronously whenever possible and synchronously when necessary.
Several performers are implemented: Sound Performer for handling sound input and output, Capture Performer for capturing video, JPEG Performer for JPEG decompression, File Performer for handling secondary storages, X Performer for X window system, and so on. For example, X Performer consists of a special server for video which cooperate with X Server. X Server manages window frames for video, but actual drawing is done by the special server.

Toolkit

In order to make it easy to develop distributed multimedia applications with friendly GUI, we extended Tcl/Tk libraries and XF interface builder.
The extended functions are available in Tcl scripts like other Tcl/Tk standard commands and Widgets. These functions work by communicating with a conductor. MPEG Widget, Video Widget, Video Capture command and so on are available. Real time video replay and QOS control are possible.

Distributed Media Base Servers

Distributed Media Servers give mechanisms for storing and providing multimedia data. They consist of Continuous Media Storage Server, Distributed Name Service and Security Server.

Continuous Media Storage Server actually stores and provides multimedia data. It is implemented as a File Performer controlled by a conductor.

Distributed Name Service locates multimedia data and services, and Security Server does authentication.

Contact

Keio MMP Project
Contact: mmp@sfc.keio.ac.jp