Project WAM

In this page: Multimedia Models and Formats | XML Processing | Multimedia Authoring | Augmented Environments

Multimedia Models and Formats

This research theme is dedicated to models and languages for representing and describing rich multimedia web documents.

Interactive audio

After some work on the SMIL language, we are now focusing on new formats for audio. More specifically, we are participating in IAsig (Interactive Audio special interest group), an international initiative for creating a new format for interactive audio called iXMF (Interactive eXtensible Music Format). We have also defined an XML tiny version of iXMF (without scripting, but with integrated 3D audio rendering) and we have implemented it in an audio engine adapted to embedded systems and based on OpenSL ES (Open Sound Library for Embedded Systems).

We are developing A2ML as an XML format for embedded interactive audio, deriving from well-established formats such as iXMF and SMIL. We use it for Augmented Reality applications, where virtual, interactive, 3D sounds are combined with the real world.

Mode details on this activity.

Timesheets

With the advent of HTML5 and its support in the most popular browsers, HTML is becoming an important multimedia language. Video and audio can be embedded in HTML pages without worrying about the availability of plugins. However, animation and synchronization of a HTML5 page still require programming skills. To address this issue, we are developing a scheduler that allows HTML documents to be animated and synchronized in a purely declarative way. This work is based on the SMIL Timing and Synchronization module and the SMIL Timesheets specification. The scheduler is implemented in JavaScript, which makes it usable in any browser. Timesheets can also be used with other document languages, such as SVG for instance.

More details on this activity — a poster

LimSee3 model

For most users, authoring multimedia documents remains a complex task. One way to deal with this problem is to provide template-based authoring tools, but with the drawback of limited functionality. The LimSee3 document model was designed specifically for the creation of authoring tools using templates while keeping rich composition capabilities. It is based on a component oriented approach integrating homogeneously logical, time and spatial structures. Templates are defined as constraints on these structures.

The LimSee3 model is implemented in the LimSee3 authoring tool.

XTiger templates

When producing a specific type of web document, authors are faced with all the possibilities provided by the (X)HTML language, and they have to make a number of difficult decisions. XTiger (eXtensible Templates for Interactive Guided Editing of Resources) tackles this problem by specifying how the document language has to be used for representing a certain type of document. A XTiger template is a skeleton representing a given type of document, expressed in the format of the final documents to be produced (XHTML, for instance). It includes some statements, expressed in the XTiger language, that specify how instances based on this minimal document can evolve and grow to make a full, well structured document.

The XTiger language is implemented in Amaya, which helps document designers to create templates, and authors to edit XHTML documents according to templates. In a joint work with EPFL, we also use XTiger for editing XML documents in the browser, thanks to the AXEL library.

XML Processing

As data and document formats for the web are based on XML, it is crucial to better understand how XML structures can be processed, and what are the theoretical tools that may help to develop an effective framework for processing XML structures.

The goal of the research published so far in the literature is often limited to establishing new theoretical properties and complexity bounds. Our work differs in that, in addition to these goals, we seek resolution algorithms, efficient implementation techniques, and concrete design that may be directly applied to XML systems. We pay special attention to issues such as type checking and efficiency.

Practical results of this research are available in the XML Reasoning Solver.

More information...

Multimedia Authoring

Authors of multimedia contents and applications need specialized tools to benefit from the many, complex possibilities offered by web multimedia formats. Methods used for static, textual documents, such as WYSIWYG or direct manipulation, do not work for dynamic multimedia web documents. New approaches have to be developed and experimented.

Authoring techniques developed in this activity are experimented in Amaya and LimSee3.

A new approach to XML editing is explored with the Media group at EPFL in the AXEL project. XTiger templates are used to edit XML documents in the browser, through a Javascript library.

We are also developing a timesheet editor, i.e. a multimedia page authoring tool for HTML5 pages that use timesheets.

Augmented Environments

The goal of our research is, using embedded sensors, to introduce personal Augmented Reality (AR) devices in ubiquitous computing, context-aware computing, and intelligent environments. We refer to these computational environments collectively as augmented environments (AE) and we believe that personal AR devices such as mobile phones will play a central role in their success. Augmented environments offer the possibility of using ubiquitous computation, communication, and sensing to enable the presentation of context-sensitive information and services to the user.

Applications we use to elaborate and validate our concepts are pedestrian navigation for visually impaired people and applications for cultural heritage visits. On the authoring side, we are interested in interactive indoor modeling, audio mobile mixing and formats for Points of Interest (POI). AE Services we consider are among others behavior analysis for accessibility, location services and indoor geographical information services.