Immersive Audio in MR

home | applications | research | people

abstract

Mixed reality experiences often demand technological solutions that are unnecessary or too complex for other applications such as desktop simulations. This is especially true of a mixed reality scenario that attempts to provide the user with an experience that is aurally as well as visually immersive. We discuss the shortcomings of technologies such as DirectSound and OpenAL, which are the typical solutions chosen for an application that must provide dynamic spatialized audio. We then present an overview of audio systems used in mixed reality platforms including our own previous efforts and describe the problems that one encounters in using such technologies in MR. We then describe the design and implementation of the audio system used in our next-generation mixed reality platform and discuss the ways in which it succeeds where other technologies fail.

Our audio library is designed to make it possible to compensate for speaker configurations which deviate from the ideal placement assumed by most 3D sound applications; this is of importance to mixed reality because a complex set may not permit ideal speaker placement. It also allows the sound designer to take advantage of speakers placed above and below the user to provide a better sense of spatialization for sounds in the vertical direction than is possible with speakers in a single plane. Further, it makes it possible for a sound system to contain point-source speakers, headphones, radios, and other unusual devices in additional to a surround sound system, and to run the full audio experience from a single machine, preventing a mixed reality exhibit from having to include several computers to run different aspects of the audio experience.

people

• Darin E. Hughes
• Scott Vogelpohl
• Charles E. Hughes

publications

• D. E. Hughes, S. Vogelpohl and C. E. Hughes, "Designing a System for Effective Use of Immersive Audio in Mixed Reality," Proceedings of 2005 International Conference on Human-Computer Interface Advances in Modeling and Simulation (SIMCHI’05), New Orleans, January 23-27, 2005, 51-57.
  [PDF (0.33 MB)]