Audio Special: Spatial awareness

[This feature was published in the May 2013 edition of Develop magazine, which is available through your browser and on iPad.]

Our ears process subtle auditory cues in everyday environments. They supply us with a constant stream of information on spaces and objects, both within and outside our field of view. Spatialisation is the process of rendering 3D audio in virtual environments.

Spatialisation of audio in game design has the potential to greatly enhance the playing experience and a diverse range of advanced technologies make this possible today, even over stereo headphones.

Firelight Technologies, the developers of Fmod, the audio engine used in recent hits Tomb Raider and Hitman: Absolution, has released FMOD Studio, a new sound solution for games. With a powerful and intuitive user interface, FMOD Studio features an inbuilt spatialiser and an open plug-in architecture which makes advanced third-party technology easily accessible to developers.

WHY SPATIALISATION?

With the increase in computing power, graphics technology has come a long way in accurately modelling light and materials, and today’s shaders are able to generate stunningly realistic visual scenes. This extra power also enables more sophisticated audio engines to accurately model acoustic environments and we are now seeing an acceleration in developments in 3D audio tech for games, even for mobile devices.

Games can achieve a high level of realism and immersion by modelling the way in which sound radiates from a game object, its propagation in the virtual environment up to how it reaches each ear.

Spatialisation may also be used creatively to emphasise the scale or mood of the space surrounding the player or even inform the player on the location or distance of objects/actions and provide useful cues, for example imminent threats, near misses, objects of interest in nearby spaces.

HOW IT WORKS

“High-quality spatial audio is about more than simple positioning, it’s also about modelling different acoustic environments, and thereby giving the user the impression of being inside for example a small room, a concert hall, or a forest,” explains Fritz Menzer, CEO of MN Signal Processing.

Spatialisation can be understood in terms of different processes:

– The process of modelling the acoustic environment is about recreating the echoes and reverberations to get a sense of the spaces in the game. The ratio of
direct-to-reverberated sound, the delays and directions of reflections and the absorption and scattering of different frequencies also help us to pinpoint the position and distance of sounds and obstacles.

Modelling may either be coarse using delays, reverb and mixer snapshots defined by the sound designer in the tool for different areas of a game level, or may be precisely calculated from real game geometry with the use of plug-ins.

– Technologies focused on accurately producing sensations of height, depth and direction exploit knowledge of human sound perception (psychoacoustics) and are delivered through plug-ins for playback on existing headphones, speakers or specialised hardware. Examples include binaural spatialisation, virtual surround over headphones and speaker arrays.

– Cinematic techniques can be used to enhance or emphasise spatial cues. This can be achieved with some creative use of reverberation, filtering and delays in the audio tool.

Solutions available today, such as those provided by GenAudio, IOSONO, DTS and Dolby, for example, address varying combinations of these aspects.

FMOD STUDIO AND SPATIALISATION

FMOD Studio is the first Digital Audio Workstation (DAW) for games, providing advanced spatialisation features through a familiar multi-track and mixing desk interface. The configurable in-built 3D panner module models sound size and performs smooth, multi-channel panning.

Pre- and post-panner sends on tracks can send to one or more simultaneous reverb return buses while mixer snapshots and automation are able to modulate send and return levels based on 3D position. The ability to automate any effect parameter also enables advanced
distance-based filtering.

FMOD Studio’s open plug-in architecture provides the developers of 3D audio technology unprecedented access to the internals of the Fmod audio engine, making these technologies available to game developers. One example of this is swapping out or adding to the built-in 3D panner with third-party or in-house plug-ins.

The FMOD Studio platform can also provide support for plug-ins which rely on dedicated hardware or software by detecting the presence of the technology on the user’s machine and enabling it in the game.

A BRIGHT FUTURE

“We are excited at the possibilities that modern systems like Fmod Studio create for 3D audio content authoring and for interactive media consumption,” says Veronique Larcher, director of research at Sennheiser North America.

FMOD Studio features a powerful built-in spatialiser and supports a diverse range of other software and hardware technologies through its open plug-in architecture. Built on a mature cross-platform audio engine, developing in FMOD Studio ensures these technologies are easily accessible to creators and supported on all game platforms.

www.fmod.org

To see other articles in our Audio Special series, visit our archive