Nevermind the Budgets

It’s a well-trodden truism that in life you have five senses and in games you have just two.

While that may flaunt the purpose of audio, it doesn’t address why an entire organoleptic channel is usually given just a fraction of a project’s budget.It seems cruel, but high-quality audio is not a vital cog in the wheels of the game development industry. Sounds won’t sell a game the way sights can, and weak audio won’t butcher a game’s reviews the way, for instance, poor AI will.Audio will simply – if ‘simply’ is the fair term – enrich and enhance an experience. It will add context, atmosphere, pace and presence. Visuals may explain how far the fall is from the edge of a cliff, but threatening lashes of whistle and wind will show how close you are to death.

At Tiga’s Games Meet Film event at Pinewood Studios, Develop sat down with two audio executives – Ubisoft Reflection’s Jon Vincent and SCEE’s Dan Bardino – to discuss the state of the second sense.

It’s widely said that there’s a Holy Grail in graphics design, and that there’s also a Holy Grail for AI, and also interface…
VINCENT: You can’t think of a Holy Grail for audio?

No.
VINCENT: I think there is one. When we can convey the emotional states that linear media such as films can, with audio, then we’d have reached a huge breakthrough.

There’s points in Half-Life 2 – incredible game – where you finish shooting up a group of people and, just then, the music kicks in. I mean, that’s awesome; the last guy literally dropped to the floor and this score kicks in.

It’s things like that which I think is the holy grail of audio design. Moments that make you take a deep breath. But of course that’s all holistic for game design – you need the gameplay and visuals to work in order for everything else to fit in.

BARDINO: I’d actually hope there isn’t a holy grail for sound design. I think it’d be very dull if there was. One of the most exciting things about games, certainly one of the reasons why I wanted to get into them, was that it has such huge potential for sound.

Typically, film directors are really fixated on picture, and they get very nervous when audio engineers start moving sounds into the surrounds – they just don’t want the principal sounds moving from the front of the screen. Videogames just don’t have those kinds of issues. There’s no legacy hang-up.

Games are not tied to the screen with audio, which means we can be so much more expressive with how sounds move around images and scenes, and we can create really immersive environments.

That’s of course if there’s a budget to match those ambition, which many in the past have said there isn’t.
BARDINO: I dunno, I think it’s… well, maybe I’m just lucky at Sony because we don’t have those kind of restraints. I got the recording of Killzone 2 at Abbey Road, I got the recording of Heavy Rain at Abbey Road . I’m just very lucky that the people I work with have a good understanding for the value of audio. And doing high-quality professional production can bring such quality to games.

You must have heard others complain about audio budgets.
BARDINO: Yeah I feel very sorry for them, I feel very lucky.

Do you think they have a point though?
BARDINO: [Joking tone] Audio people whinge quite a lot! I mean, you’ve heard them… [laughs] No I’m only kidding.

In seriousness, some movie directors are audio-focused in their films and you get audio crew in early and you always see the benefit the film gets from that. Some game developers see that benefit as well and understand what audio can bring, just as much as some developers don’t, and therefore they might not give it as much attention or – ultimately – money.

VINCENT: I don’t see the problem for myself, either. I may have been really lucky, because the studios I’ve worked with in the past have given me a great deal of budget and freedom to use it. Some of the people I’ve worked with have been fantastic. Maybe it’s the case that those who need more budget aren’t shouting loudly enough about it.

Audio doesn’t sell a unit, it sells franchises. People play through games and, if the audio is good enough to lift the experience, people are going to be interested when the sequel comes out.

At the Games Meet Film panel discussion there were suggestions that game design needs a post-production phase.
BARDINO: Well, ideally we would have an extra month to work on things. But, obviously with the way games are made, I’m not sure it’s something we can do anything about.

We just have different phases of production to films. Films have pre-production, production and post-production. Production is the shoot, and post-production the visual effects, the additional dialogue recording, and most of the sound design and music.

But in games, we have a pre-production phase [alpha], a beta phase and a master phase. Now what audio teams try to use is that master phase, because it’s very difficult to do sound in alpha and beta when everything’s still changing and iterating. So, by the end of beta everything should be fairly locked in – there’ll be very little, if any, significant changes to be made – so we as audio teams try and do is move that audio post-production phase into that mastering period.

Now that’s a very tight squeeze, publishers want projects mastered as quickly as possible, because you need to get games out to retailers and distribution chains.

What audio teams tend to do is do as much post-production work all the way through the game. We’ll be adding content, but as we do, we try and do a bit of a mix to it, so we try and do that post production as we’re doing it.

That doesn’t sound very efficient.
It’s very much to do with the necessary way a game needs to iterate. Films have it different. If you shoot footage of this auditorium we’re in – y’know, the auditorium is here. It’s built. It’s done. You might want to add lighting and sound effects in post-production, but the foundations are here.

With a game, this all has to be modeled. The seats we’re sitting in have to be modeled. The ceiling does. Everything has to be built and everything will mean that the AI has to be tweaked so that NPCs can move through these rows of seats. That’s the nature of the beast.

Of course, there are things you can prevent. There are bad practices. Getting towards the end of a game and changing something fundamental, or making sweeping edits to a cut-scene, for example, and having no respect for the impact that will have on audio, is obviously a bad thing to do.

So there are things you can do, and there are ways you can improve the efficiency of the design process. We’re obviously still a young industry, and we’re constantly looking at ways in which we can make game development more efficient, but that’s not just in audio, that’s across the board.

When you go to the Develop Conference, for instance, you’ll hear all the developers saying about how we can improve our processes and the way we build games.

Moving on to a different topic; ‘linear’ has become a bit of a dirty word in videogames; there’s a general expectancy that games offer an increasing number of options and freedom to explore. What kind of dilemma does this present to an audio team, and how is the issue tackled?
VINCENT: It can be a huge challenge. In those kind of sand box games, you either have to have everything readily available, or you have to have everything loaded – but of course you have a finite amount of memory, and usually if you have everything loaded you’ll need a better pooling system or loads of low-quality samples.

In Saints Row 2, for example, a lot of content was put in there and a lot of it was loaded. So, the game’s engine sounds and so on weren’t really that great – but that was just their focus, they wanted to offer an enormous amount of sounds. The gameplay, of course, is awesome.

GTA IV as well put a lot of content in there, but that game used a lot of spooling, so there’s a lot of good tech available to get the sound to the player just when it’s needed.

Obviously, with more linear games you can better pre-empt what the player is going to do. But with open world games, the design is more about regions and anticipating what the player is going to do from any one position.

BARDINO: Less linearity just means that you have to think more laterally about how you build more tension. So, say you have a scenario where a player is walking up the stairs towards a haunted room and you really want the player to go into that room, there will obviously be a number of design considerations that will lead them up those stairs.

In a film, it’s very simple to build that tension, because you can follow the plot in detail. That’s obviously not so easy for a game where a player is free to do what they choose. And you don’t want to lay false traps for people; you don’t want to force them anywhere or make it too obvious where they’re going, so it’s about finding more subtle ways to build tension.

One thing we can take advantage of is the game medium is a much longer form. For an actor walking up a flight of stairs that takes 15 seconds to reach the top, in a film that might take five seconds: first step, then cut to hand on banister, cut to foot at the top of the staircase, cut to the actor reaching for the door handle.

We have to look at the strengths of our own medium; we can’t really put those cuts in there – the player will actually need to spend fifteen seconds walking up the stairs. But that means we can be more subtle about how we inject tension via audio, to the point where the player only subconsciously notices it, because you can shift the mood more gently. These kind of methods are key for how we take advantage of the interactive medium.

The actions of the player can dramatically affect the tone of the game, so is it too ambitious to think that audio design can be used to reflect almost anything a player decides to do?
VINCENT: Yes sound design can totally adapt and reflect what the player is doing; but it all depends on the focus of the game. If you were to, for example, work on a game where the player can choose between being good or evil, as an audio director you would want to put assets into the game to reflect both sides.

But sometimes it simply doesn’t matter for the music to reflect the mood of what’s happening on screen. If a player would suddenly begin to hack and slash at a busy happy town market, those actions wouldn’t be seen as the fundamental part of the game anyway, so it’s not important that a specific score or sound asset is in place just to replicate that action.

We could put in assets to reflect all these possible on-screen actions and moods, but in doing so you could be detracting assets from a really important part of the game.

We have a finite amount of assets to put in, and we always want to keep load times down. So the most important issue for audio designers is to focus on what’s most important to the project. You have to pick your battles.

On the other side of the coin, does a linear game provide opportunities for the audio to flourish? Rez, a game which is constantly referred to in debates on audio design, was – essentially – an on-rails shooter.
BARDINO: Well I also think that Rez was so fantastic because it was a game built around music and sound, that was its focus. But also, it should be pointed out that, quite simply, the music in Rez was so powerful.

From a sound design point of view, games don’t need to provide us with a big open world. My proudest work on a game was the music I did for the Singstar franchise. I’ve written music for TV and short films, but the sound design – the audio branding – that I helped create for the original Singstar has been heard by millions of people all over the world. I know it won’t win awards, but I’m very, very proud of it.

What we wanted to do with that game is make an audio brand, and though I haven’t worked on the Singstar franchise for a while, even the latest games feature those same signature sounds. From the beginning we wanted it so that when someone pops in a Singstar disc, all the audio will come from the same palette of sounds.

Finally, how has audio design in games progressed over the years?
BARDINO: There were real problems before, but I think game audio has taken quite giant strides over the last few years.

Of the 2 meg of RAM we had for sound in the PS2, for The Getaway we used that to recreate the sounds of the whole of London; all the car effects, all the gun sounds, some of the dialogue, music. That was a really, really proud moment. We came away from that delighted with what we had achieved. On a standard audio CD, 2 meg accounts for twelve seconds…

With the PS3, we’re allowed to use up to… well, we’ve been benchmarking at 25 meg. I mean, So I think the PS3 has provided a real evolutionary leap in sound design, we are now able to do things we were just dreaming of on previous platforms. It’s just crazy to think about where we’re going next.

Of course, the PS3 is 7.1-native, so we’re doing interactive 5.1 music on a lot of our games. Killzone 2 and Wipeout HD were both 5.1 and interactive, there’s nothing else out there that’s doing that. That’s pushing music further than anything else.