AUDIO SPECIAL: Santa Monica's Steve Johnson looks at audio's relationship with narrative
Our industry loves Hollywood. We’re always borrowing their writers, directors, and actors.
Why? If I had to pick a single reason other than Star Power I’d say the quality storytelling. While excellent movies start with excellent stories, games often originate from good gameplay instead.
But everyone has felt the impact of a good story; the connection with the characters, caring about what happens to them, getting wrapped up in their world. So developers go to great lengths to add story, and on the list go great music, dialogue, and sound effects. In the end the music might be epic and tied to an algorithmically-derived threat level, the dialogue may be complete and star-studded, and everything makes an awesome sound.
Added together though it still might not be quite the experience you’d get in a good movie mix. Why? Probably the dialogue.
Dealbreaker. But even beyond dialogue I think there are things worth mentioning that, if kept in mind, can help us become better storytellers through our audio.
ONCE UPON A TIME
A good place to start; being truly aware of the story to begin with. Thinking about the natural rhythm in the narrative, and enhancing it with deliberate pacing and strong movement between different emotional states as the characters, settings, and circumstances change. Consciously picking the player up and setting them down somewhere else.
Easier said than done in games, because we’re colouring in a living piece of software, imperfect and lumbering, like Frankenstein. It’s not laid out for us as it would be in a linear medium; a colouring book waiting for crayons. It takes selight-of-hand and trickery under the hood, and a lot of help from programmers and designers.
What’s required then is tight collaboration between sound and music, engineering, art, and design. Working together to build the house, rather than being called in to decorate it. An environment like that fosters creative input, and is more prepared for the coordinated, dynamic movement in a game that can bring a good story to life.
There is likely more discussion about pacing, tone, perspective, mix, characters, conflict, resolution, etcetera. Crosstalk leads to more ideas, and you might just find sound inspiring art or new mechanics.
A good way of killing that approach cold is becoming fixated on a requested list of sounds and music that someone inevitably creates (including ourselves). A good starting point for sure, but the main focus can shift to addressing the checklist. Reacting to requests starts to take priority over proactive storytelling. The bigger picture fades; the forest becomes a lot of individual trees.
One ‘big picture’ idea that seems among the first to go is the idea that literal, one-to-one pairing of sight and sound is not always the best solution. Everything actually doesn’t need to make a sound, or sit evenly in the mix, or spike musically and predictably with each new wave of enemies.
It never fails that just when I’m thinking game sound has closed the quality and realism gap for good, I see a movie that blows me away, and am reminded that subjective, interpretive audio can be more provocative, and a lot cooler. How ironic that movies often do a better job at getting into the head of protagonists, while in video games you get to be the protagonist?
Conveying a story in that manner means being keenly aware of the emotional impact of sounds and instruments, and not being afraid to let go of the reality lifeline. It means thinking of what needs to be said about the character’s experience and what is significant to the moment.
Working with ThatGameCompany on Flower and their next game Journey has made me a better sound designer, because they’ve trained me to be constantly assessing emotional significance.
‘What is this trying to make me feel? What does it make me feel?’ So simple a concept that it’s easy to skate over, but it’s become my secret weapon for extracting all kinds of interesting things from games, film, literature, art, and the rest. It explains what we find lacking too – why we get bored, fatigued or frustrated by something.
With Flower, creative director Jenova Chen wanted each level to portray a particular set of emotions and the whole game to present a clear three-act structure, all without characters, text, or dialogue.
It forced composer Vincent Diamante and I to weigh the emotional worth of every sound and every note, to figure out how they fit into the greater arc, and to work closely with the rest of the team to make it happen.
The more we craft memorable moments and impressions into our work, the more they will resonate with the player. The more we can string those together into something dynamic and meaningful, the less we’ll need Hollywood to bail us out.