Small Faces

THERE’S NO EYE IN TEAM

With the other side of the Uncanny Valley clearly occupying the minds of even the most pessimistic observers, there’s an encouraging trend that is contrary to what might be expected of a discipline defined by competing technologies. Collaboration and cooperation are buzzwords in the sector, and almost every high profile firm in the field has warm things to say about its rivals, and some kind of deal with a ‘competitor’.
Xsens has partnered Image Metrics, the latter of which is hugely enthusiastic about FaceFX’s work. Elsewhere RealtimeUK has worked closely with Audiomotion to establish a flexible production pipeline to develop an internal piece called Samurai.
“It’s a small group of companies, and we still haven’t solved 100 per cent of the problems in facial animation,” admits Image Metrics CEO Michael Starkenburg. “So until somebody comes along and does solve everything we need to work together a little. To be honest with you I think that’s the same in most of the game production fields. We’re all trying to solve the same problems for the same people.
“Furthermore, it’s not a space where winner takes all. It’s not like one guy wins and the other guy looses. In fact we’re often working on the very same projects, so the more that we cooperate together, the better it will be for our clients.”
Like many of its contemporaries, Image Metrics is continually building partnerships with companies that might be seen as rivals, and additionally, it is extending its collaborations to other areas of game development. Its deal recently with localisation experts Babel provides a case in point. In fact, the entire facial animation and motion capture ecosystem is evolving through those kind of alliances.
It’s also apparent that the developers making use of facial animation services are set to gain from the sector’s willingness to collaborate, as FaceFX’s co-founder and CEO Doug Perkowski explains: “Lots of our clients are using multiple facial animation techniques on the same project.
“Audio-based technology forms the base-line approach, and important animations are hand-tweaked or authored with performance capture or mocap technologies. In many cases, our competitor’s data is loaded back into FaceFX to play in-game. So we are starting to see the various facial animation technologies as something complementary rather than competitive.”
“Using good specialists in whatever approach decided on is very important – we will always try and work with the best expertise we can from that point of view whether that means our own in house capabilities or working with external specialist contractors,” adds Richard Scott, managing director of Axis Animation.

BRIDGING THE GAP

Regardless of who you may choose to work with, and what extra help you can pull in at home or abroad, the famed Uncanny Valley conceived by roboticist Masahiro Mori in 1970 still presents a pertinent point which facial animators must cross. Whether we’re past that theoretical trough is a point of fierce debate.
“We’re absolutely there if you want to spend enough money,” proposes Starkenburg. “Look at a film like Benjamin Button, and there’s no question that it’s past it. There’s not a second in that movie where the illusion drops away.
“We’re at the point where with a certain amount of staff and a certain amount to spend you can tell a story that is completely unencumbered by technology. What we need to do now is democratise crossing the Uncanny Valley. We’re working with all kinds of people who are trying to do that kind of thing in commercials and games. We’re never going to have ‘Cameron-sized’ budgets, which is why we’re focused on a value proposition that will see quality rise and costs fall.”
If the Image Metrics CEO’s diagnosis is correct, the next big challenge for facial animation tech providers in the games industry is making crossing the Uncanny Valley affordable. However, not everyone shares Starkenburg’s optimism.
“It only really starts becoming a big issue once you are pushing render fidelity to a point where things are believably real and I think game technology still has a few years to go before it gets to that point,” opines Scott. “As such I think that Uncanny Valley issues will get more pronounced rather than less as the rendering technology gets much better.”
If that is the case then gamers may have to endure the odd rubbery smile and vacant glare for a few years more. With so much of the human brain dedicated to reading and processing the face even the slightest discrepancy will always be noticed. One solution, that James Cameron clearly made use of, was avoiding a distinctly human form.
“It seems to me that placing as near as we currently can to human face motion onto a face character face that isn’t trying to be photorealistic can produce some very pleasing results,” says Norman. “We certainly find ourselves operating in this area at Rocksteady, where the Batman world characters, in the forthcoming sequel to Batman: Arkham Asylum, are human but beautifully stylised.”
Creative thinking is all well and good, but a monster lurks in the Uncanny Valley that is just as powerful as that of financial budget; namely memory limitation. Even with a coffer as generous as Cameron’s, game developers will always be a slave to data capacity, as Klepper highlights.
“At the moment, the main limitation holding back games is the overall file size. Realism lies in subtle details, and the added layer of animation data and model boosts – for example wrinkle maps – needed to achieve this spark of life is also data expensive. As methods of compression evolve and console games are no longer limited in file size with installation to hard drive and multiple DVDs, we’ll see games getting closer and closer to true realism.”
“I don’t believe anyone in the games industry – using in-game engines – has truly beaten the Uncanny Valley yet,” adds Jones. “All the most emotive and sympathetic characters out there still tend to have that tiny degree of stylisation. They all sit on that narrow edge just before the terrible drop into that unforgiving valley.”
“I would say we are just approaching the Uncanny Valley rather than emerging from it,” agrees Perkowski. “In any case, while it’s important to create demonstrations that push the boundaries of technology, the more important question is ‘how real can we make an animation when we don’t have unlimited time and resources devoted to it?’. When we have a good answer to that question we can increase realism across the entire spectrum of video games.”
Even in a theoretical place where games makers have the technological and financial capability to straddle the Uncanny Valley, there’s another barrier to cross that presents perhaps more of a obstacle. 

STAGE FRIGHT

No matter how realistically lips twitch and cheeks stretch, companies are increasingly expected to deliver performances that are believable. With so many movement capture techniques there are real people under the markers and suits, and increasingly directing those actors is as much a skill as the working with the technology in place.
Side is a firm that specialises in performance as part of the service, and has cast and directed actors on numerous projects using a number of different facial capture methods.
“All have their own advantages and disadvantages,” declares Side’s managing director Andy Emery. “Marker-based capture works extremely well when a lot of movement is required for full performance capture, whereas video based capture works well in a controlled environment and with large volumes of lines. The most important thing with any shoot is to ensure you cast the right actor, they know exactly what to expect in the session and they are well directed.
“It’s very easy to create a facial shoot pipeline that is ‘technically’ great but inadvertently create one that is a barrier to a great performance.”
Emery isn’t alone in his opinions, and even the most sizeable facial animation specialists are aware that there’s plenty to learn from our friends in Hollywood.
“Beyond just the technology, the industry has so much to learn about how to cast and write, which is so much part of the future of good facial animation,” suggests Image Metrics’ Starkenburg.

MOVING FORWARD

As if sitting in the canvas of a director’s chair doesn’t offer enough extra work, facial animators and mocappers also have another issue to wrangle with. In short, there are two factors to realism; appearance and movement. Striking a balance between the two, and choosing which warrants more weight, is not as easy as it sounds, and when combined with other factors like virtual cinematography, the task in hand becomes an immense one.
“A lot of the trick to selling animation and mocap as reality is actually in the context in which it is shown,” insists RealtimeUK’s Jones. “Many of our most convincing pieces use realistic environments, lighting and true to life camera work. We build a situation in which the animation is strongly supported by the visual direction of the movie.”

EXPRESS YOURSELF

An extended tour of the world’s leading facial animation and mocap studios unveils a sector wrestling with a number of dichotomies, as alternative theories, creative approaches and technologies compliment and contradict one another continually.
Those employed in the field must learn to juggle the minefield of issues at the same time as picking up a new skill; storytelling.
“A major challenge for the sector is that people are still getting away with bad work,” concludes Starkenburg. “People are still saying ‘I’ll do facial animation so I can sell more games’. That’s a very short-sighted approach. In the long run, we have to solve this problem.
“Even if a game sells 100,000 more because it has great facial animation, that’s still short-term thinking. In the long term the expectations of consumers are going up, and we’re going to need to be able to tell stories at a level that the consumer wants to see. That’s the real challenge for the industry.”
Mastering the art of storytelling and directing is no mean feat when facial animators and mocappers are also charged with pushing technolgical advances so close to the cutting edge that they must trick the human eye and overcome a theoretical problem that has evaded scientists for a full four-decades.
And yet still, the sector is awash with innovation, and the public is increasingly falling for its charms. Facial animators have every reason to express optimism. 

Part 1 of this feature can be found here.