Motion capture: Moving with the times

It was the mid-nineties when motion capture was first used in video games.

Since then, the technology has become as close to a household name as any games development jargon has, in no small part thanks to the animation technique’s parallel prevalence in cinema.

Back in the previous issue of Develop (November), we turned our attention to the world of character animation, the umbrella term under which motion capture falls. But motion capture deserves special attention in its own right.

In many ways it is the most elaborate games development method, where entire warehouses, hundreds of highly specialised cameras, dozens of actors and stuntmen and even large scale physical sets are required simply to animate scenes or characters.

And yet with the arrival of Kinect, a simple motion capture technology now exists in the living rooms of millions of consumers. There lies the potential for new game experiences that offer more natural interactions for users.

Somewhere between those two ends of the spectrum, indie and small-to-medium sized studios are now making motion capture an ordinary part of their production process, and elsewhere hardware outfits are creating all kinds of varied solutions, including markerless and cameraless offerings.

So where is motion capture going, and how can your studio be sure to get the most out of it?

A MOTION FOR CHANGE

“Although the ground rules of motion capture haven’t changed much since it was first introduced, we have been witnessing some considerable improvements in the past few years,” offers Guillaume de Fondaumière, co-CEO of Beyond: Two Souls studio Quantic Dream, by way of introduction.

“Systems have been pushed forward to become more accurate, offering in particular the possibility to track very small markers from a relatively large distance. Very subtle facial movements can now be captured with fewer cameras and with far less constraints than in the past.”

Certainly, there is still room for improvement in the mo-cap realm, but the technical limitations of motion capture are somewhat less restraining than the method’s impositions to performance. Motion capture must continue to evolve to be ever more user-friendly to directors and actors, but from a technology perspective, it has come on leaps and bounds in recent years.

Prices are coming down too, and despite the convoluted nature of adopting mo-cap’s physical nature to create a virtual product, it is becoming an affordable animation option.

“Beyond that, full performance capture [where body, face and audio are recorded from a single performance] is introducing new possibilities in storytelling,” adds Jay Granier, director of technical operations and hardware and software facial animation specialist Faceware.

“Gameplay will become more story focused and the ways we as players interact with characters in-game will only get better and more enjoyable.”

Optimistic stuff indeed, and all around, despite tough economic times and an industry undergoing enough seismic shifts to shake established practices to the ground, there is positivity in the mo-cap sector.

UK Total War developer The Creative Assembly, which recently built its own mo-cap space, is home to one of those optimists; namely motion capture manager Pete Clapperton.

“Relative to early systems, the technological advancements up to now have been astounding,” he says of mo-cap.

“I believe that high fidelity markerless technology will be the next big leap forward, allowing for greater versatility from both the performers and directors, and subsequently the animators.”

‘Markerless’ and ‘cameraless’ are certainly popular words in mo-cap today, as numerous companies work to build more accurate alternatives to the established ‘markers and camera’ set-up, that while proven, can inhibit actors’ movement and performance.

One such solution is Xsens’ MVN Motion Capture system, which consists of inertial sensors attached to the body by a lycra suit.

“Motion capture used to require motion capture operators and a dedicated office space with low lighting condition,” explains Xsens product manager Hein Beute.

“Xsens MVN allowed animators to work with motion capture as just being another tool, and to be able to go use it in any place with little training. The accessibility and ease of use are pushed forward together with the ability to capture full body, but also face and hands.”

But cameras are still commonplace in the world of mo-cap, and according to those producing them, they have the power not just to impact animation, but inspire game mechanics and fuel productivity.

That’s especially the case in the example of in-house motion capture facilities, where the quality and volume of animations games designers have at their disposal is immense.

“Suddenly, the animation budget becomes significantly less constraining as designers dream up complex narratives or highly immersive player control paradigms,” says Jim Richardson, CEO of motion capture and tracking hardware specialist OptiTrack.

“Ultimately, the execution rests on the talent of the dev team, but incorporating mo-cap into storyboarding and production processes can shift character animation from a pipeline pinch point into a tool for leveraging productivity and clever new designs.”

SPEAKING THE TRUTH

Of course, it would be unrealistic to dwell exclusively on the opportunity within mo-cap, without addressing the challenges that face animators. Indeed, embracing the future means tackling the trials of today.

From Quantic Dream’s perspective, motion capture is a means to capture the ‘truth’ of an actor’s performance, which means that effort must be spent on mastering the creative end of the motion capture scale.

“From a directing and actor performance perspective, a change of paradigm would be necessary to truly mark a departure and enable us to step into new territory,” offers Quantic Dream’s de Fondaumière.

“Markerless systems could for instance one day enable us to capture actors in costume – and not in silly suits – and/or within real sets.”

Another challenge is – understandably considering the make-up of mo-cap – a matter of education, and it’s something Mark Lewis, head of sales at motion capture company Animazoo, sees as vitally important for pushing things forward.

“One of the challenges we have is educating clients on the differences between consumer and professional standard mo-cap systems,” states Lewis, who’s employer provides motion capture suits that negate the need for cameras.

“Very low cost consumer systems can provide your mo-cap data, but some users forget to consider data processing and cleaning time into their calculations. Trying to work with bad data can take longer than key framing so managing expectations is key.”

But, says Lewis, the pressure on motion capture to improve across the board is driving the discipline forward for the better.

“The economic pressures on studios mean they now have to deliver more with less,” he insists.

“This is driving us to constantly innovate and push our technology to its limits to provide better quality data at lower prices. Gamers want more realism so consumer pressure will drive studios towards tools that allow them to deliver this.”

The real challenge according to Mick Morris, MD of mo-cap service provider AudioMotion, however, is a matter staring the industry in the face. The human face is, of course, the most expressive part of our body, and at the very centre of almost any performance. Captured incorrectly, it can let an entire game down, and it is still tough for mo-cap solutions to grasp the essence of.

“One of the key challenges still involves facial capture,” explains Morris. “We can record the performance using headcams, without headcams, markers, markerless or a hybrid approach. We will typically shoot on a sound stage with a full crew to ensure the best possible results with final audio.

"However, if a face hasn’t been rigged properly or given the love and attention that is lavished on other aspects of character work then all of these efforts will be negated when it comes to the final result.”

MOVING ON UP

Despite those challenges, optimism prospers, and in one area in particular.

Today, motion capture is in reach of not just middle-sized studios, but true indies and microstudios. It’s partly down to economies of scale, which has democratised the availability of hardware, and it’s undeniably a point of much industry enthusiasm.

“Good quality mo-cap is certainly within reach of almost everyone these days thanks to easily obtainable, high quality consumer cameras like the Kinect and the abundance of software being released for it,” confirms Faceware’s Granier, before adding the qualifier: “Larger scale projects utilising full performance capture and multiple performers tend to require large volume stages and higher quality solutions due to the scope of work being done and quality of data required.”

But the mo-cap opportunity for small studios is real, and Granier is not alone in highlighting it.

“Mo-cap hasn’t hit a point of ubiquity just yet, but OptiTrack alone has enabled well over a thousand new studios across our markets to create stunning content using motion capture,” says Richardson, who’s company today serves both triple-A giants and diminutive teams.

“And it’s not just the adoption rate that we’re excited about,” he adds. “Indie studios and start-ups now have access to $10,000-to-$20,000 systems that can outperform $100,000-to-$200,000 systems sold only ten years ago.”

It’s easy to get carried away, of course, and see indies as the new champions of mo-cap, but pragmatism is still needed, as it remains the case that the animation technique is complex and intricate.

“Motion capture is now available to all but quality remains the issue,” suggests Animazoo’s Lewis, who, while positive, sees that the practice has some way to go before it is truly democratised.

“There will be a day where motion capture is the default process rather than something reserved for the biggest studios. The trickle down of technology improvements into lower cost systems means the quality of data delivered at a budget is constantly improving.”

CLEAR THE HOLODECKS

With so many methods, technologies and perspectives, motion capture is a bewildering speciality, but one that many see as set to benefit from collaboration, as Faceware’s Granier details.

“Forming partnerships is becoming more and more important for tool providers as well as service providers in today’s industry,” he says.

“Clients are often looking for a turnkey solution that services all of their needs and ultimately that puts us as a company in a better spot to give them what they’re looking for.”

“We are currently collaborating with at least half a dozen different companies who are specialists in face rigging, coding, middleware and other disciplines,” adds Morris, who argues that it is important that companies like his recognise core strengths.

“A fortress mentality is not appropriate in the creative industries. Our clients, quite rightly, expect solutions, not excuses.”

As for motion capture’s future, it absolutely looks in rude health, and it may be that Kinect has a lot to do with that. It has democratised both the hardware and understanding of motion-capture, and turned it from the reserve of the elite to a fairly normal living room function.

“Low cost controllers that use motion capture technology are already making their way into peoples living rooms, so expect motion capture to become more widely understood and accepted by the public,” says Lewis, before going on to titillate Trekkies across the globe.

“Games that fully utilise the potential of movement sensing technology will take player engagement to new levels. The holodeck may not be that distant.”

The last word, though, goes to de Fondaumière, who is already looking to the next generation of games consoles in his quest to capture that true essence of an actor’s performance.

“More memory and rendering capabilities will lift some of the constraints we face in real-time,” he says. “The next generation will certainly bring us much closer to what can be seen in the most sophisticated films or rendered CG productions.”