Building a pipeline from scratch

When you’re a new games studio, one of the many challenges your company faces is working out which are the right development platforms and tools to use. Develop Awards finalist 22Cans didn’t just decide to stick with the tools the team had used in their previous roles.

Instead, as it was starting with a clean slate, it made a strategic decision to evaluate and test the latest tools available. The end result is a mix, from the well-established (including C++ and Perforce) through to new kids on the block like Marmalade. Gary Leach, lead programmer at 22Cans, shares his insight with Develop.

Our selection process for tools breaks down into three main categories: the core engine, additional elements not included in that engine (such as FX and audio) and supporting elements, like version management, continuous integration and bug tracking. Essential criteria include the need to support our short development cycle, a highly iterative design process, and tools that are light in terms of the amount of management and intervention needed.

AT THE CORE

Agreeing on our core engine was our first step and the decision came down to the fact that Godus is highly procedural. It is based on a landscape that is designed with sophisticated mathematics behind it to allow changes to be made on the fly. So, we needed an engine that could support that. Keeping C++ ‘to the metal’ coding practices is very important to us, we wanted to be able to use C++ natively. C sharp and other managed languages just aren’t fast enough for these sorts of tasks, and even writing C++ plug-ins for C sharp often bottle-necks on things like the mesh interface.

Our highly iterative design process is all about light descriptions, followed by fast, data-driven changes. On top of that, we need to support multiple platforms, devices and specifications, from low-end phone all the way up to high-end desktop machines. We want to have just one codebase for each game that can be tweaked and tuned for each target specification.

Our shortlist was the latest version of Unity (we had used Unity for our first game, Curiosity, but since then a new version had been introduced), OGRE, Unreal, Marmalade and creating our own.

After some consideration we culled Unity and Unreal. Unity has a lot of great features, but it was not right for us because it doesn’t give us the hands-on control we want and we really need low-level mesh access. Unreal is superb as an entry-level system, but for C++ you need to have the full version and we decided that the additional control it gave for the additional cost took it out of the running.

Both OGRE and creating our own engine brought their own risks. OGRE is entirely open-source, which we don’t have a problem with for other tools, but as our core engine, we would be supporting ourselves if any problems occurred. The same would apply to creating our own engine.

So Marmalade was the front-runner, because it covered all the bases we needed – native C++, customer support, high level of control – but to be sure, we put it through its paces before we made the final decision and we liked what we found.

We implemented a system of configuration files to enable us to change any variable at run-time, by exposing that change to the config system (basically CSV spread-sheets). The design team can experiment with changes – such as different colours and values, using colour pickers and sliders, and see how those will look in the game in real-time.

The time-savings may seem minimal, but a few seconds shaved here and there add up over the course of a week and a whole design team. Another big gain is the continuity of concentration: nothing breaks creative flow more than waiting to see the results of a change. This applies to everything from basic game control to shader code.

FILLING THE GAPS

At the same time, we began to look at the additional elements, such as a particle system, a strong audio system and user interface. We started looking around for easy-to-use solutions that we could start using pretty quickly. The particle system became a priority when we brought a new VFX designer on board and he naturally wanted to know what he would be using. We came across Magic Particles, which plugs straight into Marmalade.

This provides a great particle system base for our VFX. We layered sequence control and other VFX elements on top of this using another config file based in-house creation. If our VFX designer had been less technical, we might have chosen something more visual, but he is happy having hands-on control and visibility: he can see exactly what is going on. Technical support from Magic Particles is excellent too.

The next step was to find the right audio tool and we decided on Wwise, because it has great features and the company was just so responsive to our questions. Integration with Marmalade has been time-consuming, but it has been worth it because it has given us a technology-agnostic library that we can use across any platform. The ability to use sliders to hear different sounds and implement those straight into the game has been a big advantage of Wwise.

As far as the user interface is concerned, this is something we are still working on. We built our own UI system for the prototypes and we are still using that, but we are currently evaluating the pros and cons of Scaleform and the additional features and benefits it could provide. Big considerations for us here are evaluating flexibility versus performance benefits, plus ease of integration with the rest of our environment.

SUPPORTING ROLES

The third and final category is the tools that provide the supporting infrastructure, including version management, continuous integration and bug tracking. We first focused on DVCS and used Git for some preliminary testing, but this was no good for artists, because the command lines are somewhat obscure.

We then tried Mercurial hosted on the cloud using a service called Kiln, but there were two issues with this: first, we didn’t get 100 per cent availability and second, it was very bandwidth-heavy, so it was slow at transferring gigabytes of source art assets.

The other issue with these DVCS systems is that everyone gets everything: the whole repository and all the history, which leads to extreme file system bloat, especially for binary assets. There are ways round this but that is a complicated fix. Also, without the concept of a centralised server, there’s no way to exclusively lock data, which is necessary for files that cannot be merged. There is no mechanism to prevent two people from working on the same file with the risk of one person’s changes accidentally overwriting the others.

Subversion’s interface is not particularly refined and some operations such as branching were not well visualised. So we decided to take another look at Perforce, which most of us had used in previous roles. We realised that there were some new features that we hadn’t seen before and which were ideal for 22Cans. We particularly liked the look of Streams and the ability to support intelligent branching. Also, it is a centralised versioning service and people can really get their heads around what that means: it’s a central server that can lock files, with no need to have repositories on everyone’s machines.

Perforce is a powerful mechanism that lets us customise every type of file and gives us absolute control over file coding and how much version history is stored. File loading is fast so we don’t waste time. The ability to be able to branch and control access to versions is a critical advantage of Perforce, particularly when we are releasing games out on new formats.

For continuous integration, Jenkins was the standout candidate and there didn’t seem much point looking around. The feedback and positive comments about Jenkins from the whole design community were very encouraging. There is a very active open-source community around Jenkins adding all the ‘bells and whistles’, so whenever we need something, it’s pretty likely someone has already created a plug-in.

For our bug-tracking, we used Fogbugz for Curiosity and it did the job, but we are switching over to JIRA, simply because it is a tool that most of us have used before and like. Not only is it integrated with Perforce, it is also easy to use and does exactly what we need.

So we’re pretty confident that we’ve got the right set of tools to help us complete Godus and get a great product to market. That’s not to say that we won’t continue to review our tools in the future: with each new game, we have to look at what is the best fit. But for the time being, we’ve got an infrastructure that supports our fast, iterative design process, regardless of how many platforms and devices we support.

TOP TIPS

Gary’s advice for other games studios starting out on the tools selection process

• Be careful about the extent of your project and your aspirations – think ahead: how much will you still have to write yourself? How much hands-on control do you need? How well can you manage your own codebase?
• Support – if you don’t have time to do everything yourself, then a software vendor with a responsive customer service team is essential, or at least, a user community that is very switched on.
• Don’t assume you are always going to use the same engine and other tools – while there is cost and effort involved in changing, there is no point hamstringing yourself by doggedly sticking to something that is no longer giving you what you need. Games studios need to be agile to survive.
• It’s best to try to fit tools around people, not vice versa – otherwise you will reduce people’s productivity. Of course, that’s more difficult when lots of people are involved, but sometimes it is possible to find bridges that can adapt the tool to fit the way the user works.