“My FQA testers can finish this game in 18 hours, so three days should be enough for a thorough localisation QA pass, right?” Testronic’s Head of LQA Edd Buffery discusses the challenges facing localisation specialists…
After working with hundreds of clients over 15 years supplying QA to the games industry, I can’t entirely discount the possibility that the above might, occasionally, bear some vague approximation to the truth…
Perhaps the game is designed such that a single, critical path playthrough is all that’s needed to trigger every in-game string, and there are no additional modes or optional content. We’d have to test on only one platform and of course that couldn’t be a console, since checking that the localised system messages and terminology required to pass each platform holder’s stringent submission requirements takes several hours on top of everything else. The client will need to supply solutions to any puzzles, and debug options to help get through difficult or repetitive areas.
We’d have to assume there won’t be a huge number of corrections to be made along the way, whether random linguistic mistakes by the translation team, contextual errors resulting from the translators not understanding where or how strings will be used, or a host of language-specific implementation errors, from unsuitable font choices to badly out of synch cut-scene subtitles that require careful, line by line timing adjustments. The build we test would also have to be free from major functionality errors that cause testers to lose progress or force time-consuming workarounds.
I remember talking to a client years ago who requested an estimate for a thorough LQA pass on an adventure game that had somewhere in the region of 70,000 to 80,000 words. They were adamant that we should aim to trigger 100 per cent of the strings on-screen, but had based their internal expectations on the fact that one of their testers could reach the end of the game in 10 hours by following a walkthrough – so were hoping that LQA might take just a little longer than that.
As anyone who likes adventure games knows, the vast majority of text is not seen by making progress, but by trying the ‘wrong’ conversation options, in humorous results when attempting to use inappropriate items on the environment or each other, and in the optional descriptions that appear when examining or interacting with every single red herring. In other words, everything but the steps in the walkthrough.
Ultimately, the client started to appreciate how hugely they had underestimated the time needed when we pointed out that it would take a professional proof-reader 10 days or more to review that volume of text if they were working purely with the text files and not having to play anything, so how could it take a tester a tenth of the time to review exactly the same text, while also having to find and trigger each string before they check it.
Here at Testronic, and as for any service provider, there has to be a certain ‘the customer is always right’ aspect to any discussion on how long we think we should test someone’s product for. Forging good partnerships requires us to remain flexible to each project’s unique requirements and a willingness to adjust strategy and find compromises that respect any constraints such as submission deadlines or budget limitations, while setting coverage expectations realistically.
For the earlier example of the adventure game, the client literally didn’t have the money to pay for what they asked for, so after gently guiding them towards some more realistic ways they could spend their limited budget, we ended up on a compromise whereby we agreed to check only certain strings on-screen (front end, in-game menus and UI, critical path playthrough, and compliance related strings), proof-read other commonly seen strings in the text files afterwards, using our experience thus far to flag consistency problems and shortening strings that we now knew would overrun their text boxes, with brief spot checks only of the rest.
Working with as many clients as we do, there’s a vast range of experience amongst the contacts that we plan projects with, from QA veterans that come to us with up front schedules, instructions, and realistic coverage expectations, to new indie developers that have no prior experience with localisation whatsoever. Perhaps their first mobile app has had some success in English and now they are looking at localising for additional territories, and are asking us what the difference is between Traditional and Simplified Chinese (Short answer: Simplified is for mainland China). For them we might provide a range of different scenarios representing different combinations of language and coverage expectations, while they try to figure out what their budgets are going to look like and how optimistic they feel.
For some, LQA is one of the last aspects of a project to think about, and it certainly gets less media coverage than functionality QA (compared to a visually amusing FQA bug, you have to be a bit of a language nerd to appreciate a good text error!). Nevertheless, it’s a fascinating field to be working in, with plenty of its own challenges, and I’m proud to be working with so many great clients at one of the best LQA providers in the industry.