One of the greatest challenges of a localization project manager is managing quality for a wide range of languages and subject matters. It is impossible for one person to be fluent in hundreds of languages and domains. The importance of an objective translation evaluation rubric is paramount for the success of a multilingual localization agency.
As a project manager for MTM LinguaSoft, I was tasked with improving and standardizing our own translation evaluation process and I began researching QA (Quality Assurance) models that have been accepted by the localization industry for evaluating translations into languages in which the project manager is not fluent. I consulted resources on the LISA (Localization Industry Standards Association) QA Model, the SAE J2450 QA Model, and the TMS Classic QA Model, all of which are highly touted evaluation methods in the translation community. Each method is unique, and although none specifically suited our needs, research showed that all of the models had two main objectives: categorize errors and assign them severity; or in other words, qualify and quantify.
Using this research as an outline, we at MTM LinguaSoft defined our own standards. The MTM LinguaSoft QA Model for Linguistic Quality Assurance qualifies each translation error by type (syntax, misspelling, omission, etc.) and quantifies each error by assigning it a score depending upon whether the error is minor, major, or critical.
- A critical error is one that is dangerous, misleading, or highly visible. Examples might include leaving out the word “not” in instructions for a medication; inaccurate conversion of the measurements of a piece of furniture that would lead a consumer to buy it to fill the wrong space; or an error right in the main title of an article.
- A major error includes failures to follow instructions or to adhere to an existing glossary. Other major errors are those that impact meaning, such as being fooled by what we call a “false friend”—a word that is similar in two languages. For instance, the English translation of a plaque in the Eiffel Tower that identified Thomas Edison as a “physician” because the French word for physicist is physicien.
- Minor errors are things like minor mistakes in spelling, grammar, punctuation, or formatting that are unlikely to impact a reader’s understanding.
After developing the model, our next step was identifying our “star translators” who would be used as evaluators. Our stars were ones who have received consistent positive feedback from our own reviewers and our clients’ in-country reviewers, and who regularly provide us with good reference materials and ask us important questions. These translators had already demonstrated professionalism, confidence, and authority in their language pairs and subject domains.
Using our evaluation matrix, our evaluators assess a selection of text during the editorial process. From this we are able to set a baseline average for the quality and quantity of errors found in translations, gather data on our translators, and provide them with feedback. Consistently average or low scores would raise a red flag. Everyone makes mistakes sometimes, but a translator who makes the same mistake two or three times is considered unacceptable. One single critical error would constitute an immediate failure. Consistently high scorers are rewarded and promoted to evaluator status.
Although this is part of our regular QA process, in rare cases an end client or in-country reviewer may raise a question about a deliverable. MTM LinguaSoft takes all questions and concerns very seriously, and in the case of any doubts, we will hire a third party editor who was not involved in the original workflow to review the translated text a second time. Using this evaluation tool, we can provide our clients with concrete objective feedback, rather than subjective commentary.
MTM LinguaSoft is proud to introduce the MTM LinguaSoft QA Model for Linguistic Quality Assurance, which offers our clients a cutting edge translation evaluation procedure to ensure that our products are always of the highest quality. For detailed information on our quality assurance process for documents, websites, software localization, and other materials, please contact us at firstname.lastname@example.org and request a copy of our full set of Quality Control / Quality Assurance procedures and checklists.