Data-Driven Machine Translation Evaluation for Multilingual Excellence
An official European Union agency with global reach.
The organization develops machine translation models across multiple language combinations involving the 23 official EU languages and the required professional linguistic evaluation to gain a clearer understanding of the quality achieved through model customization.
-
Objective
Evaluate the quality of customized machine translation models through systematic human assessment by professional linguists, providing the client with a comprehensive, data-driven view of MT performance across multiple EU language combinations.
-
Challenge
The project required segment-level evaluation of MT output using a four-point rating scale, with each document independently assessed by 2–3 linguists with at least C2 proficiency in both languages.
Documents averaged approximately 5,000 words, with each analysis representing around 15,000 evaluated words.
With an average of 6 jobs per month, the total project scope exceeded 5.32 million words.
A dedicated environment was also required to provide the necessary evaluation tools and real-time results. -
Implementation
Seprotec implemented human MT evaluation within the ContentQuo platform, customized to the client’s evaluation and reporting requirements.
Linguists and project managers received dedicated training.
Once the client uploads a document, a Seprotec coordinator checks the settings and assigns it to the three most suitable linguists, who assess each segment in a dedicated environment.
Results are verified by Seprotec and made available through a dashboard showing overall performance, quality evolution and results by language combination, with downloadable .csv/.xls reports. -
Result
The solution reduced the time required to launch evaluations and significantly lowered the client’s administrative workload, with Seprotec managing linguist allocation and coordination.
The client gains real-time access to evaluation results and ongoing, data-driven analysis by professional linguists to validate and improve MT output across 23 languages.
The tailored technology environment also provides secure, encrypted data transmission and access to specialized native linguists and professional project management support.