An approach to user-centered translation quality assessment of machine translation output : the case of DeepL, Google Translate, and ChatGPT in Czech-to-Spanish translation outputs

Abstract

The widespread use of free Neural Machine Translation (NMT) systems requires a greater effort on the part of the scientific community to evaluate their quality. This article presents the state of the art and the results of a pilot analysis aimed at revealing the level of satisfaction of potential users of these translations in terms of three variables: fluency, grammar, and usability. To this end, an experiment was carried out in which twenty native Spanish annotators evaluated, using a Likert rating scale, the translations generated by human professionals and by the applications DeepL, Google Translate, and ChatGPT of three Czech texts of different types (one technical, one marketing and one literary). The results show that although human translations are the best rated, there is a high degree of user satisfaction with the translations generated by NMT systems specifically designed for this purpose (DeepL and Google Translate), especially in terms of fluency and usability.

Keywords

• Neural Machine Translation
• translation quality assessment
• Czech-Spanish translation
• DeepL
• Google Translate
• ChatGPT