Google Translate vs. ChatGPT: Can non-language professionals trust them for specialized translation?

Abstract

Experts and professionals in specialized fields often need writing tools to communicate in English as a means to disseminate their knowledge or enter the international market. There are different tools to accomplish this and most of them are, lately, Machine Translation systems (MT) based on Neural Machine Translation (NMT), an approach using artificial neural networks to translate with outstanding fluency. Free and open systems such as Google Translate or, more recently, ChatGPT used as a translator, have popularized NMT to a multitude of users. However, there are experts and professionals who, due to their lack of command of English, often fail in their communication tasks by accepting NMT system’s output as correct. This paper examines these systems’ performance when translating terminology of the discourse in wine and olive oil tasting notes, specifically from Spanish into English. This domain may serve to represent lessstudied specialized languages where general language words and terms become closely intertwined. The aim is to determine whether these systems can translate terminology accurately within the domain, and, if so, whether the GPT-3.5 model outperforms Google Translate. Results will help identify or discard possible language solutions for users who need to obtain texts in specialized English with professional and internationalization purposes, but who do not have the linguistic or economic resources to ensure the quality of the English text. Results show that, although ChatGPT yields fewer terminological errors than Google Translate in terms of error severity and number of samples affected, professionals cannot rely solely on these tools just yet.