Sirvi Proceedings of the 1st Workshop on Nordic-Baltic Responsible Evaluation and Alignment of Language Models (NB-REAL 2025) Autor "Karakanta, Alina" järgi
(University of Tartu Library, 2025-03) Debess, Iben Nyholm; Karakanta, Alina; Scalvini, Barbara; Einarsson, Hafsteinn; Simonsen, Annika; Nielsen, Dan Saattrup
Machine translation (MT) for Faroese faces challenges due to limited expert annotators and a lack of robust evaluation metrics. This study addresses these challenges by developing an MQM-inspired expert annotation framework to identify key error types and a simplified crowd evaluation scheme to enable broader participation. Our findings based on an analysis of 200 sentences translated by three models demonstrate that simplified crowd evaluations align with expert assessments, paving the way for improved accessibility and democratization of MT evaluation.