Multi-level analysis as a systematic Approach to evaluating the quality of AI-generated dictionary entries

Evert S, Ganslmayer C, Rink C (2024)

Publication Language: English

Publication Type: Conference contribution

Publication year: 2024

Book Volume: Lexicography and Semantics

Pages Range: 298–315

Conference Proceedings Title: Proceedings of the XXI EURALEX International Congress

Event location: Cavtat/Dubrovnik HR

ISBN: 978‐953‐7967‐77‐2

URI: https://euralex.jezik.hr/wp-content/uploads/2021/09/Euralex-XXI-proceedings_1st.pdf

Abstract

In this paper, we report on our development of a multi-level analysis framework
that allows us to assess AI-generated lexicographic texts on both a quantitative and qualitative
level and compare them with human-written texts. We approach this problem through a
systematic and fine-grained evaluation, using dictionary articles created by human subjects
with the help of ChatGPT as an example. The levels of our framework concern the assessment
of individual entries, a comparison with existing dictionary entries written by experts, an
analysis of the writing experiment, and the discussion of AI-specific aspects. For the first
level, we propose an elaborate evaluation grid that enables a fine-grained comparison of
dictionary entries. While this grid has been developed for a specific writing experiment, it
can be adapted by metalexicographical experts for the evaluation of all kinds of dictionary
entries and all kinds of dictionary information categories.

Authors with CRIS profile

Stephanie Evert Lehrstuhl für Korpus- und Computerlinguistik Christine Ganslmayer Lehrstuhl für Germanistische Linguistik mit dem Schwerpunkt Variation und Wandel des Deutschen Christian Rink Professur für Germanistische Linguistik mit dem Schwerpunkt Lexikographie

How to cite

APA:

Evert, S., Ganslmayer, C., & Rink, C. (2024). Multi-level analysis as a systematic Approach to evaluating the quality of AI-generated dictionary entries. In Kristina Š. Despot, Ana Ostroški Anić, Ivana Brač (Eds.), Proceedings of the XXI EURALEX International Congress (pp. 298–315). Cavtat/Dubrovnik, HR.

MLA:

Evert, Stephanie, Christine Ganslmayer, and Christian Rink. "Multi-level analysis as a systematic Approach to evaluating the quality of AI-generated dictionary entries." Proceedings of the EURALEX 2024, Cavtat/Dubrovnik Ed. Kristina Š. Despot, Ana Ostroški Anić, Ivana Brač, 2024. 298–315.

BibTeX: Download