Volltext-Downloads (blau) und Frontdoor-Views (grau)

Identifying Problem Types in Automated Question Generation

  • Automated question generation holds great promise in many fields, such as education, to reduce the workload and automate an otherwise tedious task. However, major challenges remain regarding the quality of generated questions. To identify and address these challenges generated questions are evaluated either automatically or manually. While several automated metrics, mostly based on the comparison with a gold standard, exist, their usefulness is limited and human evaluation is often used for more accurate assessment. Our research generates questions using several models and methods, including fine-tuning, zero-shot and few-shot. We compare model performance by classifying the generated questions using a multi-label approach. This approach evaluates by sorting generated questions into zero or more binary problem classes and attempting to identify different problems with the generated questions. Our results show that different models tend to generate questions that fit into different problem classes. Additionally, the problem classification evaluation is capable of recognizing these differences and weighing the classes for the models accordingly, creating model-specific distribution characteristics.

Download full text files

Export metadata

Statistics

frontdoor_oas
Metadaten
Author:Joshua BergerORCiD, Markos StamatakisORCiD, Anett HoppeORCiD, Ralph EwerthORCiD, Christian WartenaORCiDGND
URN:urn:nbn:de:bsz:960-opus4-34511
DOI:https://doi.org/10.25968/opus-3451
ISBN:978-3-69018-002-3
Parent Title (English):KI-Forum 2024: AI 4 Students – AI in Research – AI Showroom
Publisher:HsH Applied Academics
Place of publication:Hannover
Document Type:Conference Proceeding
Language:English
Year of Completion:2024
Publishing Institution:Hochschule Hannover
Release Date:2025/01/24
Tag:AQG; Automated Question Generation; NLP; Problem Categorization; Transformers
GND Keyword:Künstliche IntelligenzGND; Automatische SprachanalyseGND
Page Number:4
First Page:34
Last Page:37
Institutes:Fakultät III - Medien, Information und Design
DDC classes:004 Informatik
Licence (German):License LogoCreative Commons - CC BY - Namensnennung 4.0 International