Generating Multiple-Choice Knowledge Questions with Interpretable Difficulty Estimation using Knowledge Graphs and Large Language Models<\/p>\n
Mehmet Can \u015eakiro\u011flu
\nMaster Student<\/p>\n
(Supervisor: Prof.Dr. H. Altay G\u00fcvenir) Computer Engineering Department
\nBilkent University<\/p>\n
Abstract: Generating multiple-choice questions (MCQs) with difficulty estimations remains challenging in automated MCQ-generation systems. This thesis proposes a novel methodology for generating MCQs with difficulty estimations from the given text documents by utilizing knowledge graphs (KGs) and large language models (LLMs). Given a set of documents, the approach proposed in this thesis uses an LLM to construct a KG, from which MCQs are then systematically generated. Each MCQ is generated by selecting a node from the KG as the key, sampling a related triple or quintuple\u2014optionally augmented with an extra triple\u2014and prompting an LLM to generate a corresponding stem from these graph components. Distractors, the wrong choices, are then selected from the KG. For each MCQ, nine difficulty signals are computed and combined into a unified difficulty score using a data-driven approach. Experimental results demonstrate that our method generates high-quality MCQs whose difficulty estimations are interpretable and align with human perceptions. Our approach improves MCQ generation by integrating structured knowledge representations with LLMs and a data-driven difficulty estimation model.<\/p>\n
Date: October 21, Tuesday @ 10:00
\nPlace: EA 516<\/p>\n","protected":false},"excerpt":{"rendered":"
Generating Multiple-Choice Knowledge Questions with Interpretable Difficulty Estimation using Knowledge Graphs and Large Language Models Mehmet Can \u015eakiro\u011flu Master Student (Supervisor: Prof.Dr. H. Altay G\u00fcvenir) Computer Engineering Department Bilkent University Abstract: Generating multiple-choice questions (MCQs) with difficulty estimations remains challenging in automated MCQ-generation systems. This thesis proposes a novel methodology [\u2026] <\/p>\n