Zhang, X., Zhang, L., Wu, Y., Huang, M., Wu, W., Li, B., Wang, S., Fernando, B., & Liu, J. (2025). Diagram-Driven Course Questions Generation. In (Editor), Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/2025.emnlp-main.305
Abstract:
Visual Question Generation (VQG) research focuses predominantly on natural images while neglecting the diagram, which is a critical component in educational materials. To meet the needs of pedagogical assessment, we propose the Diagram-Driven Course Questions Generation (DDCQG) task and construct DiagramQG, a comprehensive dataset with 15,720 diagrams and 25,798 questions across 37 subjects and 371 courses. Our approach employs course and input text constraints to generate course-relevant questions about specific diagram elements. We reveal three challenges of DDCQG: domain-specific knowledge requirements across courses, long-tail distribution in course coverage, and high information density in diagrams. To address these, we propose the Hierarchical Knowledge Integration framework (HKI-DDCQG), which utilizes trainable CLIP for identifying relevant diagram patches, leverages frozen vision-language models for knowledge extraction, and generates questions with trainable T5. Experiments demonstrate that HKI-DDCQG outperforms existing models on DiagramQG while maintaining strong generalizability across natural image datasets, establishing a strong baseline for DDCQG.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the National Key Research and Development Program of China - National Key Research and Development Program of China
Grant Reference no. : 2022YFC3303600
This research / project is supported by the National Natural Science Foundation of China - NA
Grant Reference no. : 62137002; 62293550; 62293553; 62293554; 62450005; 62437002; 62477036; 62477037; 62176209; 62192781; 62306229
This research / project is supported by the Shaanxi Provincial Social Science Foundation - Shaanxi Provincial Social Science Foundation Project
Grant Reference no. : 2024P041
This research / project is supported by the Natural Science Basic Research Program of Shaanxi - NA
Grant Reference no. : 2023-JC-YB-593
This research / project is supported by the National Research Foundation, Singapore - NRF Fellowship
Grant Reference no. : NRF-NRFF14-2022-0001
This research / project is supported by the China Association of Automation - Youth AI Talents Fund
Grant Reference no. : HBRC-JKYZD-2024-311