| Title | : | Leveraging Ontologies and Machine Learning to Address Challenges in Handling Transfer-Type Arithmetic Word Problems |
| Speaker | : | Suresh Kumar (IITM) |
| Details | : | Thu, 24 Apr, 2025 2:30 PM @ MR - I (SSB 233) |
| Abstract: | : | Domain knowledge is crucial in addressing challenges in Arithmetic Word Problems (AWPs), especially for solving, validity checking, repairing, and generation. Current ML-based techniques often underutilize such knowledge or neglect important aspects like validity checking. To tackle these issues, we propose hybrid methods combining machine learning and domain knowledge via ontology development. Our focus is on Transfer Case AWPs (TC-AWPs) — problems involving object transfer between agents. We develop a TC Ontology and a system called KLAUS-Tr, which uses a statistical classifier to recognize sentence types (before-transfer, transfer, after-transfer, query), extract information, represent it as RDF graphs, and solve problems using SWRL rules. KLAUS-Tr also checks for problem validity and requires fewer annotations than traditional ML approaches, achieving about 92% accuracy on datasets like All-Arith. Next, we address validity checking and repairing of machine-generated TC-AWPs. While deep learning models generate linguistically fluent problems, many are mathematically invalid. Using an extended TC Ontology, we build problem-specific representations (ABoxes) with BERT-based models to automatically detect and repair near-valid problems, ensuring mathematically correct outputs. Finally, we generate complex TC-AWPs (multiple object transfers) using ontologies and assess LLMs (ChatGPT, Gemini) on them. Results show that LLM performance declines sharply as problem complexity increases. Our extended KLAUS-Tr system outperforms LLMs on complex TC-AWPs, highlighting gaps in LLM reasoning abilities. Meeting link: https://meet.google.com/kss-bfem-xuh |
