Yang, X., Liu, F., & Lin, G. (2023). Neural Logic Vision Language Explainer. IEEE Transactions on Multimedia, 1–10. https://doi.org/10.1109/tmm.2023.3310277
Abstract:
If we compare how humans reason and how deep models reason, humans reason in a symbolic manner with a formal language called logic, while most deep models reason in black-box. A natural question to ask is “Do the trained deep models reason similar as humans?” or “Can we explain the reasoning of deep models in the language of logic?” . In this work, we present NeurLogX to explain the reasoning process of deep vision language models in the language of logic. Given a trained vision language model, our method starts by generating reasoning facts through augmenting the input data. We then develop a differentiable inductive logic programming framework to learn interpretable logic rules from the facts. We show our results on various popular vision language models. Interestingly, we observe that almost all of the tested models can reason logically.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the National Research Foundation - AI Singapore Programme
Grant Reference no. : AISG-RP-2018-003
This research / project is supported by the Ministry of Education - AcRF Tier-1 research
Grant Reference no. : RG95/20
This research / project is supported by the A*STAR - AME Programmatic Fund
Grant Reference no. : A20H6b0151