Roy, D., & Fernando, B. (2024). Predicting the Next Action by Modeling the Abstract Goal. In Pattern Recognition (pp. 162–177). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-78354-8_11
Abstract:
The problem of predicting human actions from observed videos is an inherently uncertain one. We present an action anticipation model that leverages latent goal information to reduce the uncertainty in future predictions. We develop a latent variable representing goal information called abstract goal which is conditioned on observed sequences of visual features for action anticipation. We design the abstract goal as a distribution whose parameters are estimated using a variational recurrent model. We sample multiple candidates for the next action and use goal consistency criterion to determine the best candidate that follows from the abstract goal. Our method obtains impressive results on the very challenging Epic-Kitchens55 (EK55) and good results in Epic-Kitchens100 (EK100) datasets. Code is available at https://github.com/LAHAproject/Abstract_Goal
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the National Research Foundation - NRF Fellowship
Grant Reference no. : NRF-NRFF14-2022-0001
This research / project is supported by the National Research Foundation - AI Singapore Programme
Grant Reference no. : AISG2-RP-2020-016
This research / project is supported by the A*STAR - Science and Engineering Research Council (SERC) Central Research Fund
Grant Reference no. :
Description:
This is a post-peer-review, pre-copyedit version of an article published in Lecture Notes in Computer Science. The final authenticated version is available online at: http://dx.doi.org/10.1007/978-3-031-78354-8_11