Singh, I. S., Kaza, P., Hosler Iv, P. G., Chin, Z. Y., & Ang, K. K. (2023). Real-Time Privacy Preserving Human Activity Recognition on Mobile using 1DCNN-BiLSTM Deep Learning. Proceedings of the 2023 5th International Conference on Image, Video and Signal Processing. https://doi.org/10.1145/3591156.3591159
Abstract:
Human Activity Recognition (HAR) from sensors has real-world applications such as fall detection of elderly or improvement in human-robot interactions, but privacy is of concern in videos of such applications. We investigated the feasibility of developing a real-time HAR mobile application that detects 3 different human poses: standing, sitting and lying, while preserving privacy by a facial detection method that also masks faces in the video. We proposed a hybrid 1-Dimensional Convolutional Neural Network with Bidirectional Long Short-Term Memory (1DCNN-BiLSTM) algorithm to recognise these 3 poses. We trained our proposed 1DCNNBiLSTM algorithm using RGB videos from 3 public datasets (URFall, NTU RGB+D, Fall Detection) whereby features were extracted using MoveNet. The results yielded 82.1% test accuracy and 0.930 macro average AUC-ROC score on the combined dataset, hence demonstrating feasibility of deploying the proposed algorithm for balanced as well as imbalanced datasets. We then implemented our proposed 1DCNN-BiLSTM algorithm with face detection and masking in a mobile application. The results showed that the mobile application could perform human pose detection inference in real time (3 people at 23.0-25.3 FPS). Hence this study showed promise of a larger scale-study of these HAR systems, such as monitoring potential falls in video surveillance
License type:
Publisher Copyright
Funding Info:
There was no specific funding for the research done