Zhang, M., Guo, X., Cao, X., Sun, S., Fu, H., & Guo, Q. (2025). Lighting is Unreliable: Adversarial Video Relighting Against rPPG Heart Rate Measurement. IEEE Transactions on Information Forensics and Security, 20, 8341–8356. https://doi.org/10.1109/tifs.2025.3587221
Abstract:
Facial video-based remote physiological measurement (rPPG) has gained prominence for its ability to non-invasively estimate vital signs such as heart rate (HR). rPPG measures HR by detecting variations in the diffuse reflection of light caused by blood volume changes in the skin, which is influenced by light conditions. Inspired by this property, we identify a new task, that is, to embed malicious information into facial video by subtly altering the light conditions. With this task, we can mislead state-of-the-art rPPG HR methods through natural and imperceptible lighting changes, aiming for two objectives: testing the resilience of rPPG methods against light variations and safeguarding heart rate data, which is crucial for individual privacy. However, such a task is non-trivial and should be able to adapt to different input videos automatically and generate natural and imperceptible spatial-temporal lighting perturbations. To address these challenges, we propose the AdversariaL vidEo Relighting aTtack (ALERT) method, which involves three modules: video lighting estimation (VLightE), adversarial temporal lighting prediction (AdvTLight), and adversarial temporal lighting injection (AdvTLightInj). VLightE is to estimate the spatial-temporal lighting conditions of the original video. AdvTLight predicts the adversarial spatial-temporal lighting conditions that are imperceptible but can mislead the HR detectors according to the original lighting conditions automatically. The final module (i.e., AdvTLightInj) is to inject the predicted adversarial lighting conditions into the input video and render a new one. Extensive experiments on UBFC-rPPG and PURE datasets demonstrate that ALERT generates realistic, imperceptible adversarial videos, effectively misleading 11 rPPG-based HR methods and outperforming all baseline methods. Moreover, our method can be used to protect the HR privacy of users directly and outperform two SOTA Privacy-protection-oriented methods significantly.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the National Research Foundation, Singapore - AI Singapore Programme
Grant Reference no. : AISG4-GC-2023-008-1B
This research / project is supported by the National Research Foundation, Singapore - National Large Language Models Funding Initiative
Grant Reference no. : AISG-NMLP-2024-004
This research / project is supported by the National Research Foundation, Singapore and Infocomm Media Development Authority - Trust Tech Funding Initiative
Grant Reference no. : AISG4-GC-2023-008-1B