Yang, Z., Yan, M., Zhang, Y., & Zhou, J. T. (2026). Poisoned Distillation: Injecting Backdoors into Distilled Datasets Without Raw Data Access. Proceedings of the AAAI Conference on Artificial Intelligence, 40(2), 1444–1452. https://doi.org/10.1609/aaai.v40i2.37119
Abstract:
Dataset distillation (DD) condenses large datasets into smaller synthetic ones to enhance training efficiency and reducing bandwidth. DD enables models to achieve comparable performance to those trained on the raw full dataset, making it popular for data sharing. Existing work shows that injecting backdoors during the distillation process can threaten downstream models. However, these studies assume attackers can have access to the raw dataset and interfere with the entire distillation process, which is unrealistic. In contrast, this work is the first to address a more realistic and concerning threat: attackers may intercept the dataset distribution process, inject backdoors into the distilled datasets, and redistribute them to users. While distilled datasets were previously considered resistant to backdoor attacks, we demonstrate that they remain vulnerable to such attacks. Furthermore, we show that attackers do not even require access to any raw data to inject the backdoors successfully within one minute. Specifically, our approach reconstructs conceptual archetypes for each class from the model trained on the distilled dataset. Backdoors are then injected into these archetypes to update the distilled dataset. Moreover, we ensure the updated dataset not only retains the backdoor but also preserves the original optimization trajectory, thus maintaining the knowledge of the raw dataset. To achieve this, a hybrid loss is designed to integrate backdoor information along the benign optimization trajectory, ensuring that previously learned information is not forgotten. Extensive experiments demonstrate that distilled datasets are highly vulnerable to our attack, with risks pervasive across various raw datasets, distillation methods, and downstream training strategies.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the A*STAR - Japan Science and Technology Agency (JST) and the A*STAR under the Japan-Singapore Joint Call
Grant Reference no. : R24I6IR133
This research / project is supported by the National Research Foundation - Digital Trust Centre Innovation Grant
Grant Reference no. : DTC- IGC-02
Description:
This material may not be retransmitted or redistributed without permission in writing from The Association for the Advancement of Artificial Intelligence. Permission to use document is granted, provided that (1) the copyright notice appears in all copies and that both the copyright notice and this permission notice appear, (2) use of such documents is for personal use only, and will not be copied or posted on any network computer or broadcast in any media, and (3) no modifications of any documents are made.