Zhu, Y., Jin, C., Aung, K. M. M., & Xiao, X. (2026). EncFormer: Secure and Efficient Transformer Inference Over Encrypted Data. IEEE Transactions on Dependable and Secure Computing, 23(5), 11582–11597. https://doi.org/10.1109/tdsc.2026.3714715
Abstract:
Transformer inference in machine-learning-as-a-service (MLaaS) raises privacy concerns for sensitive user inputs. Prior secure solutions that combine fully homomorphic encryption (FHE) and secure multiparty computation (MPC) are bottlenecked by inefficient FHE kernels, communication-heavy MPC protocols, and expensive FHE–MPC conversions. We present EncFormer, a two-party private Transformer inference framework that introduces Stage-Compatible Patterns so that FHE kernels compose efficiently, reducing repacking and conversions. EncFormer also provides a cost analysis model built around a minimal-conversion baseline, enabling principled selection of FHE–MPC boundaries. To further reduce communication, EncFormer proposes a secure complex CKKS–MPC conversion protocol and designs communication-efficient MPC protocols for nonlinearities. With GPU optimizations, evaluations on GPT- and BERT-style models show that EncFormer achieves 1.4×–30.4× lower inference-time communication and 1.3×–9.9× lower end-to-end latency relative to prior hybrid FHE–MPC systems, and 1.9×–3.5× lower BERT-base end-to-end latency than FHE-only pipelines under a matched backend, while maintaining near-plaintext accuracy on selected GLUE tasks.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the National Research Foundation, Singapore and Infocomm Media Development Authority - Trust Tech Funding Initiative
Grant Reference no. : DTC-IGC-01 and DTC-RGC-01