FedCTTA: A Collaborative Approach to Continual Test-Time Adaptation in Federated Learning

Abstract

Federated Learning (FL) enables collaborative model training across distributed clients without sharing raw data, making it ideal for privacy-sensitive applications. However, FL models often suffer performance degradation due to distribution shifts between training and deployment. Test-Time Adaptation (TTA) offers a promising solution by allowing models to adapt using only test samples. However, existing TTA methods in FL face challenges such as computational overhead, privacy risks from feature sharing, and scalability concerns due to memory constraints. To address these limitations, we propose Federated Continual Test-Time Adaptation (FedCTTA), a privacy-preserving and computationally efficient framework for federated adaptation. Unlike prior methods that rely on sharing local feature statistics, FedCTTA avoids direct feature exchange by leveraging similarity-aware aggregation based on model output distributions over randomly generated noise samples. This approach ensures adaptive knowledge sharing while preserving data privacy. Furthermore, FedCTTA minimizes the entropy at each client for continual adaptation, enhancing the model’s confidence in evolving target distributions. Our method eliminates the need for server-side training during adaptation and maintains a constant memory footprint, making it scalable even as the number of clients or training rounds increases. Extensive experiments show that FedCTTA surpasses existing methods across diverse temporal and spatial heterogeneity scenarios.

Keywords

CCDS Authors

M Ashraful Amin, PhD

Founder DirectorDirector, Artificial Intelligence & Machine Learing WingDirector, Human-Computer Interaction WingDirector, Data Science Wing

References

1. 1.Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun. (2016). Deep Residual Learning for Image Recognition. , 770–778[10.1109/cvpr.2016.90]
2. 2.Eric Tzeng, Judy Hoffman, Kate Saenko, Trevor Darrell. (2017). Adversarial Discriminative Domain Adaptation. , 2962–2971[10.1109/cvpr.2017.316]
3. 3.Mei Wang, Weihong Deng. (2018). Deep visual domain adaptation: A survey. Neurocomputing, 312, 135–153[10.1016/j.neucom.2018.05.083]
4. 4.Qinbin Li, Bingsheng He, Dawn Song. (2021). Model-Contrastive Federated Learning. , 10708–10717[10.1109/cvpr46437.2021.01057]
5. 5.Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Fengqiu Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, Justin Gilmer. (2021). The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 8320–8329[10.1109/iccv48922.2021.00823]
6. 6.Yutao Huang, Lingyang Chu, Zirui Zhou, Lanjun Wang, Jiangchuan Liu, Jian Pei, Yong Zhang. (2021). Personalized Cross-Silo Federated Learning on Non-IID Data. Proceedings of the AAAI Conference on Artificial Intelligence, 35(9), 7865–7873[10.1609/aaai.v35i9.16960]
7. 7.Qin Wang, Olga Fink, Luc Van Gool, Dengxin Dai. (2022). Continual Test-Time Domain Adaptation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 7191–7201[10.1109/cvpr52688.2022.00706]
8. 8.M. Jehanzeb Mirza, Jakub Micorek, Horst Possegger, Horst Bischof. (2022). The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 14745–14755[10.1109/cvpr52688.2022.01435]
9. 9.Hai Jin, Dongshan Bai, Dezhong Yao, Yutong Dai, Lin Gu, Yu Chen, Lichao Sun. (2022). Personalized Edge Intelligence via Federated Self-Knowledge Distillation. IEEE Transactions on Parallel and Distributed Systems, 34(2), 567–580[10.1109/tpds.2022.3225185]
10. 10.Robert A. Marsden, Mario Döbler, Bin Yang. (2024). Universal Test-time Adaptation through Weight Ensembling, Diversity Weighting, and Prior Correction. , 2543–2553[10.1109/wacv57701.2024.00254]
11. 11.Yasar Abbas Ur Rehman, Yan Gao, Pedro Porto Buarque de Gusmão, Mina Alibeigi, Jiajun Shen, Nicholas D. Lane. (2023). L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learning. , 16418–16427[10.1109/iccv51070.2023.01509]
12. 12.Max Klabunde, Tobias Schumacher, Markus Strohmaier, Florian Lemmerich. (2025). Similarity of Neural Network Models: A Survey of Functional and Representational Measures. ACM Computing Surveys, 57(9), 1–52[10.1145/3728458]
13. 13.Jiayuan Zhang, Xuefeng Liu, Yukang Zhang, Guogang Zhu, Jianwei Niu, Shaojie Tang. (2024). Enabling Collaborative Test-Time Adaptation in Dynamic Environment via Federated Learning. , 4191–4202[10.1145/3637528.3671908]
14. 14.Jian Xu, Shao‐Lun Huang. (2023). A Joint Training-Calibration Framework for Test-Time Personalization with Label Shift in Federated Learning. , 4370–4374[10.1145/3583780.3615173]