Our paper titled “Training-Free Layer Selection for Partial Fine-Tuning of Language Models” has been accepted for publication in the Information Sciences journal

Authors:

Aldrin Kabya Biswas, Md Fahim, Md Tahmid Hasan Fuad, Akm Moshiur Rahman Mazumder, Amin Ahsan Ali, AKM Mahbubur Rahman

In this work, the authors propose a training-free, layer-wise partial fine-tuning approach for language models using cosine similarity between representative tokens across layers . The method identifies inter-layer relationships through a single forward pass and selectively fine-tunes layers while keeping others frozen.

The proposed approach achieves competitive performance compared to full fine-tuning, with an average 1.5× training speedup and a 75% reduction in trainable parameters, and outperforms all comparative baselines on 14 out of 16 datasets.

Congratulations to all the authors and collaborators on this excellent achievement!