EDAL: Entropy based Dynamic Attention Loss for HateSpeech Classification
Md Fahim, Amin Ahsan Ali, Md Ashraful Amin, AKM Mahbubur Rahman
The 37th Pacific Asia Conference on Language, Information and Computation (PACLIC 37)
Abstract
Entropy-based Dynamic Attention Loss" (EDAL) to enhance model interpretability by incorporating an additional attention layer. EDAL encourages attention scores that provide valuable insights and boosts the performance of pretrained models during fine-tuning for downstream tasks. We conduct extensive experiments on six diverse datasets, confirming that EDAL effectively enhances classification performance while maintaining interpretability. Additionally, experiments with various pretrained models demonstrate EDAL's significant performance improvements during fine-tuning. In summary, EDAL holds promise for creating more transparent and reliable hate speech classifiers, contributing to a safer online environment.