An Approach Based on Fine-tuning Small Language Models for Fake News Detection

Abstract

Social networks have facilitated the rapid spread of fake news. This can negatively affect the society. Therefore, effective fake news detection is growing increasingly important. Recently, large language models (LLMs) have brought out impressive results. However, practical challenges remain because LLMs require significant computational resources. This paper explores the potential of high-performance small language models (SLMs) as an effective alternative in edge and low-resource settings. Particularly, we propose SLM-FND, a lightweight yet effective framework based on fine-tuning small language models. We evaluated state-of-the- art models, including DistilBERT, MiniLM, and ALBERT using both full fine-tuning and parameter-efficient fine-tuning methods on benchmark datasets. The results show that the proposed method achieves approximately 98.3% accuracy and F1-score values on balanced datasets. This indicates that SLMs yield competitive performance to LLM-based baselines, while requiring significantly fewer trainable parameters, shorter inference times, and lower hardware cost. Furthermore, parameter-efficient fine-tuning methods promote adaptability and minimize the computational cost. The code for the implementation of this work has been shared in the following GitHub repository (https://github.com/PhanKhacLapQNU/SLM-FND).