Leveraging BERT, DistilBERT, and TinyBERT for Rumor Detection

Abstract

The rapid spread of false information on social media has become a major challenge in today’s digital world. This has created a need for an effective rumor detection system that can identify and control the spread of false information in real-time. The proposed work introduces a rumor detection system by integrating transformer-based models such as BERT, DistilBERT, and TinyBERT with traditional Machine Learning (ML) techniques. The classifiers include Decision Tree (DT), Support Vector Machine (SVM), Random Forest (RF) and Naïve Bayes (NB) help in categorizing content as either rumor or non-rumor based on the patterns. The proposed work evaluated BERT, DistilBERT, TinyBERT combined with ML models (SVM, DT, RF, NB) across PHEME dataset using 70:30, 60:40, and 80:20 splits. Overall, BERT + DT and TinyBERT + SVM provided significant results, with BERT + RF and DistilBERT + NB demonstrating better classification capabilities across various events and split ratios on the dataset.

Keywords

• Social media
• rumor
• BERT
• PHEME dataset