Title: Multimodal social media disinformation detection by fusing images and BERT

Authors: Lizhu Ye; Md Gapar Md Johar; Mohammed Hazim Alkawaz

Addresses: School of Information Technology and Engineering, Guangzhou College of Commerce, Guangzhou 511363, Guangdong, China; School of Graduate Studies, Management and Science University, 40100 Shah Alam, Selangor, Malaysia ' Software Engineering and Digital Innovation Centre, Management and Science University, 40100 Shah Alam, Selangor, Malaysia ' Department of Computer Science, College of Education for Pure Science, University of Mosul, Mosul, Nineveh, Iraq

Abstract: To address the diversity and fragmentation issues in false information detection (FID) on social media, this article proposed the MM-GCN-BERT model based on the advantages of GCN and BERT models, integrating multimodal fusion and cascading detection frameworks. By designing and optimising the MM-GCN-BERT model, and combining specific experimental data to verify the performance of the MM-GCN-BERT model, experimental results were obtained. The results showed that the average F1 value of the MM-GCN-BERT model was 0.816, the average precision was 0.836, and the average recall was 0.797. The indicators of the model are all at a high level. It can be seen that the MM-GCN-BERT model has shown excellent performance in solving the problems of diversity and fragmentation in detecting false information on social media.

Keywords: MM-GCN-BERT model; multimodal fusion; cascade detection framework; false information detection; FID; social media.

DOI: 10.1504/IJIIDS.2025.147437

International Journal of Intelligent Information and Database Systems, 2025 Vol.17 No.3/4, pp.553 - 569

Received: 16 Jul 2024
Accepted: 28 Oct 2024
Published online: 15 Jul 2025 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article