Article: Large language models for vulnerability detection: a multi-use case comparative study Journal: International Journal of Applied Cryptography (IJACT) 2026 Vol.5 No.6 pp.1 - 17 Abstract: Large language models (LLMs) have shown promising capabilities in assisting researchers and developers in different fields of cybersecurity. This work investigates whether 11 state-of-the-art LLMs can be used for source code vulnerability analysis across three different use cases and four publicly available benchmark datasets. More specifically, we examined Android, smart contract and IoT source code, containing vulnerabilities from Open Worldwide Application Security Project (OWASP) Mobile Top 10, Common Weakness Enumeration (CWE) databases, and smart contract related vulnerabilities. Moreover, we explored whether LLMs could detect potentially privacy-invasive actions and if retrieval-augmented generation (RAG) could improve the performance of LLMs in vulnerability detection. Our results reveal that no single LLM is consistently better-performing compared to others across all use cases and datasets, whereas different models are the best performers in different use cases and datasets. Thus, a careful LLM selection is necessary based on the unique characteristics of each use case. Inderscience Publishers - linking academia, business and industry through research

Title: Large language models for vulnerability detection: a multi-use case comparative study

Authors: Vasileios Kouliaridis; Georgios Karopoulos; Georgios Kambourakis

Addresses: European Commission, Joint Research Centre (JRC), 21027, Ispra, Italy ' European Commission, Joint Research Centre (JRC), 21027, Ispra, Italy ' Department of Information and Communication Systems Engineering, University of the Aegean, Karlovasi, 83200, Samos, Greece

Abstract: Large language models (LLMs) have shown promising capabilities in assisting researchers and developers in different fields of cybersecurity. This work investigates whether 11 state-of-the-art LLMs can be used for source code vulnerability analysis across three different use cases and four publicly available benchmark datasets. More specifically, we examined Android, smart contract and IoT source code, containing vulnerabilities from Open Worldwide Application Security Project (OWASP) Mobile Top 10, Common Weakness Enumeration (CWE) databases, and smart contract related vulnerabilities. Moreover, we explored whether LLMs could detect potentially privacy-invasive actions and if retrieval-augmented generation (RAG) could improve the performance of LLMs in vulnerability detection. Our results reveal that no single LLM is consistently better-performing compared to others across all use cases and datasets, whereas different models are the best performers in different use cases and datasets. Thus, a careful LLM selection is necessary based on the unique characteristics of each use case.

Keywords: large language models; LLMs; vulnerability detection; vulnerability analysis; code analysis; OWASP; mobile security; Android; IoT; smart contracts; retrieval-augmented generation; RAG.

DOI: 10.1504/IJACT.2026.154618

International Journal of Applied Cryptography, 2026 Vol.5 No.6, pp.1 - 17

Received: 13 Aug 2025
Accepted: 07 Mar 2026
Published online: 07 Jul 2026 *

Title: Large language models for vulnerability detection: a multi-use case comparative study

Keep up-to-date