Title: Detecting malicious behaviour using supervised learning algorithms of the function calls

Authors: Mamoun Alazab; Sitalakshmi Venkatraman

Addresses: Regulatory Institutions Network, School of Regulation, Justice and Diplomacy, Australian National University, Acton, ACT 0200, Australia ' Victorian Institute of Technology, Level 14, 123 Queen Street, Melbourne, VIC 3000, Australia

Abstract: This paper describes our research in evaluating the use of supervised data mining algorithms for an effective detection of zero-day malware. Our aim is to design the tasks of certain popular types of supervised data mining algorithms for zero-day malware detection and compare their performance in terms of accuracy and efficiency. In this context, we propose and evaluate a novel method of employing such data mining techniques based on the frequency of Windows function calls. Our experimental investigations using large data sets to train the classifiers with a design tool to compare the performance of various data mining algorithms. Analysis of the results suggests the advantages of one data mining algorithm over the other for malware detection. Overall, data mining algorithms are employed with true positive rate as high as 98.5%, and low false positive rate of less than 0.025, indicating good applicability and future enhancements for detecting unknown and infected files with embedded stealthy malcode.

Keywords: zero-day malware; cybercrime; obfuscation; function calls; intrusion detection; data mining; malicious behaviour; supervised learning.

DOI: 10.1504/IJESDF.2013.055047

International Journal of Electronic Security and Digital Forensics, 2013 Vol.5 No.2, pp.90 - 109

Received: 17 Dec 2012
Accepted: 04 Apr 2013
Published online: 26 Jul 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article