Title: Detecting malicious files using non-signature-based methods

Authors: P. Vinod; Vijay Laxmi; Manoj Singh Gaur; Grijesh Chauhan

Addresses: Department of Computer Science and Engineering, SCMS School of Engineering and Technology, Ernakulam, India ' Department of Computer Engineering, Malaviya National Institute of Technology, Jaipur, India ' Department of Computer Engineering, Malaviya National Institute of Technology, Jaipur, India ' Department of Computer Engineering, Malaviya National Institute of Technology, Jaipur, India

Abstract: Malware or malicious code intends to harm the computer systems without the knowledge of system users. Malware are unknowingly installed by naïve users while browsing the internet. Once installed, the malicious programs perform unintentional activities like: a) steal user name, password; b) install spy software to provide remote access to the attackers; c) flood spam messages; d) perform denial of service attacks, etc. With the emergence of metamorphic malware (that uses complex obfuscation techniques), signature-based detectors fail to identify new variants of malware. In this paper, we investigate non-signature techniques for malware detection and demonstrate methods of feature selection that are best suited for detection purposes. Features are produced using mnemonic n-grams and instruction opcodes (opcodes along with addressing modes). The redundant features are eliminated using class-wise document frequency, scatter criterion and principal component analysis (PCA). The experiments are conducted on the malware dataset collected from VX Heavens and benign executables (gathered from fresh installation of Windows XP operating system and other utility software's). The experiments also demonstrate that proposed methods that do not require signatures are effective in identifying and classifying morphed malware.

Keywords: malware detection; scatter criterion; class-wise document frequency; CDF; principal component analysis; PCA; features; classifiers; malicious files; non-signature methods; feature selection; morphed malware; computer security.

DOI: 10.1504/IJICS.2014.066646

International Journal of Information and Computer Security, 2014 Vol.6 No.3, pp.199 - 240

Received: 03 Aug 2013
Accepted: 06 Apr 2014
Published online: 14 Jan 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article