Abstract
Malware, a diverse category of software specifically engineered to compromise devices, poses a serious threat to the security of computer systems and networks. Traditional malware detection methods, such as signature-based or behavior-based, rely on predefined patterns or manual analysis of mal ware characteristics or behaviors. However, these methods are ineffective against new or unknown malware, as they cannot recognize malware that does not match the existing patterns or profiles. Machine learning (ML) methods, on the other hand, can learn from data to detect malware based on complex patterns, without requiring prior knowledge or human intervention. In this paper, we propose and apply an anomaly detection approach on Programmable Executable files to detect and prevent malware installation. We evaluated our approach on a publicly available dataset, namely, Blue Hexagon Qpen Dataset for Malware AnalysiS (BODMAS) dataset using three classifiers, KNearest Neighbor, Support Vector Machine, and Random Forest to identify anomalies in the PE files. RF outperformed its counterparts and yielded highest accuracy of 99.73% with zero False Positive Rate.