Machine Learning Based Malware Detection on Encrypted Traffic: A Comprehensive Performance Study
Abstract
The increasing volume of encrypted network traffic yields a clutter for hackers to use encryption to spread their malicious software on the network. We study the problem of detecting TLS-encrypted malware on the client side using metadata and TLS protocol related flow features. We conduct a comprehensive study on a set of widely used machine learning and deep learning algorithms to detect encrypted malware on two malware flows datasets. In addition to reporting the classification accuracy of the approaches under study, we conduct comprehensive experiments to quantify their run-time performance in terms of throughput and system resource utilization such as the CPU and RAM utilization. Moreover, we further boost the speed of the detection systems using acceleration libraries such as DAAL and OpenVINO. Through the quantitative analysis, we provide a comparison on the effectiveness and run-time performance of the machine learning models, and evaluate techniques to accelerate real-world deployment.
