FlowPrint: Lightweight Behavioral Fingerprinting for Encrypted Malicious Traffic Detection

Abstract

The increasing adoption of encrypted network protocols has reduced the effectiveness of payload-based traffic inspection. This paper presents FlowPrint, a lightweight behavioral fingerprinting method for detecting malicious encrypted traffic without decrypting packet contents. FlowPrint represents each network flow using packet direction, inter-arrival time, burst structure, and packet-length transitions, and employs a multi-scale temporal encoder to distinguish malicious communication from legitimate encrypted sessions. We evaluate the method on 2.84 million network flows collected over 21 days, including 186,420 malicious flows associated with command-and-control communication, malware downloads, scanning, and data exfiltration. FlowPrint achieves an F1-score of 94.7% and AUROC of 0.982, outperforming a statistical-feature XGBoost baseline by 6.3 percentage points in F1-score and a 1D-CNN traffic classifier by 3.1 points. Under a cross-day evaluation setting, where the final seven days are excluded from training, FlowPrint maintains an F1-score of 90.8%. Runtime experiments further show a median inference latency of 1.7 ms per flow, indicating that behavioral traffic representations can provide effective detection without expensive payload inspection.