Knowledge Distillation for Lightweight and Explainable Intrusion Detection in Resource-Constrained Consumer Devices

The growing demand for deploying deep learning (DL) models for cyber-attack detection on consumer devices is constrained by their limited memory and computational resources. High-performing DL models are often too resource-intensive for on-device deployment. Knowledge distillation (KD) offers an alternative by transferring knowledge from a complex teacher model to a smaller, lightweight student model, reducing model size and inference time with minimal accuracy loss. In this paper, a KD-based framework is utilized to design and train both teacher and student models for intrusion detection using the N-BaIoT and CIC-IDS 2023 datasets. The student model achieves comparable performance to the teacher, with accuracies of 99.87% and 98.71% on the utilized datasets. It maintains accurate classification across various attack classes, achieving F1-scores above 0.98, while reducing computational complexity from 263k to 120k trainable parameters, model size from 1.00 MB to 471.67 KB, and inference time from 48 ms to 34 ms. Moreover, SHAP-based explainable AI techniques are employed to interpret both models, demonstrating that the distilled student model retains key feature attributions of the teacher model.