A Mathematical Framework for Anomaly Detection in High-Dimensional Data Using Sparse Autoencoders and Mutual Information Filtering

Abstract

In today’s world, where the volume of generated data is rapidly increasing, anomaly detection in high-dimensional datasets remains a significant challenge in data mining and artificial intelligence. With the rapid expansion of the Internet of Things (IoT) and the increasing number of connected devices, security threats in this domain have significantly intensified. Detecting anomalies in IoT network traffic has become a critical component in combating cyber-attacks and maintaining system integrity. This study aims to develop a deep learning-based approach for anomaly detection in network traffic using the Variational Autoencoder (VAE), a probabilistic generative model capable of learning hidden structures in complex data. The UNSW-NB15 benchmark dataset, which includes a wide range of normal and malicious traffic samples, was utilized. After data preprocessing—comprising cleaning, normalization, and feature selection—the VAE model was trained solely on normal data to learn the typical patterns of network behavior. Anomalies were identified by analyzing the reconstruction error between the original and generated data, where instances with high error values were flagged as anomalous. The model was optimized using a loss function combining reconstruction loss and Kullback-Leibler divergence. Experimental results showed that the proposed VAE model achieved an accuracy of 93.8%, a recall of 89.2%, and an AUC score of 0.94, demonstrating its effectiveness in detecting various types of attacks, including DoS, Fuzzing, and Exploit. This research confirms that probabilistic deep learning models, particularly VAEs, offer a robust and scalable solution for anomaly detection in IoT environments and can be instrumental in developing intelligent intrusion detection systems for modern cyber-physical infrastructures.