10.71932/ijm.2025.1205868

Parametric Activation Functions for Improved Identity Verification in IoT Devices: A Deep Learning Approach

PDF
  • Saeed Sarabadan*1,
  • Seyed Mohammad Ali Mousavi1,
  1. Department of Mathematics, Imam Hossein University, Tehran, Iran.

Received: 05-05-2025

Revised: 30-06-2025

Accepted: 09-07-2025

Published in Issue 25-07-2025

Copyright (c) -1 Saeed Sarabadan, Seyed Mohammad Ali Mousavi (Author)

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Sarabadan, S., & Ali Mousavi, S. M. (2025). Parametric Activation Functions for Improved Identity Verification in IoT Devices: A Deep Learning Approach. International Journal of Mathematical Modelling & Computations, 15(2), 0-16. https://doi.org/10.71932/ijm.2025.1205868
Crossref
Scopus
Google Scholar
Europe PMC

Abstract

One of the most important solutions in designing the architecture of a deep neural network is to use suitable activation functions in the hidden layers of the network. This function plays an important role in the back propagation algorithm and the calculation of the gradient of the cost function is based on the output of the activation function. In this paper, we will model a deep neural network to address an application problem in the Internet of Things, using experimental data recorded in a smart home, with the goal of identifying and preventing unauthorized devices from entering the Internet of Things network. The method used in this study relies on the radio frequency fingerprint of a radio device connected to the Internet of Things. The database used in this study consists of 8000 samples from 15 test series, collected using the One RF Hack radio receiver in the smart home and on 4 different connected devices. Finally ,we evaluated the performance of different activation functions in the hidden layers of the network. Ultimately, the most effective activation function was selected for the efficient and effective network. The Python code of the network architecture is located in GitHub https://github.com/SaeidSarabadan/RF_with_-ANN.

Keywords

  • Deep neural network,
  • Machine learning,
  • Activation functions,
  • Network performance improvement,
  • Internet of Things,
  • Radio frequency fingerprint
PDF

References

  1. Abayneh Bezabih, Mathematical Modeling of COVID-19 Pandemic with Treatment, International Journal of Mathematical Modelling & Computations, 11 (2021), doi:10.30495/ijm2c.2021.684815.
  2. F. Agostinelli, M. Hoffman, P. Sadowski and P. Baldi, Learning Activation Functions to Improve Deep Neural Networks, arXiv preprint arXiv:1412.6830 (2014), doi:10.48550/arXiv.1412.6830.
  3. S. Al-Hazbi, A. Hussain, S. Sciancalepore, G. Oligeri and P. Papadimitratos, Radio Frequency Fingerprinting via Deep Learning: Challenges and Opportunities, arXiv preprint arXiv:2310.16406v2 (2024), doi:10.48550/arXiv.2310.16406.
  4. E. Alcaide, E-swish: Adjusting Activations to Different Network Depths, arXiv (2018), doi:10.48550/arXiv.1801.07145.
  5. A. Apicella, F. Donnarumma, F. Isgrò and R. Prevete, A Survey on Modern Trainable Activation Functions, Neural Networks, 138 (2021) 14-32, doi:10.1016/j.neunet.2021.01.026.
  6. C. M. Bishop and H. Bishop, Deep Learning Foundations and Concepts, Springer, (2023).
  7. P. Chandra and Y. Singh, An Activation Function Adapting Training Algorithm for Sigmoidal Feedforward Networks, Neurocomputing, 61 (2004) 429-437, doi:10.1016/j.neucom.2004.04.001.
  8. P. Cheridito, A. Jentzen, A. Riekert and F. Rossmannek, A Proof of Convergence for Gradient Descent in the Training of Artificial Neural Networks for Constant Target Functions, Journal of Complexity, 101646 (2022), in press, doi:10.1016/j.jco.2022.101646.
  9. H. H. Chieng, N. Wahid, O. Pauline and S. R. K. Perla, Flatten-tswish: A Thresholded ReLU-Swish-like Activation Function for Deep Learning, International Journal of Advances in Intelligent Informatics, 4 (2) (2018) 76-86, doi:10.26555/ijain.v4i2.249.
  10. A. Dureja and P. Pahwa, Analysis of Non-linear Activation Functions for Classification Tasks Using Convolutional Neural Networks, Recent Patents on Computer Science, 12 (2019) 156-161, doi:10.2174/2213275911666181025143029.
  11. S. Elfwing, E. Uchibe and K. Doya, Sigmoid-weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning, Neural Networks, 107 (2018) 3-11, doi:10.1016/j.neunet.2017.12.012.
  12. L. B. Godfrey, An Evaluation of Parametric Activation Functions for Deep Learning, IEEE International Conference on Systems, Man and Cybernetics (SMC), (2019) 3006-3011, doi:10.1109/SMC.2019.8913972.
  13. Hossein Abbasiyan and Mohammad Jafar Doostideilami, Two methods to obtain preferred efficiency for negative data (IS), International Journal of Mathematical Modelling & Computations, 13 (2023), doi:10.30495/ijm2c.2023.1952616.1246.
  14. M. M. Hammad, Statistics for Machine Learning with Mathematica Applications, arXiv (2023), doi:10.48550/arXiv.2310.00004.
  15. K. He, X. Zhang, S. Ren and J. Sun, Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification, Proceedings of the IEEE International Conference on Computer Vision, (2015) 1026-1034, doi:10.1109/ICCV.2015.123.
  16. X. Hu, W. Liu, J. Bian and J. Pei, Measuring Model Complexity of Neural Networks with Curve Activation Functions, Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, (2020) 1521-1531, doi:10.1145/3394486.3403203.
  17. Z. Hu and H. Shao, The Study of Neural Network Adaptive Control Systems, Control and Decision, 7 (2) (1992) 361-366.
  18. A. Jagannath, J. Jagannath and P. S. P. V. Kumar, A Comprehensive Survey on Radio Frequency (RF) Fingerprinting: Traditional Approaches, Deep Learning, and Open Challenges, arXiv preprint arXiv:2201.00680v3 (2022), doi:10.48550/arXiv.2201.00680.
  19. T. Jian, B. C. Rendon, E. Ojuba et al., Deep Learning for RF Fingerprinting: A Massive Experimental Study, Northeastern University, Boston, MA, USA, (2020), doi:10.1109/IOTM.0001.1900065.
  20. G. Klambauer, T. Unterthiner, A. Mayr and S. Hochreiter, Self-normalizing neural networks, Advances in neural information processing systems, 30 (2017), doi:10.48550/arXiv.1706.02515.
  21. A. L. Maas, A. Y. Hannun and A. Y. Ng, Rectifier nonlinearities improve neural network acoustic models, Proc. icml, 30 (1) (2013) 3.
  22. F. Makhdoom and J. Ul Rahman, A Comparative Exploration of Activation Functions for Image Classification in Convolutional Neural Networks, i-Manager's Journal on Artificial Intelligence & Machine Learning, 2 (1) (2024), doi:10.26634/jaim.2.1.20225.
  23. A. Pirnat, B. Bertalanič, G. Cerar et al., Towards Sustainable Deep Learning for Wireless Fingerprinting Localization, arXiv preprint arXiv:2201.09071v1 (2022), doi:10.48550/arXiv.2201.09071.
  24. S. J. D. Prince, Understanding Deep Learning, MIT Press, Cambridge, MA, (2023).
  25. S. Qiu, X. Xu and B. Cai, Frelu: Flexible Rectified Linear Units for Improving Convolutional Neural Networks, 2018 24th International Conference on Pattern Recognition (ICPR), (2018) 1223-1228, doi:10.1109/ICPR.2018.8546022.
  26. P. Ramachandran, B. Zoph and Q. V. Le, Searching for Activation Functions, 6th International Conference on Learning Representations, (2018), doi:10.48550/arXiv.1710.05941.
  27. S. K. Roy, S. Manna, S. R. Dubey and B. B. Chaudhuri, LiSHT: Non-parametric Linearly Scaled Hyperbolic Tangent Activation Function for Neural Networks, Computer Vision and Image Processing. CVIP 2022. Communications in Computer and Information Science, 1776 (2023) 445-457, doi:10.48550/arXiv.1901.05894.
  28. S. Scardapane, S. Van Vaerenbergh, S. Totaro and A. Uncini, Kafnets: Kernel-based Non-parametric Activation Functions for Neural Networks, Neural Networks, 110 (2019) 19-32, doi:10.48550/arXiv.1707.04035.
  29. Y. Singh and P. Chandra, A Class + 1 Sigmoidal Activation Functions for FFANNs, Journal of Economic Dynamics and Control, 28 (1) (2004) 183-187, doi:10.1007/s11761-024-00423-w.
  30. G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press, (2019).
  31. L. Trottier, P. Gigu, B. Chaib-draa et al., Parametric Exponential Linear Unit for Deep Convolutional Neural Networks, Machine Learning and Applications (ICMLA), 16th IEEE International Conference on (2017) 207-214, doi:10.1109/ICMLA.2017.00038.
  32. X. Wang, H. Ren and A. Wang, Smish: A novel activation function for deep learning methods, Electronics, 11 (4) (2022) 540, doi:10.3390/electronics11040540.
  33. B. Xu, N. Wang, T. Chen and M. Li, Empirical Evaluation of Rectified Activations in Convolutional Networks, arXiv (2015), doi:10.48550/arXiv.1505.00853.
  34. Y. Ying, J. Su, P. Shan, L. Miao, X. Wang and S. Peng, Rectified Exponential Units for Convolutional Neural Networks, IEEE Access, 7 (2019) 101633-101640, doi:10.1109/ACCESS.2019.2928442.
  35. J. Zhang, G. Shen, W. Saad and K. Chowdhury, Radio Frequency Fingerprint Identification for Device Authentication in the Internet of Things, IEEE Communications Magazine, (2023), doi:10.1109/MCOM.003.2200974.

Most read articles by the same author(s)