10.57647/mathsci.2025.1904.16877

A Two-Parameter Ridge Estimators Approach to Mitigate Multicollinearity: Simulation and Application Results

PDF
  • Muhammad Haseeb*1,
  • Muhammad Yousaf Shad1,
  • Ali Rashash R Alzahrani2,
  • Asma Ahmad Alzahrani3,
  1. Department of Statistics, Quaid-i-Azam University Islamabad, Pakistan
  2. Mathematics Department, Faculty of Sciences,Umm Al-Qura University, Makkah, Saudi Arabia
  3. Department of Mathematics, Faculty of Science, Al-Baha University, Al-Baha, Saudi Arabi

Received: 2025-08-25

Accepted: 2025-11-03

Published in Issue 2025-12-31

Copyright (c) 2025 Muhammad Haseeb, Muhammad Yousaf Shad, Ali Rashash R Alzahrani, Asma Ahmad Alzahrani (Author)

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Haseeb, M., Shad, M. Y., Alzahrani, A. R. R., & Alzahrani, A. A. (2025). A Two-Parameter Ridge Estimators Approach to Mitigate Multicollinearity: Simulation and Application Results. Mathematical Sciences, 19(4 (December 2025). https://doi.org/10.57647/mathsci.2025.1904.16877
Crossref
Scopus
Google Scholar
Europe PMC

PDF views: 255

Abstract

Multicollinearity in regression analysis arises when predictor variables are highly correlated, making it difficult to accurately estimate regression coefficients. This issue can distort model interpretations and inflate coefficient variances, making estimates sensitive to small data changes. To address this issue, several ridge estimators have been developed in the past to reduce the effect of multicollinearity and improve the model stability. To overcome the negative affect of multicollinearity, we introduce three newly proposed two-parameter ridge estimators, named HITPR1, HITPR2, and HITPR3, which dynamically adjust the ridge parameter for different multicollinearity structures. We evaluate the performance of these proposed estimators through a comprehensive simulation study and employing Mean Squared Error (MSE) criterion. The numerical results show that HITPR1 estimator performs better with higher efficiency and lower MSE, outperforming the other estimators in different settings. To further investigate the performance and applicability of the newly proposed estimators, two real-world datasets, have been utilized.

Keywords

  • Regression analysis,
  • Multicollinearity issues,
  • Ridge regression methods,
  • Mean squared error evaluation,
  • Monte Carlo simulation studies
PDF

References

  1. D. A. Belsley, “A Guide to using the collinearity diagnostics,” Comput. Sci. Econ. Manag., vol. 4, no. 1, pp. 33–50, Feb. 1991, doi: 10.1007/BF00426854.
  2. A. E. Hoerl and R. W. Kennard, “Ridge Regression: Applications to Nonorthogonal Problems,” Technometrics, vol. 12, no. 1, pp. 69–82, Feb. 1970, doi: 10.1080/00401706.1970.10488635.
  3. P. Burman et al., “The estimation of prediction error: Covariance penalties and cross-validation: Comment,” J. Am. Stat. Assoc., vol. 99, no. 467, pp. 632–642, 2004, doi: 10.1198/016214504000000908.
  4. G. C. McDonald and D. I. Galarneau, “A Monte Carlo Evaluation of Some Ridge-Type Estimators,” J. Am. Stat. Assoc., vol. 70, no. 350, pp. 407–416, Jun. 1975, doi: 10.1080/01621459.1975.10479882.
  5. G. H. Golub, M. Heath, and G. Wahba, “Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter KEYWORDS Ridge regression Cross-validation Ridge parameter,” 1979.
  6. C. M. Hurvich, “Regression and time series model selection in small samples,” 1989. [Online]. Available: http://biomet.oxfordjournals.org/
  7. G. Khalaf and G. Shukur, “Choosing Ridge Parameter for Regression Problems,” Commun. Stat. - Theory Methods, vol. 34, no. 5, pp. 1177–1182, May 2005, doi: 10.1081/STA-200056836.
  8. E. J. Candès, “Ridgelets: estimating with ridge functions,” Ann. Stat., vol. 31, no. 5, Oct. 2003, doi: 10.1214/aos/1065705119.
  9. M. A. Pasha, G.R. and Shah, “Application of Ridge Regression to Multicollinear Data.,” J. Res., vol. 15, pp. 97–106, 2004.
  10. M. A. Alkhamisi and G. Shukur, “A Monte Carlo Study of Recent Ridge Parameters,” Commun. Stat. - Simul. Comput., vol. 36, no. 3, pp. 535–547, May 2007, doi: 10.1080/03610910701208619.
  11. G. Muniz and B. M. G. Kibria, “On Some Ridge Regression Estimators: An Empirical Comparisons,” Commun. Stat. - Simul. Comput., vol. 38, no. 3, pp. 621–630, Feb. 2009, doi: 10.1080/03610910802592838.
  12. B. M. Golam Kibria, “Performance of some New Ridge regression estimators,” Commun. Stat. Part B Simul. Comput., vol. 32, no. 2, pp. 419–435, May 2003, doi: 10.1081/SAC-120017499.
  13. M. Suhail, S. Chand, and B. M. G. Kibria, “Quantile-based robust ridge m-estimator for linear regression model in presence of multicollinearity and outliers,” Commun. Stat. - Simul. Comput., vol. 50, no. 11, pp. 3194–3206, Nov. 2021, doi: 10.1080/03610918.2019.1621339.
  14. M. Arashi, M. Roozbeh, N. A. Hamzah, and M. Gasparini, “Ridge regression and its applications in genetic studies,” PLoS One, vol. 16, no. 4, p. e0245376, Apr. 2021, doi: 10.1371/journal.pone.0245376.
  15. A. T. Owolabi, K. Ayinde, J. I. Idowu, O. J. Oladapo, and A. F. Lukman, “A New Two-Parameter Estimator in the Linear Regression Model with Correlated Regressors,” J. Stat. Appl. Probab., vol. 11, no. 2, pp. 499–512, 2022, doi: 10.18576/jsap/110211.
  16. N. Akhtar and M. F. Alharthi, “A comparative study of the performance of new ridge estimators for multicollinearity: Insights from simulation and real data application,” AIP Adv., vol. 14, no. 11, p. 14, Nov. 2024, doi: 10.1063/5.0236631.
  17. I. S. Dar and S. Chand, “Bootstrap-quantile ridge estimator for linear regression with applications,” PLoS One, vol. 19, no. 4, p. e0302221, Apr. 2024, doi: 10.1371/journal.pone.0302221.
  18. N. Akhtar and M. F. Alharthi, “Enhanced ridge estimators to effectively address multicollinearity challenges,” AIP Adv., vol. 15, no. 3, 2025, doi: 10.1063/5.0259923.
  19. X. Qin, K. Li, and G. Hu, “Meshless method with ridge basis functions for time fractional two-flow domain model,” Math. Sci., vol. 14, no. 4, pp. 375–385, Dec. 2020, doi: 10.1007/s40096-020-00348-3.
  20. S. Lipovetsky and W. M. Conklin, “Ridge regression in two‐parameter solution,” Appl. Stoch. Model. Bus. Ind., vol. 21, no. 6, pp. 525–540, Nov. 2005, doi: 10.1002/asmb.603.
  21. G. Khalaf, K. Månsson, and G. Shukur, “Modified Ridge Regression Estimators,” Commun. Stat. - Theory Methods, vol. 42, no. 8, pp. 1476–1487, Apr. 2013, doi: 10.1080/03610926.2011.593285.
  22. S. Toker and S. Kaçıranlar, “On the performance of two parameter ridge estimator under the mean square error criterion,” Appl. Math. Comput., vol. 219, no. 9, pp. 4718–4728, Jan. 2013, doi: 10.1016/j.amc.2012.10.088.
  23. N. Davarzani, A. Parsian, and F. Haghighi, “Estimation of P(X <= Y ) for a Bivariate Weibull Distribution Estimation of P (X ≤ Y ) for a Bivariate Weibull Distribution,” no. February 2016, 2009, [Online]. Available: https://www.researchgate.net/publication/29605176
  24. A. F. Lukman, K. Ayinde, S. Siok Kun, and E. T. Adewuyi, “A Modified New Two-Parameter Estimator in a Linear Regression Model,” Model. Simul. Eng., vol. 2019, pp. 1–10, May 2019, doi: 10.1155/2019/6342702.
  25. S. Yasin, S. Salem, H. Ayed, S. Kamal, M. Suhail, and Y. A. Khan, “Modified Robust Ridge M-Estimators in Two-Parameter Ridge Regression Model,” Math. Probl. Eng., vol. 2021, 2021, doi: 10.1155/2021/1845914.
  26. Feras Sh. Batah; Mustafa. M. Salih; Mohammed. K. Salih; Şerifenur C. Erdal, “On modified unbiased ridge regression estimator in linear regression model,” AIP Conf. Proc. 2820, 040007, vol. Volume 282, no. Issue 1, 2023, [Online]. Available: https://doi.org/10.1063/5.0150785
  27. S. Chand and B. M. G. Kibria, “A new ridge type estimator and its performance for the linear regression model: Simulation and application,” Hacettepe J. Math. Stat., vol. 53, no. 3, pp. 837–850, 2024, doi: 10.15672/hujms.1359446.
  28. M. S. Khan, A. Ali, M. Suhail, and B. M. G. Kibria, “On some two parameter estimators for the linear regression models with correlated predictors: simulation and application,” Commun. Stat. - Simul. Comput., pp. 1–15, Jul. 2024, doi: 10.1080/03610918.2024.2369809.
  29. M. M. Abdelwahab, M. R. Abonazel, A. T. Hammad, and A. M. El-Masry, “Modified Two-Parameter Liu Estimator for Addressing Multicollinearity in the Poisson Regression Model,” Axioms, vol. 13, no. 1, p. 46, Jan. 2024, doi: 10.3390/axioms13010046.
  30. M. F. Alharthi and N. Akhtar, “Newly Improved Two-Parameter Ridge Estimators: A Better Approach for Mitigating Multicollinearity in Regression Analysis,” Axioms, vol. 14, no. 3, p. 186, Mar. 2025, doi: 10.3390/axioms14030186.
  31. A. E. . K. R. W. . & B. K. F. Hoerl, “Ridge regression: Some simulations. Communications in Statistics-Theory and Methods,” pp. 195–123, 1975.
  32. M. S. Khan, A. Ali, M. Suhail, F. A. Awwad, E. A. A. Ismail, and H. Ahmad, “On the performance of two-parameter ridge estimators for handling multicollinearity problem in linear regression: Simulation and application,” AIP Adv., vol. 13, no. 11, Nov. 2023, doi: 10.1063/5.0175494.
  33. N. Akhtar and M. F. Alharthi, “Enhancing accuracy in modelling highly multicollinear data using alternative shrinkage parameters for ridge regression methods,” Sci. Rep., vol. 15, no. 1, pp. 1–13, 2025, doi: 10.1038/s41598-025-94857-7.
  34. A. M. Halawa and M. Y. El Bassiouni, “Tests of regression coefficients under ridge regression models,” J. Stat. Comput. Simul., vol. 65, no. 1–4, pp. 341–356, Jan. 2000, doi: 10.1080/00949650008812006.
  35. N. Akhtar, M. F. Alharthi, and M. S. Khan, “Mitigating Multicollinearity in Regression: A Study on Improved Ridge Estimators,” Mathematics, vol. 12, no. 19, p. 3027, Sep. 2024, doi: 10.3390/math12193027.
  36. David I. Greenbreg and Marvin Kosters, “Income Guarantees and The Woring Poor: The Effect Of Income Maintenance Programs On The Hours Of Work of Male Family Heads,” Washington, D. C., 20506, 1970. doi: http://www.rand.org/content/dam/rand/pubs/reports/2009/R579.pdf.