Optimal Biasing Parameter Selection for Shrinkage Estimators in High-Dimensional Multicollinear Systems

Abstract

Severe multicollinearity remains a challenge in regression analysis, leading to reduced estimator efficiency. A biased parameter estimator is a popular approach that provides various ‎remedies for the problem, but the performance of this estimator will depend on the selection of ‎the biasing parameter. This paper explores and compares different biasing parameter selection ‎strategies, including one-and two-parameter biased estimators, emphasizing fixed, data-driven, ‎cross-validation, and machine learning-based solutions in different sample sizes and predictor ‎degree relationships. The study used a Monte Carlo simulation that generated ‎data at the conditions of strong multicollinearity (ρx ranging from 0.7, 0.75, 0.8, 0.85, 0.9, 0.95, and ‎‎0.99), varying error variance (σ = 3, 5, 10, 15, and 20), and sample sizes (n = 100, 500, 1000, ‎‎5000, and 10,000). The performance of the estimators was analyzed with MSE, which allows the ‎systematic comparison of different methods for selection of biasing parameters under different ‎types of multicollinearity and sample sizes. The results consistently indicate that the machine ‎learning–based selection provides the lowest MSE in all cases and is increasingly advantageous as ‎increasing multicollinearity is encountered and sample size reduces. Data-driven methods provide ‎competitive outcomes and clearly improve over fixed parameter and cross-validation. Cross-validating ‎methods experience inefficiency under more complex multicollinearity scenarios. To validate the simulation studies, a real-world data setting was applied using the Ames ‎housing dataset. The machine learning approach also produced the best predictive performance, ‎recording the lowest MSE, substantially outperforming the fixed, data-driven, and cross-validation ‎methods in the real-data application. Thus, in the presence of severe ‎multicollinearity, practitioners and researchers should go beyond applying fixed or only cross-‎validation–based rules but use machine learning or data-driven guided approaches‎.