FEAST: FEATURE ENGINEERING AND STACKING FRAMEWORK FOR COMPLEX SYSTEM PREDICTION
Keywords:
Feature engineering, Stacked ensemble learning, Uncertainty quantification, High-dimensional feature optimization, Complex system predictionAbstract
The challenge of capturing high-dimensional interactions and ensuring stable predictions in complex engineering systems, characterized by rapidly growing heterogeneous data, limits traditional models. To address this, we introduce FEAST (Feature Engineering and Advanced Stacking Framework), whose core innovation lies in its integrated approach: (1) Hierarchical feature optimization using K-Means++ clustering and adaptive correlation to significantly reduce redundancy and enhance feature discrimination; (2) An advanced stacked ensemble leveraging diverse base learners (MLR, XGBoost, LightGBM, CatBoost, GBDT) fused via Shapley-weighted combination and a regularized (Elastic Net) meta-learner to robustly capture complex patterns; (3) An efficient Monte Carlo Dropout-based uncertainty quantification module providing reliable confidence intervals for risk-aware decisions. Comprehensive experiments demonstrate FEAST's superiority: it achieves 12.7% higher prediction accuracy (p<0.01), 41% lower cross-validation error variability, and a 92.3% confidence interval coverage rate, significantly outperforming baseline models in complex engineering prediction tasks.References
[1] Su P, Shang C J, Shen Q. A hierarchical fuzzy cluster ensemble approach and its application to big data clustering. Journal of Intelligent & Fuzzy Systems, 2015, 28(6): 2409-2421.
[2] Whata A, Dibeco K, Madzima K, et al. Uncertainty quantification in multi-class image classification using chest X-ray images of COVID-19 and pneumonia. Frontiers in Artificial Intelligence, 2024, 7: 1410841.
[3] Lin S, Liang Z L, Zhao S X, et al. A comprehensive evaluation of ensemble machine learning in geotechnical stability analysis and explainability. International Journal of Mechanics and Materials in Design, 2024, 20(2): 331-352.
[4] Kong G P, Ma Y C, Xing Z W, et al. Multi-view K-means clustering algorithm based on redundant and sparse feature learning. Physica A: Statistical Mechanics and Its Applications, 2024, 633: 129405.
[5] Nejati M, Amjady N. A New Solar Power Prediction Method Based on Feature Clustering and Hybrid-Classification-Regression Forecasting. IEEE Transactions on Sustainable Energy, 2022, 13(2): 1188-1198.
[6] Mienye I D, Sun Y X. A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects. IEEE Access, 2022, 10: 99129-99149.
[7] Wang S X, Shao C B, Xu S, et al. MSFSS: A whale optimization-based multiple sampling feature selection stacking ensemble algorithm for classifying imbalanced data. AIMS Mathematics, 2024, 9(7): 17504-17530.
[8] Athens N D, Caers J K. A Monte Carlo-based framework for assessing the value of information and development risk in geothermal exploration. Applied Energy, 2019, 256: 113932.
[9] Yousefian S, Jella S, Versailles P, et al. Quantification of Rare Autoignition Events in Dry Low-Emission Premixers. Journal of Engineering for Gas Turbines and Power - Transactions of the ASME, 2022, 144(11): 111012.