A Data-Driven Explainable AI Framework for Predicting Industrial Equipment Failures Using Machine Learning: A Bearing-Based Case Study
Article Sidebar
Main Article Content
Abstract
Prediction accuracy does not necessarily indicate whether a machine-learning model warns of an impending equipment event at an actionable time. This study evaluates a data-driven, explainable framework using 15 accelerated XJTU-SY bearing tests. We audited 9,216 dual-axis vibration files, constructed 16 leakage-checked features from 9,201 preterminal observations, and compared an RMS score, logistic regression, and random forest using five bearing-disjoint folds. For the primary ten-minute recorded-end-of-test horizon, mean fold average precision was 0.6491, 0.4194, and 0.3098, respectively. Nevertheless, new actionable alarm episodes occurred in only 4, 4, and 6 of 15 held-out bearings, with 9, 32, and 27 premature episodes. An exploratory operating-condition holdout reduced actionable coverage to 0, 3, and 3 bearings. TreeSHAP explanations on 449 observations showed heterogeneous between-bearing feature rankings (median pairwise Spearman correlation 0.488). Feature-group ablation further demonstrated divergence between predictive ranking and alarm timing. These findings support decision-oriented evaluation alongside conventional metrics, but concern laboratory test termination rather than independently verified first damage or demonstrated industrial savings.
Article Details
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
References
[1] Y. Lei, T. Han, B. Wang, N. Li, T. Yan, and J. Yang, “XJTU-SY Rolling Element Bearing Accelerated Life Test Datasets: A Tutorial,” J. Mech. Eng., vol. 55, no. 16, pp. 1–6, 2019, doi: 10.3901/JME.2019.16.001.
[2] B. Wang, Y. Lei, N. Li, and N. Li, “A Hybrid Prognostics Approach for Estimating Remaining Useful Life of Rolling Element Bearings,” IEEE Trans. Reliab., vol. 69, no. 1, pp. 401–412, 2020, doi: 10.1109/TR.2018.2882682.
[3] G. Zhu, Q. Huang, and Z. Zhang, “A Condition-Monitoring Method for Rolling Bearings Based on Dynamic Asynchronous Peak-Factor Ratios,” Sensors, vol. 23, no. 21, 2023, doi: 10.3390/s23218939.
[4] E. Florian, F. Sgarbossa, and I. Zennaro, “Machine learning-based predictive maintenance: A cost-oriented model for implementation,” Int. J. Prod. Econ., vol. 236, 2021, doi: 10.1016/j.ijpe.2021.108114.
[5] M. Hermansa, M. Kozielski, M. Michalak, K. Szczyrba, Ł. Wróbel, and M. Sikora, “Sensor-Based Predictive Maintenance with Reduction of False Alarms—A Case Study in Heavy Industry,” Sensors, vol. 22, no. 1, 2022, doi: 10.3390/s22010226.
[6] S. M. Lundberg and S.-I. Lee, “A Unified Approach to Interpreting Model Predictions,” in Advances in Neural Information Processing Systems, 2017. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html
[7] M. L. Baptista, K. Goebel, and E. M. P. Henriques, “Relation between prognostics predictor evaluation metrics and local interpretability SHAP values,” Artif. Intell., vol. 306, 2022, doi: 10.1016/j.artint.2022.103667.
[8] E. Brusa, L. Cibrario, C. Delprete, and L. G. Di Maggio, “Explainable AI for Machine Fault Diagnosis: Understanding Features’ Contribution in Machine Learning Models for Industrial Condition Monitoring,” Appl. Sci., vol. 13, no. 4, 2023, doi: 10.3390/app13042038.
[9] R. K. Kundu and K. A. Hoque, “Explainable Predictive Maintenance is Not Enough: Quantifying Trust in Remaining Useful Life Estimation,” in Annual Conference of the PHM Society, 2023. doi: 10.36001/phmconf.2023.v15i1.3472.
[10] Y. Deng, C. Guo, Z. Zhang, L. Zou, X. Liu, and S. Lin, “An Attention-Based Method for Remaining Useful Life Prediction of Rotating Machinery,” Appl. Sci., vol. 13, no. 4, 2023, doi: 10.3390/app13042622.
[11] A. K. S. Jardine, D. Lin, and D. Banjevic, “A review on machinery diagnostics and prognostics implementing condition-based maintenance,” Mech. Syst. Signal Process., vol. 20, no. 7, pp. 1483–1510, 2006, doi: 10.1016/j.ymssp.2005.09.012.
[12] Y. Lei, N. Li, L. Guo, N. Li, T. Yan, and J. Lin, “Machinery health prognostics: A systematic review from data acquisition to RUL prediction,” Mech. Syst. Signal Process., vol. 104, pp. 799–834, 2018, doi: 10.1016/j.ymssp.2017.11.016.
[13] T. P. Carvalho, F. A. A. M. N. Soares, R. Vita, R. da P. Francisco, J. P. T. V. Basto, and S. G. S. Alcalá, “A systematic literature review of machine learning methods applied to predictive maintenance,” Comput. Ind. Eng., vol. 137, 2019, doi: 10.1016/j.cie.2019.106024.
[14] G. A. Susto, A. Schirru, S. Pampuri, S. McLoone, and A. Beghi, “Machine Learning for Predictive Maintenance: A Multiple Classifier Approach,” IEEE Trans. Ind. Informatics, vol. 11, no. 3, pp. 812–820, 2015, doi: 10.1109/TII.2014.2349359.
[15] R. Zhao, R. Yan, Z. Chen, K. Mao, P. Wang, and R. X. Gao, “Deep learning and its applications to machine health monitoring,” Mech. Syst. Signal Process., vol. 115, pp. 213–237, 2019, doi: 10.1016/j.ymssp.2018.05.050.
[16] C. Bergmeir, R. J. Hyndman, and B. Koo, “A note on the validity of cross-validation for evaluating autoregressive time series prediction,” Comput. Stat. Data Anal., vol. 120, pp. 70–83, 2018, doi: 10.1016/j.csda.2017.11.003.
[17] L. Breiman, “Random Forests,” Mach. Learn., vol. 45, no. 1, pp. 5–32, 2001, doi: 10.1023/A:1010933404324.
[18] T. Saito and M. Rehmsmeier, “The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets,” PLoS One, vol. 10, no. 3, 2015, doi: 10.1371/journal.pone.0118432.
[19] C. Rudin, “Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead,” Nat. Mach. Intell., vol. 1, no. 5, pp. 206–215, 2019, doi: 10.1038/s42256-019-0048-x.
[20] S. M. Lundberg et al., “From local explanations to global understanding with explainable AI for trees,” Nat. Mach. Intell., vol. 2, no. 1, pp. 56–67, 2020, doi: 10.1038/s42256-019-0138-9.
[21] M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why Should I Trust You?’: Explaining the Predictions of Any Classifier,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, in KDD ’16. New York, NY, USA: Association for Computing Machinery, 2016, pp. 1135–1144. doi: 10.1145/2939672.2939778.
[22] F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” J. Mach. Learn. Res., vol. 12, pp. 2825–2830, 2011.
[23] G. Varoquaux, “Cross-validation failure: Small sample sizes lead to large error bars,” Neuroimage, vol. 180, pp. 68–77, 2018, doi: 10.1016/j.neuroimage.2017.06.061.
[24] D. Chicco and G. Jurman, “The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation,” BMC Genomics, vol. 21, no. 1, 2020, doi: 10.1186/s12864-019-6413-7.
[25] X. Li, Q. Ding, and J.-Q. Sun, “Remaining useful life estimation in prognostics using deep convolution neural networks,” Reliab. Eng. Syst. Saf., vol. 172, pp. 1–11, 2018, doi: 10.1016/j.ress.2017.11.021.
[26] J. Davis and M. Goadrich, “The relationship between Precision-Recall and ROC curves,” in Proceedings of the 23rd International Conference on Machine Learning, 2006, pp. 233–240. doi: 10.1145/1143844.1143874.
[27] H. He and E. A. Garcia, “Learning from imbalanced data,” IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp. 1263–1284, 2009.