A Difficulty-aware Approach to Fair Classification on Imbalanced Datasets

Document Type : Original Article

Authors

Niloufar Kashefi

Javad Hamidzadeh

Mona Moradi

Computer Engineering, Faculty of computer engineering and information technology, Sadjad University, Mashhad, Iran

10.22067/cke.2025.91011.1137

Abstract

Class imbalance in real-world datasets often biases standard classifiers toward the majority class, degrading performance on the minority class. While existing methods like sample re-weighting can mitigate this, they may increase overall misclassification errors or fail to consider the difficulty of training instances. To address these shortcomings, we introduce a difficulty-aware classification framework based on multi-objective evolutionary optimization. Our approach uses a specialized fitness function to simultaneously optimize for minority-class recall and overall accuracy, guiding the selection of the most informative training samples. We quantify sample difficulty using a fuzzy approach, which then modulate class-specific weights to refine the classifier's decision boundary. Furthermore, we incorporate chaotic dynamic maps into the evolutionary operators to accelerate convergence and maintain population diversity. Evaluated on various UCI benchmark datasets with 10-fold cross-validation, our method improves minority-class performance on imbalanced data without compromising accuracy on balanced data. Comparative analysis using AUC, G-mean, and F-measure confirms our approach achieves a superior trade-off between minority-class detection and overall accuracy compared to state-of-the-art methods.

Keywords

Instance reduction

Fuzzy weighted average distance-based decision surface

Chaotic imperialist competitive algorithm

Reduction rate

Subjects

• Machine learning

---

References

[1] T. Xia, T. Dang, J. Han, L. Qendro, and C. Mascolo. (2024). Uncertainty-aware health diagnostics via class-balanced evidential deep learning. IEEE Journal of Biomedical and Health Informatics. [Online]. 28(11), pp. 6417–6428. Available: https://doi.org/10.1109/JBHI.2024.3360002

[2] Z. Nouri, V. Kiani, and H. Fadishei. (2024). Rarity updated ensemble with oversampling: An ensemble approach to classification of imbalanced data streams. Statistical Analysis and Data Mining: The ASA Data Science Journal. [Online]. 17(1), p. e11662. Available: https://doi.org/10.1002/sam.11662

[3] L. Ren, R. Hu, Y. Liu, D. Li, J. Wu, Y. Zang, and W. Hu. (2024). Improving fraud detection via imbalanced graph structure learning. Machine Learning. [Online]. 113(3), pp. 1069–1090. Available: https://doi.org/10.1007/s10994-023-06464-0

[4] S. Raza, M. Garg, D. J. Reji, S. R. Bashir, and C. Ding. (2024). NBias: A natural language processing framework for bias identification in text. Expert Systems with Applications. [Online]. 237, p. 121542. Available: https://doi.org/10.1016/j.eswa.2023.121542

[5] C. Q. Zhang, Y. Deng, M. Z. Chong, Z. W. Zhang, and Y. H. Tan. (2024). Entropy-based re-sampling method on SAR class imbalance target detection. ISPRS Journal of Photogrammetry and Remote Sensing. [Online]. 209, pp. 432–447. Available: https://doi.org/10.1016/j.isprsjprs.2024.02.019

[6] Q. Zhou and B. Sun. (2024). Adaptive K-means clustering based under-sampling methods to solve the class imbalance problem. Data and Information Management. [Online]. 8(3), p. 100064. Available: https://doi.org/10.1016/j.dim.2023.100064

[7] N. Thai-Nghe, Z. Gantner, and L. Schmidt-Thieme, “Cost-sensitive learning methods for imbalanced data,” in The 2010 International Joint Conference on Neural Networks (IJCNN), Barcelona, Spain, 2010, pp. 1–8. Available: https://doi.org/10.1109/IJCNN.2010.5596486

[8] M. Moradi and J. Hamidzadeh. (2024). Handling class imbalance and overlap with a hesitation-based instance selection method. Knowledge-Based Systems. [Online]. 294, p. 111745. Available: https://doi.org/10.1016/j.knosys.2024.111745

[9] J. Bi and C. Zhang. (2018). An empirical comparison on state-of-the-art multi-class imbalance learning algorithms and a new diversified ensemble learning scheme. Knowledge-Based Systems. [Online]. 158, pp. 81–93. Available: https://doi.org/10.1016/j.knosys.2018.05.037

[10] I. Czarnowski. (2022). Weighted ensemble with one-class classification and over-sampling and instance selection (WECOI): An approach for learning from imbalanced data streams. Journal of Computational Science. [Online]. 61, p. 101614. Available: https://doi.org/10.1016/j.jocs.2022.101614

[11] M. S. E. Shahabadi, H. Tabrizchi, M. K. Rafsanjani, B. B. Gupta, and F. Palmieri. (2021). A combination of clustering-based under-sampling with ensemble methods for solving imbalanced class problem in intelligent systems. Technological Forecasting and Social Change. [Online]. 169, p. 120796. Available: https://doi.org/10.1016/j.techfore.2021.120796

[12] B. B. Hazarika and D. Gupta. (2021). Density-weighted support vector machines for binary class imbalance learning. Neural Computing and Applications. [Online]. 33(9), pp. 4243–4261. Available: https://doi.org/10.1007/s00521-020-05240-8

[13] B. B. Hazarika and D. Gupta. (2023). Affinity-based fuzzy kernel ridge regression classifier for binary class imbalance learning. Engineering Applications of Artificial Intelligence. [Online]. 117, p. 105544. Available: https://doi.org/10.1016/j.engappai.2022.105544

[14] B. B. Hazarika, D. Gupta, and P. Borah. (2023). Fuzzy twin support vector machine based on affinity and class probability for class imbalance learning. Knowledge and Information Systems. [Online]. 65(12), pp. 5259–5288. Available: https://doi.org/10.1007/s10115-023-01904-8

[15] S. Guan, X. Zhao, Y. Xue, and H. Pan. (2024). AWGAN: An adaptive weighting GAN approach for oversampling imbalanced datasets. Information Sciences. [Online]. 663, p. 120311. Available: https://doi.org/10.1016/j.ins.2024.120311

[16] Z. Wang and H. Wang, “Variational imbalanced regression: Fair uncertainty quantification via probabilistic smoothing,” Advances in Neural Information Processing Systems, vol. 36, 2024.

[17] S. Tao, P. Peng, Y. Li, H. Sun, Q. Li, and H. Wang. (2024). Supervised contrastive representation learning with tree-structured Parzen estimator Bayesian optimization for imbalanced tabular data. Expert Systems with Applications. [Online]. 237, p. 121294. Available: https://doi.org/10.1016/j.eswa.2023.121294

[18] C. Ye, R. Tsuchida, L. Petersson, and N. Barnes, “Label shift estimation for class-imbalance problem: A Bayesian approach,” in Proceeding IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024, pp. 1073–1082.

[19] A. Kumari, M. Tanveer, C. T. Lin, and Alzheimer’s Disease Neuroimaging Initiative. (2024). Class probability and generalized bell fuzzy twin SVM for imbalanced data. IEEE Transactions on Fuzzy Systems. [Online]. 32(5), pp. 3037–3048. Available: https://doi.org/10.1109/TFUZZ.2024.3366936

[20] Z. Deng, P. Cui, and J. Zhu, “Towards accelerated model training via Bayesian data selection,” in Advances in Neural Information Processing Systems, vol. 36, 2023, pp. 8513–8527.

[21] Y. Pu, W. Yao, and X. Li. (2024). EM-IFCM: Fuzzy c-means clustering algorithm based on edge modification for imbalanced data. Information Sciences. [Online]. 659, p. 120029. Available: https://doi.org/10.1016/j.ins.2023.120029

[22] S. Bano, W. Zhi, B. Qiu, M. Raza, N. Sehito, M. M. Kamal, G. Aldehim, and N. Alruwais. (2024). Self-paced ensemble and big data identification: A classification of substantial imbalance computational analysis. The Journal of Supercomputing. [Online]. 80(7), pp. 9848–9869. Available: https://doi.org/10.1007/s11227-023-05828-6

[23] N. Garcia-Pedrajas, J. M. Cuevas-Munoz, and A. de Haro-Garcia. (2024). Evolutionary simultaneous under- and oversampling of instances for dealing with class-imbalance datasets in multilabel problems. Applied Soft Computing. [Online]. 159, p. 111618. Available: https://doi.org/10.1016/j.asoc.2024.111618

[24] X. Lu, X. Ye, and Y. Cheng. (2024). An overlapping minimization-based over-sampling algorithm for binary imbalanced classification. Engineering Applications of Artificial Intelligence. [Online]. 133, p. 108107. Available: https://doi.org/10.1016/j.engappai.2024.108107

[25] Y. Liu, L. Zhu, L. Ding, H. Sui, and W. Shang. (2024). A hybrid sampling method for highly imbalanced and overlapped data classification with complex distribution. Information Sciences. [Online]. 661, p. 120117. Available: https://doi.org/10.1016/j.ins.2023.120117

[26] T. Ma, S. Lu, and C. Jiang. (2024). A membership-based resampling and cleaning algorithm for multi-class imbalanced overlapping data. Expert Systems with Applications. [Online]. 240, p. 122565. Available: https://doi.org/10.1016/j.eswa.2023.122565

[27] A. Kumar, D. Singh, and R. Shankar Yadav. (2024). Class overlap handling methods in imbalanced domain: A comprehensive survey. Multimedia Tools and Applications. [Online]. 83(23), pp. 63243–63290. Available: https://doi.org/10.1007/s11042-023-17864-8

[28] Z. Sun, W. Ying, W. Zhang, and S. Gong. (2024). Undersampling method based on minority class density for imbalanced data. Expert Systems with Applications. [Online]. 249, p. 123328. Available: https://doi.org/10.1016/j.eswa.2024.123328

[29] H. Yoon, J. Kim, and S. Lee, “Adaptive bandwidth kernel density estimation for imbalanced classification,” Pattern Recognition Letters, vol. 139, pp. 12–20, 2020.

[30] Miraj, Mahabubur Rahman, et al. "GK-SMOTE: A Hyperparameter-Free Noise-Resilient Gaussian KDE-Based Oversampling Approach." Asia-Pacific Web (APWeb) and Web-Age Information Management (WAIM) Joint International Conference on Web and Big Data. Singapore: Springer Nature Singapore, (2025).

[31] Lee, Daeun, and Hyunjoong Kim. "Adaptive oversampling via density estimation for online imbalanced classification." Information 16.1 (2025): 23. Available: https://doi.org/10.3390/info16010023.

[32] Taccaliti, Edoardo, and Jesus S. Aguilar–Ruiz. "Improving classification on imbalanced genomic data via KDE–based synthetic sampling." BioData Mining 18.1 (2025): 60. Available: https://doi.org/10.1186/s13040-025-00474-5

[33] Li, Xinqi, and Qicheng Liu. "A hybrid sampling algorithm for imbalanced and class-overlap data based on natural neighbors and density estimation." Knowledge and Information Systems 67.3 (2025): 2259-2290.

[34] C. Chen, An empirical study of adaptive kernel density estimation in detecting distributional overlap. Ph.D. dissertation, Delft University of Technology, 2023.

[35] B. Turkoglu. (2022). KDEBO: A kernel density estimation‑guided differential evolution‑based oversampling technique for imbalanced learning. Electronics. [Online]. 11(17), p. 2703.

[36] F. Kamalov, S. Moussa, and J. A. Reyes. (2022). KDE‑based ensemble learning for imbalanced data. Electronics. [Online]. 11(17), p. 2703. Available: https://doi.org/10.3390/electronics11172703

[37] F. Kamalov and A. Elnagar, “Kernel density estimation-based sampling for neural network classification,” in Proceeding 2021 International Symposium on Networks, Computers and Communications (ISNCC), 2021, pp. 1–6. Available: https://doi.org/10.1109/ISNCC52172.2021.9615715

[38] F. Kamalov. (2020). Kernel density estimation based sampling for imbalanced class distribution. Information Sciences. [Online]. 512, pp. 1192–1201. Available: https://doi.org/10.1016/j.ins.2019.10.017

[39] Y. C. Wang and C. H. Cheng. (2021). A multiple combined method for rebalancing medical data with class imbalances. Computers in Biology and Medicine. [Online]. 134, p. 104527. Available: https://doi.org/10.1016/j.compbiomed.2021.104527

[40] L. Yu, R. Zhou, L. Tang, and R. Chen. (2018). A DBN‑based resampling SVM ensemble learning paradigm for credit classification with imbalanced data. Applied Soft Computing. [Online]. 69, pp. 192–202. Available: https://doi.org/10.1016/j.asoc.2018.04.049

[41] H. Cui, Q. Wang, Y. Ye, Y. Tang, and Z. Lin. (2022). A combinational transfer learning framework for online transient stability prediction. Sustainable Energy, Grids and Networks. [Online]. 30, p. 100674. Available: https://doi.org/10.1016/j.segan.2022.100674

[42] B. Hu, Y. Liu, N. Chen, L. Wang, N. Liu, and X. Cao. (2022). SEGCN‑DCR: A syntax‑enhanced event detection framework with decoupled classification rebalance. Neurocomputing. [Online]. 481, pp. 55–66. Available: https://doi.org/10.1016/j.neucom.2022.01.069

[43] H. He and E. A. Garcia. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering. [Online]. 21(9), pp. 1263–1284. Available: https://doi.org/10.1109/TKDE.2008.239

[44] J. Liu, Y. Sun, C. Han, Z. Dou, and W. Li, “Deep representation learning on long-tailed data: A learnable embedding augmentation perspective,”  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 2970–2979.

[45] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” In Proceedings of the IEEE international conference on computer vision, 2017, pp. 2980–2988.

[46] J. Liu, F. Guo, H. Gao, Z. Huang, Y. Zhang, and H. Zhou. (2021). Image classification method on class imbalance datasets using multi‑scale CNN and two‑stage transfer learning. Neural Computing and Applications. [Online]. 33(21), pp. 14179–14197. Available: https://doi.org/10.1007/s00521-021-06066-8

[47] Z. Kang, Y. Zhang, J. Zhang, C. Li, M. Kong, Y. Zhao, and X. B. Wu. (2023). Periodic variable star classification with deep learning: Handling data imbalance in an ensemble augmentation way. Publications of the Astronomical Society of the Pacific. [Online]. 135(1051), p. 094501. Available: https://doi.org/10.1088/1538-3873/acf15e

[48] L. Xiang, G. Ding, and J. Han, “Learning from multiple experts: Self-paced knowledge distillation for long-tailed classification,” in Proceeding Springer, 2020, pp. 247–263. Available: https://doi.org/10.1007/978-3-030-58558-7_15

[49] X. Wang, L. Lian, Z. Miao, Z. Liu, and S. X. Yu. (2020). Long‑tailed recognition by routing diverse distribution‑aware experts. arXiv preprint. [Online]. arXiv:2010.01809. Available: https://doi.org/10.48550/arXiv.2010.01809 

[50] Y. Chen, X. Yang, and H. L. Dai. (2024). Cost‑sensitive continuous ensemble kernel learning for imbalanced data streams with concept drift. Knowledge‑Based Systems. [Online]. 284, p. 111272. Available: https://doi.org/10.1016/j.knosys.2023.111272

[51] E. Atashpaz-Gargari and C. Lucas, “Imperialist competitive algorithm: An algorithm for optimization inspired by imperialistic competition,” in IEEE congress on evolutionary computation, 2007, pp. 4661–4667. Available: https://doi.org/10.1109/CEC.2007.4425083

[52] S. Talatahari, B. F. Azar, R. Sheikholeslami, and A. Gandomi. (2012). Imperialist competitive algorithm combined with chaos for global optimization. Communications in Nonlinear Science and Numerical Simulation. [Online]. 17(3), pp. 1312–1319. Available: https://doi.org/10.1016/j.cnsns.2011.08.021

[53] V. H. Moghaddam and J. Hamidzadeh. (2016). New Hermite orthogonal polynomial kernel and combined kernels in support vector machine classifier. Pattern Recognition. [Online]. 60, pp. 921–935. Available: https://doi.org/10.1016/j.patcog.2016.07.004

[54] W. Pei et al. (2024). EvoSampling: A granular ball‑based evolutionary hybrid sampling with knowledge transfer for imbalanced learning. arXiv preprint. [Online]. arXiv:2412.10461. Available: https://doi.org/10.48550/arXiv.2412.10461

[55] M. A. Ganaie et al. (2024). Graph embedded intuitionistic fuzzy random vector functional link neural network for class imbalance learning. IEEE Transactions on Neural Networks and Learning Systems. [Online]. 35(9), pp. 11671–11680. Available: https://doi.org/10.1109/TNNLS.2024.3353531

[56] R. Zhang et al. (2023). A density‑based oversampling approach for class imbalance and data overlap. Computers & Industrial Engineering. [Online]. 186, p. 109747. Available: https://doi.org/10.1016/j.cie.2023.109747

[57] X. Tao et al. (2022). SVDD‑based weighted oversampling technique for imbalanced and overlapped dataset learning. Information Sciences. [Online]. 588, pp. 13–51. Available: https://doi.org/10.1016/j.ins.2021.12.066

[58] Z. Liu, Z. Li, Z. Yang, et al. (2025). CLIMB: Class‑imbalanced learning benchmark on tabular data. arXiv preprint. [Online]. arXiv:2505.17451. Available: https://doi.org/10.48550/arXiv.2505.17451

[59] T. Akiba et al., “Optuna: A next-generation hyperparameter optimization framework,” in Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2019. https://doi.org/10.1145/3292500.3330701

[60] D. Wilson. (1972). Asymptotic properties of nearest neighbor rules using edited data. IEEE Transactions on Systems, Man, and Cybernetics. [Online]. SMC‑2(3), pp. 408–421. Available: https://doi.org/10.1109/TSMC.1972.4309137

[61] Smith, Michael R., Tony Martinez, and Christophe Giraud-Carrier. "An instance level analysis of data complexity." Machine learning 95.2 (2014): 225-256. Available: https://doi.org/10.1007/s10994-013-5422-z

[62] Liu, Xu-Ying, Jianxin Wu, and Zhi-Hua Zhou. "Exploratory undersampling for class-imbalance learning." IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 39.2 (2008): 539-550. Available: https://doi.org/10.1109/TSMCB.2008.2007853.

[63] H. He, Y. Bai, E. A. Garcia, and S. Li, “ADASYN: Adaptive synthetic sampling approach for imbalanced learning,” in Proceedings of the International Joint Conference on Neural Networks, 2008, pp. 1322–1328. Available: https://doi.org/10.1109/IJCNN.2008.4633969

[64] Wang, Shuo, and Xin Yao. "Diversity analysis on imbalanced data sets by using ensemble models." 2009 IEEE symposium on computational intelligence and data mining. IEEE, 2009. Available: https://doi.org/10.1109/CIDM.2009.4938667.

[65] P. Viola and M. Jones, “Fast and robust classification using asymmetric AdaBoost,” in Neural Information Processing Systems (NeurIPS), 2001.