 |
Logistic regression (LR) model is a widely used algorithm for binary classification in statistical methods and ML contexts. The model is particularly popular in banking. It is favored for its easy interpretation and implementation [11]. LR transforms probability into odds and then applies a logarithmic function to these odds. This ensures that estimated probabilities range between zero and one. This process effectively removes both the lower and upper bounds of the probability [10]. Scholars have compared their proposed methods to LR. The findings of these studies demonstrate the power of LR as a benchmark in credit scoring [14, 23].
Decision tree (DT) algorithm incrementally builds a tree structure by splitting data into increasingly smaller subsets. It uses Chi-square, Gini-Index, and Information gain methods. DT with improved Chi-square feature selection performs especially well in terms of recall for multiclass text classification. [22]. The model follows if-then rules. When an input condition is met, it returns a decision, predicting the target value. If not, it carries out further analysis [24]
Random forest (RF) is an ensemble learning method. It operates by creating multiple decision trees during training and providing a class mode in classification or mean in regression of individual trees. [22]. In this way, RF works more than a single tree from the classification results perspective. However, while RF generally provides strong results, gradient-boosted trees often surpass RF in accuracy [13].
Synthetic Minority Over-sampling Technique for Nominal and Continuous variables (SMOTE-NC) is an oversampling technique [17]. SMOTE-ENN technique first applies SMOTE to generate synthetic samples for the minority class and then uses Edited Nearest Neighbors to clean the dataset by removing noisy samples [19].
To evaluate model performance, we use Confusion Matrix, Accuracy, Recall, Precision, F1- Score, ROC-AUC.
|
IEEE/ICACT20250136 Slide.09
[Big Slide]