 |
1) Extreme Gradient Boosting (XGBoost): Extreme Gradient
Boosting (XGBoost) [30] is an efficient open-source
project implementation of the gradient boosting algorithm.
Gradient boosting is a class of ensemble machine learning
used extensively for predictive analytics and work very well
for unstructured data. In general, the ensemble is constructed
from the addition of decision tree models one at a time to
reduce prediction error using a gradient descent optimization
algorithm. The XGBoost optimizes gradient boosting by effective
parallelization, hardware optimization, and regularization.
The algorithm is highly scalable with higher computational
speed on memory-constrained systems.
2) Tab Transformer: [31] The Transformer is widely
used for contextual modeling in Natural Language Processing
(NLP). The Tab Transformer is the adaptation of the
Transformer for tabular data. The Tab Transformer utilizes a
sequence of multi-head attention-based Transformer layers to
take input and convert it into contextual embeddings to bridge
the gap between baseline MLP and GBDT models. The highly
correlated features in the same column and cross column result
in an embedding vector close to euclidean distance. There is
no such relationship that exists in context-free embedding. The
Tab Transformer is highly robust to the random missing and
noisy data.
|
IEEE/ICACT20220078 Slide.13
[Big Slide]