 |
We produced embeddings from the SMS texts using TinyBERT in the feature extraction phase. Essential for differentiating between smishing (spam) and legal (ham) messages, TinyBERT, a lightweight and efficient variation of the BERT paradigm, detects contextual elements of the text. We project the high-dimensional TinyBERT embeddings into a 2D space using t-Distributed Stochastic Neighbour Embedding (t-SNE), a dimensionality reduction method, so visually evaluating the separability of these features.
Each point in the resulting t-SNE visualization denotes each SMS message, hence displaying clusters of data points. The colors signal the message labels; yellow denotes spam and purple denotes ham. The division of spam and ham clusters in some regions implies that the TinyBERT embeddings effectively capture important variations between the two classes, therefore offering a strong basis for our smishing detection system. This animation shows how well TinyBERT codes elements that might distinguish smishing efforts from official SMS material. |
IEEE/ICACT20250142 Slide.07
[Big Slide]