 |
In the feature extraction of Latent Dirichlet Allocation (LDA), two predominant techniques are utilized: the Bag of Words (BoW) model and the Term Frequency-Inverse Document Frequency (TF-IDF) transformation. These methods are instrumental in uncovering underlying structures and enhancing insights from text data. BoW models each document as a vector, quantifying the frequency of words within it. This enables LDA to translate textual data into numerical features, facilitating the identification of underlying topics and their distribution across documents. TF-IDF augments feature representation quality, building upon the BoW model. It assigns weights to words based on their relevance in individual documents and the entire corpus, balancing word significance. This step, following BoW, produces a more informative and distinct feature matrix, aiding LDA in assigning importance to words unique to specific documents or topics. By doing so, it diminishes the weight of common words. |
IEEE/ICACT20240447 Slide.10
[Big Slide]