 |
The goal of the Sequential Encoder is to capture how previous user behaviors influence subsequent actions by modeling item dependencies within the interaction sequence.
Because users have interaction histories of different lengths, all sequences are normalized to a fixed length L. Shorter sequences are padded at the beginning, while longer sequences keep only the most recent L items. Next, each item in the sequence is represented using the Hybrid Item Encoder. As shown in Equation (6), this step produces a unified embedding that combines ID-based and review-based information. Since the Transformer cannot model order by itself, positional information is explicitly added. In Equation (7), a positional embedding is added to each item representation to form a position-aware vector. Finally, as defined in Equation (8), these vectors are stacked to form the input sequence for the sequential encoder.
With the input sequence prepared, we now model sequential dependencies. HiSeR adopts a Transformer-based encoder to capture user sequential behavior. The input sequence is processed through a stack of Transformer encoder layers. Each layer consists of: Multi-Head Self-Attention, which learns dependencies among all items in the sequence, and
Feed-Forward Networks, which transform the extracted contextual features. Residual connections and layer normalization are applied to stabilize training and maintain information flow across layers. After passing through multiple layers, the model outputs contextualized item representations, as shown in Equation (9). Each representation reflects both the behavior at a specific position and the influence of the entire interaction history. Finally, we summarize the encoded sequence into a single user representation. As shown in Equation (10), the user embedding is obtained by averaging all contextualized vectors in the sequence. This mean-pooling strategy leverages the Transformer¡¯s ability to encode global information at every position, resulting in a stable representation that reflects the user¡¯s overall interaction history.
|
IEEE/ICACT20260275 Slide.08
[Big Slide]