 |
Besides heuristic features, we also incorporate lightweight language-model signals.
we use three pretrained models: BERT, RoBERTa, and XLNet.
In simple terms, BERT captures context from both directions.
RoBERTa is an improved version of BERT, trained with more data and more stable training strategies.
XLNet uses a permutation-based objective to better model sequence dependencies.
Although these models differ in their designs, in our system they all play the same role. They are used only to generate sentence embeddings.
These embeddings help us identify semantically meaningful sentences, rather than directly generating summaries. |
IEEE/ICACT20260145 Slide.10
[Big Slide]