model¶
news_recommendation.model
PLM-NR (Wu et al., 2021) with a BERT news encoder, adapted from https://github.com/wuch15/PLM4NewsRec.
Only the title encoder is kept: each input variant packs its text into the title field. Attribute names match the state-dict keys of the trained checkpoints, and modules are built in the upstream order, so a seeded run draws the same initial weights.
AdditiveAttention
¶
Bases: Module
Attention pooling over the second dimension.
Source code in src/news_recommendation/model.py
13 14 15 16 17 18 19 20 21 22 23 24 | |
att_fc1 = nn.Linear(dimension, hidden_size)
instance-attribute
¶
att_fc2 = nn.Linear(hidden_size, 1)
instance-attribute
¶
forward(x)
¶
Source code in src/news_recommendation/model.py
21 22 23 24 | |
MultiHeadAttention
¶
Bases: Module
Unmasked multi-head self-attention with exp-normalized scores, as upstream.
Source code in src/news_recommendation/model.py
27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | |
W_Q = nn.Linear(dimension, head_size * heads)
instance-attribute
¶
W_K = nn.Linear(dimension, head_size * heads)
instance-attribute
¶
W_V = nn.Linear(dimension, head_size * heads)
instance-attribute
¶
forward(x)
¶
Source code in src/news_recommendation/model.py
39 40 41 42 43 44 45 | |
TextEncoder
¶
Bases: Module
BERT token states, multi-head self-attention, and attention pooling of one packed text.
Source code in src/news_recommendation/model.py
48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | |
bert_model = bert_model
instance-attribute
¶
dropout_rate = config['drop_rate']
instance-attribute
¶
multihead_attention = MultiHeadAttention(config['word_embedding_dim'], config['num_attention_heads'], 20)
instance-attribute
¶
additive_attention = AdditiveAttention(config['num_attention_heads'] * 20, config['news_query_vector_dim'])
instance-attribute
¶
forward(text)
¶
text packs input ids, token type ids, and attention mask side by side.
Source code in src/news_recommendation/model.py
58 59 60 61 62 63 64 | |
NewsEncoder
¶
Bases: Module
Source code in src/news_recommendation/model.py
67 68 69 70 71 72 73 74 | |
text_encoders = nn.ModuleDict({'title': TextEncoder(bert_model, config)})
instance-attribute
¶
reduce_dim_linear = nn.Linear(config['num_attention_heads'] * 20, config['news_dim'])
instance-attribute
¶
forward(news)
¶
Source code in src/news_recommendation/model.py
73 74 | |
UserEncoder
¶
Bases: Module
Attention pooling over all history slots; padded slots hold the encoded all-zero news row.
Source code in src/news_recommendation/model.py
77 78 79 80 81 82 83 84 85 | |
ModelBert
¶
Bases: Module
Click-prediction loss over 1 + K candidates and the user's history.
Source code in src/news_recommendation/model.py
88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 | |
news_encoder = NewsEncoder(bert_model, config)
instance-attribute
¶
user_encoder = UserEncoder(config)
instance-attribute
¶
criterion = nn.CrossEntropyLoss()
instance-attribute
¶
forward(candidates, history, history_mask, targets)
¶
Source code in src/news_recommendation/model.py
98 99 100 101 102 103 104 105 106 107 108 109 110 | |