inputs¶
knowledge_graph.news.inputs
Build DocRED inputs from DWIE reference entities or predicted singleton mentions.
tokenize(text: str, begin: int, end: int) -> tuple[list[str], list[tuple[int, int]]]
¶
Source code in src/knowledge_graph/news/inputs.py
22 23 24 | |
sentence_of(mention: dict, sentences: list[dict]) -> int | None
¶
Source code in src/knowledge_graph/news/inputs.py
27 28 29 30 31 | |
evidence_sentences(subject_mentions: list[dict], object_mentions: list[dict]) -> list[int]
¶
Shared sentences, otherwise the nearest pair; break ties by subject then object sentence ID.
Source code in src/knowledge_graph/news/inputs.py
34 35 36 37 38 39 40 41 42 43 44 45 46 | |
to_docred(doc: dict) -> tuple[dict, list]
¶
Tokenize a document, align its mentions, and map entities to document concepts.
Source code in src/knowledge_graph/news/inputs.py
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | |
select_sentences(document: dict, sentence_ids, title: str) -> tuple[dict, list[int]]
¶
Select sentences from a DocRED document and return their surviving entity indices.
Source code in src/knowledge_graph/news/inputs.py
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 | |
subword_lengths(doc: dict, tokenizer) -> list[int]
¶
Subwords per DocRED sentence, including the entity markers used by the models.
Source code in src/knowledge_graph/news/inputs.py
108 109 110 111 | |
pack_sentences(sentence_ids: list[int], lengths: list[int], limit: int) -> list[list[int]]
¶
Consecutive sentences packed greedily up to limit subwords; a longer sentence stands alone.
Source code in src/knowledge_graph/news/inputs.py
114 115 116 117 118 119 120 121 122 123 | |
semantic_windows(units: list[list[int]], lengths: list[int]) -> list[list[int]]
¶
Pack consecutive segments into windows; each window repeats the previous window's last segment.
Source code in src/knowledge_graph/news/inputs.py
126 127 128 129 130 131 132 133 134 135 136 137 138 139 | |
sentence_groups(condition: str, partitions: dict, lengths: list[int]) -> list
¶
Sentence groups for one DocRED input condition.
Source code in src/knowledge_graph/news/inputs.py
157 158 159 160 161 162 163 164 165 166 167 168 169 | |
condition_inputs(documents: dict, condition: str, partitions: dict, lengths: dict) -> list[tuple]
¶
(document id, DocRED input, document concepts of its entities) of one condition; inputs need two entities.
Source code in src/knowledge_graph/news/inputs.py
172 173 174 175 176 177 178 179 180 | |
document_groups(documents, tokenizer) -> dict[str, set[str]]
¶
All documents, those longer than 1,024 subwords, and the others.
Source code in src/knowledge_graph/news/inputs.py
183 184 185 186 187 188 189 190 191 | |
write_conditions(out_dir, documents, conditions, partitions, tokenizer, gold_of=None)
¶
test_gold_of
maps the predicted concepts of each document to gold concepts (gold-entity inputs need none).
Source code in src/knowledge_graph/news/inputs.py
194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 | |
observable_relations(documents) -> set[tuple]
¶
Gold relations whose two distinct endpoints both have mentions.
Source code in src/knowledge_graph/news/inputs.py
211 212 213 214 215 216 217 218 219 220 221 | |
singleton_document(gold_doc: dict, spans: set[tuple]) -> dict
¶
The gold document with each predicted span inside a sentence as a singleton concept and no relations.
Source code in src/knowledge_graph/news/inputs.py
224 225 226 227 228 229 230 231 | |
gold_concept(gold_doc: dict, predicted_doc: dict) -> dict[int, int | None]
¶
Gold concept of each predicted mention with an exact, unambiguous span match (scoring only).
Source code in src/knowledge_graph/news/inputs.py
234 235 236 237 238 239 240 241 242 243 | |
gold_entities(args: argparse.Namespace) -> None
¶
Train and dev inputs and one test file per input condition, with gold entity clusters and DREEAM evidence.
Source code in src/knowledge_graph/news/inputs.py
246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 | |
spans_by_document(payload)
¶
(begin, end) entity spans of every document.
Source code in src/knowledge_graph/news/inputs.py
264 265 266 | |
predicted_mentions(args: argparse.Namespace) -> None
¶
Build singleton-mention inputs and exact-span gold mappings for scoring.
Source code in src/knowledge_graph/news/inputs.py
269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 | |
main() -> None
¶
Source code in src/knowledge_graph/news/inputs.py
287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 | |