Serial KinderMiner (SKiM) discovers and annotates biomedical knowledge using co-occurrence and transformer models

BMC Bioinformatics

Table 4 Precision, recall, and F₁ score of the ML models used to build the knowledge graph

Class	Precision	Recall	F₁
GGP	0.947	0.881	0.913
BIO_PROCESS	0.833	0.669	0.742
DRUG	0.880	0.849	0.864
CHEMICAL	0.849	0.814	0.832
CONDITION	0.826	0.791	0.808
NER Micro-Average	0.885	0.814	0.848
NER Macro-Average	0.867	0.801	0.832
Weighted Average	0.893	0.834	0.862
POS_ASSOCIATION	0.733	0.487	0.585
REGULATES	0.829	0.447	0.581
COREF	0.865	0.533	0.660
MUTATION_AFFECTS	0.667	0.333	0.444
ACTIVATES	0.632	0.255	0.364
BINDS	0.667	1.000	0.800
INHIBITS	0.548	0.500	0.523
TREATS	0.417	0.455	0.435
NEG_ASSOCIATION	0.250	0.053	0.087
DRUG_ASSOCIATION_WITH	1.000	0.773	0.872
REL Micro-Average	0.777	0.428	0.552
REL Macro-Average	0.661	0.484	0.535
REL Weighted Average	0.722	0.462	0.555

The PubMedBERT model from Microsoft was fine-tuned on NER and RE tasks using spaCy. The recall of the RE model is likely low because of sparse data, resulting in an unbalanced training set (i.e., most entities do not have a relationship with each other)

ISSN: 1471-2105