View in EDS

Dynamic knowledge graph evaluation: Semantic and syntactic metrics for evaluating changes.

Saved in:

Bibliographic Details
Title:	Dynamic knowledge graph evaluation: Semantic and syntactic metrics for evaluating changes.
Authors:	Bakker, Roos M.^1,2 (AUTHOR) roos.bakker@tno.nl, de Boer, Maaike H.T.¹ (AUTHOR)
Source:	Data & Knowledge Engineering. Jul2026, Vol. 164, pN.PAG-N.PAG. 1p.
Subjects:	Knowledge graphs, Evaluation methodology, Knowledge representation (Information theory)
Abstract:	In a world where information is exchanged at an increasing pace, knowledge becomes quickly outdated. Formal constructs that capture human knowledge, such as knowledge graphs and ontologies, need to be updated and evaluated to stay relevant and functioning. However, evaluating knowledge models is labour-intensive and prone to errors. This study addresses the challenge of automatically evaluating changes in existing knowledge graphs. We introduce syntactic and semantic metrics tailored for change evaluation. The metrics are implemented and validated through experiments on knowledge graphs across various domains. In these experiments, real-world changes are simulated by removing concepts and introducing faulty ones before evaluation with the syntactic and semantic metrics. The hypothesis is that such changes decrease the quality aspects of the knowledge graph: removing concepts influences syntactic qualities such as the structure of the model, while adding faulty concepts affects semantic qualities like model consistency. The validation results support this hypothesis, demonstrating that the introduced metrics effectively reflect the intended performance differences when making changes to the graph. Additionally, the experiments show that the size and domain specialisation of a knowledge graph influence how well the metrics detect changes. Overall, this study proposes a novel set of evaluation metrics and provides evidence of their effectiveness for assessing modifications to knowledge graphs across different domains. These metrics can help developers detect errors, highlight unintended side effects, and flag other quality changes that might otherwise go unnoticed. • Evaluating changes in knowledge models is resource-intensive. • We introduce novel metrics to automatically assess such changes. • Metrics capture both structural and conceptual quality aspects. • Experiments across domains confirm the metrics' effectiveness. • Metrics support developers in detecting errors and side effects of changes. [ABSTRACT FROM AUTHOR]
	Copyright of Data & Knowledge Engineering is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Engineering Source

Description
Abstract:	In a world where information is exchanged at an increasing pace, knowledge becomes quickly outdated. Formal constructs that capture human knowledge, such as knowledge graphs and ontologies, need to be updated and evaluated to stay relevant and functioning. However, evaluating knowledge models is labour-intensive and prone to errors. This study addresses the challenge of automatically evaluating changes in existing knowledge graphs. We introduce syntactic and semantic metrics tailored for change evaluation. The metrics are implemented and validated through experiments on knowledge graphs across various domains. In these experiments, real-world changes are simulated by removing concepts and introducing faulty ones before evaluation with the syntactic and semantic metrics. The hypothesis is that such changes decrease the quality aspects of the knowledge graph: removing concepts influences syntactic qualities such as the structure of the model, while adding faulty concepts affects semantic qualities like model consistency. The validation results support this hypothesis, demonstrating that the introduced metrics effectively reflect the intended performance differences when making changes to the graph. Additionally, the experiments show that the size and domain specialisation of a knowledge graph influence how well the metrics detect changes. Overall, this study proposes a novel set of evaluation metrics and provides evidence of their effectiveness for assessing modifications to knowledge graphs across different domains. These metrics can help developers detect errors, highlight unintended side effects, and flag other quality changes that might otherwise go unnoticed. • Evaluating changes in knowledge models is resource-intensive. • We introduce novel metrics to automatically assess such changes. • Metrics capture both structural and conceptual quality aspects. • Experiments across domains confirm the metrics' effectiveness. • Metrics support developers in detecting errors and side effects of changes. [ABSTRACT FROM AUTHOR]
ISSN:	0169023X
DOI:	10.1016/j.datak.2026.102611