Dynamic knowledge graph evaluation: Semantic and syntactic metrics for evaluating changes.
Saved in:
| Title: | Dynamic knowledge graph evaluation: Semantic and syntactic metrics for evaluating changes. |
|---|---|
| Authors: | Bakker, Roos M.1,2 (AUTHOR) roos.bakker@tno.nl, de Boer, Maaike H.T.1 (AUTHOR) |
| Source: | Data & Knowledge Engineering. Jul2026, Vol. 164, pN.PAG-N.PAG. 1p. |
| Subjects: | Knowledge graphs, Evaluation methodology, Knowledge representation (Information theory) |
| Abstract: | In a world where information is exchanged at an increasing pace, knowledge becomes quickly outdated. Formal constructs that capture human knowledge, such as knowledge graphs and ontologies, need to be updated and evaluated to stay relevant and functioning. However, evaluating knowledge models is labour-intensive and prone to errors. This study addresses the challenge of automatically evaluating changes in existing knowledge graphs. We introduce syntactic and semantic metrics tailored for change evaluation. The metrics are implemented and validated through experiments on knowledge graphs across various domains. In these experiments, real-world changes are simulated by removing concepts and introducing faulty ones before evaluation with the syntactic and semantic metrics. The hypothesis is that such changes decrease the quality aspects of the knowledge graph: removing concepts influences syntactic qualities such as the structure of the model, while adding faulty concepts affects semantic qualities like model consistency. The validation results support this hypothesis, demonstrating that the introduced metrics effectively reflect the intended performance differences when making changes to the graph. Additionally, the experiments show that the size and domain specialisation of a knowledge graph influence how well the metrics detect changes. Overall, this study proposes a novel set of evaluation metrics and provides evidence of their effectiveness for assessing modifications to knowledge graphs across different domains. These metrics can help developers detect errors, highlight unintended side effects, and flag other quality changes that might otherwise go unnoticed. • Evaluating changes in knowledge models is resource-intensive. • We introduce novel metrics to automatically assess such changes. • Metrics capture both structural and conceptual quality aspects. • Experiments across domains confirm the metrics' effectiveness. • Metrics support developers in detecting errors and side effects of changes. [ABSTRACT FROM AUTHOR] |
| Copyright of Data & Knowledge Engineering is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 194525328 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Dynamic knowledge graph evaluation: Semantic and syntactic metrics for evaluating changes. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Bakker%2C+Roos+M%2E%22">Bakker, Roos M.</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> roos.bakker@tno.nl</i><br /><searchLink fieldCode="AR" term="%22de+Boer%2C+Maaike+H%2ET%2E%22">de Boer, Maaike H.T.</searchLink><relatesTo>1</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Data+%26+Knowledge+Engineering%22">Data & Knowledge Engineering</searchLink>. Jul2026, Vol. 164, pN.PAG-N.PAG. 1p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Knowledge+graphs%22">Knowledge graphs</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+methodology%22">Evaluation methodology</searchLink><br /><searchLink fieldCode="DE" term="%22Knowledge+representation+%28Information+theory%29%22">Knowledge representation (Information theory)</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: In a world where information is exchanged at an increasing pace, knowledge becomes quickly outdated. Formal constructs that capture human knowledge, such as knowledge graphs and ontologies, need to be updated and evaluated to stay relevant and functioning. However, evaluating knowledge models is labour-intensive and prone to errors. This study addresses the challenge of automatically evaluating changes in existing knowledge graphs. We introduce syntactic and semantic metrics tailored for change evaluation. The metrics are implemented and validated through experiments on knowledge graphs across various domains. In these experiments, real-world changes are simulated by removing concepts and introducing faulty ones before evaluation with the syntactic and semantic metrics. The hypothesis is that such changes decrease the quality aspects of the knowledge graph: removing concepts influences syntactic qualities such as the structure of the model, while adding faulty concepts affects semantic qualities like model consistency. The validation results support this hypothesis, demonstrating that the introduced metrics effectively reflect the intended performance differences when making changes to the graph. Additionally, the experiments show that the size and domain specialisation of a knowledge graph influence how well the metrics detect changes. Overall, this study proposes a novel set of evaluation metrics and provides evidence of their effectiveness for assessing modifications to knowledge graphs across different domains. These metrics can help developers detect errors, highlight unintended side effects, and flag other quality changes that might otherwise go unnoticed. • Evaluating changes in knowledge models is resource-intensive. • We introduce novel metrics to automatically assess such changes. • Metrics capture both structural and conceptual quality aspects. • Experiments across domains confirm the metrics' effectiveness. • Metrics support developers in detecting errors and side effects of changes. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Data & Knowledge Engineering is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=194525328 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1016/j.datak.2026.102611 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 1 StartPage: N.PAG Subjects: – SubjectFull: Knowledge graphs Type: general – SubjectFull: Evaluation methodology Type: general – SubjectFull: Knowledge representation (Information theory) Type: general Titles: – TitleFull: Dynamic knowledge graph evaluation: Semantic and syntactic metrics for evaluating changes. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Bakker, Roos M. – PersonEntity: Name: NameFull: de Boer, Maaike H.T. IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 07 Text: Jul2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 0169023X Numbering: – Type: volume Value: 164 Titles: – TitleFull: Data & Knowledge Engineering Type: main |
| ResultId | 1 |
