View in EDS HTML Full Text PDF Full Text

Self-reported checklists and quality scoring tools in radiomics: a meta-research.

Saved in:
Bibliographic Details
Title: Self-reported checklists and quality scoring tools in radiomics: a meta-research.
Authors: Kocak, Burak1 (AUTHOR) drburakkocak@gmail.com, Akinci D'Antonoli, Tugba2 (AUTHOR), Ates Kus, Ece3 (AUTHOR), Keles, Ali1 (AUTHOR), Kala, Ahmet1 (AUTHOR), Kose, Fadime1 (AUTHOR), Kadioglu, Mehmet1 (AUTHOR), Solak, Sila1 (AUTHOR), Sunman, Seyma1 (AUTHOR), Temiz, Zisan Hayriye1 (AUTHOR)
Source: European Radiology. Aug2024, Vol. 34 Issue 8, p5028-5040. 13p.
Subjects: Radiomics, Computer-assisted image analysis (Medicine), Artificial intelligence, Reproducible research, Sample size (Statistics)
Abstract: Objective: To evaluate the use of reporting checklists and quality scoring tools for self-reporting purposes in radiomics literature. Methods: Literature search was conducted in PubMed (date, April 23, 2023). The radiomics literature was sampled at random after a sample size calculation with a priori power analysis. A systematic assessment for self-reporting, including the use of documentation such as completed checklists or quality scoring tools, was conducted in original research papers. These eligible papers underwent independent evaluation by a panel of nine readers, with three readers assigned to each paper. Automatic annotation was used to assist in this process. Then, a detailed item-by-item confirmation analysis was carried out on papers with checklist documentation, with independent evaluation of two readers. Results: The sample size calculation yielded 117 papers. Most of the included papers were retrospective (94%; 110/117), single-center (68%; 80/117), based on their private data (89%; 104/117), and lacked external validation (79%; 93/117). Only seven papers (6%) had at least one self-reported document (Radiomics Quality Score (RQS), Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD), or Checklist for Artificial Intelligence in Medical Imaging (CLAIM)), with a statistically significant binomial test (p < 0.001). Median rate of confirmed items for all three documents was 81% (interquartile range, 6). For quality scoring tools, documented scores were higher than suggested scores, with a mean difference of − 7.2 (standard deviation, 6.8). Conclusion: Radiomic publications often lack self-reported checklists or quality scoring tools. Even when such documents are provided, it is essential to be cautious, as the accuracy of the reported items or scores may be questionable. Clinical relevance statement: Current state of radiomic literature reveals a notable absence of self-reporting with documentation and inaccurate reporting practices. This critical observation may serve as a catalyst for motivating the radiomics community to adopt and utilize such tools appropriately, thereby fostering rigor, transparency, and reproducibility of their research, moving the field forward. Key Points: • In radiomics literature, there has been a notable absence of self-reporting with documentation. • Even if such documents are provided, it is critical to exercise caution because the accuracy of the reported items or scores may be questionable. • Radiomics community needs to be motivated to adopt and appropriately utilize the reporting checklists and quality scoring tools. [ABSTRACT FROM AUTHOR]
Copyright of European Radiology is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: egs
DbLabel: Engineering Source
An: 178483232
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Self-reported checklists and quality scoring tools in radiomics: a meta-research.
– Name: Author
  Label: Authors
  Group: Au
  Data: &lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Kocak%2C+Burak%22&quot;&gt;Kocak, Burak&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;i&gt; drburakkocak@gmail.com&lt;/i&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Akinci+D&#39;Antonoli%2C+Tugba%22&quot;&gt;Akinci D&#39;Antonoli, Tugba&lt;/searchLink&gt;&lt;relatesTo&gt;2&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Ates+Kus%2C+Ece%22&quot;&gt;Ates Kus, Ece&lt;/searchLink&gt;&lt;relatesTo&gt;3&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Keles%2C+Ali%22&quot;&gt;Keles, Ali&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Kala%2C+Ahmet%22&quot;&gt;Kala, Ahmet&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Kose%2C+Fadime%22&quot;&gt;Kose, Fadime&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Kadioglu%2C+Mehmet%22&quot;&gt;Kadioglu, Mehmet&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Solak%2C+Sila%22&quot;&gt;Solak, Sila&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Sunman%2C+Seyma%22&quot;&gt;Sunman, Seyma&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Temiz%2C+Zisan+Hayriye%22&quot;&gt;Temiz, Zisan Hayriye&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: &lt;searchLink fieldCode=&quot;JN&quot; term=&quot;%22European+Radiology%22&quot;&gt;European Radiology&lt;/searchLink&gt;. Aug2024, Vol. 34 Issue 8, p5028-5040. 13p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: &lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Radiomics%22&quot;&gt;Radiomics&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Computer-assisted+image+analysis+%28Medicine%29%22&quot;&gt;Computer-assisted image analysis (Medicine)&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Artificial+intelligence%22&quot;&gt;Artificial intelligence&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Reproducible+research%22&quot;&gt;Reproducible research&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Sample+size+%28Statistics%29%22&quot;&gt;Sample size (Statistics)&lt;/searchLink&gt;
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Objective: To evaluate the use of reporting checklists and quality scoring tools for self-reporting purposes in radiomics literature. Methods: Literature search was conducted in PubMed (date, April 23, 2023). The radiomics literature was sampled at random after a sample size calculation with a priori power analysis. A systematic assessment for self-reporting, including the use of documentation such as completed checklists or quality scoring tools, was conducted in original research papers. These eligible papers underwent independent evaluation by a panel of nine readers, with three readers assigned to each paper. Automatic annotation was used to assist in this process. Then, a detailed item-by-item confirmation analysis was carried out on papers with checklist documentation, with independent evaluation of two readers. Results: The sample size calculation yielded 117 papers. Most of the included papers were retrospective (94%; 110/117), single-center (68%; 80/117), based on their private data (89%; 104/117), and lacked external validation (79%; 93/117). Only seven papers (6%) had at least one self-reported document (Radiomics Quality Score (RQS), Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD), or Checklist for Artificial Intelligence in Medical Imaging (CLAIM)), with a statistically significant binomial test (p &lt; 0.001). Median rate of confirmed items for all three documents was 81% (interquartile range, 6). For quality scoring tools, documented scores were higher than suggested scores, with a mean difference of − 7.2 (standard deviation, 6.8). Conclusion: Radiomic publications often lack self-reported checklists or quality scoring tools. Even when such documents are provided, it is essential to be cautious, as the accuracy of the reported items or scores may be questionable. Clinical relevance statement: Current state of radiomic literature reveals a notable absence of self-reporting with documentation and inaccurate reporting practices. This critical observation may serve as a catalyst for motivating the radiomics community to adopt and utilize such tools appropriately, thereby fostering rigor, transparency, and reproducibility of their research, moving the field forward. Key Points: • In radiomics literature, there has been a notable absence of self-reporting with documentation. • Even if such documents are provided, it is critical to exercise caution because the accuracy of the reported items or scores may be questionable. • Radiomics community needs to be motivated to adopt and appropriately utilize the reporting checklists and quality scoring tools. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: &lt;i&gt;Copyright of European Radiology is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder&#39;s express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.&lt;/i&gt; (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=178483232
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s00330-023-10487-5
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 13
        StartPage: 5028
    Subjects:
      – SubjectFull: Radiomics
        Type: general
      – SubjectFull: Computer-assisted image analysis (Medicine)
        Type: general
      – SubjectFull: Artificial intelligence
        Type: general
      – SubjectFull: Reproducible research
        Type: general
      – SubjectFull: Sample size (Statistics)
        Type: general
    Titles:
      – TitleFull: Self-reported checklists and quality scoring tools in radiomics: a meta-research.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Kocak, Burak
      – PersonEntity:
          Name:
            NameFull: Akinci D'Antonoli, Tugba
      – PersonEntity:
          Name:
            NameFull: Ates Kus, Ece
      – PersonEntity:
          Name:
            NameFull: Keles, Ali
      – PersonEntity:
          Name:
            NameFull: Kala, Ahmet
      – PersonEntity:
          Name:
            NameFull: Kose, Fadime
      – PersonEntity:
          Name:
            NameFull: Kadioglu, Mehmet
      – PersonEntity:
          Name:
            NameFull: Solak, Sila
      – PersonEntity:
          Name:
            NameFull: Sunman, Seyma
      – PersonEntity:
          Name:
            NameFull: Temiz, Zisan Hayriye
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Text: Aug2024
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 09387994
          Numbering:
            – Type: volume
              Value: 34
            – Type: issue
              Value: 8
          Titles:
            – TitleFull: European Radiology
              Type: main
ResultId 1