View in EDS HTML Full Text PDF Full Text

Automation Tools to Support Undertaking Scoping Reviews

Saved in:
Bibliographic Details
Title: Automation Tools to Support Undertaking Scoping Reviews
Language: English
Authors: Hanan Khalil (ORCID 0000-0002-3302-2009), Danielle Pollock (ORCID 0000-0002-6604-0609), Patricia McInerney, Catrin Evans, Erica B. Moraes, Christina M. Godfrey, Lyndsay Alexander, Andrea Tricco, Micah D. J. Peters, Dawid Pieper (ORCID 0000-0002-0715-5182), Ashrita Saran (ORCID 0000-0003-3261-6654), Daniel Ameen, Petek Eylul Taneri, Zachary Munn
Source: Research Synthesis Methods. 2024 15(6):839-850.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 12
Publication Date: 2024
Document Type: Journal Articles
Reports - Descriptive
Descriptors: Automation, Computer Software, Synthesis, Protocol Analysis, Team Training, Journal Articles, Literature Reviews, Online Searching, Duplication, Data Processing
DOI: 10.1002/jrsm.1731
ISSN: 1759-2879
1759-2887
Abstract: Objective: This paper describes several automation tools and software that can be considered during evidence synthesis projects and provides guidance for their integration in the conduct of scoping reviews. Study Design and Setting: The guidance presented in this work is adapted from the results of a scoping review and consultations with the JBI Scoping Review Methodology group. Results: This paper describes several reliable, validated automation tools and software that can be used to enhance the conduct of scoping reviews. Developments in the automation of systematic reviews, and more recently scoping reviews, are continuously evolving. We detail several helpful tools in order of the key steps recommended by the JBI's methodological guidance for undertaking scoping reviews including team establishment, protocol development, searching, de-duplication, screening titles and abstracts, data extraction, data charting, and report writing. While we include several reliable tools and software that can be used for the automation of scoping reviews, there are some limitations to the tools mentioned. For example, some are available in English only and their lack of integration with other tools results in limited interoperability. Conclusion: This paper highlighted several useful automation tools and software programs to use in undertaking each step of a scoping review. This guidance has the potential to inform collaborative efforts aiming at the development of evidence informed, integrated automation tools and software packages for enhancing the conduct of high-quality scoping reviews.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1447299
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGJHGlQ_GDxAIJkZnPnAJMbAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDE35XBURnr2wlZColAIBEICBmokqdPeZBRv-zr8LO3T52pYDUzUK8v11-wBKtxRe3Yv3YQOOThjpKxpu3aQHXIfQoULqfa2jC-BSNFApnew2qKZYi_SaCsXzZnojn37IQ4hM0W-KtVu6htXomZ6ePaC3_r7yTyLxvKLaNnNt0qzFVq5S54qZkoWslvzUGePTMP44Qc5ZmHzXaeKNcBOUmQY38qork4i2_mWf-Ps=
Text:
  Availability: 1
  Value: <anid>AN0180736596;[bdct]01nov.24;2024Nov11.04:50;v2.2.500</anid> <title id="AN0180736596-1">Automation tools to support undertaking scoping reviews </title> <p>Objective: This paper describes several automation tools and software that can be considered during evidence synthesis projects and provides guidance for their integration in the conduct of scoping reviews. Study Design and Setting: The guidance presented in this work is adapted from the results of a scoping review and consultations with the JBI Scoping Review Methodology group. Results: This paper describes several reliable, validated automation tools and software that can be used to enhance the conduct of scoping reviews. Developments in the automation of systematic reviews, and more recently scoping reviews, are continuously evolving. We detail several helpful tools in order of the key steps recommended by the JBI's methodological guidance for undertaking scoping reviews including team establishment, protocol development, searching, de‐duplication, screening titles and abstracts, data extraction, data charting, and report writing. While we include several reliable tools and software that can be used for the automation of scoping reviews, there are some limitations to the tools mentioned. For example, some are available in English only and their lack of integration with other tools results in limited interoperability. Conclusion: This paper highlighted several useful automation tools and software programs to use in undertaking each step of a scoping review. This guidance has the potential to inform collaborative efforts aiming at the development of evidence informed, integrated automation tools and software packages for enhancing the conduct of high‐quality scoping reviews.</p> <p>Keywords: automation tools; guidance; methodology; quality; scoping reviews</p> <hd id="AN0180736596-2">Highlights</hd> <p></p> <hd id="AN0180736596-3">What is already known?</hd> <p></p> <ulist> <item> The International Collaboration for Automation of Systematic Reviews (ICASR) has put forward a set of principles for researchers to use automation for systematic reviews highlighting that automation has the potential to facilitate the production of systematic reviews that adhere to high standards for the reporting, conduct, and updating of rigorous reviews.</item> <p></p> <item> As scoping reviews have broader research questions and include a variety of evidence sources such as primary studies, reviews, editorials, etc. specific guidance is required for undertaking them using automation tools.</item> </ulist> <hd id="AN0180736596-4">What is new?</hd> <p></p> <ulist> <item> This paper describes several reliable and validated automation tools that can be used to undertake scoping reviews.</item> <p></p> <item> Automation of systematic and scoping reviews are continuously evolving, and further work is still being undertaken in this area.</item> <p></p> <item> This guidance constitutes an important step in supporting reviewers to undertake scoping reviews by integrating both automation tools and available software with human capabilities.</item> </ulist> <hd id="AN0180736596-5">Potential impact for Research Synthesis Methods readers</hd> <p></p> <ulist> <item> This guidance describes various tools and software that can facilitate the process of undertaking scoping reviews.</item> <p></p> <item> The main limitations of these tools are their lack of availability in languages other than English and lack of direct comparability and limited integration with one another which makes the process of undertaking reviews challenging.</item> </ulist> <hd id="AN0180736596-6">INTRODUCTION</hd> <p>Scoping reviews are an increasingly common type of evidence synthesis. To date, there are more than 465,000 results in Google Scholar with the term 'scoping review' in the title.[[<reflink idref="bib1" id="ref1">1</reflink>]] Their use has increased as a result of several factors, including the existence of robust methodological guidance, a standardised reporting checklist, and their utility in scoping the breadth of literature available in a field.[[<reflink idref="bib3" id="ref2">3</reflink>], [<reflink idref="bib5" id="ref3">5</reflink>]] They also served a unique role during the COVID‐19 pandemic for mapping the available evidence and helping to inform researchers regarding what further research needed to be conducted.[<reflink idref="bib6" id="ref4">6</reflink>]</p> <p>Scoping reviews have broad research questions and include a variety of evidence sources.[<reflink idref="bib7" id="ref5">7</reflink>] As such, they can potentially include a variety of study types in addition to non‐traditional forms of evidence (when appropriate), such as websites, guidelines, handbooks, reports, evaluations, editorials and commentaries, amongst other information sources. Even with a clear review question and robust search strategy, scoping reviews may be large, including up to 2000 studies.[[<reflink idref="bib8" id="ref6">8</reflink>]] A large number of included studies presents several challenges for reviewers regarding human resources, time constraints, and the speed at which a review can be completed and disseminated. The ongoing challenge is to balance rigorous and robust scoping review methods against the potential volume of evidence. The use of machine technologies to reduce human effort and therefore reduce the overall time taken in systematic reviews continues to grow.[<reflink idref="bib10" id="ref7">10</reflink>] Scoping reviews are promising candidates for utilising review automation approaches and software tools to enable a faster and timely process.</p> <p>We define automation as computerised systems that require 'intelligence' to perform tasks. This could be independent of human interaction (often termed deep learning or unsupervised learning), or supervised learning, where the computer uses training data to calibrate the tool. O'Connor et al. detail four levels of automation for human computer interactions.[<reflink idref="bib11" id="ref8">11</reflink>] These include Level 1 which includes tools to improve the file management process such as citation databases, reference management software and systematic review management software. Level 2 includes tools to enable workflow prioritisation such as prioritisation of relevant abstracts. Level 3 includes tools used to perform a task automatically but unreliably and require human supervision or otherwise provide the option to override the tools decisions manually. Finally, Level 4 includes tools that eliminate the need for human participation altogether.[<reflink idref="bib12" id="ref9">12</reflink>]</p> <p>The adoption of automation tools for systematic reviews has been relatively slow. There are several reasons for this, including: the fact that very few tools have data on their validity and reliability available, licencing issues or costs, the steep learning curve in using the tools, lack of support for some freely available tools, and lack of integration of some tools used into one streamlined platform.[[<reflink idref="bib13" id="ref10">13</reflink>], [<reflink idref="bib15" id="ref11">15</reflink>]] A survey of researchers using automated tools to conduct their systematic reviews found that the most common tools used were Endnote, review manager, Rayyan and EPPI‐reviewer.[[<reflink idref="bib12" id="ref12">12</reflink>], [<reflink idref="bib16" id="ref13">16</reflink>]] Other less popular tools included Abstrackr, Epistemonikos method of searching, SWIFT‐Review, RevMan HAL and RobotReviewer.[[<reflink idref="bib17" id="ref14">17</reflink>]] The authors of the survey suggested that the lack of uptake of some of the less commonly used tools was due to lack of awareness of these tools rather than due to perceptions of lower quality or utility.[<reflink idref="bib12" id="ref15">12</reflink>]</p> <p>The International Collaboration for Automation of Systematic Reviews (ICASR) has put forward a set of principles for researchers to use automation in systematic reviews.[<reflink idref="bib12" id="ref16">12</reflink>] The collaboration highlighted that automation has the potential to facilitate the production of systematic reviews that adhere to high standards for the reporting, conduct, and updating of rigorous reviews. They also advocate for sharing all automation techniques by making code, evaluation data, and corpora freely available.[<reflink idref="bib19" id="ref17">19</reflink>] More importantly, the collaboration encourage researchers to focus on approaches that could integrate existing tools into one platform rather than having a variety of tools for specific steps or for a particular discipline.[<reflink idref="bib12" id="ref18">12</reflink>] For example, having a platform connecting all of the smaller tools into a more seamless package for the conduct of reviews.</p> <p>The idea of automating some steps of systematic reviews was proposed more than a decade ago and is evolving at a rapid pace.[<reflink idref="bib11" id="ref19">11</reflink>] This paper discusses the current state of automation and semi‐automation in the field of scoping reviews now that both scoping reviews and automation are becoming more common. Specifically, it outlines key automation approaches and existent software and how they can be used for each step of the scoping review process, discussing the relative strengths and weaknesses of each different approach.</p> <p>Several authors have published work associated with automation of systematic reviews.[[<reflink idref="bib12" id="ref20">12</reflink>], [<reflink idref="bib20" id="ref21">20</reflink>]] However, to date, there is no guidance for conducting scoping reviews using automation tools or existing software to facilitate the process of scoping reviews. Scoping reviews differ from systematic reviews in many ways including their research questions, methodology, data analysis and reporting.[[<reflink idref="bib3" id="ref22">3</reflink>], [<reflink idref="bib22" id="ref23">22</reflink>]] Therefore, clear guidance on how to use automation for each step of scoping reviews is warranted.</p> <hd id="AN0180736596-7">AIM</hd> <p>This paper aims to provide guidance on the automation tools that are used for scoping reviews using the definition by O′ Connor, et al. and based on the experience of the authorship team.[<reflink idref="bib12" id="ref24">12</reflink>]</p> <hd id="AN0180736596-8">METHOD</hd> <p>The methodology underpinning the work presented in this paper has been published by Khalil et al.[<reflink idref="bib13" id="ref25">13</reflink>] Since then, the work has been updated to include research published until June 2023 and this paper includes the updated results. Briefly, a scoping review methodology was used to map all automation tools used for systematic reviews. The review included any automation tools that were used in the process of automation of any steps of the systematic review. The concept of interest was the automation process, and the context was systematic reviews.[<reflink idref="bib7" id="ref26">7</reflink>] This was followed by input from the JBI scoping review methodology group. Scoping reviews are protocol driven types of evidence synthesis, as opposed to literature reviews. We chose a scoping review methodology in order to map the tools that are currently used in the literature. We only included peer review literature as opposed to articles that have not been peer reviewed to ensure only high‐quality research has been included. Quality appraisal of included studies is not recommended for scoping reviews as per the JBI 2020 guidance on scoping reviews.</p> <p>JBI is a global research organisation that focusses on the development of methodological guidance.[<reflink idref="bib23" id="ref27">23</reflink>] The group includes methodologists, researchers and clinicians who are all evidence synthesis experts.[<reflink idref="bib4" id="ref28">4</reflink>] The guidance provided in this article is presented in the order of the key steps that are normally undertaken when conducting and reporting a scoping review in line with the JBI's methodological guidance.[[<reflink idref="bib7" id="ref29">7</reflink>], [<reflink idref="bib24" id="ref30">24</reflink>]] For each step, only validated and tested tools associated with high specificity, reliability, validity, area under the curve or F score and recall were included as detailed by Khalil et al.[<reflink idref="bib25" id="ref31">25</reflink>] The results addressing the aforementioned measures are extracted from recent published work addressing automation tools for systematic reviews.[<reflink idref="bib13" id="ref32">13</reflink>] Khalil et al. presented automation tools based on a scoping review of all the automation tools published from 47 citations. The same search was updated on June 13, 2023 using previously used databases (Medline, Embase, Global Heath and JBI database) and search terms (automation, method improvement, method acceleration, natural language processing, technology assessment, systematic review acceleration, knowledge synthesis and evidence synthesis).</p> <p>The tools included in this review were based on whether they were tested for validity and/ or reliability using different measures. The authors included any tools if they were either freely available or subscription based. In addition, only tools that were readily available and published with clear guidelines regarding their use were included. Tools from websites such as Systematic reviews toolbox (available at <ulink href="http://www.systematicreviewtools.com/">http://www.systematicreviewtools.com/</ulink>) were excluded unless they were included in peer‐reviewed publications. The scoping review protocol for this review was registered in Open Science Framework on the April 08, 2021 (retrieved from osf.io/9gzd7).</p> <hd id="AN0180736596-9">GUIDANCE</hd> <p>There was a total of 60 studies retrieved addressing automation tools describing 70 tools after searching for new updated on the June 13, 2024. Most of these tools were used for more than one step of the scoping review.[<reflink idref="bib13" id="ref33">13</reflink>] In this section, we will sequentially present the various steps of a scoping review and will consider the automation tools that can be used for each step (see Table 1). The section below addresses each step of the scoping review process and present all the relevant tools that could be used to perform each stage.</p> <p>1 TABLE Tasks of a scoping review and how automation tools could support their efficiency.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">Scoping review task</th><th align="left">Method used</th><th align="left">Suggested automation tools to increase efficiency or gain</th><th align="left">Level of automation</th><th align="left">Availability of the tool (open access/proprietary)</th></tr></thead><tbody valign="top"><tr><td align="left">Team establishment</td><td align="left">Personnel needed for the completion of the review</td><td align="left">Crowd sourcing or using task sharing platforms i.e., JBI SUMARI and Covidence.Management teams—Trello</td><td align="left">Level 1</td><td align="left">Proprietary</td></tr><tr><td align="left">Protocol development</td><td align="left">Templated reports of some report items</td><td align="left">JBI SUMARI and Revman</td><td align="left">Level 1</td><td align="left">Proprietary</td></tr><tr><td align="left">Searches</td><td align="left">Running searches on databases</td><td align="left">Automatic alerts from databases. Database (i.e., Cochrane, JBI and Campbell, Health Database Advanced Search (HDAS)) Automatic retrieval of full‐text papers (e.g., CrossRef), Polyglot Search TranslatorLitsuggest, eSuRFr</td><td align="left">Level 4</td><td align="left">CrossRef and Litsuggest (Open access)The rest of the tools are proprietary.</td></tr><tr><td align="left">Deduplication</td><td align="left">Deleting duplicates</td><td align="left">Endnote, Systematic Review Accelerator (SRA De‐duplicate), Abstrackr, Rayyan, EPPI‐Reviewer, Deduklick, Covidence and DistillerSR.</td><td align="left">Level 3</td><td align="left">Abstrackr (Open access)Rayyan (Open Access to early career researchers)The remainder of tools are proprietary.</td></tr><tr><td align="left">Screening</td><td align="left">Selecting studies for inclusion</td><td align="left">Crowdsourced inclusion decision and machine learning classifier (Abstrackr, Rayyan, BIBOT, EPPI‐Reviewer, ASReview and DistillerSR). Covidence is also used for this step.</td><td align="left">Level 2</td><td align="left">Abstrackr (Open access)Rayyan (Open Access to early career researchers)The remainder of tools are proprietary.</td></tr><tr><td align="left">Data extraction</td><td align="left">Extracting information on participants, Concept and context</td><td align="left">Machine learning‐information extraction systems such as DistillerSR and ExaCT), Tabulizer, UDPipe, Webplotdigitizer. For non‐machine learning but wanting a faster extraction process‐ Covidence</td><td align="left">Level 4</td><td align="left">UDPipe (open access) and the remainder of the tools are proprietary.</td></tr><tr><td align="left">Data charting</td><td align="left">Presentation of data</td><td align="left">GoogleCharts, Tableau Public, Google Data studio, Drawio, Eppi‐ reviewer, NVivo.</td><td align="left">Level 1</td><td align="left">Googlecharts, Google data studio and Drawio (open access). The remainder of the tools are proprietary.</td></tr><tr><td align="left">Report writing</td><td align="left">Templated reports of some report items</td><td align="left">JBI SUMARI, Revman and Robotreviewer</td><td align="left">Level 1</td><td align="left">Proprietary</td></tr></tbody></table> </ephtml> </p> <hd id="AN0180736596-10">Step 1: Team establishment</hd> <p>This is the first step of undertaking any review and it is important to choose well‐established software that can sustain collaboration and data input from several users despite geographical location. The chosen software should be available to the entire review team so that each member has access via their institution or ability to acquire an individual licence. JBI SUMARI, Covidence[<reflink idref="bib26" id="ref34">26</reflink>] and Revman, Distiller SR[<reflink idref="bib27" id="ref35">27</reflink>] are all suitable and user friendly tools that support reviews from the team establishment stage.[[<reflink idref="bib26" id="ref36">26</reflink>], [<reflink idref="bib28" id="ref37">28</reflink>]] JBI SUMARI was developed by JBI (https://<ulink href="http://www.jbisumari.org/">www.jbisumari.org/</ulink>) and supports the entire systematic review process for scoping reviews as well as several different types of systematic reviews. The software was developed through an agile software development approach, wide consultation, and user testing.[[<reflink idref="bib29" id="ref38">29</reflink>]]</p> <p>Covidence is another collaborative platform and was developed by an Australian company. In 2015, Cochrane initiated a collaboration with Covidence that made it the standard production platform for Cochrane reviews.[<reflink idref="bib26" id="ref39">26</reflink>] Covidence operates by subscription and many universities make it available to their staff and students. Covidence can be used by reviewers for both systematic and scoping reviews.[<reflink idref="bib26" id="ref40">26</reflink>]</p> <p>Another important consideration in the formation of the team is ensuring there are enough members to complete the various tasks in a scoping review within a reasonable timeframe. The use of crowdsourcing to establish larger review collaborations has been rising consistently in the areas of systematic reviews and could be beneficial for scoping reviews.[[<reflink idref="bib31" id="ref41">31</reflink>]] It has been used to reduce the workload for undertaking systematic reviews by enabling non‐experts to complete tasks. This strategy was shown not only to be efficient but also has benefits in cost reduction. Mortensen et al. showed that crowdsourcing was able to reduce the cost of using experts by 88%.[<reflink idref="bib31" id="ref42">31</reflink>] Crowdsourcing for systematic review tasks such as review citations and screening has been validated by Nama et al. The authors found that there was a high degree of accuracy for these tasks using crowdsourcing.[[<reflink idref="bib32" id="ref43">32</reflink>]]</p> <p>For researchers, managing multiple scoping reviews or vast numbers of included sources of evidence is challenging. Using team management tools like Trello Project Management Software is useful for organising review steps and roles for each reviewer.[<reflink idref="bib34" id="ref44">34</reflink>] This tool is useful for evidence synthesis in general and other types of reviews. For example, the users can visually organise projects into boards, divide projects into groups, and subdivide groups into tasks. It also allows integration with cloud files to be shared with the review team.[<reflink idref="bib34" id="ref45">34</reflink>] This automation feature follows the agile methodology of projects based on the organisation in three primary columns: the column of tasks to be performed in the future, tasks that are being performed in the present, and tasks that have already been completed.[<reflink idref="bib34" id="ref46">34</reflink>]</p> <hd id="AN0180736596-11">Step 2: Protocol development</hd> <p>The protocol development stage has been described in detail by Peters et al. and includes a proposed reporting checklist.[<reflink idref="bib35" id="ref47">35</reflink>] The guidance provides clear and transparent steps to follow to formulate the research question, for searching, data extraction and charting. JBI SUMARI may be used for this step as it includes pre‐populated text that can support with formulation of the title, research question, database searching, screening, and data extraction.[<reflink idref="bib28" id="ref48">28</reflink>]</p> <hd id="AN0180736596-12">Step 3: Searching</hd> <p>Searching the literature for sources to include in a scoping review should be comprehensive as the aim is to include all the available literature to comprehensively map the evidence. The JBI scoping review methodology recommends three stages of searching. Firstly, a general search of a database such as MEDLINE should identify relevant key terms for the research question. This is then followed by a second search using all the key words retrieved from the first search across all the relevant databases and grey literature sources identified for the scoping reviews. Finally, a search of the reference lists of all the included studies is also undertaken.[<reflink idref="bib36" id="ref49">36</reflink>] From previously published work of automation of scoping reviews, we have identified a range of validated tools that can be used to search the literature, these include Litsuggest (https://www.ncbi.nlm.nih.gov/research/litsuggest/).[<reflink idref="bib37" id="ref50">37</reflink>] This is a web‐based server that provides literature recommendation and curation services to help researchers stay up to date with the literature. Litsuggest was shown to have a relatively high precision, recall, and AUC.[<reflink idref="bib37" id="ref51">37</reflink>] However, it can only retrieve publications from Pubmed and is not suitable for non‐peer reviewed publications. As it relies on abstracts and not full texts, only articles with abstracts will be retrieved. Another useful tool that has been used by our team is Citation chaser.[<reflink idref="bib38" id="ref52">38</reflink>] The tool is very useful in obtaining lists of references from across studies.[<reflink idref="bib38" id="ref53">38</reflink>] It is able to undertake both forward citation which includes looking for all records citing one or more articles of known relevance and backward citation chasing looking for all records referenced in one or more articles.[[<reflink idref="bib6" id="ref54">6</reflink>], [<reflink idref="bib38" id="ref55">38</reflink>]] The two‐week systematic review is a concept supported by the Institute for Evidence Based Healthcare at Bond University.[<reflink idref="bib39" id="ref56">39</reflink>] They have developed a suite of tools that support the automation and rapid completion of systematic reviews, however, these tools can also be applied to scoping reviews. In relation to searching, both the Word Frequency Analyser and Polygot search translator are useful tools in the development and translation of search strings across multiple databases.[<reflink idref="bib40" id="ref57">40</reflink>] The Polyglot Search Translator (PST) performs the automatic translation of a search performed in a database to other databases, aligning descriptors, keywords and syntax.[<reflink idref="bib40" id="ref58">40</reflink>]</p> <p>eSuRFr‐snowball citation is another tool that can also be used for snowballing citations.[<reflink idref="bib13" id="ref59">13</reflink>] The tool was shown to be useful and accurate in obtaining the full texts and abstracts for a large number of scholarly citations in review articles. Using this method of snowballing, the time for searching relevant articles can be reduced.[[<reflink idref="bib41" id="ref60">41</reflink>]]</p> <p>Searches can also be supported by automatic alerts from several databases (i.e., Cochrane, JBI and Campbell, Health Database Advanced Search—HDAS).[<reflink idref="bib38" id="ref61">38</reflink>] These alerts can be set in advance to enable updates of scoping reviews.[<reflink idref="bib10" id="ref62">10</reflink>]</p> <p>Google Translate has been used to identify articles in several languages,[<reflink idref="bib43" id="ref63">43</reflink>] while it can also be helpful in study selection or data extraction. However, the accuracy of translation varied depending on the language translated. Google Translate was found to have the highest accuracy for Spanish (93%), followed by German and Japanese (89%) and French (85%).[<reflink idref="bib43" id="ref64">43</reflink>] DeepL is also another translating software that can detect languages, and reviewers can upload full pdfs to be easily translated and then accessed if they meet the inclusion and exclusion criteria (available at https://www.deepl.com/translator).[<reflink idref="bib44" id="ref65">44</reflink>]</p> <p>Other tools that can be used to manage references from webpages and other sources include Zotero and Mendeley. Zotero is similar to Endnote in that it extracts references from bibliographic databases. It is a free and open source that can be downloaded from the intranet. Its advantage includes its ease to save snapshots of web pages and annotate them within any citation library. However, it does not have as many citation styles as Endnote.[<reflink idref="bib45" id="ref66">45</reflink>] Another citation reference is Mendeley, it can automatically generate bibliographies and enable easier collaboration between researchers online. It is also a freely accessible tool online.[<reflink idref="bib46" id="ref67">46</reflink>] While these tools were not validated but they were peer reviewed.[<reflink idref="bib46" id="ref68">46</reflink>]</p> <p>Although automation tools are extremely useful within the development of the search strategy, it should be used in conjunction with expertise from an information specialist (Librarian) and if possible, peer‐reviewed according to the Peer review of Electronic Search Strategies (PRESS) 2015 guideline statement.[<reflink idref="bib47" id="ref69">47</reflink>] All searches should also be reported using best‐practice methods which align with the Preferred Reporting Items for Systematic Reviews (PRISMA) for reporting Literature searches in systematic reviews.[<reflink idref="bib48" id="ref70">48</reflink>]</p> <hd id="AN0180736596-13">Step 4: De‐duplication</hd> <p>Removing duplicates before screening titles and abstracts makes the initial screening easier and less laborious, and it also enables the tools used for screening titles and abstracts to work at their optimum.[<reflink idref="bib49" id="ref71">49</reflink>] Neither Abstrackr nor Rayyan can identify duplicates as it confuses their algorithms.[<reflink idref="bib13" id="ref72">13</reflink>] Several tools can be used to remove duplicates successfully, these include Endnote, Mendeley, Zotero, Systematic Review Accelerator (SRA De‐duplicator), EPPI‐Reviewer, and DistillerSR.[[<reflink idref="bib10" id="ref73">10</reflink>], [<reflink idref="bib21" id="ref74">21</reflink>], [<reflink idref="bib30" id="ref75">30</reflink>], [<reflink idref="bib46" id="ref76">46</reflink>], [<reflink idref="bib50" id="ref77">50</reflink>]] Covidence software also identifies duplicates and removes them on initial import of searches to the system. Whilst tools can support de‐duplication, it must be recognised that they are not 100% effective and duplicates will still require to be removed manually by reviewers during the screening process. Covidence also easily manages duplicates if identified during screening, subsequently, the PRISMA chart will have a more accurate representation of the included evidence sources. Lastly, expert information specialists recently created Deduklick, an automated, effective, and quick artificial intelligence‐based algorithm.[[<reflink idref="bib51" id="ref78">51</reflink>]]</p> <hd id="AN0180736596-14">Step 5: Screening titles abstracts and full‐texts</hd> <p>Study selection usually follows a pre‐specified protocol where the population, concept, and context of the scoping review is well defined in advance. This step requires at least two reviewers to go through titles and abstracts of all the articles. Any disagreements should be settled by either consensus or with a third reviewer. Pilot testing of this step is usually recommended for very large reviews to ensure all reviewers can consistently apply the inclusion criteria. In addition, pilot testing could reduce the time then required for conflict management. Depending on the research question, this step of the scoping review could include more than 5000 citations which can be very time consuming.[<reflink idref="bib9" id="ref79">9</reflink>] Tools that have been used both successfully and reliably for this step include Abstrackr, Rayyan, BIBOT, EPPI‐Reviewer and DistillerSR.[[<reflink idref="bib21" id="ref80">21</reflink>], [<reflink idref="bib30" id="ref81">30</reflink>], [<reflink idref="bib50" id="ref82">50</reflink>], [<reflink idref="bib53" id="ref83">53</reflink>]] These four tools had the highest reliability and validity amongst the others that were introduced as undertaking this step. Abstrackr had an accuracy of 95% and a precision of 97.7%,[<reflink idref="bib52" id="ref84">52</reflink>] Rayyan had an area under the curve of 0.87[<reflink idref="bib20" id="ref85">20</reflink>] and BIBOT had a reliability (<emph>k</emph> = 0.84).[<reflink idref="bib54" id="ref86">54</reflink>] The mean sensitivity of DistillerSR is 78% and the mean specificity of 95% for this approach. The area under the ROC curve was 0.87.[[<reflink idref="bib6" id="ref87">6</reflink>], [<reflink idref="bib27" id="ref88">27</reflink>]]</p> <p>ASReview is another tool that has been used in systematic reviews to title and abstract screening. It aims at minimising the number of articles to be screened by the researcher, while still identifying the majority of relevant articles. A study by Ferdinands compared the use of ASReview randomly and statistically for small sample size, the author found the tool to reduce screening by up to 82%.[<reflink idref="bib55" id="ref89">55</reflink>]</p> <p>Covidence systematic review management software can support all stages of screening as well as data extraction in scoping reviews.[<reflink idref="bib26" id="ref90">26</reflink>] The software is user friendly and can reduce time spent per scoping review. The system can automatically detect and upload full‐text open access articles, support bulk uploads for.pdf files as well as report on the reliability of reviewers in each stage of screening as well as manage conflicts. The screening results can then be exported to complete the PRISMA flowchart, the excluded studies with reasons for exclusion can also be exported for inclusion in the final scoping review.[<reflink idref="bib26" id="ref91">26</reflink>]</p> <p>The 'Cochrane RCT Classifier' was created as a machine learning classifier for extracting randomised controlled trials (RCTs) to minimise the time and effort associated with study identification in Cochrane while conducting systematic reviews.[<reflink idref="bib56" id="ref92">56</reflink>] Additionally, a machine learning classifier called the 'Cochrane COVID‐19 Study Classifier' was created to reduce the workload associated with study identification in order to maintain the Cochrane COVID‐19 Study Register (CCSR), a continually updated register of COVID‐19 research papers.[<reflink idref="bib57" id="ref93">57</reflink>]</p> <p>In the academic literature, several other automation tools have been referenced for facilitating the screening process, which encompass the Evidence‐Based Medicine NLP (EBM‐NLP) corpus,[<reflink idref="bib58" id="ref94">58</reflink>] GRADEpro, MAGICapp, SRA‐Helper, TerMIne, Lingo3G, GAPScreener, Revis, Pimiento, RapidMine, RobotAnalyst, Sherlock, Metta Snowballing‐ParsCit, Quick Clinical, Support for Systematic Review[<reflink idref="bib59" id="ref95">59</reflink>] and Research Screener.[<reflink idref="bib60" id="ref96">60</reflink>]</p> <hd id="AN0180736596-15">Step 6: Data extraction</hd> <p>Data extraction should use a standardised process and tools where relevant information is extracted from included sources to address the review's question related to the PCC (Population, Concept and Context). This step is usually done by at least two reviewers especially when there is a large number of included studies.[<reflink idref="bib7" id="ref97">7</reflink>] Examples of items included in data extraction sheets are citation details, country of origin of publications, methods, methodology, relevant findings, and conclusions. However, other examples of data extraction could include participants, types of intervention, and barriers and facilitators.[[<reflink idref="bib8" id="ref98">8</reflink>], [<reflink idref="bib22" id="ref99">22</reflink>]]</p> <p>DistillerSR, ExaCT, and Covidence have been used by the JBI scoping review methodology group for data extraction.[[<reflink idref="bib10" id="ref100">10</reflink>], [<reflink idref="bib27" id="ref101">27</reflink>], [<reflink idref="bib61" id="ref102">61</reflink>]] DistillerSR is a web‐based tool that aids in reference screening and data extraction and ExaCT assists the location and extraction of key trial characteristics from journal articles for Randomised Controlled Trials (RCTs).[<reflink idref="bib62" id="ref103">62</reflink>] ExaCT tool had very high recall (72%–100%) for data extraction; however, it is only specific for RCTs which may not be ideal for scoping reviews that include other study designs.[[<reflink idref="bib13" id="ref104">13</reflink>], [<reflink idref="bib61" id="ref105">61</reflink>]] While, it is hard to find the specificity and reliability of DistillerSR for data extraction, it is currently being used by researchers in their systematic reviews[[<reflink idref="bib63" id="ref106">63</reflink>]] Covidence Data Extraction 2.0 tool can support extraction of data and also piloting of data extraction by all reviewers. Also, it has been noted that both Tabulizer and UDPipe tools are useful for the data extraction process.[<reflink idref="bib65" id="ref107">65</reflink>] Webplotdigitizer has also been used effectively and reliably to extract data from studies. Its validity has been tested and was found to be comparable to other data extraction tools that are used to extract data from tables.[[<reflink idref="bib14" id="ref108">14</reflink>], [<reflink idref="bib66" id="ref109">66</reflink>]]</p> <p>Piloting data extraction is recommended to ensure the data captured addresses the research questions adequately and accurately by all reviewers, as this can be further modified to obtain the appropriate data for the review. Furthermore, data extraction that is easy to interpret can assist in a quicker data extraction stage.</p> <hd id="AN0180736596-16">Step 7: Data charting and analysis</hd> <p>This step of the scoping review is dependent on the results obtained to address the scoping review questions. This step can also be prespecified in the protocol stage. Data charting can be done in Excel or other programs with automated functions for data transformation that enable data to be presented visually. Examples of these programs include GoogleCharts, Tableau Public, Tableau Software, Eppi‐reviewer, Nvivo, and Google Data studio amongst others.[[<reflink idref="bib50" id="ref110">50</reflink>], [<reflink idref="bib67" id="ref111">67</reflink>], [<reflink idref="bib69" id="ref112">69</reflink>]] For diagram construction, Diagrams.net by Draw.io software offers several resources and possibilities.[<reflink idref="bib68" id="ref113">68</reflink>] While these tools are not peer reviewed, they are useful for data presentation.</p> <hd id="AN0180736596-17">Step 8: Report writing</hd> <p>This step of the scoping review can be undertaken in platforms such as JBI SUMARI and Revman as they provide structured headings and prepopulated text for some sections of the final review. Covidence can support this step by exporting the data to populate the PRISMA flowchart as well as exporting studies excluded at full text screening with reason to populate the supporting scoping review Appendices.</p> <p>In conjunction with the aforementioned steps and tools, a tool named RobotReviewer LIVE was developed for updating reviews. In a pilot study, this tool was demonstrated to effectively decrease the reliance on manual screening work while enabling timely rolling updates upon the publication of new primary research.[<reflink idref="bib69" id="ref114">69</reflink>]</p> <hd id="AN0180736596-18">DISCUSSION</hd> <p>This paper has described several reliable and validated automation tools and software that can be used to undertake scoping reviews[[<reflink idref="bib13" id="ref115">13</reflink>], [<reflink idref="bib70" id="ref116">70</reflink>]] Automation of systematic and scoping reviews are continuously evolving and further work is still being undertaken in this area.[<reflink idref="bib71" id="ref117">71</reflink>] This guidance constitutes an important step in supporting reviewers to undertake scoping reviews by integrating both automation tools and available software with human capabilities. It may support joint efforts aiming at the development of evidence informed integrative automated tools and platforms for conducting high quality scoping reviews.</p> <p>While we included several reliable tools and software that can be used for the automation of scoping reviews, there are some limitations to the tools mentioned. For example, some are available in English only. Another disadvantage is the lack of direct comparison between the tools in a head‐to‐head experiment which limits their use due to researchers not having a realistic view of their capability as compared to other tools. Other limitations for some include the lack of integration with other tools and limited interoperability except for SRA tool integrator that has been used to automatically exchanges data between various software currently used in reviews.[<reflink idref="bib39" id="ref118">39</reflink>] This tool was only cited in one reference so increasing its use will determine its benefit for automation tools integration. This renders the scoping review tasks more time consuming as researchers navigate between various platforms to undertake scoping reviews. These limitations could discourage reviewers from using available tools and delay the uptake and enhancement of innovative new scoping review approaches. We also did not comprehensively cover all tools available in the SR Toolbox that might be relevant to scoping reviews (due to the lack of data on their reliability) and we encourage the reader to use this website as a great resource for additional tools.</p> <p>When planning a scoping review, additional time should be included to ensure all reviewers are trained and proficient in using each automated tool. For each tool that will be utilised, it should be referred to in the scoping review protocol as well as the final scoping review. While automated tools have the capacity to speed up scoping reviews, trialling, training, and piloting of the tools with the review team should be factored into a review or conducted before the review begins in the set‐up phase to ensure efficient application.</p> <p>While automation tools and software can be useful and save time for many review steps, human knowledge and expertise are still vital. When authors choose to use some of these tools, especially those for searching and data extraction, we recommend that human reviewers are involved to enable checking of the process.</p> <p>Automation and innovative software have the potential to substantially enhance and ease the conduct of scoping reviews. Further research is needed in the areas of data extraction as most tools were only validated with small data sets, which makes it difficult to generalise their use and applicability to various topics. Additionally, if a large number of sources are included in a scoping review, steps to ensure the software can accommodate large volumes of evidence need to be established early on in the extraction process.</p> <p>Is it worthwhile discussing ChatGPT was explored to complete systematic‐review tasks with a focus on tasks relevant to interpretation of language.[<reflink idref="bib72" id="ref119">72</reflink>] The authors found that it was able to complete some tasks such as formulating a review question, performing a preliminary PubMed search strategy and basic synthesis of three study results. Significant work is still needed to improve its functionality to undertake the whole process.</p> <p>It is also important to note that most of the tools presented in this study are all proprietary and need subscriptions. However, we found that only a few tools are open access including CrossRef, Litsuggest, Abstrackr, Rayyan, UDPipe, Googlecharts, Google data studio and Drawio. Needless to say, we have not used the FAIRness principles to evaluate these tools in terms of Findable, Accessible, Interoperable and reusable as it is beyond the scope of this study. Further work should be undertaken to compare and contrast tool characteristics, evaluation metrics, and metrics tests for public datasets.[<reflink idref="bib73" id="ref120">73</reflink>]</p> <p>This guidance has a few limitations including the cost of some of the tools described may not be feasible for research teams from Low‐ and middle‐income countries due to licencing requirements and the availability of the tools to researchers without subscriptions. A useful list of the open source automation tools have been recently compiled by researchers.[<reflink idref="bib70" id="ref121">70</reflink>]</p> <p>Moreover, the performance indicators of screening tools mentioned are based on different benchmark datasets, making direct comparisons are challenging and not currently available.[<reflink idref="bib74" id="ref122">74</reflink>] Another limitation is that we only included some of the tools that have been validated and published in the literature and from our experience in doing scoping reviews, we may have omitted other tools that are currently being used by other researchers. Moreover, the numbers presented in this study for validation and reliability are based on the datasets used by researchers and therefore used different datasets may generate different numbers of validity and reliability. In addition to this, these were self‐identified by the author of these tools that they were valid and reliable. Future research should include a systematic review which can include critical appraisal and an appropriate assessment of validity and reliability of all the tools.</p> <p>Nevertheless, this guidance represents the most used tools by many researchers including our team. Needless to say, that this this guidance will need to be updated regularly in light of new evidence becoming available.</p> <hd id="AN0180736596-19">CONCLUSION</hd> <p>This report highlighted several useful automation tools and software to use in the undertaking of each step of a scoping review. The main limitation of these tools is their lack of integration with one another which makes the process of undertaking reviews challenging. This guidance has the potential to inform collaborative efforts aimed at the development of evidence informed integrative automated tools and platforms for conducting high quality scoping reviews.</p> <hd id="AN0180736596-20">AUTHOR CONTRIBUTIONS</hd> <p> <bold>Hanan Khalil:</bold> Conceptualization; methodology; software; funding acquisition; validation; formal analysis; data curation; supervision; project administration; visualization; writing – review and editing; writing – original draft; resources; investigation. <bold>Danielle Pollock:</bold> Writing – original draft; writing – review and editing. <bold>Patricia McInerney:</bold> Writing – original draft. <bold>Catrin Evans:</bold> Writing – review and editing. <bold>Erica B. Moraes:</bold> Writing – review and editing. <bold>Christina M. Godfrey:</bold> Writing – review and editing. <bold>Lyndsay Alexander:</bold> Writing – review and editing. <bold>Andrea Tricco:</bold> Writing – original draft; writing – review and editing. <bold>Micah D. J. Peters:</bold> Writing – review and editing. <bold>Dawid Pieper:</bold> Writing – review and editing. <bold>Ashrita Saran:</bold> Writing – review and editing. <bold>Daniel Ameen:</bold> Investigation; writing – original draft; writing – review and editing; visualization; formal analysis; data curation; software. <bold>Petek Eylul Taneri:</bold> Writing – review and editing; formal analysis. <bold>Zachary Munn:</bold> Writing – review and editing.</p> <hd id="AN0180736596-21">ACKNOWLEDGEMENTS</hd> <p>The authors would like to thank Radhika Arunkumar for her support with some of the data extraction for the update of the review. Open access publishing facilitated by La Trobe University, as part of the Wiley ‐ La Trobe University agreement via the Council of Australian University Librarians.</p> <hd id="AN0180736596-22">FUNDING INFORMATION</hd> <p>No funding was received for this work. ACT is funded by a Tier 2 Canada Research Chair in Knowledge Synthesis. ZM is supported by an NHMRC investigator grant APP1195676.</p> <hd id="AN0180736596-23">CONFLICT OF INTEREST STATEMENT</hd> <p>The authors declare no conflicts of interest.</p> <hd id="AN0180736596-24">DATA AVAILABILITY STATEMENT</hd> <p>The data that support the findings of this study are available from the corresponding author upon reasonable request.</p> <ref id="AN0180736596-25"> <title> REFERENCES </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> Horbach SP. Pandemic publishing: medical journals strongly speed up their publication process for COVID‐19. Quant Sci Stud. 2020 ; 1 (3): 1056 ‐ 1067.</bibtext> </blist> <blist> <bibl id="bib2" type="bt">2</bibl> <bibtext> Müller SM, Mueller GF, Navarini AA, Brandt O. National publication productivity during the COVID‐19 pandemic—a preliminary exploratory analysis of the 30 countries most affected. Biology. 2020 ; 9 (9): 271.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref2" type="bt">3</bibl> <bibtext> Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018 ; 18 (1): 143.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref28" type="bt">4</bibl> <bibtext> Aromataris E, Stern C, Lockwood C, et al. JBI series paper 2: tailored evidence synthesis approaches are required to answer diverse questions: a pragmatic evidence synthesis toolkit from JBI. J Clin Epidemiol. 2022 ; 150 : 196 ‐ 202.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref3" type="bt">5</bibl> <bibtext> Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA‐ScR): checklist and explanation. Ann Intern Med. 2018 ; 169 (7): 467 ‐ 473.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref4" type="bt">6</bibl> <bibtext> Khalil H, Tamara L, Rada G, Akl EA. Challenges of evidence synthesis during the 2020 COVID pandemic: a scoping review. J Clin Epidemiol. 2022 ; 142 : 10 ‐ 18.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref5" type="bt">7</bibl> <bibtext> Peters MD, Marnie C, Tricco AC, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Implement. 2021 ; 19 (1): 3 ‐ 10.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref6" type="bt">8</bibl> <bibtext> Khalil H, Shahid M, Roughead L. Medication safety programs in primary care: a scoping review. JBI Evid Synth. 2017 ; 15 (10): 2512 ‐ 2526.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref79" type="bt">9</bibl> <bibtext> Khalil H, Downie A, Ristevski E. Mapping palliative and end of care research in Australia (2000–2018). Palliat Support Care. 2020 ; 18 (6): 713 ‐ 721.</bibtext> </blist> <blist> <bibtext> Marshall IJ, Wallace BC. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev. 2019 ; 8 (1): 163.</bibtext> </blist> <blist> <bibtext> Wallace BC, Small K, Brodley CE, Lau J, Trikalinos TA. Deploying an interactive machine learning system in an evidence‐based practice center: abstrackr. Proceedings of the 2nd ACM SIGHIT international health informatics symposium. 2012.</bibtext> </blist> <blist> <bibtext> O'Connor AM, Tsafnat G, Gilbert SB, Thayer KA, Wolfe MS. Moving toward the automation of the systematic review process: a summary of discussions at the second meeting of international collaboration for the automation of systematic reviews (ICASR). Syst Rev. 2018 ; 7 (1): 1 ‐ 5.</bibtext> </blist> <blist> <bibtext> Khalil H, Ameen D, Zarnegar A. Tools to support the automation of systematic reviews: a scoping review. J Clin Epidemiol. 2021 ; 144 : 22 ‐ 42.</bibtext> </blist> <blist> <bibtext> Scott AM, Forbes C, Clark J, Carter M, Glasziou P, Munn Z. Systematic review automation tools improve efficiency but lack of knowledge impedes their adoption: a survey. J Clin Epidemiol. 2021 ; 138 : 80 ‐ 94.</bibtext> </blist> <blist> <bibtext> Munn Z, Brandt L, Kuijpers T, Whittington C, Wiles L, Karge T. Are systematic review and guideline development tools useful? A guidelines international network survey of user preferences. JBI Evid Implement. 2020 ; 18 (3): 345 ‐ 352.</bibtext> </blist> <blist> <bibtext> Van Altena A, Spijker R, Olabarriaga S. Usage of automation tools in systematic reviews. Res Synth Methods. 2019 ; 10 (1): 72 ‐ 82.</bibtext> </blist> <blist> <bibtext> Przybyła P, Brockmeier AJ, Kontonatsios G, et al. Prioritising references for systematic reviews with RobotAnalyst: a user study. Res Synth Methods. 2018 ; 9 (3): 470 ‐ 488.</bibtext> </blist> <blist> <bibtext> Howard BE, Phillips J, Miller K, et al. SWIFT‐review: a text‐mining workbench for systematic review. Syst Rev. 2016 ; 5 (1): 87.</bibtext> </blist> <blist> <bibtext> Aguinis H, Ramani RS, Alabduljader N. Best‐practice recommendations for producers, evaluators, and users of methodological literature reviews. Organ Res Methods. 2023 ; 26 (1): 46 ‐ 76.</bibtext> </blist> <blist> <bibtext> Cleo G, Scott AM, Islam F, Julien B, Beller E. Usability and acceptability of four systematic review automation software packages: a mixed method design. Syst Rev. 2019 ; 8 (1): 1 ‐ 5.</bibtext> </blist> <blist> <bibtext> Gates A, Gates M, Sebastianski M, Guitard S, Elliott SA, Hartling L. The semi‐automation of title and abstract screening: a retrospective exploration of ways to leverage Abstrackr's relevance predictions in systematic and rapid reviews. BMC Med Res Methodol. 2020 ; 20 (1): 139.</bibtext> </blist> <blist> <bibtext> Munn Z, Pollock D, Khalil H, et al. What are scoping reviews? Providing a formal definition of scoping reviews as a type of evidence synthesis. JBI evidence. Synthesis. 2022 ; 20 : 950 ‐ 952.</bibtext> </blist> <blist> <bibtext> Jordan Z, Lockwood C, Aromataris E, et al. JBI series paper 1: introducing JBI and the JBI model of EHBC. J Clin Epidemiol. 2022 ; 150 : 191 ‐ 195.</bibtext> </blist> <blist> <bibtext> Nyanchoka L, Tudur‐Smith C, Thu VN, Iversen V, Tricco AC, Porcher R. A scoping review describes methods used to identify, prioritize and display gaps in health research. J Clin Epidemiol. 2019 ; 109 : 99 ‐ 110.</bibtext> </blist> <blist> <bibtext> Khalil H, Peters MD, Tricco AC, et al. Conducting high quality scoping reviews‐challenges and solutions. J Clin Epidemiol. 2021 ; 130 : 156 ‐ 160.</bibtext> </blist> <blist> <bibtext> Kellermeyer L, Harnke B, Knight S. Covidence and rayyan. J Med Library Assoc. 2018 ; 106 (4): 580.</bibtext> </blist> <blist> <bibtext> Hamel C, Kelly S, Thavorn K, Rice D, Wells G, Hutton B. An evaluation of DistillerSR's machine learning‐based prioritization tool for title/abstract screening–impact on reviewer‐relevant outcomes. BMC Med Res Methodol. 2020 ; 20 (1): 1 ‐ 14.</bibtext> </blist> <blist> <bibtext> Munn Z, Aromataris E, Tufanaru C, et al. The development of software to support multiple systematic review types: the Joanna Briggs institute system for the unified management, assessment and review of information (JBI SUMARI). JBI Evid Implement. 2019 ; 17 (1): 36 ‐ 43.</bibtext> </blist> <blist> <bibtext> Munn Z, Aromataris E, Tufanaru C, et al. The development of software to support multiple systematic review types: the Joanna Briggs institute system for the unified management, assessment and review of information (JBI SUMARI). Int J Evid Based Healthc. 2019 ; 17 (1): 36 ‐ 43.</bibtext> </blist> <blist> <bibtext> Gates A, Guitard S, Pillay J, et al. Performance and usability of machine learning for screening in systematic reviews: a comparative evaluation of three tools. Syst Rev. 2019 ; 8 (1): 278.</bibtext> </blist> <blist> <bibtext> Mortensen ML, Adam GP, Trikalinos TA, Kraska T, Wallace BC. An exploration of crowdsourcing citation screening for systematic reviews. Res Synth Methods. 2017 ; 8 (3): 366 ‐ 386.</bibtext> </blist> <blist> <bibtext> Nama N, Sampson M, Barrowman N, et al. Crowdsourcing the citation screening process for systematic reviews: validation study. J Med Internet Res. 2019 ; 21 (4): e12953.</bibtext> </blist> <blist> <bibtext> Nama N, Iliriani K, Xia MY, et al. A pilot validation study of crowdsourcing systematic reviews: update of a searchable database of pediatric clinical trials of high‐dose vitamin D. Transl Pediatr. 2017 ; 6 (1): 18 ‐ 26.</bibtext> </blist> <blist> <bibtext> Johnson HA. Trello. J Med Library Assoc. 2017 ; 105 (2): 209.</bibtext> </blist> <blist> <bibtext> Peters MD, Godfrey C, McInerney P, et al. Best practice guidance and reporting items for the development of scoping review protocols. JBI Evid Synth. 2022 ; 20 (4): 953 ‐ 968.</bibtext> </blist> <blist> <bibtext> Aromataris E, Riitano D. Constructing a search strategy and searching for evidence. Am J Nurs. 2014 ; 114 (5): 49 ‐ 56.</bibtext> </blist> <blist> <bibtext> Allot A, Lee K, Chen Q, Luo L, Lu Z. LitSuggest: a web‐based system for literature recommendation and curation using machine learning. Nucleic Acids Res. 2021 ; 49 (W1): W352 ‐ W358.</bibtext> </blist> <blist> <bibtext> Haddaway NR, Grainger MJ, Gray CT. citationchaser: An R package and Shiny app for forward and backward citations chasing in academic searching. 2021.</bibtext> </blist> <blist> <bibtext> Clark J, Glasziou P, Del Mar C, Bannach‐Brown A, Stehlik P, Scott AM. A full systematic review was completed in 2 weeks using automation tools: a case study. J Clin Epidemiol. 2020 ; 121 : 81 ‐ 90.</bibtext> </blist> <blist> <bibtext> Clark JM, Sanders S, Carter M, et al. Improving the translation of search strategies using the polyglot search translator: a randomized controlled trial. J Med Libr Assoc. 2020 ; 108 (2): 195 ‐ 207.</bibtext> </blist> <blist> <bibtext> Choong MK, Galgani F, Dunn AG, Tsafnat G. Automatic evidence retrieval for systematic reviews. J Med Internet Res. 2014 ; 16 (10): e223.</bibtext> </blist> <blist> <bibtext> Beller E, Clark J, Tsafnat G, et al. Making progress with the automation of systematic reviews: principles of the international collaboration for the automation of systematic reviews (ICASR). Syst Rev. 2018 ; 7 (1): 1 ‐ 7.</bibtext> </blist> <blist> <bibtext> Balk EM, Chung M, Chen ML, Chang LKW, Trikalinos TA. Data extraction from machine‐translated versus original language randomized trial reports: a comparative study. Syst Rev. 2013 ; 2 (1): 1 ‐ 6.</bibtext> </blist> <blist> <bibtext> Yulianto A, Supriatnaningsih R. Google translate vs. DeepL: a quantitative evaluation of close‐language pair translation (French to English). Asian J Engl Lang Pedagogy. 2021 ; 9 (2): 109 ‐ 127.</bibtext> </blist> <blist> <bibtext> Idri N. Zotero software: A means of bibliographic research and data organisation; teaching bibliographic research. Arab World English Journal (AWEJ) Special Issue on CALL. 2015.</bibtext> </blist> <blist> <bibtext> Basak SK. Comparison of Researcher’ s reference management software: Refworks, Mendeley, and EndNote. J Econ Behav Stud. 2014 ; 6 (7): 561 ‐ 568.</bibtext> </blist> <blist> <bibtext> McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016 ; 75 : 40 ‐ 46.</bibtext> </blist> <blist> <bibtext> Rethlefsen ML, Kirtley S, Waffenschmidt S, et al. PRISMA‐S: an extension to the PRISMA statement for reporting literature searches in systematic reviews. Syst Rev. 2021 ; 10 : 1 ‐ 19.</bibtext> </blist> <blist> <bibtext> McKeown S, Mir ZM. Considerations for conducting systematic reviews: evaluating the performance of different methods for de‐duplicating references. Syst Rev. 2021 ; 10 : 1 ‐ 8.</bibtext> </blist> <blist> <bibtext> Tsou AY, Treadwell JR, Erinoff E, Schoelles K. Machine learning for screening prioritization in systematic reviews: comparative performance of Abstrackr and EPPI‐reviewer. Syst Rev. 2020 ; 9 (1): 1 ‐ 14.</bibtext> </blist> <blist> <bibtext> Borissov N, Haas Q, Minder B, et al. Reducing systematic review burden using Deduklick: a novel, automated, reliable, and explainable deduplication algorithm to foster medical research. Syst Rev. 2022 ; 11 (1): 172.</bibtext> </blist> <blist> <bibtext> Kwon Y, Lemieux M, McTavish J, Wathen N. Identifying and removing duplicate records from systematic review searches. J Med Libr Assoc. 2015 ; 103 (4): 184 ‐ 188.</bibtext> </blist> <blist> <bibtext> Rathbone J, Hoffmann T, Glasziou P. Faster title and abstract screening? Evaluating Abstrackr, a semi‐automated online screening program for systematic reviewers. Syst Rev. 2015 ; 4 (1): 80.</bibtext> </blist> <blist> <bibtext> Orgeolet L, Foulquier N, Misery L, et al. Can artificial intelligence replace manual search for systematic literature? Review on cutaneous manifestations in primary Sjögren's syndrome. Rheumatology. 2020 ; 59 (4): 811 ‐ 819.</bibtext> </blist> <blist> <bibtext> Ferdinands G. AI‐assisted systematic reviewing: selecting studies to compare Bayesian versus frequentist SEM for small sample sizes. Multivar Behav Res. 2021 ; 56 (1): 153 ‐ 154.</bibtext> </blist> <blist> <bibtext> Thomas J, McDonald S, Noel‐Storr A, et al. Machine learning reduced workload with minimal risk of missing studies: development and evaluation of a randomized controlled trial classifier for Cochrane reviews. J Clin Epidemiol. 2021 ; 133 : 140 ‐ 151.</bibtext> </blist> <blist> <bibtext> Shemilt I, Noel‐Storr A, Thomas J, Featherstone R, Mavergames C. Machine learning reduced workload for the Cochrane COVID‐19 study register: development and evaluation of the Cochrane COVID‐19 study classifier. Syst Rev. 2022 ; 11 (1): 15.</bibtext> </blist> <blist> <bibtext> Tsubota T, Bollegala D, Zhao Y, Jin Y, Kozu T. Improvement of intervention information detection for automated clinical literature screening during systematic review. J Biomed Inform. 2022 ; 104185 : 104185.</bibtext> </blist> <blist> <bibtext> Escaldelai FMD, Escaldelai L, Bergamaschi DP. Systematic review support software system: web‐based solution for managing duplicates and screening eligible studies. Rev Bras Epidemiol. 2022 ; 25 : 25.</bibtext> </blist> <blist> <bibtext> Chai KE, Lines RL, Gucciardi DF, Ng L. Research screener: a machine learning tool to semi‐automate abstract screening for systematic reviews. Syst Rev. 2021 ; 10 : 1 ‐ 13.</bibtext> </blist> <blist> <bibtext> Marshall IJ, Johnson BT, Wang Z, Rajasekaran S, Wallace BC. Semi‐automated evidence synthesis in health psychology: current methods and future prospects. Health Psychol Rev. 2020 ; 14 (1): 145 ‐ 158.</bibtext> </blist> <blist> <bibtext> Marshall IJ, Wallace BC. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev. 2019 ; 8 (1): 1 ‐ 10.</bibtext> </blist> <blist> <bibtext> Squires JE, Graham ID, Grinspun D, et al. Inappropriateness of health care in Canada: a systematic review protocol. Syst Rev. 2019 ; 8 (1): 1 ‐ 8.</bibtext> </blist> <blist> <bibtext> Van Grootven B, Jepma P, Rijpkema C, et al. Prediction models for hospital readmissions in patients with heart disease: a systematic review and meta‐analysis. BMJ Open. 2021 ; 11 (8): e047576.</bibtext> </blist> <blist> <bibtext> Halamoda‐Kenzaoui B, Rolland E, Piovesan J, Puertas Gallardo A, Bremer‐Hoffmann S. Toxic effects of nanomaterials for health applications: how automation can support a systematic review of the literature? J Appl Toxicol. 2022 ; 42 (1): 41 ‐ 51.</bibtext> </blist> <blist> <bibtext> Mohan K, Alam MA, Patnaik R, Mohan K. A review on use of automation in systematic reviews for scientific evidence generation. Eur J Mol Clin Med. 2020; 8 (2): 2021.</bibtext> </blist> <blist> <bibtext> Park SE, Thomas J. Evidence synthesis software. BMJ Evide Med. 2018 ; 23 : 140 ‐ 141.</bibtext> </blist> <blist> <bibtext> Dhakal K. NVivo. J Med Libr Assoc. 2022 ; 110 (2): 270 ‐ 272.</bibtext> </blist> <blist> <bibtext> Jena B. An approach for forecast prediction in data analytics field by tableau software. Int J Inform Eng Electr Bus. 2019 ; 11 (1): 19 ‐ 26.</bibtext> </blist> <blist> <bibtext> Stoel L, Mourits G, van de Schoot R. Procedure and Results for the Initial Selection of Software for Systematically Screening Large Amounts of Textual Data Implementing Active Learning.OSF. 2022.</bibtext> </blist> <blist> <bibtext> Cierco Jimenez R, Lee T, Rosillo N, et al. Machine learning computational tools to assist the performance of systematic reviews: a mapping review. BMC Med Res Methodol. 2022 ; 22 (1): 322.</bibtext> </blist> <blist> <bibtext> Qureshi R, Shaughnessy D, Gill KA, Robinson KA, Li T, Agai E. Are ChatGPT and large language models "the answer" to bringing us closer to systematic review automation? Syst Rev. 2023 ; 12 (1): 72.</bibtext> </blist> <blist> <bibtext> Sun C, Emonet V, Dumontier M. A comprehensive comparison of automated FAIRness Evaluation Tools. 13th International Conference on Semantic Web Applications and Tools for Health Care and Life Sciences. 2022.</bibtext> </blist> <blist> <bibtext> Teijema JJ, Seuren S, Anadria D, Bagheri A, van de Schoot R. Simulation‐based Active Learning for Systematic Reviews: A Systematic Review of the Literature. Retrieved from psyarxiv com/67zmt. 2023.</bibtext> </blist> </ref> <aug> <p>By Hanan Khalil; Danielle Pollock; Patricia McInerney; Catrin Evans; Erica B. Moraes; Christina M. Godfrey; Lyndsay Alexander; Andrea Tricco; Micah D. J. Peters; Dawid Pieper; Ashrita Saran; Daniel Ameen; Petek Eylul Taneri and Zachary Munn</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib10" firstref="ref7"></nolink> <nolink nlid="nl2" bibid="bib11" firstref="ref8"></nolink> <nolink nlid="nl3" bibid="bib12" firstref="ref9"></nolink> <nolink nlid="nl4" bibid="bib13" firstref="ref10"></nolink> <nolink nlid="nl5" bibid="bib15" firstref="ref11"></nolink> <nolink nlid="nl6" bibid="bib16" firstref="ref13"></nolink> <nolink nlid="nl7" bibid="bib17" firstref="ref14"></nolink> <nolink nlid="nl8" bibid="bib19" firstref="ref17"></nolink> <nolink nlid="nl9" bibid="bib20" firstref="ref21"></nolink> <nolink nlid="nl10" bibid="bib22" firstref="ref23"></nolink> <nolink nlid="nl11" bibid="bib23" firstref="ref27"></nolink> <nolink nlid="nl12" bibid="bib24" firstref="ref30"></nolink> <nolink nlid="nl13" bibid="bib25" firstref="ref31"></nolink> <nolink nlid="nl14" bibid="bib26" firstref="ref34"></nolink> <nolink nlid="nl15" bibid="bib27" firstref="ref35"></nolink> <nolink nlid="nl16" bibid="bib28" firstref="ref37"></nolink> <nolink nlid="nl17" bibid="bib29" firstref="ref38"></nolink> <nolink nlid="nl18" bibid="bib31" firstref="ref41"></nolink> <nolink nlid="nl19" bibid="bib32" firstref="ref43"></nolink> <nolink nlid="nl20" bibid="bib34" firstref="ref44"></nolink> <nolink nlid="nl21" bibid="bib35" firstref="ref47"></nolink> <nolink nlid="nl22" bibid="bib36" firstref="ref49"></nolink> <nolink nlid="nl23" bibid="bib37" firstref="ref50"></nolink> <nolink nlid="nl24" bibid="bib38" firstref="ref52"></nolink> <nolink nlid="nl25" bibid="bib39" firstref="ref56"></nolink> <nolink nlid="nl26" bibid="bib40" firstref="ref57"></nolink> <nolink nlid="nl27" bibid="bib41" firstref="ref60"></nolink> <nolink nlid="nl28" bibid="bib43" firstref="ref63"></nolink> <nolink nlid="nl29" bibid="bib44" firstref="ref65"></nolink> <nolink nlid="nl30" bibid="bib45" firstref="ref66"></nolink> <nolink nlid="nl31" bibid="bib46" firstref="ref67"></nolink> <nolink nlid="nl32" bibid="bib47" firstref="ref69"></nolink> <nolink nlid="nl33" bibid="bib48" firstref="ref70"></nolink> <nolink nlid="nl34" bibid="bib49" firstref="ref71"></nolink> <nolink nlid="nl35" bibid="bib21" firstref="ref74"></nolink> <nolink nlid="nl36" bibid="bib30" firstref="ref75"></nolink> <nolink nlid="nl37" bibid="bib50" firstref="ref77"></nolink> <nolink nlid="nl38" bibid="bib51" firstref="ref78"></nolink> <nolink nlid="nl39" bibid="bib53" firstref="ref83"></nolink> <nolink nlid="nl40" bibid="bib52" firstref="ref84"></nolink> <nolink nlid="nl41" bibid="bib54" firstref="ref86"></nolink> <nolink nlid="nl42" bibid="bib55" firstref="ref89"></nolink> <nolink nlid="nl43" bibid="bib56" firstref="ref92"></nolink> <nolink nlid="nl44" bibid="bib57" firstref="ref93"></nolink> <nolink nlid="nl45" bibid="bib58" firstref="ref94"></nolink> <nolink nlid="nl46" bibid="bib59" firstref="ref95"></nolink> <nolink nlid="nl47" bibid="bib60" firstref="ref96"></nolink> <nolink nlid="nl48" bibid="bib61" firstref="ref102"></nolink> <nolink nlid="nl49" bibid="bib62" firstref="ref103"></nolink> <nolink nlid="nl50" bibid="bib63" firstref="ref106"></nolink> <nolink nlid="nl51" bibid="bib65" firstref="ref107"></nolink> <nolink nlid="nl52" bibid="bib14" firstref="ref108"></nolink> <nolink nlid="nl53" bibid="bib66" firstref="ref109"></nolink> <nolink nlid="nl54" bibid="bib67" firstref="ref111"></nolink> <nolink nlid="nl55" bibid="bib69" firstref="ref112"></nolink> <nolink nlid="nl56" bibid="bib68" firstref="ref113"></nolink> <nolink nlid="nl57" bibid="bib70" firstref="ref116"></nolink> <nolink nlid="nl58" bibid="bib71" firstref="ref117"></nolink> <nolink nlid="nl59" bibid="bib72" firstref="ref119"></nolink> <nolink nlid="nl60" bibid="bib73" firstref="ref120"></nolink> <nolink nlid="nl61" bibid="bib74" firstref="ref122"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1447299
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Automation Tools to Support Undertaking Scoping Reviews
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Hanan+Khalil%22">Hanan Khalil</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-3302-2009">0000-0002-3302-2009</externalLink>)<br /><searchLink fieldCode="AR" term="%22Danielle+Pollock%22">Danielle Pollock</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-6604-0609">0000-0002-6604-0609</externalLink>)<br /><searchLink fieldCode="AR" term="%22Patricia+McInerney%22">Patricia McInerney</searchLink><br /><searchLink fieldCode="AR" term="%22Catrin+Evans%22">Catrin Evans</searchLink><br /><searchLink fieldCode="AR" term="%22Erica+B%2E+Moraes%22">Erica B. Moraes</searchLink><br /><searchLink fieldCode="AR" term="%22Christina+M%2E+Godfrey%22">Christina M. Godfrey</searchLink><br /><searchLink fieldCode="AR" term="%22Lyndsay+Alexander%22">Lyndsay Alexander</searchLink><br /><searchLink fieldCode="AR" term="%22Andrea+Tricco%22">Andrea Tricco</searchLink><br /><searchLink fieldCode="AR" term="%22Micah+D%2E+J%2E+Peters%22">Micah D. J. Peters</searchLink><br /><searchLink fieldCode="AR" term="%22Dawid+Pieper%22">Dawid Pieper</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-0715-5182">0000-0002-0715-5182</externalLink>)<br /><searchLink fieldCode="AR" term="%22Ashrita+Saran%22">Ashrita Saran</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3261-6654">0000-0003-3261-6654</externalLink>)<br /><searchLink fieldCode="AR" term="%22Daniel+Ameen%22">Daniel Ameen</searchLink><br /><searchLink fieldCode="AR" term="%22Petek+Eylul+Taneri%22">Petek Eylul Taneri</searchLink><br /><searchLink fieldCode="AR" term="%22Zachary+Munn%22">Zachary Munn</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Research+Synthesis+Methods%22"><i>Research Synthesis Methods</i></searchLink>. 2024 15(6):839-850.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 12
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Descriptive
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Software%22">Computer Software</searchLink><br /><searchLink fieldCode="DE" term="%22Synthesis%22">Synthesis</searchLink><br /><searchLink fieldCode="DE" term="%22Protocol+Analysis%22">Protocol Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Team+Training%22">Team Training</searchLink><br /><searchLink fieldCode="DE" term="%22Journal+Articles%22">Journal Articles</searchLink><br /><searchLink fieldCode="DE" term="%22Literature+Reviews%22">Literature Reviews</searchLink><br /><searchLink fieldCode="DE" term="%22Online+Searching%22">Online Searching</searchLink><br /><searchLink fieldCode="DE" term="%22Duplication%22">Duplication</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Processing%22">Data Processing</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1002/jrsm.1731
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1759-2879<br />1759-2887
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Objective: This paper describes several automation tools and software that can be considered during evidence synthesis projects and provides guidance for their integration in the conduct of scoping reviews. Study Design and Setting: The guidance presented in this work is adapted from the results of a scoping review and consultations with the JBI Scoping Review Methodology group. Results: This paper describes several reliable, validated automation tools and software that can be used to enhance the conduct of scoping reviews. Developments in the automation of systematic reviews, and more recently scoping reviews, are continuously evolving. We detail several helpful tools in order of the key steps recommended by the JBI's methodological guidance for undertaking scoping reviews including team establishment, protocol development, searching, de-duplication, screening titles and abstracts, data extraction, data charting, and report writing. While we include several reliable tools and software that can be used for the automation of scoping reviews, there are some limitations to the tools mentioned. For example, some are available in English only and their lack of integration with other tools results in limited interoperability. Conclusion: This paper highlighted several useful automation tools and software programs to use in undertaking each step of a scoping review. This guidance has the potential to inform collaborative efforts aiming at the development of evidence informed, integrated automation tools and software packages for enhancing the conduct of high-quality scoping reviews.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1447299
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1447299
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1002/jrsm.1731
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 12
        StartPage: 839
    Subjects:
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Computer Software
        Type: general
      – SubjectFull: Synthesis
        Type: general
      – SubjectFull: Protocol Analysis
        Type: general
      – SubjectFull: Team Training
        Type: general
      – SubjectFull: Journal Articles
        Type: general
      – SubjectFull: Literature Reviews
        Type: general
      – SubjectFull: Online Searching
        Type: general
      – SubjectFull: Duplication
        Type: general
      – SubjectFull: Data Processing
        Type: general
    Titles:
      – TitleFull: Automation Tools to Support Undertaking Scoping Reviews
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Hanan Khalil
      – PersonEntity:
          Name:
            NameFull: Danielle Pollock
      – PersonEntity:
          Name:
            NameFull: Patricia McInerney
      – PersonEntity:
          Name:
            NameFull: Catrin Evans
      – PersonEntity:
          Name:
            NameFull: Erica B. Moraes
      – PersonEntity:
          Name:
            NameFull: Christina M. Godfrey
      – PersonEntity:
          Name:
            NameFull: Lyndsay Alexander
      – PersonEntity:
          Name:
            NameFull: Andrea Tricco
      – PersonEntity:
          Name:
            NameFull: Micah D. J. Peters
      – PersonEntity:
          Name:
            NameFull: Dawid Pieper
      – PersonEntity:
          Name:
            NameFull: Ashrita Saran
      – PersonEntity:
          Name:
            NameFull: Daniel Ameen
      – PersonEntity:
          Name:
            NameFull: Petek Eylul Taneri
      – PersonEntity:
          Name:
            NameFull: Zachary Munn
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 11
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 1759-2879
            – Type: issn-electronic
              Value: 1759-2887
          Numbering:
            – Type: volume
              Value: 15
            – Type: issue
              Value: 6
          Titles:
            – TitleFull: Research Synthesis Methods
              Type: main
ResultId 1