View in EDS PDF Full Text

Footprint of Publication Selection Bias on Meta-Analyses in Medicine, Environmental Sciences, Psychology, and Economics

Saved in:
Bibliographic Details
Title: Footprint of Publication Selection Bias on Meta-Analyses in Medicine, Environmental Sciences, Psychology, and Economics
Language: English
Authors: František Bartoš (ORCID 0000-0002-0018-5573), Maximilian Maier (ORCID 0000-0002-9873-6096), Eric-Jan Wagenmakers (ORCID 0000-0003-1596-1034), Franziska Nippold, Hristos Doucouliagos (ORCID 0000-0001-5269-3556), John P. A. Ioannidis (ORCID 0000-0003-3118-6859), Willem M. Otte (ORCID 0000-0003-1511-6834), Martina Sladekova (ORCID 0000-0001-5059-6576), Teshome K. Deresssa (ORCID 0000-0003-1351-1849), Stephan B. Bruns (ORCID 0000-0002-3028-9699), Daniele Fanelli (ORCID 0000-0003-1780-1958), T. D. Stanley (ORCID 0000-0002-3205-1983)
Source: Research Synthesis Methods. 2024 15(3):500-511.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 12
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Descriptors: Publications, Selection, Bias, Meta Analysis, Medicine, Environmental Education, Psychology, Economics, Effect Size, Probability
DOI: 10.1002/jrsm.1703
ISSN: 1759-2879
1759-2887
Abstract: Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.
Abstractor: As Provided
Notes: https://osf.io/bgfzp
Entry Date: 2024
Accession Number: EJ1421864
Database: ERIC
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwG0lWxoMGnL9fAet9esVYxKAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDB_j4YKApdn_LSPZTQIBEICBmn-3mpNS4U38OpXp_2zQ08iHbtp2Eit_IZy7xeR_gyONu2tCSb0RZ6qz7j6_Pwh4K4EtIpbdcMinAjEHXW2dmbQi2TF5tSwvMcQ5ix18e-6v23A2PihVCtequs4jaGK-m_MBi-gtiHRP0QnZI0nLUlaYz8QTaM9xVpwtqDFy323indLtW1xpRyuiCkqonEYse4nzmvyqNi2R1YY=
Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1421864
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Footprint of Publication Selection Bias on Meta-Analyses in Medicine, Environmental Sciences, Psychology, and Economics
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22František+Bartoš%22">František Bartoš</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-0018-5573">0000-0002-0018-5573</externalLink>)<br /><searchLink fieldCode="AR" term="%22Maximilian+Maier%22">Maximilian Maier</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-9873-6096">0000-0002-9873-6096</externalLink>)<br /><searchLink fieldCode="AR" term="%22Eric-Jan+Wagenmakers%22">Eric-Jan Wagenmakers</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-1596-1034">0000-0003-1596-1034</externalLink>)<br /><searchLink fieldCode="AR" term="%22Franziska+Nippold%22">Franziska Nippold</searchLink><br /><searchLink fieldCode="AR" term="%22Hristos+Doucouliagos%22">Hristos Doucouliagos</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-5269-3556">0000-0001-5269-3556</externalLink>)<br /><searchLink fieldCode="AR" term="%22John+P%2E+A%2E+Ioannidis%22">John P. A. Ioannidis</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3118-6859">0000-0003-3118-6859</externalLink>)<br /><searchLink fieldCode="AR" term="%22Willem+M%2E+Otte%22">Willem M. Otte</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-1511-6834">0000-0003-1511-6834</externalLink>)<br /><searchLink fieldCode="AR" term="%22Martina+Sladekova%22">Martina Sladekova</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-5059-6576">0000-0001-5059-6576</externalLink>)<br /><searchLink fieldCode="AR" term="%22Teshome+K%2E+Deresssa%22">Teshome K. Deresssa</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-1351-1849">0000-0003-1351-1849</externalLink>)<br /><searchLink fieldCode="AR" term="%22Stephan+B%2E+Bruns%22">Stephan B. Bruns</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-3028-9699">0000-0002-3028-9699</externalLink>)<br /><searchLink fieldCode="AR" term="%22Daniele+Fanelli%22">Daniele Fanelli</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-1780-1958">0000-0003-1780-1958</externalLink>)<br /><searchLink fieldCode="AR" term="%22T%2E+D%2E+Stanley%22">T. D. Stanley</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-3205-1983">0000-0002-3205-1983</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Research+Synthesis+Methods%22"><i>Research Synthesis Methods</i></searchLink>. 2024 15(3):500-511.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 12
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Publications%22">Publications</searchLink><br /><searchLink fieldCode="DE" term="%22Selection%22">Selection</searchLink><br /><searchLink fieldCode="DE" term="%22Bias%22">Bias</searchLink><br /><searchLink fieldCode="DE" term="%22Meta+Analysis%22">Meta Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Medicine%22">Medicine</searchLink><br /><searchLink fieldCode="DE" term="%22Environmental+Education%22">Environmental Education</searchLink><br /><searchLink fieldCode="DE" term="%22Psychology%22">Psychology</searchLink><br /><searchLink fieldCode="DE" term="%22Economics%22">Economics</searchLink><br /><searchLink fieldCode="DE" term="%22Effect+Size%22">Effect Size</searchLink><br /><searchLink fieldCode="DE" term="%22Probability%22">Probability</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1002/jrsm.1703
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1759-2879<br />1759-2887
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Note
  Label: Notes
  Group: Note
  Data: https://osf.io/bgfzp
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1421864
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1421864
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1002/jrsm.1703
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 12
        StartPage: 500
    Subjects:
      – SubjectFull: Publications
        Type: general
      – SubjectFull: Selection
        Type: general
      – SubjectFull: Bias
        Type: general
      – SubjectFull: Meta Analysis
        Type: general
      – SubjectFull: Medicine
        Type: general
      – SubjectFull: Environmental Education
        Type: general
      – SubjectFull: Psychology
        Type: general
      – SubjectFull: Economics
        Type: general
      – SubjectFull: Effect Size
        Type: general
      – SubjectFull: Probability
        Type: general
    Titles:
      – TitleFull: Footprint of Publication Selection Bias on Meta-Analyses in Medicine, Environmental Sciences, Psychology, and Economics
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: František Bartoš
      – PersonEntity:
          Name:
            NameFull: Maximilian Maier
      – PersonEntity:
          Name:
            NameFull: Eric-Jan Wagenmakers
      – PersonEntity:
          Name:
            NameFull: Franziska Nippold
      – PersonEntity:
          Name:
            NameFull: Hristos Doucouliagos
      – PersonEntity:
          Name:
            NameFull: John P. A. Ioannidis
      – PersonEntity:
          Name:
            NameFull: Willem M. Otte
      – PersonEntity:
          Name:
            NameFull: Martina Sladekova
      – PersonEntity:
          Name:
            NameFull: Teshome K. Deresssa
      – PersonEntity:
          Name:
            NameFull: Stephan B. Bruns
      – PersonEntity:
          Name:
            NameFull: Daniele Fanelli
      – PersonEntity:
          Name:
            NameFull: T. D. Stanley
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 05
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 1759-2879
            – Type: issn-electronic
              Value: 1759-2887
          Numbering:
            – Type: volume
              Value: 15
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Research Synthesis Methods
              Type: main
ResultId 1