View in EDS

ChatGPT as an Education and Learning Tool for Engineering, Technology and General Studies: Performance Analysis of ChatGPT 3.0 on CSE, GATE and JEE Examinations of India

Saved in:
Bibliographic Details
Title: ChatGPT as an Education and Learning Tool for Engineering, Technology and General Studies: Performance Analysis of ChatGPT 3.0 on CSE, GATE and JEE Examinations of India
Language: English
Authors: Ravindra Giriraj Bhardwaj (ORCID 0000-0003-3816-9437), Harpreet Singh Bedi (ORCID 0000-0001-5662-0071)
Source: Interactive Learning Environments. 2025 33(1):321-334.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 14
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Foreign Countries, Artificial Intelligence, Occupational Tests, College Entrance Examinations, Undergraduate Study, Graduate Study, Government Employees, Engineering Education, Computer Assisted Testing, Technology Uses in Education, Computer Software, Standardized Tests, Testing, Accuracy, Training
Geographic Terms: India
DOI: 10.1080/10494820.2024.2344054
ISSN: 1049-4820
1744-5191
Abstract: The quantitative and qualitative performance analysis of ChatGPT-3.0, a large language model, is carried out on three important and highly competitive examinations held in India: civil services examination (CSE, prelims), graduate aptitude test in engineering (GATE), and joint entrance examination (JEE). These examinations cover general knowledge, current affairs, history, geography, Indian polity, economics, mathematics, physics, chemistry, engineering, and technology aspects at the undergraduate and graduate levels. The Accuracy, Concordance, and Insight (ACI) criteria is used to analyze the performance of ChatGPT. ChatGPT passed CSE without much specialized training and reinforcement, however, underperformed in GATE and JEE. Overall, the average accuracy rate of ChatGPT is 48.71%, with a 44.45% concordance for all explanations. However, the concordance for accurate explanations is found to be 91.87% with a high level of insights given in the explanations. Moreover, the average accuracy of ChatGPT improves to 77.69% after training. The results suggest that large language models have great potential to assist with education technology and act as an instructor for the preparation of technical, aptitude and general studies topics for competitive examinations. Drawing insights from the findings of the current research, some limitations in the present study and possible future research directions are suggested.
Abstractor: As Provided
Entry Date: 2026
Accession Number: EJ1500342
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1500342
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: ChatGPT as an Education and Learning Tool for Engineering, Technology and General Studies: Performance Analysis of ChatGPT 3.0 on CSE, GATE and JEE Examinations of India
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Ravindra+Giriraj+Bhardwaj%22">Ravindra Giriraj Bhardwaj</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3816-9437">0000-0003-3816-9437</externalLink>)<br /><searchLink fieldCode="AR" term="%22Harpreet+Singh+Bedi%22">Harpreet Singh Bedi</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-5662-0071">0000-0001-5662-0071</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Interactive+Learning+Environments%22"><i>Interactive Learning Environments</i></searchLink>. 2025 33(1):321-334.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 14
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Occupational+Tests%22">Occupational Tests</searchLink><br /><searchLink fieldCode="DE" term="%22College+Entrance+Examinations%22">College Entrance Examinations</searchLink><br /><searchLink fieldCode="DE" term="%22Undergraduate+Study%22">Undergraduate Study</searchLink><br /><searchLink fieldCode="DE" term="%22Graduate+Study%22">Graduate Study</searchLink><br /><searchLink fieldCode="DE" term="%22Government+Employees%22">Government Employees</searchLink><br /><searchLink fieldCode="DE" term="%22Engineering+Education%22">Engineering Education</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Uses+in+Education%22">Technology Uses in Education</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Software%22">Computer Software</searchLink><br /><searchLink fieldCode="DE" term="%22Standardized+Tests%22">Standardized Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Testing%22">Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Accuracy%22">Accuracy</searchLink><br /><searchLink fieldCode="DE" term="%22Training%22">Training</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22India%22">India</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/10494820.2024.2344054
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1049-4820<br />1744-5191
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The quantitative and qualitative performance analysis of ChatGPT-3.0, a large language model, is carried out on three important and highly competitive examinations held in India: civil services examination (CSE, prelims), graduate aptitude test in engineering (GATE), and joint entrance examination (JEE). These examinations cover general knowledge, current affairs, history, geography, Indian polity, economics, mathematics, physics, chemistry, engineering, and technology aspects at the undergraduate and graduate levels. The Accuracy, Concordance, and Insight (ACI) criteria is used to analyze the performance of ChatGPT. ChatGPT passed CSE without much specialized training and reinforcement, however, underperformed in GATE and JEE. Overall, the average accuracy rate of ChatGPT is 48.71%, with a 44.45% concordance for all explanations. However, the concordance for accurate explanations is found to be 91.87% with a high level of insights given in the explanations. Moreover, the average accuracy of ChatGPT improves to 77.69% after training. The results suggest that large language models have great potential to assist with education technology and act as an instructor for the preparation of technical, aptitude and general studies topics for competitive examinations. Drawing insights from the findings of the current research, some limitations in the present study and possible future research directions are suggested.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1500342
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1500342
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/10494820.2024.2344054
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 14
        StartPage: 321
    Subjects:
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Occupational Tests
        Type: general
      – SubjectFull: College Entrance Examinations
        Type: general
      – SubjectFull: Undergraduate Study
        Type: general
      – SubjectFull: Graduate Study
        Type: general
      – SubjectFull: Government Employees
        Type: general
      – SubjectFull: Engineering Education
        Type: general
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Technology Uses in Education
        Type: general
      – SubjectFull: Computer Software
        Type: general
      – SubjectFull: Standardized Tests
        Type: general
      – SubjectFull: Testing
        Type: general
      – SubjectFull: Accuracy
        Type: general
      – SubjectFull: Training
        Type: general
      – SubjectFull: India
        Type: general
    Titles:
      – TitleFull: ChatGPT as an Education and Learning Tool for Engineering, Technology and General Studies: Performance Analysis of ChatGPT 3.0 on CSE, GATE and JEE Examinations of India
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Ravindra Giriraj Bhardwaj
      – PersonEntity:
          Name:
            NameFull: Harpreet Singh Bedi
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 1049-4820
            – Type: issn-electronic
              Value: 1744-5191
          Numbering:
            – Type: volume
              Value: 33
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Interactive Learning Environments
              Type: main
ResultId 1