ChatGPT as an Education and Learning Tool for Engineering, Technology and General Studies: Performance Analysis of ChatGPT 3.0 on CSE, GATE and JEE Examinations of India
Saved in:
| Title: | ChatGPT as an Education and Learning Tool for Engineering, Technology and General Studies: Performance Analysis of ChatGPT 3.0 on CSE, GATE and JEE Examinations of India |
|---|---|
| Language: | English |
| Authors: | Ravindra Giriraj Bhardwaj (ORCID |
| Source: | Interactive Learning Environments. 2025 33(1):321-334. |
| Availability: | Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals |
| Peer Reviewed: | Y |
| Page Count: | 14 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | Foreign Countries, Artificial Intelligence, Occupational Tests, College Entrance Examinations, Undergraduate Study, Graduate Study, Government Employees, Engineering Education, Computer Assisted Testing, Technology Uses in Education, Computer Software, Standardized Tests, Testing, Accuracy, Training |
| Geographic Terms: | India |
| DOI: | 10.1080/10494820.2024.2344054 |
| ISSN: | 1049-4820 1744-5191 |
| Abstract: | The quantitative and qualitative performance analysis of ChatGPT-3.0, a large language model, is carried out on three important and highly competitive examinations held in India: civil services examination (CSE, prelims), graduate aptitude test in engineering (GATE), and joint entrance examination (JEE). These examinations cover general knowledge, current affairs, history, geography, Indian polity, economics, mathematics, physics, chemistry, engineering, and technology aspects at the undergraduate and graduate levels. The Accuracy, Concordance, and Insight (ACI) criteria is used to analyze the performance of ChatGPT. ChatGPT passed CSE without much specialized training and reinforcement, however, underperformed in GATE and JEE. Overall, the average accuracy rate of ChatGPT is 48.71%, with a 44.45% concordance for all explanations. However, the concordance for accurate explanations is found to be 91.87% with a high level of insights given in the explanations. Moreover, the average accuracy of ChatGPT improves to 77.69% after training. The results suggest that large language models have great potential to assist with education technology and act as an instructor for the preparation of technical, aptitude and general studies topics for competitive examinations. Drawing insights from the findings of the current research, some limitations in the present study and possible future research directions are suggested. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | EJ1500342 |
| Database: | ERIC |
| Abstract: | The quantitative and qualitative performance analysis of ChatGPT-3.0, a large language model, is carried out on three important and highly competitive examinations held in India: civil services examination (CSE, prelims), graduate aptitude test in engineering (GATE), and joint entrance examination (JEE). These examinations cover general knowledge, current affairs, history, geography, Indian polity, economics, mathematics, physics, chemistry, engineering, and technology aspects at the undergraduate and graduate levels. The Accuracy, Concordance, and Insight (ACI) criteria is used to analyze the performance of ChatGPT. ChatGPT passed CSE without much specialized training and reinforcement, however, underperformed in GATE and JEE. Overall, the average accuracy rate of ChatGPT is 48.71%, with a 44.45% concordance for all explanations. However, the concordance for accurate explanations is found to be 91.87% with a high level of insights given in the explanations. Moreover, the average accuracy of ChatGPT improves to 77.69% after training. The results suggest that large language models have great potential to assist with education technology and act as an instructor for the preparation of technical, aptitude and general studies topics for competitive examinations. Drawing insights from the findings of the current research, some limitations in the present study and possible future research directions are suggested. |
|---|---|
| ISSN: | 1049-4820 1744-5191 |
| DOI: | 10.1080/10494820.2024.2344054 |
