Surprises from the Shadows: Confronting Intrusions from Bots and Generative AI in Survey Research
Saved in:
| Title: | Surprises from the Shadows: Confronting Intrusions from Bots and Generative AI in Survey Research |
|---|---|
| Language: | English |
| Authors: | Vincent Cho (ORCID |
| Source: | Journal of Educational Research and Practice. 2025 15. |
| Availability: | Walden University, LLC. 100 Washington Avenue South Suite 900, Minneapolis, MN 55401. Tel: 800-925-3368; Fax: 612-338-5092; e-mail: JERAP@waldenu.edu; Web site: http://scholarworks.waldenu.edu/jerap |
| Peer Reviewed: | Y |
| Page Count: | 21 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Artificial Intelligence, Online Surveys, Research Methodology, Research Projects, Educational Research, Data Collection, Educational Technology, Teacher Attitudes, Parent Attitudes, Incentives, Deception, Technological Advancement, Integrity, Computer Software |
| ISSN: | 2167-8693 |
| Abstract: | Unbeknownst to many researchers, online surveys can be vulnerable to attacks from bots and generative artificial intelligence (AI), which can generate hundreds of fraudulent responses instantly. Confronting this challenge to data integrity, we employed an adaptive approach to test the effectiveness of various anti-fraud tactics (e.g., CAPTCHA; honeypot questions; question pairs; open-ended questions) and to distinguish between good-faith human respondents versus bots or fraudsters. Findings revealed that bots with generative AI capabilities bypassed some conventional defenses, including CAPTCHA, and effectively supplied human-like responses to open-ended questions. Other tactics, like removing response incentives and verifying IP addresses, showed promise but came with trade-offs. This study highlights the increasing complexity of online survey fraud and advocates for innovating new defenses and training. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | EJ1492339 |
| Database: | ERIC |
| Abstract: | Unbeknownst to many researchers, online surveys can be vulnerable to attacks from bots and generative artificial intelligence (AI), which can generate hundreds of fraudulent responses instantly. Confronting this challenge to data integrity, we employed an adaptive approach to test the effectiveness of various anti-fraud tactics (e.g., CAPTCHA; honeypot questions; question pairs; open-ended questions) and to distinguish between good-faith human respondents versus bots or fraudsters. Findings revealed that bots with generative AI capabilities bypassed some conventional defenses, including CAPTCHA, and effectively supplied human-like responses to open-ended questions. Other tactics, like removing response incentives and verifying IP addresses, showed promise but came with trade-offs. This study highlights the increasing complexity of online survey fraud and advocates for innovating new defenses and training. |
|---|---|
| ISSN: | 2167-8693 |
