An Instrument for Observing Teams to Explicate Regulation Strategies in Computer Science
Saved in:
| Title: | An Instrument for Observing Teams to Explicate Regulation Strategies in Computer Science |
|---|---|
| Language: | English |
| Authors: | Carolin Wortmann (ORCID |
| Source: | ACM Transactions on Computing Education. 2026 26(2). |
| Availability: | Association for Computing Machinery. 1601 Broadway 10th Floor, New York, NY 10119. Tel: 800-342-6626; Tel: 212-626-0500; Fax: 212-944-1318; e-mail: acmhelp@acm.org; Web site: http://toce.acm.org/ |
| Peer Reviewed: | Y |
| Page Count: | 29 |
| Publication Date: | 2026 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Computer Science Education, Computer Software, Cooperative Learning, Active Learning, Laboratories, Capstone Experiences, Learning Strategies, Observation, Measures (Individuals), Self Management, Metacognition |
| DOI: | 10.1145/3779308 |
| ISSN: | 1946-6226 |
| Abstract: | Motivation and Objectives: Informed by a long tradition of studying regulation strategies in general education, recent work in Computing Education Research has highlighted the importance and effects of such strategies in computing-related classrooms as well. Most of this work has focused on self-regulation of individuals or dyads, for example, in pair programming. Little, however, is known about regulation in larger groups which naturally occur in, for example, active-learning classrooms, Software Engineering lab courses, or capstone projects. Moreover, research involving regulation strategies, including, but not limited to computing contexts, predominantly relies on self-reported data. While recent work has advocated using trace data from learning management systems to assess self-regulation strategies more objectively, such systems cannot capture regulation as it occurs in many versions of group work, for example, in group discussions and interactions in a classroom. We thus aim to develop an observation instrument that can help researchers to explicate regulation strategies in Computer Science and, more generally, computing-related group work. Together with self-reported data and--where applicable--trace data, such an instrument would facilitate obtaining a more complete picture of where and how regulation strategies emerge and develop. Methods: Using data from semi-structured interviews with students from two capstone projects, we conducted a deductive qualitative analysis to add concrete descriptors to Miller and Hadwin's framework for regulated learning. We refined the resulting coding scheme into an observation instrument that was field-tested for saturation, ease-of-use, and reliability in a Software Engineering lab course. Results: Our main result is OTTERS, an instrument for observing teams to explicate regulation strategies in computing contexts. Initial field-testing indicates that this instrument, despite its granularity, is easy to use and reliable. Furthermore, first proof-of-concept observations lead to patterns of regulation activities that are clearly distinguishable, thus suggesting construct validity. Discussion. Whereas previous work focused on self-reported data or trace data obtained from learning management systems, our observation instrument adds another facet to assessing regulation strategies. Compared to self-reported data, it trades off the bias of possibly unreliable self-assessment and the limitations of external observations. Compared to analyses based on high-resolution, objective log data, it yields observations on a coarser granularity but is applicable not only to self-regulation but to socially shared regulation and co-regulation as well. Our observation instrument thus complements the current methodological portfolio of assessing regulation strategies in Computer Science and related contexts. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | EJ1504217 |
| Database: | ERIC |
| Abstract: | Motivation and Objectives: Informed by a long tradition of studying regulation strategies in general education, recent work in Computing Education Research has highlighted the importance and effects of such strategies in computing-related classrooms as well. Most of this work has focused on self-regulation of individuals or dyads, for example, in pair programming. Little, however, is known about regulation in larger groups which naturally occur in, for example, active-learning classrooms, Software Engineering lab courses, or capstone projects. Moreover, research involving regulation strategies, including, but not limited to computing contexts, predominantly relies on self-reported data. While recent work has advocated using trace data from learning management systems to assess self-regulation strategies more objectively, such systems cannot capture regulation as it occurs in many versions of group work, for example, in group discussions and interactions in a classroom. We thus aim to develop an observation instrument that can help researchers to explicate regulation strategies in Computer Science and, more generally, computing-related group work. Together with self-reported data and--where applicable--trace data, such an instrument would facilitate obtaining a more complete picture of where and how regulation strategies emerge and develop. Methods: Using data from semi-structured interviews with students from two capstone projects, we conducted a deductive qualitative analysis to add concrete descriptors to Miller and Hadwin's framework for regulated learning. We refined the resulting coding scheme into an observation instrument that was field-tested for saturation, ease-of-use, and reliability in a Software Engineering lab course. Results: Our main result is OTTERS, an instrument for observing teams to explicate regulation strategies in computing contexts. Initial field-testing indicates that this instrument, despite its granularity, is easy to use and reliable. Furthermore, first proof-of-concept observations lead to patterns of regulation activities that are clearly distinguishable, thus suggesting construct validity. Discussion. Whereas previous work focused on self-reported data or trace data obtained from learning management systems, our observation instrument adds another facet to assessing regulation strategies. Compared to self-reported data, it trades off the bias of possibly unreliable self-assessment and the limitations of external observations. Compared to analyses based on high-resolution, objective log data, it yields observations on a coarser granularity but is applicable not only to self-regulation but to socially shared regulation and co-regulation as well. Our observation instrument thus complements the current methodological portfolio of assessing regulation strategies in Computer Science and related contexts. |
|---|---|
| ISSN: | 1946-6226 |
| DOI: | 10.1145/3779308 |
