The Effects of Email Performance-Based Feedback on Teacher Candidate Implementation of a Multiple Stimulus without Replacement Preference Assessment
Saved in:
| Title: | The Effects of Email Performance-Based Feedback on Teacher Candidate Implementation of a Multiple Stimulus without Replacement Preference Assessment |
|---|---|
| Language: | English |
| Authors: | Emma S. Sipila-Thomas, Matthew T. Brodhead (ORCID |
| Source: | Journal of Behavioral Education. 2024 33(2):297-319. |
| Availability: | Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/ |
| Peer Reviewed: | Y |
| Page Count: | 23 |
| Publication Date: | 2024 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | Electronic Mail, Feedback (Response), Preservice Teachers, Special Education, Evaluation, Fidelity, Improvement |
| DOI: | 10.1007/s10864-022-09496-z |
| ISSN: | 1053-0819 1573-3513 |
| Abstract: | The purpose of this study was to evaluate the effects of performance-based feedback delivered via email on teacher candidates' implementation of a multiple stimulus without replacement (MSWO) preference assessment with an adult confederate. Six participants enrolled in a special education teacher preparation program were recruited to participate. After exposure to an introductory MSWO training video, participant fidelity of MSWO administration was measured. Following baseline, performance feedback was delivered via e-mail to each participant after the completion of each research session. Results indicate that performance feedback delivered via e-mail improved fidelity of implementation for all six participants. Implications and future directions are discussed. |
| Abstractor: | As Provided |
| Entry Date: | 2024 |
| Accession Number: | EJ1428316 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHgY_SOiVYGXOddYhfYwp_JAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDENdVMlx7jYq_ixMdwIBEICBmhwvKnLdlMhsoLOOGj0gNd6TPVFVV09iS5XQ27XyHp9RBbMzxDM7nI7iON9R4qNrXFUbsSbh8fhZvrEuVOvNiu96V4s_995M_K-Bvbox6-THmqWUze65eit2BPFBC29Peo5-2Ta4EgSstQtq87CtkKCXXMph8F4tt0O-3ZoYw9hZMEYM_lIJ9rxMKxmMufS5b-5hcgctvRjzH9U= Text: Availability: 1 Value: <anid>AN0177897748;41z01jun.24;2024Jun18.05:03;v2.2.500</anid> <title id="AN0177897748-1">The Effects of Email Performance-Based Feedback on Teacher Candidate Implementation of a Multiple Stimulus Without Replacement Preference Assessment </title> <p>The purpose of this study was to evaluate the effects of performance-based feedback delivered via email on teacher candidates' implementation of a multiple stimulus without replacement (MSWO) preference assessment with an adult confederate. Six participants enrolled in a special education teacher preparation program were recruited to participate. After exposure to an introductory MSWO training video, participant fidelity of MSWO administration was measured. Following baseline, performance feedback was delivered via e-mail to each participant after the completion of each research session. Results indicate that performance feedback delivered via e-mail improved fidelity of implementation for all six participants. Implications and future directions are discussed.</p> <p>Keywords: E-mail; Multiple stimulus without replacement; Performance-based feedback; Telehealth</p> <p>This study was completed in partial fulfillment of program requirements for a PhD in Special Education by the first author, under the direction of the second author, at Michigan State University. The authors thank Grace Medlin and Brianna Smith for their support in conducting this study.</p> <hd id="AN0177897748-2">Introduction</hd> <p>Performance feedback is a critical component of professional development (Barton et al., [<reflink idref="bib3" id="ref1">3</reflink>]; Miltenberger, [<reflink idref="bib35" id="ref2">35</reflink>]). Performance feedback involves the use of data, derived from an observation occurring during supervision, to inform the delivery of feedback in order to change and sustain the individual's behavior (Barton et al., [<reflink idref="bib2" id="ref3">2</reflink>], [<reflink idref="bib3" id="ref4">3</reflink>]; Hemmeter et al., [<reflink idref="bib24" id="ref5">24</reflink>]; Novak et al., [<reflink idref="bib36" id="ref6">36</reflink>]). Within school settings, researchers found performance feedback to increase procedural fidelity and maintain the teacher's use of effective practices, which in turn increased the quality of instruction provided and improves child learning outcomes (Barton et al., [<reflink idref="bib3" id="ref7">3</reflink>]; Schles &amp; Robertson, [<reflink idref="bib43" id="ref8">43</reflink>]). Without additional support (e.g., performance feedback), new and returning teachers may implement evidence-based practices with low or variable levels of fidelity and negatively impact child learning outcomes for students with disabilities (Schles &amp; Robertson, [<reflink idref="bib43" id="ref9">43</reflink>]).</p> <p>Performance feedback can be delivered in many forms, such as in verbal or written formats, and it can be provided during or after a supervisory observation of the target individual implementing an intervention or engaging in an activity (Barton &amp; Wolery, [<reflink idref="bib4" id="ref10">4</reflink>]). In addition, performance feedback can be delivered in a variety of modalities such as bug-in-ear as well as visible counters, public wall postings, and personal interactions (Coogle et al., [<reflink idref="bib10" id="ref11">10</reflink>], [<reflink idref="bib8" id="ref12">8</reflink>]; Warrilow et al., [<reflink idref="bib50" id="ref13">50</reflink>]). As technology has evolved and become more available, so have modalities for delivering performance feedback, such as computer displays, text messages, video conferencing, social media communications, and emails (e.g., Barton &amp; Wolery, [<reflink idref="bib4" id="ref14">4</reflink>]; Hemmeter et al., [<reflink idref="bib24" id="ref15">24</reflink>]; Krick Oborn &amp; Johnson, [<reflink idref="bib28" id="ref16">28</reflink>]; Zhu et al., [<reflink idref="bib51" id="ref17">51</reflink>]).</p> <p>One technology-based modality that has been shown to have several advantages for delivering performance feedback is email (Gorton et al., [<reflink idref="bib21" id="ref18">21</reflink>]). Email feedback allows supervisors to save time by sending an email following the observation rather than scheduling a time to meet in person to review that feedback (Warrilow et al., [<reflink idref="bib50" id="ref19">50</reflink>]). Related, the observer is able to send the email feedback immediately to the individual after the observation is completed, without interrupting the individual and the activity they are engaging in (e.g., implementing an intervention with a child; Barton &amp; Wolery, [<reflink idref="bib4" id="ref20">4</reflink>]; Gorton et al., [<reflink idref="bib21" id="ref21">21</reflink>]; Zhu et al., [<reflink idref="bib51" id="ref22">51</reflink>]). Second, verbal and some forms of written feedback (e.g., handwritten notes) may be seen as obsolete and ineffective by the individuals receiving the feedback, whereas wireless communication forms of feedback (e.g., email, video, bug-in-ear) are seen as more current or up-to-date (Barton et al., [<reflink idref="bib3" id="ref23">3</reflink>]; Gomez et al., [<reflink idref="bib20" id="ref24">20</reflink>]; Zhu et al., [<reflink idref="bib51" id="ref25">51</reflink>]) and may be seen as more socially acceptable (Barton &amp; Wolery, [<reflink idref="bib4" id="ref26">4</reflink>]). Additionally, using emails results in an electronic record of the feedback provided (Zhu et al., [<reflink idref="bib51" id="ref27">51</reflink>]), which can be reviewed more than once and can be used for future performance reviews. Overall, email feedback may be a strategy for interacting with individuals quickly and more efficiently (Barton et al., [<reflink idref="bib2" id="ref28">2</reflink>]; Warrilow et al., [<reflink idref="bib50" id="ref29">50</reflink>]).</p> <p>A limitation of previous email feedback research is that the studies did not isolate the effects of email feedback, because the email feedback was always provided in conjunction with a training (Artman-Meecker &amp; Hemmeter, [<reflink idref="bib1" id="ref30">1</reflink>]; Gomez et al., [<reflink idref="bib20" id="ref31">20</reflink>]; Gorton et al., [<reflink idref="bib21" id="ref32">21</reflink>]; Hemmeter et al., [<reflink idref="bib24" id="ref33">24</reflink>]; Martinez Cueto et al., [<reflink idref="bib34" id="ref34">34</reflink>]). Researchers then sought to isolate and evaluate the effects of email feedback alone (Barton et al., [<reflink idref="bib2" id="ref35">2</reflink>], [<reflink idref="bib3" id="ref36">3</reflink>]) and also compared email feedback alone to other forms of feedback such as immediate bug-in-ear feedback (Coogle et al., [<reflink idref="bib9" id="ref37">9</reflink>]) and videoconference feedback (Zhu et al., [<reflink idref="bib51" id="ref38">51</reflink>]). Overall, the studies found that email feedback alone was effective in increasing teachers' behaviors.</p> <p>Though the aforementioned studies evaluated providing performance feedback via email, there were several limitations to the studies that the present study hopes to address. First, within the recent studies evaluating email feedback, researchers did not control for the amount or quality of the performance feedback provided in each email. This lack of experimental control presents problems because researchers have found that more frequent and specific feedback produces more significant changes in performance (Park et al., [<reflink idref="bib38" id="ref39">38</reflink>]; Sleiman et al., [<reflink idref="bib47" id="ref40">47</reflink>]). It is unclear if variation in amount or quality of performance feedback in previously published research differentially affected participant outcomes.</p> <p>Second, several of the aforementioned studies included a training component between the baseline condition and the intervention condition (i.e., Artman-Meecker &amp; Hemmeter, [<reflink idref="bib1" id="ref41">1</reflink>]; Gomez et al., [<reflink idref="bib20" id="ref42">20</reflink>]; Gorton et al., [<reflink idref="bib21" id="ref43">21</reflink>]; Hemmeter et al., [<reflink idref="bib24" id="ref44">24</reflink>]; Martinez Cueto et al., [<reflink idref="bib34" id="ref45">34</reflink>]). Thus, it is unclear if improvements in instructional behaviors were a result of email feedback alone, training alone, or a combination of the email feedback and the training.</p> <p>Third, the previous literature did not control for variation in the instructional environment. Specifically, the studies reviewed above involved classroom settings where teachers engaged with students. Opportunities for teacher participants to respond, then, were at least particularly affected by student behavior in those settings, and it is unclear to what extent participant behavior was at least partially affected by student behavior (e.g., frequency of teacher descriptive praise is dependent on the frequency of student engagement).</p> <p>In summary, the purpose of the present study is to extend previous research on performance feedback delivered via email by evaluating the effects of email feedback alone on teacher candidates' implementation of a multiple stimulus without replacement (MSWO) preference assessment. An MSWO preference assessment was selected to assess the effects of email feedback because an MSWO is a common preference assessment used within the field of behavior analysis that is well described in the literature and consists of discrete steps that can be easily measured (e.g., represents remaining items and rotates by taking the item at the left end of the line and moving to the right end, then shifting the other items so they are equally spaced on the table), and the duration to conduct an MSWO is shorter in comparison to other preference assessments (e.g., paired-choice preference assessments; Graff &amp; Karsten, [<reflink idref="bib22" id="ref46">22</reflink>]; Kang et al., [<reflink idref="bib26" id="ref47">26</reflink>]; Leaf et al., [<reflink idref="bib29" id="ref48">29</reflink>]; Lill et al., [<reflink idref="bib31" id="ref49">31</reflink>]). Therefore, researchers evaluated the following research question: Given email feedback, to what extent did teacher candidates' implement an MSWO preference assessment with fidelity?</p> <hd id="AN0177897748-3">Method</hd> <p></p> <hd id="AN0177897748-4">Participants</hd> <p>Participants were recruited from three undergraduate courses offered through a special education program at a Midwestern University. A total of six participants (all identified as female) were recruited and agreed to participate in the study. Participants were between 20 and 23 years of age. Participants were included in the study if they: (a) were enrolled at the university at the start of the study, (b) had no prior experience implementing an MSWO preference assessment, (c) had access to an email service, (d) had access to a computer with audio and video capabilities, and (d) were willing to read and respond to emails daily. Participants were screened for the inclusion criteria through a questionnaire prior to the start of the study. In addition, each participant was required to respond to three non-study-related emails sent over three days prior to the start of the study.</p> <p>Riley was a 20-year-old female who identified as White and not of Latinx, Hispanic, or Spanish origin. Riley had been enrolled at the university for less than one year, was a transfer student, and was majoring in Special Education.</p> <p>Olivia was a 23-year-old female who identified as Asian and not of Latinx, Hispanic, or Spanish origin. Olivia had been enrolled at the university for four years and was majoring in Special Education.</p> <p>Ava was a 20-year-old female who identified as White and not of Latinx, Hispanic, or Spanish origin. Ava had been enrolled at the university for three years and was majoring in Special Education.</p> <p>Layla was a 20-year-old female who identified as White and not of Latinx, Hispanic, or Spanish origin. Layla had been enrolled at the university for two years, was a transfer student, and was majoring in Special Education.</p> <p>Ellie was a 21-year-old female who identified as White and not of Latinx, Hispanic, or Spanish origin. Ellie had been enrolled at the university for four years and was majoring in Special Education and Elementary Education.</p> <p>Kennedy was a 20-year-old female who identified as Black or African American and Hispanic, and of Latinx, Hispanic, or Spanish origin. Kennedy had been enrolled at the university for three years and was majoring in Special Education.</p> <hd id="AN0177897748-5">Confederate</hd> <p>The primary researcher, a doctoral candidate who held a BCBA credential and had prior experience conducting research on preference assessments with children with ASD, served as the confederate throughout the duration of the study. The confederate played the role of a student (i.e., the learner) and engaged in specific responses during each research session (see Table 1 and see Confederate Response Data Sheet section for further description).</p> <p>Table 1 Confederate responses during MSWO research sessions</p> <p> <ephtml> &lt;table frame="hsides" rules="groups"&gt;&lt;thead&gt;&lt;tr&gt;&lt;th align="left"&gt;&lt;p&gt;Response After the Participant Says, "Pick One or Choose One"&lt;/p&gt;&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Immediately select one stimulus from the array presented (one item)&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Wait 7 s and then select one stimulus from the array presented (one item)&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Immediately select two stimuli simultaneously (simultaneous)&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Wait 7 s and then select two stimuli simultaneously (simultaneous)&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Immediately select one stimulus and then select another stimulus from the array presented (sequential)&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Wait 7 s and then select one stimulus and then select another stimulus from the array presented (sequential)&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Do not select a stimulus from the array presented (no response)&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <hd id="AN0177897748-6">Primary Data Collector</hd> <p>One graduate student served as the primary data collector and measured the dependent variable (participant procedural fidelity) for all research sessions across all conditions for each participant. The graduate student was trained by the primary researcher on the data collection process prior to the start of the study. The training was conducted across two different days, for 1 h each day. During the training session, the primary researcher reviewed each step of the participant procedural fidelity data sheet (see Dependent Variable section for further description), virtually displayed the MSWO training video that the participants were presented (see Participant Materials section for further description) and practiced scoring a research session. Following the training sessions, both the primary researcher and the primary data collector scored three research sessions independently. If the primary data collector achieved 100% accuracy across all three research sessions, they were considered to pass the reliability checks and were assigned to score the remaining research sessions. The primary data collector achieved 100% accuracy across the three research sessions the first time.</p> <hd id="AN0177897748-7">Setting and Materials</hd> <p>Due to the COVID-19 pandemic, all research sessions, data collection, and feedback were conducted or provided remotely. For each research session, the participant and the primary researcher/confederate were present. Additionally, each research session was recorded for data collection purposes, and no data collection occurred during the session. Instead, research sessions were reviewed later by the primary data collector. The primary data collector reviewed and scored research sessions in the order in which they were recorded.</p> <hd id="AN0177897748-8">Participant Materials</hd> <p>Participant materials consisted of a computer with audio and video capabilities, an email service, a video conferencing software (e.g., Zoom), a pen or pencil, a calculator, a timer (e.g., phone), five leisure stimuli, and data sheets. The five leisure stimuli (e.g., ball, stapler) for the preference assessment were arbitrarily selected by each participant based on the stimuli they had available in their households for each research session. Twenty data sheets (see Supplementary Materials, Appendix A) for participants to use for data collection while conducting the MSWO preference assessment were mailed to the participant's household or emailed to the participant (based on each participant's preference) prior to the start of the study.</p> <hd id="AN0177897748-9">Primary Researcher/Confederate Materials</hd> <p>Primary researcher/confederate materials consisted of a computer with audio and video capabilities, a video conferencing software (e.g., Zoom), an email service, a timer (e.g., digital stopwatch), and a second computer monitor. Additional materials consisted of an MSWO training video (presented once during the initial research meeting), an MSWO excerpt, a session script, a list of responses (here after referred to as the confederate response data sheet) to perform during each research session (see Table 1), and a pool of specific responses for each component of the email feedback. During each research session, the primary researcher/confederate used a second monitor to display the assigned confederate response data sheet, timer, and the session script. The primary monitor was used to display the video conferencing software in gallery view, to allow both the primary researcher/confederate and participant to be visible side by side.</p> <p> <bold>MSWO Training Video</bold> The purpose of the MSWO training video was to approximate an "onboarding" or initial training a professional may receive on the topic and to provide basic information about how to implement an MSWO preference assessment, so that each participant would have baseline level knowledge of the MSWO <emph>prior</emph> to the start of the study. The MSWO training video was informed by an MSWO video created by researchers at Vanderbilt University (Chazin &amp; Ledford, [<reflink idref="bib7" id="ref50">7</reflink>]) and by the primary researcher's eight years of ABA experience and implementing preference assessments. The training video was created by the primary researcher and included a step-by-step guide of how to conduct an MSWO preference assessment within a remote context. When an MSWO preference assessment is conducted in-person, the learner may be able to engage with the physical item following a selection response. However, when an MSWO preference assessment is conducted within a remote context, the learner may not be able to engage with the physical item because they may not have access to it. Instead, the implementer may engage with the item and the learner may observe. The training video included specific steps that were tailored to meet the unique aspects of conducting an MSWO preference assessment within a remote context.</p> <p>The video included a video model where the primary researcher played the role of the participant and a research assistant played the role of the learner (i.e., confederate), written text that indicated the target behavior for each step, and a voice over narration completed by the primary researcher. The training video (see Brodhead, [<reflink idref="bib6" id="ref51">6</reflink>]) was 21 min in length and was displayed virtually to the participants by the primary researcher one time prior to the start of the study (i.e., during the initial research meeting that occurred one week before each participant's first research session). The participants did not have access to the training video outside of the initial research meeting.</p> <p> <bold>MSWO Excerpt</bold> The MSWO excerpt was derived from a relevant peer-reviewed article (DeLeon &amp; Iwata, [<reflink idref="bib14" id="ref52">14</reflink>]) and consisted of sections that outlined correct implementation of the MSWO preference assessment (see Supplementary Materials, Appendix B). The purpose of the MSWO excerpt was to provide the participants with a brief description of the MSWO procedures in order to mimic a description someone would find if they were to conduct an internet search or access notes on MSWO preference assessments. Additionally, this was an identical approach that previous preference assessment literature has used (e.g., O'Handley et al., [<reflink idref="bib37" id="ref53">37</reflink>]; Rosales et al., [<reflink idref="bib41" id="ref54">41</reflink>]).</p> <p>The MSWO excerpt was displayed virtually to each participant at the beginning of each research session for up to 15 min. During the 15 min, the participants could read and reread the entire excerpt or specific sentences. For one participant, due to participant characteristics that cannot be disclosed for confidentiality reasons, the MSWO excerpt was displayed virtually to the participant and was read aloud to the participant by the primary researcher for up to 15 min. Similar to the other participants, the participant could ask the primary researcher to read or reread the entire excerpt or specific sentences aloud. At any time during the 15 min, the participants could indicate to the primary researcher that they were finished reviewing the excerpt and the primary researcher would stop displaying or reading aloud the excerpt. The participants did not have access to the excerpt outside of the research sessions.</p> <p> <bold>Session Script</bold> The session script consisted of step-by-step instructions for the primary researcher to engage in during each research session. It included directions for the primary researcher to share the MSWO excerpt and how to respond if a participant asked a question which ensured consistency in researcher responses within and across participants. See Supplementary Materials, Appendix C for a complete description of the session script.</p> <p> <bold>Confederate Response Data Sheet</bold> Each confederate response data sheet consisted of a sequence of responses for the confederate to engage in during the research session (see Supplementary Materials, Appendix D for an example). Specifically, each data sheet included 15 responses, one for each trial of the MSWO. There were a total of 10 variations in the confederate response data sheet.</p> <p>The sequence of the responses on each confederate data sheet were randomly generated. There were seven different responses the confederate could engage in (see Table 1) throughout a research session. Given that each research session consisted of 15 total trials, some of the responses were included in the sequence more than once and some were included only one time. To ensure that each of the seven responses were included at least once within each of the confederate response data sheet variations, the primary researcher used a random number generator (Maple Tech International LLC, [<reflink idref="bib33" id="ref55">33</reflink>]) to identify a trial number for each potential response to occur in. Once all seven responses were included in the sequence, the primary researcher then used the random number generator to determine how many additional times each response would be included in the data sheet, with no response being repeated more than three times. Once the frequency of each response was determined, the primary researcher continued to use the random number generator to determine which trial each response would occur in until all trials had a response.</p> <p>For each research session, the assigned confederate response data sheet was randomly selected using a random number generator (Maple Tech International LLC, [<reflink idref="bib33" id="ref56">33</reflink>]). Once a confederate response data sheet (e.g., assigned confederate response data sheet #8) was used for a participant, it was not replaced into the potential selections for future research sessions, until all 10 confederate data sheets had been used for that participant. Once all 10 confederate response data sheets had been used for that participant, all 10 had the opportunity to be selected again.</p> <hd id="AN0177897748-10">Primary Data Collector Materials</hd> <p>The primary data collector materials consisted of a pen or pencil, a timer, the video recording of the research session, an email service, and data sheets. The primary data collector used three different data sheets. The first data sheet was used to record the confederate's stimulus selections, which was identical to the participant's data sheet. The second data sheet was used to record participant's adherence to the steps of the MSWO preference assessment, the dependent variable (i.e., procedural fidelity). Finally, the third data sheet was the assigned confederate response data sheet, so the primary data collector would know the sequence of responses the primary researcher/confederate engaged in during the research session.</p> <hd id="AN0177897748-11">Measurement</hd> <p></p> <hd id="AN0177897748-12">Dependent Variable</hd> <p>The primary dependent variable was <emph>fidelity of participant implementation</emph> (procedural fidelity). Procedural fidelity was the degree to which the participant implemented the MSWO as intended (Cooper et al., [<reflink idref="bib11" id="ref57">11</reflink>]; Gast &amp; Ledford, [<reflink idref="bib18" id="ref58">18</reflink>]). The dependent variable comprised of component responses derived from a task analysis that depicted the instructional behaviors for the MSWO preference assessment (see Supplementary Materials, Appendix E). The task analysis was developed using the MSWO steps provided in DeLeon and Iwata ([<reflink idref="bib14" id="ref59">14</reflink>]), DeLeon et al. ([<reflink idref="bib15" id="ref60">15</reflink>]), and Sipila-Thomas et al. ([<reflink idref="bib46" id="ref61">46</reflink>]) and modified for delivery in a remote context. The task analysis for the MSWO contained 24 total steps. Each step in the task analysis was coded as a whole occurrence (i.e., always occurred) or a whole nonoccurrence (i.e., never occurred). In order for a participant's response to be scored as a whole occurrence, the participant had to correctly engage in the response multiple times (e.g., engaged in the response correctly seven out of seven times) throughout the research session. If at any point the participant did not engage in the response correctly (e.g., engaged in the response correctly five out of seven times), the entire response was scored as a whole nonoccurrence for that session. The percentages of occurrences for each session were calculated by dividing the total number of occurrences (steps implemented correct in the task analysis) by the sum of occurrences and non-occurrences. The quotient was then multiplied by 100 to yield a percentage (Cooper et al., [<reflink idref="bib11" id="ref62">11</reflink>]).</p> <p>For one participant, due to participant characteristics that cannot be disclosed for confidentiality reasons, the task analysis for the MSWO was modified to contain 23 total steps, instead of 24 total steps. Step 5, which was to ensure that the confederate was attending to the items (looking at the participant or the items), was eliminated from the task analysis and the participant was not required to engage in this step. All other 23 steps remained the same.</p> <hd id="AN0177897748-13">Interobserver Agreement</hd> <p>A second graduate student served as a second observer and measured participant procedural fidelity for at least 30% of sessions for all baseline and intervention conditions across all participants. The graduate student was trained by the primary researcher on the data collection process in a manner identical to that of the primary data collector. Interobserver agreement (IOA) was calculated for the dependent variable and all participants and met standards for single-case research (Kratochwill et al., [<reflink idref="bib27" id="ref63">27</reflink>]). When calculating IOA for participant procedural fidelity, an agreement was scored if the primary data collector and the second observer recorded the same behavior in the task analysis as a whole occurrence or a whole nonoccurrence. For example, an agreement was scored if both the primary data collector and second observer recorded "always occurred" for Step 1 on the task analysis. A disagreement was recorded if the primary data collector and the second observer did not record the same behavior in the task analysis as a whole occurrence or a whole nonoccurrence. For example, a disagreement was recorded if the primary data collector recorded "always occurred" for Step 1, but the second observer recorded "never occurred" for Step 1. IOA was calculated by dividing the total number of agreements by the sum of agreements and disagreements. The quotient was then multiplied by 100 to yield a percentage (Cooper et al., [<reflink idref="bib11" id="ref64">11</reflink>]).</p> <p>Total IOA across all conditions was 96.7% for Riley (range: 87.5–100.0%), 99.0% for Olivia (range: 95.8–100.0%), and 99.3% for Ava (range: 95.8–100.0%). Total IOA across all conditions was 98.6% for Layla (range: 95.8–100.0%), 98.3% for Ellie (range: 95.8–100.0%), and 95.8% for Kennedy (range: 91.7–100.0%). Average instructor IOA scores for each participant, across the two conditions (i.e., baseline and intervention), are displayed in Table 2.</p> <p>Table 2 Average IOA scores for each participant</p> <p> <ephtml> &lt;table frame="hsides" rules="groups"&gt;&lt;thead&gt;&lt;tr&gt;&lt;th align="left"&gt;&lt;p&gt;Participant&lt;/p&gt;&lt;/th&gt;&lt;th align="left"&gt;&lt;p&gt;Baseline (%)&lt;/p&gt;&lt;/th&gt;&lt;th align="left"&gt;&lt;p&gt;Intervention (%)&lt;/p&gt;&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Riley&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;91.7&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;100.0&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Olivia&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;97.9&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;100.0&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Ava&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;98.6&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;100.0&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Layla&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;97.9&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;99.0&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Ellie&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;97.9&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;98.6&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Kennedy&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;94.5&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;97.9&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <hd id="AN0177897748-14">Primary Researcher/Confederate Procedural Fidelity</hd> <p>A third graduate student measured primary researcher/confederate procedural fidelity for at least 30% of sessions for all baseline and intervention conditions across all participants. The graduate student was trained by the primary researcher on the data collection process prior to the start of the study in a manner similar to that of the primary data collector. The training was identical, except that the primary researcher reviewed each step of the researcher procedural fidelity data sheet with the graduate student, instead of the participant procedural fidelity data sheet.</p> <p>The primary researcher/confederate procedural fidelity was the degree to which the primary researcher/confederate implemented the independent variable (i.e., email feedback), followed the session script, and engaged in the predetermined sequence behaviors (i.e., assigned confederate response data sheet) as intended (Cooper et al., [<reflink idref="bib11" id="ref65">11</reflink>]; Gast &amp; Ledford, [<reflink idref="bib18" id="ref66">18</reflink>]). Primary researcher/confederate procedural fidelity was derived from a task analysis that depicted the behaviors the primary researcher/confederate engaged in before, during, and after each research session (see Supplementary Materials, Appendix F). Each step in the task analysis was scored "yes" if the primary researcher/confederate implemented that step correctly and "no" if the step was implemented incorrectly or was omitted. The primary researcher/confederate researcher procedural fidelity was calculated by dividing the sum of "yes" scores by the sum of "yes" plus "no" scores. The quotient was then multiplied by 100 to yield a percentage (Cooper et al., [<reflink idref="bib11" id="ref67">11</reflink>]). Procedural fidelity across all conditions was 100% for all participants.</p> <hd id="AN0177897748-15">Experimental Design</hd> <p>A multiple probe design across participants design was used to evaluate the effects of the MSWO preference assessment with email feedback on the participants' procedural fidelity (Gast et al., [<reflink idref="bib19" id="ref68">19</reflink>]). A multiple probe design systematically introduces the independent variable (i.e., email feedback) to evaluate its effects on the dependent variable (i.e., procedural fidelity) and controls for threats to internal validity. The independent variable was introduced on one occasion for each participant, for a total of six opportunities to demonstrate experimental control and treatment effect across all six participants (Cooper et al., [<reflink idref="bib11" id="ref69">11</reflink>]). The multiple probe design consisted of two experimental conditions: (a) baseline and (b) intervention. Participants moved from the baseline condition to the intervention condition once visual analysis of data suggested a steady state of responding had been achieved (see Sidman, [<reflink idref="bib45" id="ref70">45</reflink>]). The intervention condition for each participant ended once at least five research sessions were completed and a stead state of responding had been achieved.</p> <p>Additionally, based on when participants were enrolled in the study, the multiple probe design was conducted either nonconcurrently or concurrently across participants. Participants were assigned to either a non-concurrent or concurrent design based on the order in which they enrolled in the study. For the first set of participants to enroll in the study, Riley, Olivia, and Ava (see Fig. 1), the multiple probe design was nonconcurrent as they began the study on different days (i.e., Olivia started two days after Riley and Ava started six days after Olivia). For the second set of participants to enroll in the study, Layla, Ellie, and Kennedy (see Fig. 2), the multiple probe design was concurrent as they all began the study on the same day (see Slocum et al., [<reflink idref="bib48" id="ref71">48</reflink>] for a primer on concurrent and nonconcurrent multiple baseline designs and variations).</p> <p>Graph: Fig. 1 Percentage of correct implementation for three participants (Riley, Olivia, and Amy) across conditions depicting the nonconcurrent session schedule</p> <p>Graph: Fig. 2 Percentage of correct implementation for three participants (Layla, Ellie, and Kennedy) across conditions</p> <hd id="AN0177897748-16">Procedure</hd> <p>Each research session lasted approximately 15–20 min. Baseline sessions were conducted two-to-three times per week as close in days as possible, with the exception of baseline probes which consisted of at least five days (based on recommendations from Gast &amp; Ledford, [<reflink idref="bib18" id="ref72">18</reflink>]) between the previous baseline session and previous baseline probe session. Identical to baseline sessions, intervention sessions were conducted two-to-three times per week as close in days as possible. All research sessions occurred at times and dates convenient for the participant and when the primary researcher was available. Research sessions always began when the primary researcher gave the instruction to begin (i.e., "Okay, now you can begin implementing the assessment. Once you are finished, please let me know").</p> <p>All research sessions continued until: (a) the participant indicated they completed the assessment or (b) the participant did not engage in a target response from the task analysis for two min. At the end of each session, the participant was asked to display their data sheet on the screen and the primary researcher thanked the participant for attending. All research sessions were conducted via Zoom and were recorded for data collection purposes. The recording started when the participant joined Zoom and concluded immediately after the participant displayed their data sheet on the screen and the primary researcher thanked them for attending.</p> <hd id="AN0177897748-17">Initial Research Meeting and Training</hd> <p>Prior to the start of the first baseline session, all participants attended an initial research meeting with the primary researcher. The initial meeting was approximately 40 min in length. During the initial research meeting, the primary researcher reviewed the informed consent, shared a demographic questionnaire with the participant, displayed the MSWO training video in its entirety, gathered information for data sheet preference (i.e., mailed or emailed), and obtained the participant's availability for session scheduling purposes.</p> <hd id="AN0177897748-18">Baseline</hd> <p>The purpose of this condition was to measure participant behavior prior to the introduction of the email performance-based feedback. At the beginning of each research session, participants were given up to 15 min to review the MSWO excerpt derived from DeLeon and Iwata ([<reflink idref="bib14" id="ref73">14</reflink>]) described above. After 15 min, or when the participant reported they finished reviewing the excerpt, the participant was asked to implement the MSWO preference assessment with the primary researcher/confederate.</p> <p>During implementation of the MSWO preference assessment, the primary researcher/confederate engaged in the sequence of responses determined by the assigned confederate response data sheet (see Supplementary Materials, Appendix D for an example). However, if the participant did not engage in the steps following a specific type of selection correctly, the primary researcher/confederate systematically adjusted their next selection response. For example, following a two-item sequential selection response, the participant should have engaged in Steps 16–21 on the task analysis. However, occasionally instead of ignoring the second selected item and continuing to engage with the first selected item for 30 s (Step 16), participants engaged with both of the selected items and then re-presented the remaining non-selected items. When this occurred, the primary researcher/confederate skipped the next predetermined trial (e.g., Trial #2) and moved onto the next trial (e.g., Trial #3) to match the number of items that were represented in the array. As another example, following a no selection response, if the participant continued to represent the remaining items, instead of ending that round of trials (i.e., one round equals five trials, one trial for each of the five items) and recording them as not selected (Step #22), the primary researcher/confederate continued to engage in no selection responses until the next round of trials began. Finally, as a third example, if the participant engaged in errors such as those noted above and continued to represent items in the array and the primary researcher/confederate had completed all of the predetermined trials, the primary researcher/confederate engaged in no selection responses until the participant indicated that had completed the MSWO.</p> <p>After the participant indicated they had completed the MSWO (e.g., stating "I'm done"), the primary researcher/confederate provided the participant time to calculate their results. Once the participant reported they had completed their results, the primary researcher/confederate instructed the participant to display their data sheet on the screen for a few seconds. No further instructions were given, and no feedback was provided. If a participant asked the primary researcher/confederate researcher a question during the research session, they were informed that the primary researcher/confederate could not answer their question at that time and that they should do the best they could.</p> <p>After each baseline research session ended, the primary researcher sent the participant an email without any feedback components. The components that were included in the baseline emails were: (a) a general positive opening statement, (b) a request for a response (i.e., posing a scheduling question and asking for a reply), and (c) a positive closing statement. Emails were sent to participants during the baseline condition to ensure that participants were accessing, reading, and responding to emails and to further isolate the effects of feedback during the subsequent condition (see Supplementary Materials, Appendix G, for an example e-mail).</p> <hd id="AN0177897748-19">Intervention</hd> <p>The purpose of this condition was to evaluate participant behavior when the participant was provided with email feedback. Experimental procedures in the intervention condition were identical to that of the baseline condition, with the exception of the email sent to participants following each research session. Email followed a format similar to that of baseline but also included components specific to supportive (i.e., the number of steps the participant engaged in correctly and comments about the participants implementation) and corrective feedback (i.e., the steps the participant did not engage in correctly). The components included in the intervention emails were: (a) a general positive opening statement, (b) supportive feedback, (c) corrective feedback, (d) a request for a response (i.e., posing a scheduling question and asking for a reply), and (e) a positive closing statement. (See Supplementary Materials, Appendix H for an example intervention email.)</p> <hd id="AN0177897748-20">Data Analysis</hd> <p>Following each research session, data for each participant were graphed and reviewed by the primary researcher for trend, level, and variability of data to evaluate intervention effects (Cooper et al., [<reflink idref="bib11" id="ref74">11</reflink>]). Following the completion of the study, Tau-<emph>U</emph> was calculated to supplement visual analysis of data and to provide a secondary measure of treatment effect. Tau-<emph>U</emph> is a statistical analysis that combines non-overlap analysis between phases with trend from within the intervention phase. Tau-<emph>U</emph> is dynamic in that it can calculate trend only, non-overlap between phases only, as well as combinations of trend and overlap between multiple phases (e.g., baseline and intervention conditions; Parker et al., [<reflink idref="bib39" id="ref75">39</reflink>]).</p> <hd id="AN0177897748-21">Results</hd> <p></p> <hd id="AN0177897748-22">Participant Procedural Fidelity</hd> <p></p> <hd id="AN0177897748-23">Nonconcurrent Set of Participants</hd> <p> <bold>Riley</bold> Riley's percentage of correct implementation data are displayed in Fig. 1, top panel. In the baseline condition, Riley's percentage of correct implementation ranged from 12.5 to 33.0% (<emph>M</emph> = 27.3%), with the last three sessions remaining stable at 33.0%. During the first intervention session, Riley's percentage of correct implementation increased to 66.7% and then varied from 62.5 to 91d.7%, until it remained stable at 100% for two sessions and 95.8% in the last session (<emph>M</emph> = 86.5%; range: 62.5–100.0%). See Supplementary Materials for raw data for Riley and the remaining participants.</p> <p> <bold>Olivia</bold> Olivia's percentage of correct implementation data are displayed in Fig. 1, middle panel. In the baseline condition, Olivia's percentage of correct implementation was at 54.0% in the first session and then decreased and remained stable at 41.7% for the last four sessions (<emph>M</emph> = 43.8%). During the first intervention session, Olivia's percentage of correct implementation increased to 87.5% and then remained stable between 95.8 and 100% for the last four sessions (<emph>M</emph> = 96.7%; range: 87.5–100.0%).</p> <p> <bold>Ava</bold> Ava's percentage of correct implementation data are displayed in Fig. 1, bottom panel. In the baseline condition, Ava's percentage of correct implementation ranged from 41.7 to 54.2% (<emph>M</emph> = 48.8%). During the first intervention session, Ava's percentage of correct implementation increased to 70.8% and then varied from 70.8 to 87.5%, until it remained stable between 95.8 and 100.0% for the last three sessions (<emph>M</emph> = 86.9%; range: 70.8–100.0%).</p> <hd id="AN0177897748-24">Concurrent Set of Participants</hd> <p> <bold>Layla</bold> Layla's percentage of correct implementation data are displayed in Fig. 2, top panel. In the baseline condition, Layla's percentage of correct implementation ranged from 0.0 to 12.5% (<emph>M</emph> = 3.34%). During the first intervention session, Layla's percentage of correct implementation increased to 33.3% and then varied from 16.6 to 79.2%, until it remained stable between 87.5 and 95.8% for the last three sessions (<emph>M</emph> = 69.3%; range: 16.6–95.8%).</p> <p> <bold>Ellie</bold> Ellie's percentage of correct implementation data are displayed in Fig. 2, middle panel. In the baseline condition, Ellie's percentage of correct implementation began at 30.4% in the first session and then varied between 17.4 and 21.7% across three sessions, with the last two sessions at 13.0% (<emph>M</emph> = 19.5%). During the first intervention session, Ellie's percentage of correct implementation increased to 86.9% and then varied from 86.9 to 95.6%, until it remained stable between 95.6 and 100.0% for the last four sessions (<emph>M</emph> = 94.5%; range: 86.9–100.0%).</p> <p> <bold>Kennedy</bold> Kennedy's percentage of correct implementation data are displayed in Fig. 2, bottom panel. In the baseline condition, Kennedy's percentage of correct implementation ranged from 25.0 to 41.6% (<emph>M</emph> = 32.7%). During the first intervention session, Kennedy's percentage of correct implementation increased to 75.0% and then varied from 79.2 to 91.6%, until it remained stable at 95.8% for the last three sessions (<emph>M</emph> = 88.9%; range: 75.0–95.8%).</p> <hd id="AN0177897748-25">Tau-U</hd> <p>The researchers calculated Tau-<emph>U</emph> using a web-based Tau-<emph>U</emph> calculator (<ulink href="http://singlecaseresearch.org/calculators/tau-u">http://singlecaseresearch.org/calculators/tau-u</ulink>) for the participants. The Tau-<emph>U</emph> for Riley was 1 (<emph>p</emph> &lt; 0.05, z = 2.93), the Tau-<emph>U</emph> for Olivia was 1 (<emph>p</emph> &lt; 0.05, z = 2.74), the Tau-<emph>U</emph> for Ava was 1 (<emph>p</emph> &lt; 0.05, z = 3.13), the Tau-<emph>U</emph> for Layla was 1 (<emph>p</emph> &lt; 0.05, z = 3.12), the Tau-<emph>U</emph> for Ellie was 1 (<emph>p</emph> &lt; 0.05, z = 3.10), and the Tau-<emph>U</emph> for Kennedy was also 1 (<emph>p</emph> &lt; 0.05, z = 3.00). Based on the weighted Tau-<emph>U</emph> for all participants, the intervention had a large or strong effect with 100% of participants' data showing significant improvement (<emph>p</emph> &lt; 0.001, z = 7.33) from baseline to intervention with 95% CIs [0.73, 1]. See Supplementary Materials for raw outputs of Tau-<emph>U</emph> calculations.</p> <hd id="AN0177897748-26">Discussion</hd> <p>Overall, the findings indicate the email performance-based feedback was effective in increasing procedural fidelity of MSWO preference assessment implementation in the pre-service teacher participants. All six participants implemented the MSWO preference assessment with high levels of procedural fidelity following email performance-based feedback. These results extend previous findings (i.e., Artman-Meecker &amp; Hemmeter, [<reflink idref="bib1" id="ref76">1</reflink>]; Gomez et al., [<reflink idref="bib20" id="ref77">20</reflink>]; Gorton et al., [<reflink idref="bib21" id="ref78">21</reflink>]; Hemmeter et al., [<reflink idref="bib24" id="ref79">24</reflink>]; Martinez Cueto et al., [<reflink idref="bib34" id="ref80">34</reflink>]) by isolating the effects of email feedback. Additionally, these results support previous findings (i.e., Barton et al., [<reflink idref="bib2" id="ref81">2</reflink>], [<reflink idref="bib3" id="ref82">3</reflink>]; Coogle et al., [<reflink idref="bib9" id="ref83">9</reflink>]) suggesting that email performance-based feedback alone is effective in increasing target behavior(s).</p> <p>In the present study, we used a conservative "all or nothing" whole occurrence measure (similar approach used in Sipila-Thomas et al., [<reflink idref="bib46" id="ref84">46</reflink>]) that was biased toward deflating participant performance because a single error resulted in the entire step being marked as incorrectly implemented. An alternative measure would have been a per opportunity measure; however, the use of this measure can be biased toward inflating participant performance (Ledford et al., [<reflink idref="bib30" id="ref85">30</reflink>]). For example, in this study there were 15 total trials and each trial consisted of 24 different behaviors the participant could engage in. Therefore, the participant would have 24 opportunities that would be included in the calculation of a whole occurrence measure and 360 opportunities that would be included in the calculation of a per opportunity measure. When calculating the whole occurrence measure, if the participant incorrectly engaged in one specific behavior one time, during the 15 trials, and incorrectly engages in four other specific behaviors one time each, all five behaviors would be marked incorrect and would result in a score of 79.2% or 19 out of 24 correct, whereas, if a per opportunity measure was used for each trial, the participant would have scored a 98.6% or 355 out of 360 correct. Future research may evaluate the extent to which the whole occurrence measure deflates participant performance and determine if practitioners should proceed with caution in using the whole occurrence measure over the per opportunity measure in practice.</p> <p>Although the email performance-based feedback increased procedural fidelity for all six participants and Tau-<emph>U</emph> results indicated the intervention had a large or strong effect, the accuracy of responding varied across participants. Layla and Kennedy did not reach 100% procedural fidelity, though they both reached 95.8% which was implementing 23 out of the 24 steps correctly. Additionally, the number of sessions to reach high levels of fidelity varied across participants. Olivia, Ava, Ellie, and Riley reached 100% procedural fidelity after being exposed to two to six intervention research sessions, while Kennedy and Layla were exposed to six and 11 intervention research sessions, respectively, but both did not reach 100% procedural fidelity. Previous research on training individuals to implement behavior analytic procedures found similar results when using written or vocal instructions (i.e., instruction-based methods) alone (Iwata et al., [<reflink idref="bib25" id="ref86">25</reflink>]; Roscoe et al., [<reflink idref="bib42" id="ref87">42</reflink>]; Shapiro &amp; Kazemi, [<reflink idref="bib44" id="ref88">44</reflink>]; Vladescu et al., [<reflink idref="bib49" id="ref89">49</reflink>]). This variation in responding (i.e., participants do not always achieve 100% fidelity) is an important finding because it demonstrates that rate of performance improvements are idiosyncratic across participants. Therefore, some participants may require additional support (e.g., video models of correct implementation) beyond email performance-based feedback in order to reach 100% procedural fidelity. Previous researchers have found the use of multiple procedures (e.g., video modeling, written self-instruction packages, feedback) to train individuals to implement an intervention have been effective in increasing procedural fidelity (Shapiro &amp; Kazemi, [<reflink idref="bib44" id="ref90">44</reflink>]). Future research could conduct a component analysis to evaluate what additional procedures alongside email feedback may be needed to achieve 100% procedural fidelity. For example, a study could evaluate the effects of (a) email feedback alone, (b) email feedback with video modeling, and (c) email feedback, video modeling, and roleplaying on procedural fidelity to determine if one or all components are necessary to achieve 100% procedural fidelity. Future researchers may also consider evaluating whether 100% procedural fidelity is necessary in order to increase client outcomes (see Groskreutz et al., [<reflink idref="bib23" id="ref91">23</reflink>]). If it is determined that 100% procedural fidelity is not necessary, future research could conduct a component analysis to evaluate additional procedures alongside email feedback in order to achieve the ideal procedural fidelity percentage.</p> <p>When conducting a post hoc analysis of participant errors, the two steps most frequently implemented incorrectly were (<reflink idref="bib1" id="ref92">1</reflink>) calculating stimulus rankings (Step 23) and (<reflink idref="bib2" id="ref93">2</reflink>) accurately transcribing data (Step 24). Incorrectly implementing these two steps when conducting an MSWO preference assessment can be a major issue for two reasons. First, if the implementer used an item that was miscalculated as a top item when providing treatment to an individual with ASD, there may be noticeably different effects in treatment delivery because for treatment delivery to be successful (i.e., increase desired behavior or decrease undesired behavior), an effective reinforcer is required (Bottini &amp; Gillis, [<reflink idref="bib5" id="ref94">5</reflink>]; Cooper et al., [<reflink idref="bib12" id="ref95">12</reflink>]). Second, if the implementer did not accurately record data during the implementation of the MSWO, the inaccurate data would result in incorrect decision-making (see Cox &amp; Brodhead, [<reflink idref="bib13" id="ref96">13</reflink>]) and may impact the effects of treatment delivery. However, it is unclear why these two specific steps were the two most common errors. One potential cause for the errors could be that the feedback was less effective in explaining how to correctly implement these steps. A second potential cause could be session fatigue as these two steps came at the end of the research session. Future research could evaluate if participants experience similar difficulty when calculating results and recording data when implementing other preference assessment or behavior analytic procedures. For example, a study could evaluate the use of email performance-based feedback on the implementation of a multiple stimulus <emph>with</emph> replacement preference assessment (DeLeon &amp; Iwata, [<reflink idref="bib14" id="ref97">14</reflink>]), paired stimulus preference assessment (Fisher et al., [<reflink idref="bib17" id="ref98">17</reflink>]), or free operant preference assessment (Roane et al., [<reflink idref="bib40" id="ref99">40</reflink>]) and assess if participants engage in incorrect responses when calculating stimulus rankings or recording data. Additionally, future research could evaluate if another form of feedback (e.g., video model, see DiGennaro-Reed et al., [<reflink idref="bib16" id="ref100">16</reflink>]) is required in order to correctly calculate results.</p> <hd id="AN0177897748-27">Extension of Previous Literature</hd> <p>The present study extends previous research in at least three ways. First, we standardized the email responses provided to each of the participants to control for variations in the email feedback. In previous studies, emails were not standardized beyond their general frameworks (e.g., general positive opening statements, supportive feedback, corrective feedback, request for response; Artman-Meecker &amp; Hemmeter, [<reflink idref="bib1" id="ref101">1</reflink>]; Barton et al., [<reflink idref="bib2" id="ref102">2</reflink>], [<reflink idref="bib3" id="ref103">3</reflink>]; Coogle et al., [<reflink idref="bib9" id="ref104">9</reflink>]; Gorton et al., [<reflink idref="bib21" id="ref105">21</reflink>]; Hemmeter et al., [<reflink idref="bib24" id="ref106">24</reflink>]; Martinez Cueto et al., [<reflink idref="bib34" id="ref107">34</reflink>]; Zhu et al., [<reflink idref="bib51" id="ref108">51</reflink>]). As a result, it was unclear how researchers composed the email feedback for each participant and whether the feedback was specific or general feedback, making replication of these studies difficult. In the present study, we used a pool of specific responses for each component of the email that the primary researcher pulled from. As a result, we were able to rule out researcher variation as a potential source of influence in the present study and demonstrated that providing standardized email responses can improve target behaviors. However, comparisons between customized and standardized email performance-based feedback would still need to be conducted. Additionally, future researchers should consider evaluating other feedback components (e.g., specific vs. general feedback, frequent vs. infrequent feedback, group vs. individual feedback, and source of feedback; Novak et al., [<reflink idref="bib36" id="ref109">36</reflink>]; Sleiman et al., [<reflink idref="bib47" id="ref110">47</reflink>]) within email feedback in order to further refine email performance-based feedback interventions.</p> <p>Second, we implemented the training component (i.e., the MSWO training video) prior to the baseline condition in order to isolate the effects of the email feedback and to ensure that any changes that occurred between the baseline and intervention conditions were likely a result of the email feedback alone. The results of the present study demonstrated that providing email performance-based feedback alone can improve implementation of an MSWO preference assessment. Future research could evaluate if email feedback would increase individuals' procedural fidelity when implementing procedures such as match-to-sample, manding, and imitation, and other behavioral interventions. Additionally, future researchers could continue to evaluate the use of email feedback with behavior analytic procedures that consist of varying number of steps of increasing difficulty to understand the conditions under which it may or may not have functional value in improving employee performance (e.g., functional analyses, functional communication training).</p> <p>Third, we used a confederate with a specific list of responses to perform during each research session. Previous literature (i.e., Artman-Meecker &amp; Hemmeter, [<reflink idref="bib1" id="ref111">1</reflink>]; Barton et al., [<reflink idref="bib2" id="ref112">2</reflink>], [<reflink idref="bib3" id="ref113">3</reflink>]; Coogle et al., [<reflink idref="bib9" id="ref114">9</reflink>]; Gomez et al., [<reflink idref="bib20" id="ref115">20</reflink>]; Gorton et al., [<reflink idref="bib21" id="ref116">21</reflink>]; Hemmeter et al., [<reflink idref="bib24" id="ref117">24</reflink>]; Martinez Cueto et al., [<reflink idref="bib34" id="ref118">34</reflink>]) did not control for variation in the instructional environment as those studies involved classroom settings where teachers engaged with students. As a result, opportunities for participants to respond may have been at least partially affected by student behavior within those settings. By using a confederate, we eliminated or reduced variation in the instructional environment by holding errors and correct responses constant across all research sessions. Additionally, participants were exposed to each confederate response during each research session, which mimicked the responses a child with ASD may engage in during an MSWO preference assessment. As a result, we were able to evaluate participant behavior in the presence of responses likely encountered in an applied setting across the entire experiment. However, in the present study we did not assess if high procedural fidelity in the presence of a confederate would generalize to a child with ASD. Future research could evaluate if the participant's fidelity of implementation would generalize to implementing an MSWO preference assessment with a child with ASD after receiving email feedback based on their performance implementing the MSWO with a confederate.</p> <hd id="AN0177897748-28">Limitations</hd> <p>Several limitations of the present study should be noted. The first limitation of this study was that it was unclear what specific components (e.g., supportive feedback, corrective feedback) of the email performance-based feedback were responsible for increases in participant procedural fidelity. Second, though all participants achieved high levels of procedural fidelity, the extent to which these gains maintain or persist long term is unknown because we only evaluated the immediate effects the email performance-based feedback had on implementation of the MSWO preference assessment. Third, the primary data collector reviewed and scored research session videos in the order in which they were recorded; as a result, the primary data collector was aware of the experimental conditions the participants were exposed to during each research session. Finally, we evaluated participant responding in the presence of a confederate instead of an individual with ASD. Although the use of a confederate allowed the researchers to control for variation in the instructional environment and provided opportunities for participants to respond to multiple learner responses, the extent to which high levels of procedural fidelity would generalize to individuals with ASD is unclear. Results of previous training studies (e.g., Lipschultz et al., [<reflink idref="bib32" id="ref119">32</reflink>]) that have used a confederate and then evaluated participant performance in the presence of a child found high levels could be achieved. However, future research could evaluate the effects of email performance-based feedback on the implementation of an MSWO preference assessment with individuals with ASD in person or within a remote context. Finally, to our knowledge, this was the first study to evaluate email performance-based feedback effects on a procedure that consists of more than eight discrete steps. Future research could continue to evaluate email performance-based feedback on other multiple step behavior analytic procedures (e.g., implementing a gross motor imitation program, conducting a functional analysis) in order to understand the extent to which email feedback can be used. Though email performance-based feedback appears to be helpful, until future research is conducted, we urge caution in viewing it as substitute for in-person feedback. Instead, email may be considered another tool to delivering high-quality feedback to help teachers improve fidelity of implementation of behavioral procedures.</p> <hd id="AN0177897748-29">Funding</hd> <p>Michigan State University's College of Education provided financial support for participant recruitment.</p> <hd id="AN0177897748-30">Data Availability</hd> <p>The data that support the findings of this study are available as supplementary materials.</p> <hd id="AN0177897748-31">Declarations</hd> <p></p> <hd id="AN0177897748-32">Conflict of interest</hd> <p>The authors do not have any conflict of interest to disclose.</p> <hd id="AN0177897748-33">Ethical Approval</hd> <p>All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.</p> <hd id="AN0177897748-34">Informed Consent</hd> <p>All participants provided written informed consent prior to commencement of the study.</p> <hd id="AN0177897748-35">Supplementary Information</hd> <p>Below is the link to the electronic supplementary material.</p> <p>Graph: Supplementary file1 (DOCX 431 KB)</p> <p>Graph: Supplementary file2 (XLSX 10 KB)</p> <p>Graph: Supplementary file3 (XLS 29 KB)</p> <hd id="AN0177897748-36">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0177897748-37"> <title> References </title> <blist> <bibl id="bib1" idref="ref30" type="bt">1</bibl> <bibtext> Artman-Meeker KM, Hemmeter LM. Effects of training and feedback on teachers' use of classroom preventative practices. Topics in Early Childhood Special Education. 2013; 33; 2: 112-123. 10.1177/02711214122447115</bibtext> </blist> <blist> <bibl id="bib2" idref="ref3" type="bt">2</bibl> <bibtext> Barton EE, Fuller EA, Schnitz A. The use of email to coach preservice early childhood teachers. Topics in Early Childhood Special Education. 2016; 36; 2: 78-90. 10.1177/0271121415612728</bibtext> </blist> <blist> <bibl id="bib3" idref="ref1" type="bt">3</bibl> <bibtext> Barton EE, Velez M, Pokorski EA, Domingo M. The effects of email performance based feedback delivered to teaching teams: A systematic replication. Journal of Early Intervention. 2020; 42; 2: 143-162. 10.1177/1053815119872451</bibtext> </blist> <blist> <bibl id="bib4" idref="ref10" type="bt">4</bibl> <bibtext> Barton EE, Wolery M. Evaluation of e-mail feedback on the verbal behaviors of pre-service teachers. Journal of Early Intervention. 2007; 30; 1: 55-72. 10.1177/105381510703000105</bibtext> </blist> <blist> <bibl id="bib5" idref="ref94" type="bt">5</bibl> <bibtext> Bottini S, Gillis J. A comparison of the feedback sandwich, constructive-positive feedback, and within session feedback for training preference assessment implementation. Journal of Organizational Behavior Management. 2021; 41; 1: 83-93. 10.1080/01608061.2020.1862019</bibtext> </blist> <blist> <bibl id="bib6" idref="ref51" type="bt">6</bibl> <bibtext> Brodhead, M. T. (2022). Training video. osf.io/xhdzw</bibtext> </blist> <blist> <bibl id="bib7" idref="ref50" type="bt">7</bibl> <bibtext> Chazin, K.T, &amp; Ledford, J. R. (2016). Multiple stimulus without replacement (MSWO) preference assessment. In Evidence-based instructional practices for young children with autism and other disabilities. Retrieved from <ulink href="http://ebip.vkcsites.org/multiple-stimulus-without-replacement">http://ebip.vkcsites.org/multiple-stimulus-without-replacement</ulink></bibtext> </blist> <blist> <bibl id="bib8" idref="ref12" type="bt">8</bibl> <bibtext> Coogle CG, Ottley JR, Storie S, Rahn NL, Kurowski-Burt A. eCoaching to enhance special educator practice and child outcomes. Infants and Young Children. 2017; 30; 1: 58-75. 10.1097/IYC.0000000000000082</bibtext> </blist> <blist> <bibl id="bib9" idref="ref37" type="bt">9</bibl> <bibtext> Coogle CG, Ottley JR, Storie S, Rahn NL, Kurowski-Burt A. Performance-based feedback to enhance preservice teachers' practice and preschool children's expressive communication. Journal of Teacher Education. 2020; 7; 2: 188-202. 10.1177/0022487118803583</bibtext> </blist> <blist> <bibtext> Coogle CG, Rahn NL, Ottley JR, Storie S. eCoaching across routines to enhance teachers' use of modeling. Teacher Education and Special Education. 2016; 39; 4: 227-245. 10.1177/0888406415621959</bibtext> </blist> <blist> <bibtext> Cooper JO, Heron TE, Heward WL. Applied behavior analysis. 20072; Pearson Education</bibtext> </blist> <blist> <bibtext> Cooper JO, Heron TE, Heward WL. Applied behavior analysis. 20203; Pearson Education. 10.26741/abaespana/2020.cooper3e</bibtext> </blist> <blist> <bibtext> Cox DJ, Brodhead MT. A proof of concept analysis of decision-making with time-series data. The Psychological Record. 2021; 71; 2: 349-366. 10.1007/s40732-020-00451-w</bibtext> </blist> <blist> <bibtext> DeLeon IG, Iwata BA. Evaluation of a multiple-stimulus presentation format for assessing reinforcer preferences. Journal of Applied Behavior Analysis. 1996; 29; 4: 519-533. 10.1901/jaba.1996.29.519. 8995834. 1284008</bibtext> </blist> <blist> <bibtext> DeLeon IG, Iwata BA, Roscoe EM. Displacement of leisure reinforcers by food during preference assessments. Journal of Applied Behavior Analysis. 1997; 30; 3: 475-484. 10.1901/jaba.1997.30-475. 9316260. 1284063</bibtext> </blist> <blist> <bibtext> DiGennero-Reed FD, Codding R, Catania CN, Maguire H. Effects of video modeling on treatment integrity of behavioral interventions. Journal of Applied Behavior Analysis. 2010; 43; 2: 291-295. 10.1901/jaba.2010.43-291</bibtext> </blist> <blist> <bibtext> Fisher W, Piazza CC, Bowman LG, Hagopian LP, Owens JC, Slevin I. A comparison of two approaches for identifying reinforcers for persons with severe and profound disabilities. Journal of Applied Behavior Analysis. 1992; 25; 2: 491-498. 10.1901/jaba.1992.25-491. 1634435. 1279726</bibtext> </blist> <blist> <bibtext> Gast DL, Ledford JR. Single case research methodology: Applications in special education and behavioral sciences. 20142; Routledge. 10.4324/9780203521892</bibtext> </blist> <blist> <bibtext> Gast DL, Llyod BP, Ledford JRGast DL, Ledford JR. Multiple baseline and multiple probe designs. Single subject research methodology: Applications in special education and behavioral sciences. 2014; Routledge: 252-296. 10.4324/9780203521892</bibtext> </blist> <blist> <bibtext> Gomez L, Barton EE, Winchester C, Locchetta B. Effects of email performance feedback on teachers' use of play expansions. Journal of Early Intervention. 2021; 43; 3: 235-254. 10.1177/1053815120969821</bibtext> </blist> <blist> <bibtext> Gorton K, Allday RA, Lane JD, Ault MJ. Effects of brief training plus electronic feedback on increasing quantity and intonation of behavior specific praise among preschool teachers. Journal of Behavioral Education. 2021. 10.1007/s10864-020-09427-w</bibtext> </blist> <blist> <bibtext> Graff RB, Karsten AM. Assessing preferences of individuals with developmental disabilities: A survey of current practices. Behavior Analysis in Practice. 2012; 5; 2: 37-48. 10.1007/bf03391822. 23730465. 3592488</bibtext> </blist> <blist> <bibtext> Groskreutz NC, Groskreutz MP, Higbee TS. Effects of varied levels of treatment integrity on appropriate toy manipulation in children with autism. Research in Autism Spectrum Disorders. 2011; 5; 4: 1358-1369. 10.1016/j.rasd.2011.01.018</bibtext> </blist> <blist> <bibtext> Hemmeter ML, Snyder P, Kinder K, Artman K. Impact of performance feedback delivered via electronic mail on preschool teachers' use of descriptive praise. Early Childhood Research Quarterly. 2011; 26; 1: 96-109. 10.1016/j.ecresq.2010.05.004</bibtext> </blist> <blist> <bibtext> Iwata BA, Wallace MD, Kahng S, Lindberg JS, Roscoe EM, Connners J, Hanley GP, Thompson RH, Worsdell AS. Skill acquisition in the implementation of functional analysis methodology. Journal of Applied Behavior Analysis. 2000; 33; 2: 181-194. 10.1901/jaba.2000.33-181. 10885526. 1284237</bibtext> </blist> <blist> <bibtext> Kang S, O'Reilly M, Lancioni G, Falcomata TS, Sigafoos J, Xu Z. Comparison of the predictive validity and consistency among preference assessment procedures: A review of the literature. Research in Developmental Disabilities. 2013; 34; 4: 1125-1133. 10.1016/j.ridd.2012.12.021. 23357675</bibtext> </blist> <blist> <bibtext> Kratochwill TR, Hitchcock JH, Horner RH, Levin JR, Odom SL, Rindskopf DM, Shadish WR. Single-case intervention research design standards. Remedial and Special Education. 2013; 34: 26-38. 10.1177/0741932512452794</bibtext> </blist> <blist> <bibtext> Krick Oborn KM, Johnson LD. Coaching via electronic performance feedback to support home visitors' use of caregiver coaching strategies. Topics in Early Childhood Special Education. 2015; 35; 3: 157-169. 10.1177/0271121415592411</bibtext> </blist> <blist> <bibtext> Leaf JB, Leaf R, Alcalay A, Leaf JA, Ravid D, Dale S, Kassardjian A, Tsuji K, Taubman M, McEachin H, Oppenheim-Leaf M. Utility of formal preference assessment for individuals diagnosed with autism spectrum disorder. Education and Training in Autism and Developmental Disabilities. 2015; 50; 2: 199-212</bibtext> </blist> <blist> <bibtext> Ledford JR, Wolery M, Gast DLGast DL, Ledford JR. Controversial and critical issues in single case research. Single subject research methodology: Applications in special education and behavioral sciences. 2014; Routledge: 377-396. 10.4324/9780203521892-14</bibtext> </blist> <blist> <bibtext> Lill JD, Shriver MD, Allen KD. Stimulus preference assessment decision-making system (SPADS): A decision-making model for practitioners. Behavior Analysis in Practice. 2021; 14: 1144-1156. 10.1007/s40617-020-00539-3. 34868818. 8586311</bibtext> </blist> <blist> <bibtext> Lipschultz JL, Vladescu JC, Reeve KF, Reeve SA, Dipsey CR. Using video modeling with voiceover instruction to train staff to conduct stimulus preference assessments. Journal of Developmental and Physical Disabilities. 2015; 27; 4: 505-532. 10.1007/s10882-015-9434-4</bibtext> </blist> <blist> <bibtext> Maple Tech International LLC. (2022). Random number generator. https://<ulink href="http://www.calculator.net/random-number-generator.html">www.calculator.net/random-number-generator.html</ulink></bibtext> </blist> <blist> <bibtext> Martinez Cueto AP, Barton EE, Bancroft JC. The effects of training and performance feedback on preservice teachers' use of statements that promote preschool children's social interactions. Journal of Positive Behavior Interventions. 2021. 10.1177/1098300721994200</bibtext> </blist> <blist> <bibtext> Miltenberger RGMiltenberger RG. Behavior skills training procedures. Behavior modification: Principles and procedures. 20125; Wadsworth: 217</bibtext> </blist> <blist> <bibtext> Novak MD, DiGennaro Reed FD, Erath TG, Blackman AL, Ruby SA, Pellegrino AJ. Evidence-based performance management: Applying behavioral science to support practitioners. Perspectives on Behavior Science. 2019; 42; 4: 955-972. 10.1007/s40614-019-00232-z. 31976469. 6901648</bibtext> </blist> <blist> <bibtext> O'Handley RD, Pearson S, Taylor C, Congdon M. Training preservice school psychologists to conduct a stimulus preference assessment. Behavior Analysis in Practice. 2021; 14; 2: 445-450. 10.1007/s40617-020-00537-5. 34150457. 8149553</bibtext> </blist> <blist> <bibtext> Park J, Johnson DA, Moon K, Lee J. The interaction effects of frequency and specificity of feedback on work performance. Journal of Organizational Behavior Management. 2019; 39; 3–4: 164-178. 10.1080/01608061.2019.1632242</bibtext> </blist> <blist> <bibtext> Parker RI, Vannest KJ, Davis JL, Sauber SB. Combining nonoverlap and trend for single case research: Tau-U. Behavior Therapy. 2011; 42; 2: 284-299. 10.1016/j.beth.2010.08.006. 21496513</bibtext> </blist> <blist> <bibtext> Roane HS, Vollmer TR, Ringdahl JE, Marcus BA. Evaluation of a brief stimulus preference assessment. Journal of Applied Behavior Analysis. 1998; 31; 4: 605-620. 10.1901/jaba.1998.31-605. 9891397. 1284151</bibtext> </blist> <blist> <bibtext> Rosales R, Gongola L, Homlitas C. An evaluation of video modeling with embedded instructions to teach implementation of stimulus preference assessments. Journal of Applied Behavior Analysis. 2015; 48; 1: 209-214. 10.1002/jaba.174. 25927112</bibtext> </blist> <blist> <bibtext> Roscoe EM, Fisher WW, Glover AC, Volkert VM. Evaluating the relative effects of feedback and contingent money for staff training of stimulus preference assessments. Journal of Applied Behavior Analysis. 2006; 39; 1: 63-77. 10.1901/jaba.2006.7-05. 16602386. 1389611</bibtext> </blist> <blist> <bibtext> Schles RA, Robertson RE. The role of performance feedback and implementation of evidence-based practices for preservice special education teachers and student outcomes: A review of the literature. Teacher Education and Special Education. 2019; 42; 1: 36-48. 10.1177/0888406417736571</bibtext> </blist> <blist> <bibtext> Shapiro M, Kazemi E. A review of training strategies to teach individuals implementation of behavioral interventions. Journal of Organizational Management. 2017; 37; 1: 32-62. 10.1080/01608061.2016.1267066</bibtext> </blist> <blist> <bibtext> Sidman, M. (1960). Tactics of scientific research: evaluating experimental data in psychology. Authors Cooperative.</bibtext> </blist> <blist> <bibtext> Brodhead MT. An evaluation of fidelity of implementation of a manualized social-play intervention. Journal of Developmental and Physical Disabilities. 2021; 33; 1: 391-412. 10.1007/s10882-020-09754-w</bibtext> </blist> <blist> <bibtext> Sleiman AA, Sigurjonsdottir S, Elnes A, Gage NA, Gravina NE. A quantitative review of performance feedback in organizational settings (1998–2018). Journal of Organization Behavior Management. 2020; 40; 3–4: 303-332. 10.1080/01608061.2020.1823300</bibtext> </blist> <blist> <bibtext> Slocum TA, Pinkelman SE, Joslyn PR, Nichols B. Threats to internal validity in multiple-baseline design variations. Perspectives on Behavior Science. 2022. 10.1007/s40614-022-00326-1. 36249172. 9458797</bibtext> </blist> <blist> <bibtext> Vladescu JC, Caroll R, Paden A, Kodak TM. The effects of video modeling with voiceover instruction on accurate implementation of discrete-trial instruction. Journal of Applied Behavior Analysis. 2012; 45; 2: 419-423. 10.1901/jaba.2012.45-419. 22844149. 3405937</bibtext> </blist> <blist> <bibtext> Warrilow GD, Johnson DA, Eagle LM. The effects of feedback modality on performance. Journal of Organizational Behavior Management. 2020; 40; 3–4: 233-248. 10.1080/01608061.2020.1784351</bibtext> </blist> <blist> <bibtext> Zhu J, Bruhn A, Yuan C, Wang L. Comparing the effects of videoconference and email feedback on treatment integrity. Journal of Applied Behavior Analysis. 2021; 54; 2: 618-635. 10.1002/jaba.810. 33472275</bibtext> </blist> </ref> <aug> <p>By Emma S. Sipila-Thomas; Matthew T. Brodhead and Ashley N. Walker</p> <p>Reported by Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib35" firstref="ref2"></nolink> <nolink nlid="nl2" bibid="bib24" firstref="ref5"></nolink> <nolink nlid="nl3" bibid="bib36" firstref="ref6"></nolink> <nolink nlid="nl4" bibid="bib43" firstref="ref8"></nolink> <nolink nlid="nl5" bibid="bib10" firstref="ref11"></nolink> <nolink nlid="nl6" bibid="bib50" firstref="ref13"></nolink> <nolink nlid="nl7" bibid="bib28" firstref="ref16"></nolink> <nolink nlid="nl8" bibid="bib51" firstref="ref17"></nolink> <nolink nlid="nl9" bibid="bib21" firstref="ref18"></nolink> <nolink nlid="nl10" bibid="bib20" firstref="ref24"></nolink> <nolink nlid="nl11" bibid="bib34" firstref="ref34"></nolink> <nolink nlid="nl12" bibid="bib38" firstref="ref39"></nolink> <nolink nlid="nl13" bibid="bib47" firstref="ref40"></nolink> <nolink nlid="nl14" bibid="bib22" firstref="ref46"></nolink> <nolink nlid="nl15" bibid="bib26" firstref="ref47"></nolink> <nolink nlid="nl16" bibid="bib29" firstref="ref48"></nolink> <nolink nlid="nl17" bibid="bib31" firstref="ref49"></nolink> <nolink nlid="nl18" bibid="bib14" firstref="ref52"></nolink> <nolink nlid="nl19" bibid="bib37" firstref="ref53"></nolink> <nolink nlid="nl20" bibid="bib41" firstref="ref54"></nolink> <nolink nlid="nl21" bibid="bib33" firstref="ref55"></nolink> <nolink nlid="nl22" bibid="bib11" firstref="ref57"></nolink> <nolink nlid="nl23" bibid="bib18" firstref="ref58"></nolink> <nolink nlid="nl24" bibid="bib15" firstref="ref60"></nolink> <nolink nlid="nl25" bibid="bib46" firstref="ref61"></nolink> <nolink nlid="nl26" bibid="bib27" firstref="ref63"></nolink> <nolink nlid="nl27" bibid="bib19" firstref="ref68"></nolink> <nolink nlid="nl28" bibid="bib45" firstref="ref70"></nolink> <nolink nlid="nl29" bibid="bib48" firstref="ref71"></nolink> <nolink nlid="nl30" bibid="bib39" firstref="ref75"></nolink> <nolink nlid="nl31" bibid="bib30" firstref="ref85"></nolink> <nolink nlid="nl32" bibid="bib25" firstref="ref86"></nolink> <nolink nlid="nl33" bibid="bib42" firstref="ref87"></nolink> <nolink nlid="nl34" bibid="bib44" firstref="ref88"></nolink> <nolink nlid="nl35" bibid="bib49" firstref="ref89"></nolink> <nolink nlid="nl36" bibid="bib23" firstref="ref91"></nolink> <nolink nlid="nl37" bibid="bib12" firstref="ref95"></nolink> <nolink nlid="nl38" bibid="bib13" firstref="ref96"></nolink> <nolink nlid="nl39" bibid="bib17" firstref="ref98"></nolink> <nolink nlid="nl40" bibid="bib40" firstref="ref99"></nolink> <nolink nlid="nl41" bibid="bib16" firstref="ref100"></nolink> <nolink nlid="nl42" bibid="bib32" firstref="ref119"></nolink> |
|---|---|
| Header | DbId: eric DbLabel: ERIC An: EJ1428316 AccessLevel: 3 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: The Effects of Email Performance-Based Feedback on Teacher Candidate Implementation of a Multiple Stimulus without Replacement Preference Assessment – Name: Language Label: Language Group: Lang Data: English – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Emma+S%2E+Sipila-Thomas%22">Emma S. Sipila-Thomas</searchLink><br /><searchLink fieldCode="AR" term="%22Matthew+T%2E+Brodhead%22">Matthew T. Brodhead</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0003-4567-0051">0000-0003-4567-0051</externalLink>)<br /><searchLink fieldCode="AR" term="%22Ashley+N%2E+Walker%22">Ashley N. Walker</searchLink> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="SO" term="%22Journal+of+Behavioral+Education%22"><i>Journal of Behavioral Education</i></searchLink>. 2024 33(2):297-319. – Name: Avail Label: Availability Group: Avail Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/ – Name: PeerReviewed Label: Peer Reviewed Group: SrcInfo Data: Y – Name: Pages Label: Page Count Group: Src Data: 23 – Name: DatePubCY Label: Publication Date Group: Date Data: 2024 – Name: TypeDocument Label: Document Type Group: TypDoc Data: Journal Articles<br />Reports - Research – Name: Audience Label: Education Level Group: Audnce Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink> – Name: Subject Label: Descriptors Group: Su Data: <searchLink fieldCode="DE" term="%22Electronic+Mail%22">Electronic Mail</searchLink><br /><searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Preservice+Teachers%22">Preservice Teachers</searchLink><br /><searchLink fieldCode="DE" term="%22Special+Education%22">Special Education</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation%22">Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Fidelity%22">Fidelity</searchLink><br /><searchLink fieldCode="DE" term="%22Improvement%22">Improvement</searchLink> – Name: DOI Label: DOI Group: ID Data: 10.1007/s10864-022-09496-z – Name: ISSN Label: ISSN Group: ISSN Data: 1053-0819<br />1573-3513 – Name: Abstract Label: Abstract Group: Ab Data: The purpose of this study was to evaluate the effects of performance-based feedback delivered via email on teacher candidates' implementation of a multiple stimulus without replacement (MSWO) preference assessment with an adult confederate. Six participants enrolled in a special education teacher preparation program were recruited to participate. After exposure to an introductory MSWO training video, participant fidelity of MSWO administration was measured. Following baseline, performance feedback was delivered via e-mail to each participant after the completion of each research session. Results indicate that performance feedback delivered via e-mail improved fidelity of implementation for all six participants. Implications and future directions are discussed. – Name: AbstractInfo Label: Abstractor Group: Ab Data: As Provided – Name: DateEntry Label: Entry Date Group: Date Data: 2024 – Name: AN Label: Accession Number Group: ID Data: EJ1428316 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1428316 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s10864-022-09496-z Languages: – Text: English PhysicalDescription: Pagination: PageCount: 23 StartPage: 297 Subjects: – SubjectFull: Electronic Mail Type: general – SubjectFull: Feedback (Response) Type: general – SubjectFull: Preservice Teachers Type: general – SubjectFull: Special Education Type: general – SubjectFull: Evaluation Type: general – SubjectFull: Fidelity Type: general – SubjectFull: Improvement Type: general Titles: – TitleFull: The Effects of Email Performance-Based Feedback on Teacher Candidate Implementation of a Multiple Stimulus without Replacement Preference Assessment Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Emma S. Sipila-Thomas – PersonEntity: Name: NameFull: Matthew T. Brodhead – PersonEntity: Name: NameFull: Ashley N. Walker IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 06 Type: published Y: 2024 Identifiers: – Type: issn-print Value: 1053-0819 – Type: issn-electronic Value: 1573-3513 Numbering: – Type: volume Value: 33 – Type: issue Value: 2 Titles: – TitleFull: Journal of Behavioral Education Type: main |
| ResultId | 1 |
