View in EDS HTML Full Text PDF Full Text

Identifying Characteristics of Dissemination Success Using an Expert Panel

Saved in:
Bibliographic Details
Title: Identifying Characteristics of Dissemination Success Using an Expert Panel
Language: English
Authors: Bourrie, David M., Cegielski, Casey G., Jones-Farmer, L. Allison
Source: Decision Sciences Journal of Innovative Education. Oct 2014 12(4):357-380.
Availability: Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Peer Reviewed: Y
Page Count: 24
Publication Date: 2014
Sponsoring Agency: National Science Foundation
Contract Number: 1140542
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Expertise, Adoption (Ideas), Educational Innovation, Communication (Thought Transfer), Delphi Technique, Student Characteristics, Teacher Characteristics, Administrator Characteristics, Information Dissemination, College Students, College Faculty
DOI: 10.1111/dsji.12049
ISSN: 1540-4595
Abstract: Although considerable work has been done to develop new educational innovations, few have found widespread acceptance in the classroom. To improve the likelihood of adoption of educational innovations, researchers need to understand why some innovations are adopted and routinely used, while others are not. An initial aspect of the diffusion of innovations, as defined in the classical sociological literature, involves the communication of ideas and concepts related to innovations between individuals. This article presents an expert panel's answer to the following question: "What are the most important characteristics that relate to the dissemination of educational innovations?" As dissemination is a critical facet of the diffusion of an innovation, 45 researchers who received technology and engineering grants from the National Science Foundation (NSF) participated in a Delphi study designed to address this research question. In three rounds, the experts identified and ranked 11 characteristics of educational innovations, 6 characteristics of students, 13 characteristics of faculty members, and 5 characteristics of administrators that can relate to the successful dissemination of educational innovations. The results of this study led to the formation of a Characteristics of Dissemination Success (CODS) framework. This framework offers useful guidance for educational innovators seeking a better understanding of the influences on the dissemination of educational innovations.
Abstractor: As Provided
Entry Date: 2014
Accession Number: EJ1043427
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwENNCDUfropESTjq_9kQRemAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDI8Df2LCXtVnA6BnhgIBEICBmrzXLeJ6BJXzXeKdGeWj39iuAluDYvzp_g-D-LUZYisOrAQAoAUsAFltY15raLCAw5CUHQr-zLbiEHdXJ0Zmo-DtIqm9YTAmtkexvaPDc05Gxic6Yalbzvmf5ksCJYMz6AxigrZ4egmREbRRVHsAF_MGCCRe8ouUsauW2MdYCquGN1it2KCBCXvcFl0s5sBLi0gqzjE90LocoX8=
Text:
  Availability: 1
  Value: <anid>AN0099000348;q1n01oct.14;2018Jul12.08:25;v2.2.500</anid> <title id="AN0099000348-1">Identifying Characteristics of Dissemination Success Using an Expert Panel. </title> <p>Although considerable work has been done to develop new educational innovations, few have found widespread acceptance in the classroom. To improve the likelihood of adoption of educational innovations, researchers need to understand why some innovations are adopted and routinely used, while others are not. An initial aspect of the diffusion of innovations, as defined in the classical sociological literature, involves the communication of ideas and concepts related to innovations between individuals. This article presents an expert panel's answer to the following question: “What are the most important characteristics that relate to the dissemination of educational innovations?” As dissemination is a critical facet of the diffusion of an innovation, 45 researchers who received technology and engineering grants from the National Science Foundation (NSF) participated in a Delphi study designed to address this research question. In three rounds, the experts identified and ranked 11 characteristics of educational innovations, 6 characteristics of students, 13 characteristics of faculty members, and 5 characteristics of administrators that can relate to the successful dissemination of educational innovations. The results of this study led to the formation of a Characteristics of Dissemination Success (CODS) framework. This framework offers useful guidance for educational innovators seeking a better understanding of the influences on the dissemination of educational innovations.</p> <p>Delphi Method; Diffusion; Dissemination; Educational Innovations; and Framework</p> <p>During 2010, U.S. federal agencies invested $3.4 billion in science, technology, engineering and mathematics (STEM) education (National Science and Technology Council, [<reflink idref="bib56" id="ref1">56</reflink>] ). The programs funded included efforts to increase the number of students involved in degree programs and careers, develop evidence‐based education models and practices, develop technical skills or knowledge in students, and increase learner engagement, interest, and value in technology areas (National Science and Technology Council, [<reflink idref="bib55" id="ref2">55</reflink>] ). During the same period, 12 private foundations committed $506 million through the “Investing in Innovation” fund to improve STEM education (Blankinship, [<reflink idref="bib6" id="ref3">6</reflink>] ). However, despite the many recent advances in STEM teaching and learning know‐how, it is felt by many policy makers and practitioners (and evident in accounts published in articles in academic journals) that highly effective teaching and learning practices are still not in widespread use in most institutions of higher education (National Science Foundation, [<reflink idref="bib57" id="ref4">57</reflink>] ). Dissemination is defined as the process of communication that creates awareness of the artifact of interest—in this research, technology and engineering educational innovations—which may subsequently influence the intention to adopt, actual adoption, and perhaps the routine use of an innovation (Fincher, [<reflink idref="bib22" id="ref5">22</reflink>] ; Gravestock, [<reflink idref="bib25" id="ref6">25</reflink>] ; Hutchinson & Huberman, [<reflink idref="bib35" id="ref7">35</reflink>] ; King, [<reflink idref="bib38" id="ref8">38</reflink>] ).</p> <p>In business education, the Academy of Management, the Association for Information Systems, the Decision Sciences Institute, and the Institute for Operations Research and Management Sciences (INFORMS) all have conference tracks on educational innovation. In addition, they publish academic journals such as the Academy of Management Learning and Education (AMLE), the Decision Sciences Journal of Innovative Education (DSJIE), and INFORMS Transactions on Education that disseminate research‐based educational innovations. Recently, several articles from these journals have discussed the utility of incorporating academic games (D'Amours & Rönnqvist, [<reflink idref="bib14" id="ref9">14</reflink>] ), active learning (Dufresne & Offstein, [<reflink idref="bib18" id="ref10">18</reflink>] ), case studies (Erzurumlu & Rollag, [<reflink idref="bib19" id="ref11">19</reflink>] ; Hu & Kumar, [<reflink idref="bib34" id="ref12">34</reflink>] ; Nkhoma, Sriratanaviriyakul, Pham, & Lam, [<reflink idref="bib58" id="ref13">58</reflink>] ), computer simulations (Leemkuil & De Jong, [<reflink idref="bib42" id="ref14">42</reflink>] ; Legner, Estier, Avdiji, & Boillat, [<reflink idref="bib43" id="ref15">43</reflink>] ; Snider & Balakrishnan, [<reflink idref="bib75" id="ref16">75</reflink>] ), and experiential learning (Taras et al., [<reflink idref="bib76" id="ref17">76</reflink>] ; Wu & Sankar, [<reflink idref="bib80" id="ref18">80</reflink>] ) into business education. In engineering disciplines, many educational associations and government commissions, including the American Society for Engineering Education ([<reflink idref="bib2" id="ref19">2</reflink>] , [<reflink idref="bib3" id="ref20">3</reflink>] ), the National Academy of Education ([<reflink idref="bib53" id="ref21">53</reflink>] ), the National Academy of Sciences ([<reflink idref="bib54" id="ref22">54</reflink>] ), and the National Research Council ([<reflink idref="bib55" id="ref23">55</reflink>] ), have suggested that scenario‐, problem‐, and case‐based teaching and learning need to be incorporated into engineering and technology classrooms.</p> <p>Faculty members who have used new instructional materials and strategies in the classroom report improved student engagement and motivation on both an intellectual and personal level (Balasubramanian, Lohani, Puri, Case, & Mahajan, [<reflink idref="bib5" id="ref24">5</reflink>] ; Krishnan, Paulik, Yost, & Stoltz, [<reflink idref="bib40" id="ref25">40</reflink>] ; Mykytyn, Pearson, Paul, & Mykytyn, [<reflink idref="bib51" id="ref26">51</reflink>] ; Schunk, Pintrich, & Meece, [<reflink idref="bib71" id="ref27">71</reflink>] ; Smith, Sheppard, Johnson, & Johnson, [<reflink idref="bib74" id="ref28">74</reflink>] ). The President's Council of Advisors on Science and Technology ([<reflink idref="bib64" id="ref29">64</reflink>] ) concurs, recommending the widespread adoption of such empirically validated teaching practices to improve the enrolment of students in STEM education. The Organization for Economic Cooperation and Development (OECD, 2010) suggests that improving all U.S. students to a baseline level of STEM proficiency could result in an increase in the gross domestic product by $72 billion based on historical growth patterns. It is therefore critical that empirically validated educational innovations are widely diffused, disseminated, and adopted (National Science Foundation, [<reflink idref="bib57" id="ref30">57</reflink>] ).</p> <p>We formulated the research question for this study as: “What are the most important characteristics that relate to the dissemination of educational innovations?” To answer this question, the opinions of 45 faculty members who received grants from the National Science Foundation (NSF) in the areas of engineering and technology were solicited via a three‐round Delphi study. The opinions were synthesized to develop the Characteristics of Dissemination Success (CODS) framework. In the next section, the literature regarding educational innovations and factors that relate to the dissemination process are discussed. The Delphi methodology and its application within the context of the current study are then discussed, after which the analysis of the opinions of the experts is presented. The analysis is then used to develop the CODS framework. The article concludes with a set of recommendations for future research and a summary of this study's contributions.</p> <hd id="AN0099000348-2">BACKGROUND</hd> <hd id="AN0099000348-3">Diffusion of Innovations and Education</hd> <p>In the broadest context, Rogers ([<reflink idref="bib67" id="ref31">67</reflink>] ) described an innovation as an idea, practice, or object that is perceived as new by an individual or group. Katz ([<reflink idref="bib37" id="ref32">37</reflink>] ) defined diffusion of innovation as the process by which communication regarding an innovation occurs over time among the members of a society. The definition contained four elements: (<reflink idref="bib1" id="ref33">1</reflink>) the innovation, (<reflink idref="bib2" id="ref34">2</reflink>) communication, (<reflink idref="bib3" id="ref35">3</reflink>) time, and (<reflink idref="bib4" id="ref36">4</reflink>) the social system. Some or all of these elements are identifiable in all previous innovation diffusion studies. In this exploratory study, aspect of communication generally referred to in sociological research as dissemination (the act of transferring information regarding an innovation between those individuals who possess knowledge of an innovation and those who do not) is examined. Dissemination typically occurs at the Knowledge phase of the innovation‐decision process (Figure [NaN] ). This phase is the point in the innovation‐decision process when an individual learns of the innovation's existence and accumulates some understanding of its function. Persuasion, the second phase of the process, refers to an individual's formulation of an attitude (favorable/unfavorable) toward the innovation. Subsequently, the Decision phase involves the individual undertaking activities leading to a decision to adopt or reject the innovation.</p> <p>As described, significant efforts have focused on the development of educational innovations. However, despite promising potential benefits, many educational innovations fail to find significant adoption, therefore we ask why this is the case. Although the process of diffusion for educational innovations is multidimensional, a well‐structured comprehensive research agenda should address the broader context of this topic in a systematic and methodological fashion. To frame these efforts in this study and to narrow the focus of investigation, we suggest it is most prudent to begin at the beginning of the diffusion of innovation process as it relates to educational innovations. This research therefore focuses on the dissemination of information regarding educational innovations. Next, we describe the various forms of educational innovations that have been examined by other academicians.</p> <p>Tornatzky and Fleischer ([<reflink idref="bib79" id="ref37">79</reflink>] ) described two primary types of innovations, product innovations, defined as “those that are ends in themselves,” and process innovations that are “adopted as instrumental to some other end.” Both Rogers’ (2003) general and Tornatzky and Fleischer's (1990) more granular characterization of innovations provided the basis from which additional researchers have described educational innovations. In pedagogical research, Hazen, Wu, and Sankar ([<reflink idref="bib29" id="ref38">29</reflink>] ) defined an educational innovation as any new instructional material, strategy, or pedagogical approach that differs from current or previous materials or methods employed by an educator. Relating this characterization to Tornatzky and Fleischer's ([<reflink idref="bib79" id="ref39">79</reflink>] ) product categorization, examples of educational product innovations include multimedia resources such as podcasts, videos, laboratory simulations, and Web sites with subject resources (McKenzie, Alexander, Harper, & Anderson, [<reflink idref="bib48" id="ref40">48</reflink>] ), while educational process innovations include online role‐playing simulations, the use of problem‐based learning, and peer‐ or criterion‐based assessments. From these definitions and examples, researchers further subdivide aspects of educational innovations into new curricula, instructional materials, faculty expertise, and/or instructional strategies.</p> <p>Curriculum development deals with overarching changes planned or implemented for an educational program or curriculum, while instructional materials are specifically designed for use in a particular course or laboratory session. The development of faculty expertise refers to methods that can be used to help faculty members improve their teaching such as training workshops or mentoring. Finally, instructional strategies, such as experiential learning or student centered learning, consist of both instructional design and pedagogy and include approaches that enhance the acquisition of knowledge (Gropper, [<reflink idref="bib26" id="ref41">26</reflink>] ).</p> <hd id="AN0099000348-4">Factors That Relate to the Dissemination Process</hd> <p>A number of different theories, models, and frameworks have been proposed to describe the complicated processes associated with the dissemination of educational innovations (Fairweather, [<reflink idref="bib20" id="ref42">20</reflink>] ; Hazen, Wu, Sankar, & Jones‐Farmer, [<reflink idref="bib30" id="ref43">30</reflink>] ; Henderson, Finkelstein, & Beach, [<reflink idref="bib33" id="ref44">33</reflink>] ). Dancy and Henderson ([<reflink idref="bib16" id="ref45">16</reflink>] ) asserted that current approaches to the dissemination of educational innovations, such as the publication of journal articles and participation in conferences and workshops, are successful in raising general awareness regarding innovative instructional practices, but fail to robustly support faculty seeking to adopt and implement innovations. Borrego, Froyd, and Hall ([<reflink idref="bib7" id="ref46">7</reflink>] ) concurred, reporting that although 82% of department chairs surveyed were aware of seven recent educational innovations, only 47% reported actual adoption of the seven innovations at their respective institutions.</p> <p>These findings appear to signal a distinct disconnect between the information available regarding educational innovations and educators’ subsequent adoption of the innovations. Interestingly, the department chairs that participated in the 2010 survey noted that they saw student perceptions, faculty attitudes, and administrative barriers as major considerations in educational innovation adoption decisions. A number of researchers have found evidence of significant resistance from students regarding the potential impact of disseminating information about educational innovations (Dancy & Henderson, [<reflink idref="bib16" id="ref47">16</reflink>] ; Hazen et al., [<reflink idref="bib30" id="ref48">30</reflink>] ; Henderson et al., [<reflink idref="bib33" id="ref49">33</reflink>] ). Henderson and Dancy ([<reflink idref="bib31" id="ref50">31</reflink>] ) reported student attitude to be a prominent barrier to the adoption of new instructional strategies. Similar studies have also identified the lack of student motivation as a significant factor in the adoption decision related to new instructional materials and strategies (Henderson & Dancy, [<reflink idref="bib31" id="ref51">31</reflink>] ; Kolmos, Kofoed, & Du, [<reflink idref="bib39" id="ref52">39</reflink>] ; Morgan & Barroso, [<reflink idref="bib50" id="ref53">50</reflink>] ).</p> <p>Another possible reason for the apparent disconnect in the adoption of educational innovations may be a gap between a faculty member's teaching style and his or her students’ learning styles (Felder & Silverman, [<reflink idref="bib21" id="ref54">21</reflink>] ). Faculty resistance to educational innovations has also been suggested to pose a significant impediment in several recent pedagogical studies (Cheville & Bunting, [<reflink idref="bib11" id="ref55">11</reflink>] ; Henderson, Dancy, & Niewiadomska‐Bugaj, [<reflink idref="bib32" id="ref56">32</reflink>] ; Riley, Davis, Cox, & Maciukenas, [<reflink idref="bib66" id="ref57">66</reflink>] ). Last but not least, administration is cited by The President's Council of Advisors on Science and Technology ([<reflink idref="bib64" id="ref58">64</reflink>] ) as a barrier to educational innovation adoption, administrators generally failing to provide faculty rewards and incentives, or not investing in classroom renovations.</p> <hd id="AN0099000348-5">THE DELPHI METHOD</hd> <p>The current study seeks to identify those characteristics that relate to the successful dissemination of educational innovations. We define “successful dissemination” as the communication of information regarding an educational innovation that results in a decision to adopt and use it in a routine manner. Methodologically, the most appropriate means to conduct this type of research is through the use of the Delphi technique. The Delphi method is a qualitative research technique developed by the Rand Corporation in the early 1950s (Dalkey & Helmer, [<reflink idref="bib15" id="ref59">15</reflink>] ; Linstone & Turoff, [<reflink idref="bib44" id="ref60">44</reflink>] ; Malhotra, Steele, & Grover, [<reflink idref="bib46" id="ref61">46</reflink>] ), and Delphi studies have been used to prioritize the qualities, attributes, and best practices of expert panels and leaders (Cegielski, Bourrie, & Hazen, [<reflink idref="bib10" id="ref62">10</reflink>] ; Loo, [<reflink idref="bib45" id="ref63">45</reflink>] ; Romano, [<reflink idref="bib69" id="ref64">69</reflink>] ). Given the unique characteristics of the methodology, the Delphi technique is well suited to highly multidimensional research questions that deal with uncertainty in a domain of imperfect knowledge (Cegielski et al., [<reflink idref="bib10" id="ref65">10</reflink>] ; Churchman & Schainblatt, [<reflink idref="bib12" id="ref66">12</reflink>] ). The objective of the technique is to achieve consensus among experts regarding a specific topic (Okoli & Pawlowski, [<reflink idref="bib60" id="ref67">60</reflink>] ; Taylor & Meinhardt, [<reflink idref="bib77" id="ref68">77</reflink>] ). In previous comparative analyses of group survey techniques, the Delphi technique was shown to achieve a greater level of accuracy than other group consensus techniques (Linstone & Turoff, [<reflink idref="bib44" id="ref69">44</reflink>] ; Riggs, [<reflink idref="bib65" id="ref70">65</reflink>] ; Rohrbaugh, [<reflink idref="bib68" id="ref71">68</reflink>] ).</p> <p>Operationally, the Delphi Method involves three phases: the selection of expert panelists, the collection of topic‐relevant issues, and the ranking of reported issues (Cegielski et al., [<reflink idref="bib10" id="ref72">10</reflink>] ; Okoli & Pawlowski, [<reflink idref="bib60" id="ref73">60</reflink>] ). The term “expert” is subjective and thus requires researchers to specify, utilizing measurable terms, what constitutes an expert for the purposes of their study. Experts are typically selected based on factors such as years of professional experience, job or position title, level of education, and/or professional certifications. To collect topic‐relevant issues, an initial, open‐ended Delphi questionnaire is used (Delbecq, Van de Ven, & Gustafson, [<reflink idref="bib17" id="ref74">17</reflink>] ). In early research, questionnaires were sent to between 5 and 15 experts, but current standards suggest sample sizes ranging from 15 to 30 panelists for heterogeneous populations and five to ten panelists for homogeneous populations (Linstone & Turoff, [<reflink idref="bib44" id="ref75">44</reflink>] ; Loo, [<reflink idref="bib45" id="ref76">45</reflink>] ; Martino, [<reflink idref="bib47" id="ref77">47</reflink>] ). The purpose of the first questionnaire is to aggregate information for use in subsequent rounds of the study (Brancheau & Wetherby, [<reflink idref="bib8" id="ref78">8</reflink>] ). The panel of experts each contributes whatever input they feel is pertinent to the focal question of the study (Nambisan, Agarwal, & Tanniru, [<reflink idref="bib52" id="ref79">52</reflink>] ). In the second round, they rank each of the issues identified in the first round (Paliwoda, [<reflink idref="bib62" id="ref80">62</reflink>] ). From the data gathered, the study administrator scores the issues (typically using the weighted average method) and redistributes the results to the panelists (Nambisan et al., [<reflink idref="bib52" id="ref81">52</reflink>] ). In the third as well as any subsequent rounds of the study, the experts review the group rankings and re‐rank the issues based on the aggregated responses of the group. The process of ranking and re‐ranking continues until the panel achieves consensus (Delbecq et al., [<reflink idref="bib17" id="ref82">17</reflink>] ; Linstone & Turoff, [<reflink idref="bib44" id="ref83">44</reflink>] ).</p> <p>In the current study, a three‐round Delphi process was employed to aggregate four separate consensuses regarding the research question. The three rounds were all conducted in a distributed fashion via e‐mail correspondence and the online survey software Qualtrics. A total of 307 potential expert participants were identified, all of whom were Principal Investigators (139 individuals) or Co‐Principal Investigators (168 individuals) from the NSF's Course, Curriculum, and Laboratory Improvements (CCLI) or Transforming Undergraduate Education in STEM (TUES) programs in engineering, technology, and computer science. The list of potential subjects was assembled by analyzing past and current CCLI and TUES grants and then searching the Internet for Co‐Principal Investigators’ e‐mail addresses that were not listed on the NSF Web site. As part of the subject solicitation process for this study, each of the 307 potential panelists received an e‐mail describing the study along with a definition of the relevant terms, and was invited to participate by responding to the following four questions:</p> <p>What characteristics of educational innovations relate to their successful dissemination?</p> <p>What characteristics of students relate to the successful dissemination of educational innovations?</p> <p>What characteristics of faculty members relate to the successful dissemination of educational innovations?</p> <p>What characteristics of the college or university administration relate to the successful dissemination of educational innovations?</p> <hd id="AN0099000348-6">Participation By Round</hd> <p>The participants were given two weeks to respond to the initial invitation and the open‐ended research questions. Forty‐five responses were received, a response rate of 14.7%, of which 23 were from Principal Investigators and 22 were from Co‐Principal Investigators (Table [NaN] ). There were 18 non‐deliverable e‐mail addresses, and 15 individuals who declined to participate. Members of the panel included at least one member from 25.8% of all CCLI and TUES grant projects that were used as the basis for the sampling frame.</p> <p>Distribution of subjects and response rates</p> <p> <ephtml> <table><tr><th /><th>No. of</th><th>No. of</th><th>Initial</th><th>No. of</th><th>Round 2</th><th>No. of</th><th>Round 3</th></tr><tr><th>Position</th><th>Invited</th><th>Respondents</th><th>Response</th><th>Respondents</th><th>Response</th><th>Respondents</th><th>Response</th></tr><tr><th>Title</th><th>Participants</th><th>Round 1</th><th>Rate</th><th>Round 2</th><th>Rate</th><th>Round 3</th><th>Rate</th></tr><tr><td>Principal Investigator</td><td>139</td><td>23</td><td>16.75%</td><td>17</td><td>73.91%</td><td>17</td><td>100%</td></tr><tr><td>Co‐Principal Investigator</td><td>168</td><td>22</td><td>13.10%</td><td>18</td><td>81.82%</td><td>17</td><td>94.44%</td></tr><tr><td>Total</td><td>307</td><td>45</td><td>14.66%</td><td>35</td><td>77.78%</td><td>34</td><td>97.14%</td></tr></table> </ephtml> </p> <p>A total of 593 comments were generated by the participants (Table [NaN] ). Of these, 174, 119, 165, and 135 comments were related to characteristics of the innovation, students, faculty, and administrators, respectively. As is typical with the Delphi technique, many of the comments generated by the members were repeated. To synthesize the comments, a three‐member committee was established. One researcher has experience in the research domain, over 15 years of experience in educational innovation and dissemination, and had received funding through several NSF CCLI/TUES grants. A second researcher was an expert in the diffusion of innovation and the Delphi approach, and the third researcher, while not an expert in the research domain, was proficient in qualitative research methodologies. To maintain a high level of inter‐rater reliability, each of the committee members individually reviewed all the contributed comments and classified them into their own respective categories. Upon completing their individual classifications, the committee met to discuss each individual's classifications and subsequently agreed on a common classification schema. Table [NaN] shows examples of the comments made by the experts for each of the classifications and the resulting classification of the comments. These classifications form the basis for identifying the characteristics of the educational innovations, students, faculty members, and administration that the expert panel considered important for the successful dissemination of educational innovations. For each characteristic, a contextual definition was created based on the comments. A full list of the comments is available from the authors.</p> <p>Number of unique comments generated in round one of the Delphi study</p> <p> <ephtml> <table><tr><th /><th>Total No. of</th><th align="center">Number of</th></tr><tr><th /><th>Comments Generated</th><th align="center">Unique Constructs</th></tr><tr><td>Characteristics of the Innovation</td><td>174</td><td>11</td></tr><tr><td>Characteristics of Students</td><td>119</td><td>6</td></tr><tr><td>Characteristics of Faculty</td><td>165</td><td>13</td></tr><tr><td>Characteristics of Administration</td><td>135</td><td>4</td></tr><tr><td>Total</td><td>593</td><td>34</td></tr><tr><td>N = 47</td><td /><td /></tr></table> </ephtml> </p> <p>Classifications derived from participants’ comments</p> <p> <ephtml> <table><tr><th>Example of Comment by Expert</th><th /><th /></tr><tr><th>That Illustrates the Classification</th><th align="center">Characteristic</th><th align="center">Contextual Definition</th></tr><tr><td>Characteristics of the Educational Innovation</td><td /><td /></tr><tr><td>Must be sufficiently flexible so that instructors may tailor it to their needs, Adaptable so that an instructor can adapt to his or her style and environment, etc.</td><td>Adaptability</td><td>The degree to which the user can modify the pedagogical innovation as deemed necessary.</td></tr><tr><td>Ability to communicate the results, the ability to articulate the problem addressed by the innovation, etc.</td><td>Communicability</td><td>The ease with which the results of using the pedagogical innovation can be easily described to others.</td></tr><tr><td>Match of innovation goal with goal of faculty and students, the innovation is not that different from what is already being done, etc.</td><td>Compatibility</td><td>The consistency of the pedagogical innovation with current pedagogy.</td></tr><tr><td>Conceptually simple so that it can be easily understood by adaptors, relatively simple and user friendly, etc.</td><td>Ease of Use</td><td>The degree to which a pedagogical innovation is perceived as relatively easy to use and understand.</td></tr><tr><td>Ease of adoption (how much work do "I" have to do to adopt the innovation, easy to adopt, etc.</td><td>Ease of Implementation</td><td>The degree to which a pedagogical innovation is perceived as relatively easy to implement and adopt.</td></tr><tr><td>Their benefit to student learning is measureable and quantified, innovation must have solid evaluation data documenting success, statistical significance of results, etc.</td><td>Measurability</td><td>The degree to which the impact of the pedagogical innovation can be assessed in particular the ability to clearly attribute the effects to the innovation.</td></tr><tr><td>Innovation is rooted in a documented need for the practice of engineering, a clear link between the innovation and a prevalent problem, etc.</td><td>Practicality of the Concept</td><td>The degree to which the innovation is based on a clear definition of basic terms and clear methodology applicable for the specific situation.</td></tr><tr><td>Completeness of the “package” on how to use the innovation, high quality descriptions, excellent formatting, visuals, etc.</td><td>Quality of Initial Information</td><td>The degree to which the initial information regarding the pedagogical innovation is relevant, timely, complete, and appropriate in terms of amount so as to add value.</td></tr><tr><td>Don't require a lot of time (compared to current techniques) to prepare and administer, cost benefit, it is clearly an improvement, etc.</td><td>Relative Advantage</td><td>The degree to which the pedagogical innovation is perceived as being better than its predecessor.</td></tr><tr><td>Innovation seems useful to my work and will make my job easier, professional relevancy, etc.</td><td>Relevance to Job</td><td>The ability of the pedagogical innovation to enhance the educator's or student's performance.</td></tr><tr><td>Innovations that are based on theoretical understandings of pedagogy or learning , highly based on a strong literature review, etc.</td><td>Theoretical Coherence</td><td>The degree to which pedagogical innovations are based on a theoretical understanding of pedagogy or learning.</td></tr><tr><td>Characteristics of the Students</td><td /><td /></tr><tr><td>Diverse population of students, ethnic/social background, fluency in spoken English, etc.</td><td>Cultural Differences</td><td>The varying intrinsic and extrinsic attributes within and between dissemination groups; accounts for the differences between different, teachers, schools, ethnicities, and locations.</td></tr><tr><td>Engagement with innovative products, willing to ask questions, student's willingness to engage with others, etc.</td><td>Engagement</td><td>The degree to which students study or practice a subject to learn about the subject.</td></tr><tr><td>Will the innovation advance student learning, etc.</td><td>Intent to Enhance Learning</td><td>The degree to which the innovation will enhance student learning.</td></tr><tr><td>Possess the meta‐cognitive traits that value improved problem solving, creativity, or complex thinking skills; It is important to form a teaching philosophy where there is a good mix of audio, visual, and interactive learning, etc.</td><td>Learning Styles</td><td>Accounts for the belief differences between different students, faculty, schools, ethnicities, and locations.</td></tr><tr><td>Who would like to learn the concepts and apply them, they are not concerned about grades only, motivated to learn the material, etc.</td><td>Motivation to Learn</td><td>The meaningfulness, value, and benefits of academic tasks to the learner—regardless of whether or not they are intrinsically interesting.</td></tr><tr><td>Openness to change from standard lectures and problem sets, willing to follow different paradigms, etc.</td><td>Receptivity to Change</td><td>The degree of receptivity from the students toward the pedagogical innovation.</td></tr><tr><td>Characteristics of the faculty members</td><td /><td /></tr><tr><td>I am looking for imaginative things, those that use technology, or are strongly visual or event things that have an entertainment factor, I hear these kinds of things, it engages me, and so I go try them in my classroom.</td><td>Attitude to Innovation</td><td>The opinion that a person holds toward the pedagogical innovation.</td></tr><tr><td>Awareness of innovations through reading, conference attendance, word‐of mouth; amount of media attention, especially TV.</td><td>Awareness of Innovation</td><td>The degree of awareness from the faculty toward the pedagogical innovation.</td></tr><tr><td>A devoted faculty who has a passion of teaching and learning and is willing to go the extra mile to ensure the integrity of the students' learning outcomes, faculty's personal desire to improve their students’ learning and success, etc.</td><td>Care about Student Learning Outcomes</td><td>The degree to which faculty focuses on the learning process and not the results.</td></tr><tr><td>If they reflect approved minority/special needs status, the innovation is given consideration, etc.</td><td>Cultural Acceptance of Innovation</td><td>The degree to which the cultural differences of the students and faculty influence the acceptance of the pedagogical innovation.</td></tr><tr><td>Provide logistical support to adopters; workshops, webinars, Web sites, etc.</td><td>Degree of Support</td><td>The degree of support from the educators and the students toward the pedagogical innovation.</td></tr><tr><td>Being willing to take the risk to try something new, creativity to come up with new ideas, tolerance for early failure, etc.</td><td>Innovativeness</td><td>A faculty member's willingness to try out a pedagogical innovation.</td></tr><tr><td>Getting bored if I have to do the same thing, exactly, over and over again; funds to disseminate, the innovation clearly is going to improve an outcome that matters to me or my campus (i.e., retention, students success), etc.</td><td>Motivation to be Innovative</td><td>The wants, needs, and set of circumstances that drive a person to be innovative.</td></tr><tr><td>Strong ties with other faculty within their own department as well as at other institutions, faculty who participate in many newsgroups, workshops, seminars, and conferences, etc.</td><td>Professional Social System</td><td>A set of interrelated units that are engaged in joint problem solving to accomplish common professional goals.</td></tr><tr><td>Openness to change, willingness to try new things, Faculty should be open‐minded for such dissemination, etc.</td><td>Receptivity to Change</td><td>The degree of receptivity from the faculty toward the pedagogical innovation.</td></tr><tr><td>Confidence in using the innovation, confidence in their own teaching ability, etc.</td><td>Self‐Efficacy</td><td>The belief that one has the capability to perform the pedagogical innovation.</td></tr><tr><td>Belonging to a supportive community willing and able to try out others' innovations, faculty need to be supportive when an innovation does fail, etc.</td><td>Supportive Community</td><td>The degree to which the faculty community supports and encourages continuous teaching improvements regardless of measureable or successful outcomes.</td></tr><tr><td>One needs the security of tenure to be able to devote the time and resources to such things, less desirable for non‐tenured faculty to engage in introducing innovations, etc.</td><td>Tenure Status</td><td>Whether the individual that is adopting the pedagogical innovation is a tenured or non‐tenured faculty member.</td></tr><tr><td>Recognition that teaching is of real value, willingness to keep learning about teaching, strong commitment to teaching, etc.</td><td>Value Teaching in Addition to Research</td><td>The degree to which the faculty balances the priorities of teaching and research.</td></tr><tr><td>Characteristics of the Administration</td><td /><td /></tr><tr><td>Travel funds for conferences and presentations; infrastructure to support the use of the innovation, etc.</td><td>Degree of Facilitating Conditions</td><td>The degree of availability of resources necessary to facilitate dissemination of pedagogical innovations.</td></tr><tr><td>Release time for instructors to learn about innovations and to create or modify their courses; creating an atmosphere of innovation in education; interest in development and support of innovations, etc.</td><td>Management Support</td><td>The degree of support provided by institutional authorities (i.e., government, university administration, college deans, department chairs, etc.).</td></tr><tr><td>Promotion, tenure, and merit systems that reward innovation; recognition of the effort required to disseminate and adopt innovations, etc.</td><td>Promote and Reward Innovation</td><td>The degree to which the administration rewards and encourages innovation in the classroom.</td></tr><tr><td>Open mindedness, cooperation, and collaboration; willingness to try new things, etc.</td><td>Receptivity to Change</td><td>The degree of receptivity from the administration toward the pedagogical innovation.</td></tr></table> </ephtml> </p> <p>To begin round 2 of the study, the specific unique results were reported to the experts via an online survey, and instructions given to review the characteristics and rank them in order from most important (<reflink idref="bib1" id="ref84">1</reflink>) to least important. Each of the participants was given two weeks to individually evaluate the characteristics related to the successful dissemination of educational innovations and individually rank the associated items. The response rate was 73.9% (17/23) for Principal Investigators, 81.8% (18/22) for Co‐Principal Investigators, and 77.8% (35/45) overall. The ranking for the overall group was computed using the weighted average method. The fact that a majority of the experts responded to the second round without suggesting additions, deletions, or other modifications of the comments, while ranking the items that were returned to them provides a qualitative verification that the classifications made by the researchers was reasonable.</p> <p>To begin round three of the Delphi survey, the participants from round two were invited to participate in an online survey and given the rankings from round two. Each of the participants was asked to review the group's weighted average ranking of the characteristics, and rank the characteristics in light of the group rankings. The response rate was 100% (17/17) for Principal Investigators, 94.4% (17/18) for Co‐Principal Investigators, and 97.1% (34/35) overall. The ranking of this group was computed using the weighted average method.</p> <p>Following the completion of the third round of the study, an inspection of the results revealed that none of the rankings of the items changed between the second and third rounds of the study, thus a concordance assessment was made. The computation of Kendall's Coefficient of Concordance (W) revealed that consensus existed among the participants with respect to the round‐three rankings of the importance of the items. Kendall's W is a measure designed to determine the degree of agreement among individual sets of ranked scores (Siegel, [<reflink idref="bib73" id="ref85">73</reflink>] ). A significant value of W indicates that the participants applied essentially the same standard in judging the importance of the issues and that they are in consensus. For the third‐round rankings of the Delphi survey, all values of W (W<subs>(Educational innovations)</subs> =.90, p <.001, W<subs>(Students)</subs> =.84, p <.001, W<subs>(Faculty)</subs> =.93, p <.001, and W<subs>(Administration)</subs> =.79, p <.001) were statistically significant.</p> <p>An additional estimate of group consensus, the percentage of respondents whose characteristics rank matched the group rank, provided further support for the assertion that each group was in accordance. Based on the values of Kendall's W and the rank percentages, it was clear that consensus was achieved for all four research questions, hence no additional rounds of the Delphi approach were deemed necessary.</p> <hd id="AN0099000348-7">DEVELOPMENT OF THE CODS FRAMEWORK</hd> <p>Based on the results of the Delphi study, the CODS framework, shown in Figure [NaN] , was developed. The figure shows the characteristics and ranks that were agreed upon by the experts in round 3 of the Delphi study. The framework is described in more detail in this section.</p> <hd id="AN0099000348-8">The Stages of Disseminating Educational Innovations</hd> <p>The CODS framework is grounded in the Theory of Reasoned Action (Fishbein & Ajzen, [<reflink idref="bib23" id="ref86">23</reflink>] , [<reflink idref="bib24" id="ref87">24</reflink>] ), the Theory of Planned Behavior (Ajzen, [<reflink idref="bib1" id="ref88">1</reflink>] ), and other research (Compeau, Meister, & Higgins, [<reflink idref="bib13" id="ref89">13</reflink>] ; Moore & Benbasat, [<reflink idref="bib49" id="ref90">49</reflink>] ; Rogers, [<reflink idref="bib67" id="ref91">67</reflink>] ).</p> <p>In the framework, the characteristics of the innovation, faculty, students, and administration affect the intention to adopt an innovation. Within this context and based on the Delphi results, the rank order of the items identified in each categorization are included in the framework.</p> <p>In grounding the framework, we rely on the work of Taylor and Todd ([<reflink idref="bib78" id="ref92">78</reflink>] ) who analyzed the theory of planned behavior and found that characteristics of innovations such as relative advantage, complexity, and compatibility are antecedents to an individual's intention to adopt, which is in turn an antecedent to actual adoption. The users, environment, and culture in which the innovation is to be used have also been shown to affect intention to adopt, and are likely to affect actual adoption and routine use (Hazen et al., [<reflink idref="bib30" id="ref93">30</reflink>] ; Lee, Kozar, & Larsen, [<reflink idref="bib41" id="ref94">41</reflink>] ; Rogers, [<reflink idref="bib67" id="ref95">67</reflink>] ). Faculty members may intend to adopt an innovation, but due to tenure and promotion criteria, poor facilitating conditions, or fear of poor student evaluations due to student resistance may not actually adopt the innovation; an adopted innovation may not become routinely used for similar reasons. Figure [NaN] portrays the characteristics of educational innovations, students, faculty members, and administrators as antecedents based on the theory of planned behavior.</p> <hd id="AN0099000348-9">Ranking Characteristics of Educational Innovations, Students, Faculty Members, and ...</hd> <hd id="AN0099000348-10">Characteristics of the educational innovation</hd> <p>Based on the 174 comments made by the experts, the researchers synthesized 11 characteristics and asked the expert panel to rank them in the second and third rounds of the Delphi study. Relative advantage was identified as the most important characteristic, followed by ease of implementation and ease of use. Many of the characteristics identified by the experts (i.e., relative advantage, ease of use, compatibility, communicability, and measurability) are similar to those discussed in the literature on the diffusion of innovations (Compeau et al., [<reflink idref="bib13" id="ref96">13</reflink>] ; Moore & Benbasat, [<reflink idref="bib49" id="ref97">49</reflink>] ; Rogers, [<reflink idref="bib67" id="ref98">67</reflink>] ). One member of the expert panel noted</p> <p>“It's the same characteristics specified in Roger's diffusion of innovations theory: they have to be useful, easy to use, people must have a chance to try them at no risk, etc.”</p> <p>It is not surprising that characteristics such as the practicality of the concept, relevance to the job, quality of initial information, and theoretical coherence were considered to be important to the experts. The experts also recognized that the ease of implementation and adaptability of the innovations are particularly important when integrating educational innovations in the classroom. These characteristics have also been identified as important in the education, information systems, and marketing literature (Kaleelur Rahuman, Wikramanayake, & Hewagamage, [<reflink idref="bib36" id="ref99">36</reflink>] ; Oppermann & Rasher, [<reflink idref="bib61" id="ref100">61</reflink>] ; Ross, Rhodes, & Hastings, [<reflink idref="bib70" id="ref101">70</reflink>] ).</p> <hd id="AN0099000348-11">Characteristics of the students</hd> <p>The experts’ 119 comments regarding the characteristics of students that relate to dissemination were classified into six characteristics. These were subsequently ranked by the experts in the order engagement, motivation to learn, intent to enhance learning, receptivity to change, learning styles, and cultural differences. Engagement was mentioned in five unique comments during the first round of the Delphi study. One of the experts stated</p> <p>“Students should be active toward the direction of education. In order to improve the dissemination of educational innovations, I believe a student driven classroom environment assisted by [the] professor is important.”</p> <p>Students are the end users of any educational innovation that is taught in the classroom, and their receptivity to change and evaluations have an important influence over whether faculty members continue to use or even adopt educational innovations (Hannon, [<reflink idref="bib27" id="ref102">27</reflink>] ). Students are often interested in student‐focused, rather than teacher‐focused, approaches to learning that involve more interaction with classmates and faculty members. These approaches have been shown to improve student engagement and motivate students to learn (McKenzie et al., [<reflink idref="bib48" id="ref103">48</reflink>] ). In addition, students fall into one of sixteen different learning styles that influence how they learn (Cagiltay, [<reflink idref="bib9" id="ref104">9</reflink>] ; Felder & Silverman, [<reflink idref="bib21" id="ref105">21</reflink>] ; Hawk & Shah, [<reflink idref="bib28" id="ref106">28</reflink>] ). The experts noted that educational innovations that are compatible with students’ learning styles and cultural differences can enhance students’ overall learning.</p> <hd id="AN0099000348-12">Characteristics of faculty members</hd> <p>The 165 comments made by the experts were synthesized into 13 characteristics that the experts subsequently ranked in the order shown in Figure [NaN] . The highest ranked characteristic of was receptivity to change, which received 16 unique comments. One member of the expert panel commented</p> <p>“Some people are more amenable to change than others in any arena. Often those faculty most willing to change have both a deep‐seated understanding of the content impacted by the educational innovation as well as a realization of why the educational innovation will be of benefit.”</p> <p>Most of the faculty‐related characteristics break down into three common themes: the faculty member's personal attributes (i.e., receptivity to change, innovativeness, self‐efficacy, and tenure status), their professional network (i.e., a supportive community, professional social system, and cultural acceptance of innovation), and their attitudes toward teaching (i.e., concern for student learning outcomes and valuing teaching in addition to research). Other characteristics are awareness of innovation and degree of support.</p> <hd id="AN0099000348-13">Characteristics of administrators</hd> <p>The 135 comments were classified into 4 characteristics that were then ranked by the experts. The most important characteristic was promoting and rewarding innovation in the classroom. This construct received 26 unique comments during the first round, with a panel member describing it in the following terms:</p> <p>“Most educational innovations require substantial time, energy, and commitment on the part of the innovator. It also typically requires a collaborative effort of several practitioners. These kinds of efforts and collaborations are not always rewarded in current workload or promotion and tenure considerations. One would not, or at least should not, expect faculty to engage in sustained practice of innovation if it is not valued in terms of workload or promotion and tenure considerations.”</p> <p>This opinion echoes that expressed by the President's Council of Advisors on Science and Technology ([<reflink idref="bib64" id="ref107">64</reflink>] ), which identified the lack of departmental rewards and expectations for good teaching as one of the main barriers and challenges to producing more STEM graduates. The American Society for Engineering Education ([<reflink idref="bib2" id="ref108">2</reflink>] ) has suggested that hiring, tenure and promotion guidelines, policies, and practices need to be reviewed to ensure that educational innovations and pedagogical preparations beyond teaching excellence are recognized and rewarded. The Association to Advance Collegiate Schools of Business (AACSB) ([<reflink idref="bib4" id="ref109">4</reflink>] ) has also recognized the importance of pedagogical innovations, listing it as one of the three scholarly outputs expected of faculty members in colleges of business across the nation. Other important characteristics identified as important for administrators were receptivity to change, management support, and facilitating conditions. The majority of faculty committees, department chairs, and deans feel that management support and the degree of facilitating conditions fall far short of supporting educational innovations (American Society for Engineering Education, [<reflink idref="bib3" id="ref110">3</reflink>] ). This finding shows that for educational innovations to be used routinely, administrators will need to invest time to change policies and practices, update the infrastructure of their institutions, and make resources more readily available in the classrooms.</p> <hd id="AN0099000348-14">LIMITATIONS AND FUTURE RESEARCH</hd> <p>There are several limitations to this study. First, the sample consisted of a relatively homogenous group of participants, each of whom had been a recipient of a CCLI or TUES grant from the NSF in the areas of engineering and technology in the United States. As such, the data collected may not reflect concerns that would have been expressed by faculty members in other fields. Second, while the sample was sufficiently large for a Delphi study, some subject attrition through the rounds of data collection was experienced. This attrition may have limited the scope and perspective of the resulting data and subsequent conclusions. Third, the process of synthesizing the characteristics from the comments was inductive. While this was necessary to develop the CODS framework, the process is inevitably subjective, and the data could have been synthesized differently. This limitation was addressed to some extent since the experts agreed to proceed using the characteristics identified by the researchers in rounds 2 and 3, suggesting that there was overall agreement among the researchers and the experts regarding the naming of the characteristics.</p> <p>Future research that operationalizes the proposed CODS framework, in part or in its entirety, will serve to allay the aforementioned limitations and enhance researchers understanding of the dissemination process. By operationalizing the CODS characteristics, future research may be able to create a score card, index, or decision support tool that can be used to apply the framework in a prescriptive manner. Although the results of this research can be of use to scholars and practitioners, future research needs to examine specific types or categories of educational innovations (i.e., study instructional materials alone or instructional strategies alone) to provide more specific guidance. Using the results of this study as a starting point, additional research may be able to develop a more refined version of the CODS framework for specific categories of educational innovations. In a similar vein, it would be interesting to conduct research that focuses specifically on students, faculty members, or administrators.</p> <p>There remain many unanswered questions which we encourage other researchers to examine. Most notably, appropriate measures need to be developed for the assessment of the characteristics of the CODS model. Empirical studies examining the diffusion of innovation theory, the theory of reasoned action, the theory of planned behavior, the technology acceptance model, and the concerns‐based acceptance model (CBAM) can all contribute to the development of suitable measures. Furthermore, the literature on organizational change may offer a way to operationalize measures of the characteristics of various stakeholders, and facilitate the quantification of their contributions toward organizational readiness. The levels of use of the innovation from CBAM can also be applied to develop measures of the extent to which faculty members actually use new technologies in the classroom (Patton, [<reflink idref="bib63" id="ref111">63</reflink>] ; Shenton, [<reflink idref="bib72" id="ref112">72</reflink>] ).</p> <hd id="AN0099000348-15">CONCLUSIONS</hd> <p>Many educators design, evaluate, and adopt educational innovations that will help their students develop the knowledge and skills they will need to compete in today's economy. Although public and private funding has been available to develop these innovations, there is an apparent disconnect between publishing information regarding these innovations and their subsequent dissemination and adoption for use in classrooms. For example, teaching briefs have been published regularly in the Decision Sciences Journal of Innovative Education for the past ten years but few follow‐up articles analyze whether they are actually being adopted in classrooms. This disconnect was addressed in this study by seeking answers to the research question “What are the most important characteristics that relate to the dissemination of educational innovations?” A Delphi methodology was used to elicit insights from experts regarding these characteristics. The ranking of the characteristics via a three‐round Delphi survey was then formulated into a conceptual CODS framework. This study's major contribution is that the CODS framework is based on experts’ comments, and identifies the characteristics that encourage/hinder the dissemination of educational innovations in engineering and technology disciplines.</p> <ref id="AN0099000348-16"> <title>Footnotes</title> <blist> <bibl id="bib1" idref="ref33" type="bt">1</bibl> <bibtext>We thank the Division of Undergraduate Education, National Science Foundation for funding this research under grant # 1140542. Any opinions, findings, and conclusions or recommendations expressed in this article are those of the authors and do not necessarily reflect the view of the National Science Foundation. </bibtext> </blist> </ref> <ref id="AN0099000348-17"> <title>REFERENCES</title> <blist> <bibtext>Ajzen, I. ( 1991 ). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50 ( 2 ), 179 – 211. </bibtext> </blist> <blist> <bibl id="bib2" idref="ref19" type="bt">2</bibl> <bibtext>American Society for Engineering Education ( 2009 ). Creating a culture for scholarly and systematic innovation in engineering education, accessed June 20, 2013, available at <ulink href="http://www.asee.org/about‐us/the‐organization/advisory‐committees/CCSSIE/CCSSIEE%5fPhase1Report%5fJune2009.pdf">http://www.asee.org/about‐us/the‐organization/advisory‐committees/CCSSIE/CCSSIEE%5fPhase1Report%5fJune2009.pdf</ulink>. </bibtext> </blist> <blist> <bibl id="bib3" idref="ref20" type="bt">3</bibl> <bibtext>American Society for Engineering Education ( 2012 ). Innovation with impact, accessed May 16, 2013, available at <ulink href="http://www.asee.org/Innovation">http://www.asee.org/Innovation</ulink> with Impact. </bibtext> </blist> <blist> <bibl id="bib4" idref="ref36" type="bt">4</bibl> <bibtext>Association to Advance Collegiate Schools of Business ( 2013 ). 2013 Accreditation standards, accessed January 28, 2014, available at <ulink href="http://aacsb.edu/accreditation/2013standards/">http://aacsb.edu/accreditation/2013standards/</ulink>. </bibtext> </blist> <blist> <bibl id="bib5" idref="ref24" type="bt">5</bibl> <bibtext>Balasubramanian, G., Lohani, V. K., Puri, I. K., Case, S. W., & Mahajan, R. L. ( 2011 ). Nanotechnology education: First step in implementing a spiral curriculum. The International Journal of Engineering Education, 27 ( 2 ), 333 – 353. </bibtext> </blist> <blist> <bibl id="bib6" idref="ref3" type="bt">6</bibl> <bibtext>Blankinship, D. G. ( 2010 ). Foundations offer $506 million for education innovation, accessed November 5, 2013, available at https://<ulink href="http://www.ncstem.org/blog/foundations‐offer‐506‐million‐for‐education‐innovation">www.ncstem.org/blog/foundations‐offer‐506‐million‐for‐education‐innovation</ulink>. </bibtext> </blist> <blist> <bibl id="bib7" idref="ref46" type="bt">7</bibl> <bibtext>Borrego, M. J., Froyd, J. E., & Hall, T. S. ( 2010 ). Diffusion of engineering education innovations: A survey of awareness and adoption rates in U.S. engineering departments. Journal of Engineering Education, 99 ( 3 ), 185 – 207. </bibtext> </blist> <blist> <bibl id="bib8" idref="ref78" type="bt">8</bibl> <bibtext>Brancheau, J. C., & Wetherby, J. C. ( 1987 ). Key issues in information systems managment. MIS Quarterly, 1 ( 11 ), 23 – 46. </bibtext> </blist> <blist> <bibl id="bib9" idref="ref104" type="bt">9</bibl> <bibtext>Cagiltay, N. E. ( 2008 ). Using learning styles theory in engineering education. European Journal of Engineering Education, 33 ( 4 ), 415 – 424. </bibtext> </blist> <blist> <bibl id="bib10" idref="ref62" type="bt">10</bibl> <bibtext>Cegielski, C. G., Bourrie, D. M., & Hazen, B. T. ( 2013 ). Evaluating adoption of emerging IT for corporate IT strategy: Developing a model using a qualitative method. Information Systems Management, 30 ( 3 ), 235 – 249. </bibtext> </blist> <blist> <bibl id="bib11" idref="ref55" type="bt">11</bibl> <bibtext>Cheville, A., & Bunting, C. ( 2011 ). Engineering students for the 21 st century: Student development through the curriculum. Advances in Engineering Education, 2 ( 4 ), 1 – 37. </bibtext> </blist> <blist> <bibl id="bib12" idref="ref66" type="bt">12</bibl> <bibtext>Churchman, C. W., & Schainblatt, A. ( 1965 ). The researcher and the manager: A dialectic of implementation. Management Science, 11 ( 4 ), 469 – 487. </bibtext> </blist> <blist> <bibl id="bib13" idref="ref89" type="bt">13</bibl> <bibtext>Compeau, D. R., Meister, D. B., & Higgins, C. A. ( 2007 ). From prediction to explanation: Reconceptualizing and extending the perceived characteristics of innovating. Journal of the Association for Information Systems, 8 ( 8 ), 409 – 439. </bibtext> </blist> <blist> <bibl id="bib14" idref="ref9" type="bt">14</bibl> <bibtext>D'Amours, S., & Rönnqvist, M. ( 2013 ). An educational game in collaborative logistics. INFORMS Transactions on Education, 13 ( 2 ), 102 – 113. </bibtext> </blist> <blist> <bibl id="bib15" idref="ref59" type="bt">15</bibl> <bibtext>Dalkey, N., & Helmer, O. ( 1963 ). An experimental application of the Delphi method to the use of experts. Management Science, 9 ( 3 ), 458 – 467. </bibtext> </blist> <blist> <bibl id="bib16" idref="ref45" type="bt">16</bibl> <bibtext>Dancy, M. H., & Henderson, C. ( 2010 ). Pedagogical practices and instructional change in physics faculty. American Journal of Physics, 78 ( 10 ), 1056 – 1063. </bibtext> </blist> <blist> <bibl id="bib17" idref="ref74" type="bt">17</bibl> <bibtext>Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. ( 1975 ). Group techniques for program planning: A guide to nominal group and Delphi processes : Glenview, IL : Scott, Foresman. </bibtext> </blist> <blist> <bibl id="bib18" idref="ref10" type="bt">18</bibl> <bibtext>Dufresne, R. L., & Offstein, E. H. ( 2012 ). Holistic and intentional student character development process: Learning from West Point. Academy of Management Learning & Education, 11 ( 4 ), 570 – 590. </bibtext> </blist> <blist> <bibl id="bib19" idref="ref11" type="bt">19</bibl> <bibtext>Erzurumlu, S. S., & Rollag, K. ( 2013 ). Increasing student interest and engagement with business cases by turning them into consulting exercises. Decision Sciences Journal of Innovative Education, 11 ( 4 ), 359 – 381. </bibtext> </blist> <blist> <bibl id="bib20" idref="ref42" type="bt">20</bibl> <bibtext>Fairweather, J. ( 2008 ). Linking evidence and promising practices in science, technology, engineering, and mathematics (STEM) undergraduate education. Washington, DC: Board of Science Education, National Research Council, The National Academies. </bibtext> </blist> <blist> <bibl id="bib21" idref="ref54" type="bt">21</bibl> <bibtext>Felder, R. M., & Silverman, L. ( 1988 ). Learning and teaching styles in engineering education. Engineering Education, 91 ( 1 ), 3 – 17. </bibtext> </blist> <blist> <bibl id="bib22" idref="ref5" type="bt">22</bibl> <bibtext>Fincher, S. ( 2000 ). From transfer to transformation: Towards a framework for successful dissemination of engineering education. Frontiers in Education Conference. Kansas City, MO : IEEE, F4F/14‐F4F/19 vol. 2. </bibtext> </blist> <blist> <bibl id="bib23" idref="ref86" type="bt">23</bibl> <bibtext>Fishbein, M., & Ajzen, I. ( 1975 ). Belief, attitude, intention and behavior: An introduction to theory and research. Reading, MA.: Addison‐Wesley. </bibtext> </blist> <blist> <bibl id="bib24" idref="ref87" type="bt">24</bibl> <bibtext>Fishbein, M., & Ajzen, I. ( 1981 ). Attitudes and voting behavior: An application of the theory of reasoned action. In G. M. Stephenson & J. M. Davis (Eds.), Progress in applied social psychology. London, UK : Wiley, 253 – 313. </bibtext> </blist> <blist> <bibl id="bib25" idref="ref6" type="bt">25</bibl> <bibtext>Gravestock, P. ( 2002 ). Making an impact through dissemination. In C. Baume, P. Martin, & M. Yorke (Eds.), Managing educational development projects: Effective management for maximum impact. London, UK : Kogan Page, 109 – 124. </bibtext> </blist> <blist> <bibl id="bib26" idref="ref41" type="bt">26</bibl> <bibtext>Gropper, G. L. ( 1974 ). Instructional strategies. Englewood Cliffs, NJ : Educational Technology Publications, Inc. </bibtext> </blist> <blist> <bibl id="bib27" idref="ref102" type="bt">27</bibl> <bibtext>Hannon, J. ( 2009 ). Breaking down online teaching: Innovation and resistance. Australasian Journal of Educational Technology, 25 ( 1 ), 14 – 29. </bibtext> </blist> <blist> <bibl id="bib28" idref="ref106" type="bt">28</bibl> <bibtext>Hawk, T. F., & Shah, A. J. ( 2007 ). Using learning style instruments to enhance student learning. Decision Sciences Journal of Innovative Education, 5 ( 1 ), 1 – 19. </bibtext> </blist> <blist> <bibl id="bib29" idref="ref38" type="bt">29</bibl> <bibtext>Hazen, B. T., Wu, Y., & Sankar, C. S. ( 2012a ). Factors that influence dissemination in engineering education. IEEE Transactions on Education, 55 ( 3 ), 384 – 393. </bibtext> </blist> <blist> <bibl id="bib30" idref="ref43" type="bt">30</bibl> <bibtext>Hazen, B. T., Wu, Y., Sankar, C. S., & Jones‐Farmer, L. A. ( 2012b ). A proposed framework for educational innovation dissemination. Journal of Educational Technology Systems, 40 ( 3 ), 301 – 321. </bibtext> </blist> <blist> <bibl id="bib31" idref="ref50" type="bt">31</bibl> <bibtext>Henderson, C., & Dancy, M. H. ( 2007 ). Barriers to the use of research‐based instructional strategies: The influence of both individual and situational characteristics. Physical Review Special Topics: Physics Education Research, 3 ( 2 ), 200102. </bibtext> </blist> <blist> <bibl id="bib32" idref="ref56" type="bt">32</bibl> <bibtext>Henderson, C., Dancy, M. H., & Niewiadomska‐Bugaj, M. ( 2012 ). Use of research‐based instructional strategies in introductory physics: Where do faculty leave the innovation‐decision process? Physical Review Special Topics – Physics Education Research, 8 ( 2 ), 020104. </bibtext> </blist> <blist> <bibl id="bib33" idref="ref44" type="bt">33</bibl> <bibtext>Henderson, C., Finkelstein, N., & Beach, A. ( 2010 ). Beyond dissemination in college science teaching: An introduction to four core change strategies. Journal of College Science Teaching, 39 ( 5 ), 18 – 25. </bibtext> </blist> <blist> <bibl id="bib34" idref="ref12" type="bt">34</bibl> <bibtext>Hu, X., & Kumar, A. ( 2013 ). Case—Beijing Opera mask sales in four cities in the United States. INFORMS Transactions on Education, 13 ( 3 ), 145 – 151. </bibtext> </blist> <blist> <bibl id="bib35" idref="ref7" type="bt">35</bibl> <bibtext>Hutchinson, J. R., & Huberman, M. ( 1994 ). Knowledge dissemination and use in science and mathematics education: A literature review. Journal of Science Education and Technology, 3 ( 1 ), 27 – 47. </bibtext> </blist> <blist> <bibl id="bib36" idref="ref99" type="bt">36</bibl> <bibtext>Kaleelur Rahuman, M. A., Wikramanayake, G. N., & Hewagamage, K. P. ( 2011 ). Case study on adaptability to ICT enabled childhood education in Sri Lanka. 2nd International Conference on Advances in ICT for Emerging Regions. Colombo, Sri Lanka : IEEE Computer Society, 112 ‐ 116. </bibtext> </blist> <blist> <bibl id="bib37" idref="ref32" type="bt">37</bibl> <bibtext>Katz, E. ( 1961 ). The social itineray of social change: Two studies on the diffusion of innovation. In W. Scharamm (Ed.), Studies of Innovation and Communication to the Public. Stanford, CA : Stanford University. </bibtext> </blist> <blist> <bibl id="bib38" idref="ref8" type="bt">38</bibl> <bibtext>King, H. ( 2003 ). Disseminating educational developments. In P. Kahn & D. Baume (Eds.), A guide to staff and educational development. London, UK : Kogan Page, 79 – 94. </bibtext> </blist> <blist> <bibl id="bib39" idref="ref52" type="bt">39</bibl> <bibtext>Kolmos, A., Kofoed, L. B., & Du, X. Y. ( 2008 ). Ph.D. students' work conditions and study environment in university‐ and industry‐based PhD programmes. European Journal of Engineering Education, 33 ( 5/6 ), 539 – 550. </bibtext> </blist> <blist> <bibl id="bib40" idref="ref25" type="bt">40</bibl> <bibtext>Krishnan, M., Paulik, M. J., Yost, S., & Stoltz, T. ( 2008 ). Shared projects with a multi‐sub‐disciplinary flavor: Providing integration and context in a new ECE spiral curriculum. ASEE/IEEE Frontiers in Education Conference. Saratoga Springs, NY : Institute of Electrical and Electronics Engineers Inc. F1H1–F1H6. </bibtext> </blist> <blist> <bibl id="bib41" idref="ref94" type="bt">41</bibl> <bibtext>Lee, Y., Kozar, K. A., & Larsen, K. R. T. ( 2003 ). The technology acceptance model: Past, present, and future. Communications of the Association for Information Systems, 12 ( 50 ), 752 – 780. </bibtext> </blist> <blist> <bibl id="bib42" idref="ref14" type="bt">42</bibl> <bibtext>Leemkuil, H., & De Jong, T. O. N. ( 2012 ). Adaptive advice in learning with a computer‐based knowledge management simulation game. Academy of Management Learning & Education, 11 ( 4 ), 653 – 665. </bibtext> </blist> <blist> <bibl id="bib43" idref="ref15" type="bt">43</bibl> <bibtext>Legner, C., Estier, T., Avdiji, H., & Boillat, T. ( 2013 ). Designing Capstone courses in management education: Knowledge activation and integration using and ERP‐based simulation game. Thirty Fourth International Conference on Information Systems. Milan, accessed January 31, 2014, available at <ulink href="http://aisel.aisnet.org/icis2013/proceedings/ISCurriculum/8/">http://aisel.aisnet.org/icis2013/proceedings/ISCurriculum/8/</ulink>. </bibtext> </blist> <blist> <bibl id="bib44" idref="ref60" type="bt">44</bibl> <bibtext>Linstone, H. A., & Turoff, M. ( 2011 ). Delphi: A brief look backward and forward. Technological Forecasting and Social Change, 78 ( 9 ), 1712 – 1719. </bibtext> </blist> <blist> <bibl id="bib45" idref="ref63" type="bt">45</bibl> <bibtext>Loo, R. ( 2002 ). The Delphi method: A powerful tool for strategic management. Policing: An International Journal of Police Strategies & Management, 25 ( 4 ), 762 – 769. </bibtext> </blist> <blist> <bibl id="bib46" idref="ref61" type="bt">46</bibl> <bibtext>Malhotra, M. K., Steele, D. C., & Grover, V. ( 1994 ). Important strategic and tactical manufacturing issues in the 1990s. Decision Sciences, 25 ( 2 ), 189 – 214. </bibtext> </blist> <blist> <bibl id="bib47" idref="ref77" type="bt">47</bibl> <bibtext>Martino, J. P. ( 1972 ). Technological forecasting for decision making. New York, NY : Elsevier Publishing. </bibtext> </blist> <blist> <bibl id="bib48" idref="ref40" type="bt">48</bibl> <bibtext>McKenzie, J., Alexander, S., Harper, C., & Anderson, S. ( 2005 ). Dissemination, adoption, & adaptation of project innovations in higher education. A report for the Carrick Institute for Learning and Teaching in Higher Education: University of Technology Sydney. </bibtext> </blist> <blist> <bibl id="bib49" idref="ref90" type="bt">49</bibl> <bibtext>Moore, G. C., & Benbasat, I. ( 1991 ). Development of an instrument to measure the perceptions of adopting an information technology innovation. Information Systems Research, 2 ( 3 ), 192 – 222. </bibtext> </blist> <blist> <bibl id="bib50" idref="ref53" type="bt">50</bibl> <bibtext>Morgan, J., & Barroso, L. ( 2008 ). North Texas STEM center: An engineer's perspective. ASEE Annual Conference and Exposition. Pittsburg, PA : American Society for Engineering Education. </bibtext> </blist> <blist> <bibl id="bib51" idref="ref26" type="bt">51</bibl> <bibtext>Mykytyn, K., Pearson, A., Paul, S., & Mykytyn, P. P. ( 2008 ). The use of problem‐based learning to enhance MIS education. Decision Sciences Journal of Innovative Education, 6 ( 1 ), 89 – 113. </bibtext> </blist> <blist> <bibl id="bib52" idref="ref79" type="bt">52</bibl> <bibtext>Nambisan, S., Agarwal, R., & Tanniru, M. ( 1999 ). Organizational mechanisms for enhancing user innovation in information technology. MIS Quarterly, 23 ( 3 ), 365 – 395. </bibtext> </blist> <blist> <bibl id="bib53" idref="ref21" type="bt">53</bibl> <bibtext>National Academy of Education. ( 2009 ). Education policy white paper on science and mathematics education, accessed August 20, 2012, available at <ulink href="http://www.naeducation.org/xpedio/groups/naedsite/documents/webpage/NAED%5f080865.pdf">http://www.naeducation.org/xpedio/groups/naedsite/documents/webpage/NAED%5f080865.pdf</ulink>. </bibtext> </blist> <blist> <bibl id="bib54" idref="ref22" type="bt">54</bibl> <bibtext>National Academy of Sciences. ( 2012 ). Discipline‐based education research: Understanding and improving learning in undergraduate science and engineering. Washington, DC : National Academies Press. </bibtext> </blist> <blist> <bibl id="bib55" idref="ref2" type="bt">55</bibl> <bibtext>National Research Council. ( 2011 ). Promising practices in undergraduate science, technology, engineering, and mathematics education: Summary of two workshops, accessed November 15, 2012, available at <ulink href="http://crpde.ou.edu/sites/crpde.ou.edu/files/NRC%20Report%20on%20Promising%20Practices%20in%20STEM%20Education.pdf">http://crpde.ou.edu/sites/crpde.ou.edu/files/NRC%20Report%20on%20Promising%20Practices%20in%20STEM%20Education.pdf</ulink>. </bibtext> </blist> <blist> <bibl id="bib56" idref="ref1" type="bt">56</bibl> <bibtext>National Science and Technology Council. ( 2011 ). The federal science, technology, engineering, and mathematics (STEM) education portfolio, accessed November 3, 2013, available at <ulink href="http://www.whitehouse.gov/sites/default/files/microsites/ostp/costem%5f%5ffederal%5fstem%5feducation%5fportfolio%5freport.pdf">http://www.whitehouse.gov/sites/default/files/microsites/ostp/costem%5f%5ffederal%5fstem%5feducation%5fportfolio%5freport.pdf</ulink>. </bibtext> </blist> <blist> <bibl id="bib57" idref="ref4" type="bt">57</bibl> <bibtext>National Science Foundation ( 2013 ). Widening implementation and demonstration of evidence‐based reforms (WIDER): Program soliciation, accessed May 14, 2013, available at <ulink href="http://www.nsf.gov/pubs/2013/nsf13552/nsf13552.htm">http://www.nsf.gov/pubs/2013/nsf13552/nsf13552.htm</ulink>. </bibtext> </blist> <blist> <bibl id="bib58" idref="ref13" type="bt">58</bibl> <bibtext>Nkhoma, M., Sriratanaviriyakul, N., Pham, H. C., & Lam, T. K. ( 2013 ). Understanding the paradox of information systems case studies. Nineteenth Americas Conference on Information Systems. Chicago, IL, accessed January 30, 2014, available at <ulink href="http://aisel.aisnet.org/amcis2013/ISEducation/GeneralPresentations/23/">http://aisel.aisnet.org/amcis2013/ISEducation/GeneralPresentations/23/</ulink>. </bibtext> </blist> <blist> <bibl id="bib59" type="bt">59</bibl> <bibtext>OECD ( 2010 ). OECD economic surveys: United States 2010. OECD Publishing. </bibtext> </blist> <blist> <bibl id="bib60" idref="ref67" type="bt">60</bibl> <bibtext>Okoli, C., & Pawlowski, S. D. ( 2004 ). The Delphi method as a research tool: An example, design considerations and applications. Information & Management, 42 ( 1 ), 15 – 29. </bibtext> </blist> <blist> <bibl id="bib61" idref="ref100" type="bt">61</bibl> <bibtext>Oppermann, R., & Rasher, R. ( 1997 ). Adaptability and adaptivity in learning systems. In A. Behrooz (Ed.), Knowledge transfer. London, UK : pAce, 173 – 179. </bibtext> </blist> <blist> <bibl id="bib62" idref="ref80" type="bt">62</bibl> <bibtext>Paliwoda, S. J. ( 1983 ). Predicting the future using Delphi. Management Decision, 21 ( 1 ), 31 – 38. </bibtext> </blist> <blist> <bibl id="bib63" idref="ref111" type="bt">63</bibl> <bibtext>Patton, M. Q. ( 2002 ). Qualitative evaluation and research methods Thousand Oaks, CA : Sage Publications. </bibtext> </blist> <blist> <bibl id="bib64" idref="ref29" type="bt">64</bibl> <bibtext>President's Council of Advisors on Science and Technology. ( 2012 ). Report to the President: Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. Washington DC : Executive Office of the President, accessed October 21, 2012, available at <ulink href="http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast‐engage‐to‐excel‐final%5ffeb.pdf">http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast‐engage‐to‐excel‐final%5ffeb.pdf</ulink>. </bibtext> </blist> <blist> <bibl id="bib65" idref="ref70" type="bt">65</bibl> <bibtext>Riggs, W. E. ( 1983 ). The Delphi technique: An experimental evaluation. Technological Forecasting and Social Change, 23 ( 1 ), 89 – 94. </bibtext> </blist> <blist> <bibl id="bib66" idref="ref57" type="bt">66</bibl> <bibtext>Riley, K., Davis, M., Cox, A., & Maciukenas, J. ( 2007 ). “Ethics in the details”: An NSF project to integrate ethics into the graduate engineering curriculum. IEEE International Professional Communication Conference, Seattle, WA : Institute of Electrical and Electronics Engineers. </bibtext> </blist> <blist> <bibl id="bib67" idref="ref31" type="bt">67</bibl> <bibtext>Rogers, E. M. ( 2003 ). Diffusion of innovations ( 5th ed. ). New York, NY : Free Press/Simon & Schuster. </bibtext> </blist> <blist> <bibl id="bib68" idref="ref71" type="bt">68</bibl> <bibtext>Rohrbaugh, J. ( 1979 ). Improving the quality of group judgment: Social judgment analysis and the Delphi technique. Organizational Behavior and Human Performance, 24 ( 1 ), 73 – 92. </bibtext> </blist> <blist> <bibl id="bib69" idref="ref64" type="bt">69</bibl> <bibtext>Romano, A. R. ( 2010 ). Malleable Delphi: Delphi research technique, its evolution, and business applications. International Review of Business Research Papers, 6 ( 5 ), 235 – 243. </bibtext> </blist> <blist> <bibl id="bib70" idref="ref101" type="bt">70</bibl> <bibtext>Ross, A. M., Rhodes, D. H., & Hastings, D. E. ( 2008 ). Defining changeability: Reconciling flexibility, adaptability, scalability, modifiability, and robustness for maintaining system lifecycle value. Systems Engineering, 11 ( 3 ), 246 – 262. </bibtext> </blist> <blist> <bibl id="bib71" idref="ref27" type="bt">71</bibl> <bibtext>Schunk, D. H., Pintrich, P. R., & Meece, J. L. ( 2008 ). Motivation in education: Theory, research, and applications (3rd ed.). Upper Saddle River, NJ : Merrill Prentice Hall. </bibtext> </blist> <blist> <bibl id="bib72" idref="ref112" type="bt">72</bibl> <bibtext>Shenton, A. K. ( 2004 ). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 ( 2 ), 63 – 75. </bibtext> </blist> <blist> <bibl id="bib73" idref="ref85" type="bt">73</bibl> <bibtext>Siegel, S. ( 1956 ). Nonparametric statistics for the behavioral sciences. New York : McGraw‐Hill. </bibtext> </blist> <blist> <bibl id="bib74" idref="ref28" type="bt">74</bibl> <bibtext>Smith, K. A., Sheppard, S. D., Johnson, D. W., & Johnson, R. T. ( 2005 ). Pedagogies of engagement: Classroom‐based practices. Journal of Engineering Education, 94 ( 1 ), 87 – 101. </bibtext> </blist> <blist> <bibl id="bib75" idref="ref16" type="bt">75</bibl> <bibtext>Snider, B., & Balakrishnan, J. ( 2013 ). Lessons learned from implementing web‐based simulations to teach operations management concepts. INFORMS Transactions on Education, 13 ( 3 ), 152 – 161. </bibtext> </blist> <blist> <bibl id="bib76" idref="ref17" type="bt">76</bibl> <bibtext>Taras, V. A. S., Caprar, D. V., Rottig, D., Sarala, R. M., Zakaria, N., Zhao, F., Jimenez, A., Wankel, C., Si Lei, W., Minor, M. S., Bryla, P., Ordenana, X., Bode, A., Schuster, A., Vaiginiene, E., Jintae Froese, F., Bathula, H., Yajnik, N., Baldegger, R., & Zengyu Huang, V. ( 2013 ). A global classroom? Evaluating the effectiveness of global virtual collaboration as a teaching tool in management education. Academy of Management Learning & Education, 12 ( 3 ), 414 – 435. </bibtext> </blist> <blist> <bibl id="bib77" idref="ref68" type="bt">77</bibl> <bibtext>Taylor, R. E., & Meinhardt, D. J. ( 1985 ). Defining computer information needs for small business: A Delphi method. Journal of Small Business Management, 23 ( 2 ), 3 – 10. </bibtext> </blist> <blist> <bibl id="bib78" idref="ref92" type="bt">78</bibl> <bibtext>Taylor, S., & Todd, P. ( 1995 ). Decomposition and crossover effects in the theory of planned behavior: A study of consumer adoption intentions. International Journal of Research in Marketing, 12 ( 2 ), 137 – 155. </bibtext> </blist> <blist> <bibl id="bib79" idref="ref37" type="bt">79</bibl> <bibtext>Tornatzky, L. G., & Fleischer, M. ( 1990 ). The processes of technological innovation. Lexington, MA : Lexington Books. </bibtext> </blist> <blist> <bibl id="bib80" idref="ref18" type="bt">80</bibl> <bibtext>Wu, Y., & Sankar, C. S. ( 2013 ). Impact of hands‐on research experience on students’ learning in an introductory management information system course. Decision Sciences Journal of Innovative Education, 11 ( 4 ), 335 – 358. </bibtext> </blist> </ref> <p>Graph: Innovation‐diffusion process (adapted from Rogers, 67 ).</p> <p>Graph: Characteristics of Dissemination Success (CODS) framework.</p> <aug> <p>By David M. Bourrie; Casey G. Cegielski; L. Allison Jones‐Farmer and Chetan S. Sankar</p> <p></p> <p>David M. Bourrie (Ph.D. Auburn University) is an Assistant Professor in the School of Computing at the University of South Alabama. His research interests include information technology capabilities, innovation diffusion, innovation dissemination, health information systems, and how information technology can improve decision making and performance. His research has appeared in international information systems journals including Information Systems Management and the International Journal of Information Systems and Change Management.</p> <p>Casey G. Cegielski is a Professor of Information Systems Management and former KPMG Information Assurance Faculty Fellow in the Harbert College of Business. He has authored more than 35 peer‐reviewed articles in areas that include innovation diffusion, inter‐organizational information systems, information security, and the strategic use of information technology. His research has appeared in numerous international information systems journals including Communications of the AIS, Information & Management, Decision Support Systems, and the Information Systems Journal. Additionally, Dr. Cegielski is the co‐author of a vastly popular information systems text book – Introduction to Management Information Systems: Supporting and Transforming Business, 5eds, John Wiley & Sons, 2014. Dr. Cegielski is an active professional consultant and holds several professional certifications including CISSP, CISA, and CRISC.</p> <p>L. Allison Jones‐Farmer (Ph.D. University of Alabama) currently holds the Van Andel Endowed Professorship in Business Analytics at Miami University. She is an active researcher, developing practical methods for analyzing data in industrial and business settings. She is currently on the editorial board of Journal of Quality Technology, and is a past Associate Editor for Technometrics. Dr. Jones‐Farmer enjoys developing innovative curricula and teaching analytics and statistics to undergraduates, master's and Ph.D. students. Dr. Jones‐Farmer is the 2005 recipient of the Colonial Company Teaching Excellence Award, the 2008 recipient of the MBA Teacher of the Year Award, and the 2012 recipient of the Outstanding Teacher in the Department of Aviation and Supply Chain Management at Auburn University. She has consulted with numerous organizations including Southwestern Bell Corporation, Regions Bank, Blue Cross Blue Shield, HID, PHH, and HealthSouth.</p> <p>Chetan S. Sankar is the Harbert College of Business Advisory Council Professor of Management Information Systems at Auburn University. He has received more than three million dollars from several National Science Foundation grants to develop exceptional instructional materials that bring real‐world issues into classrooms. He has won awards for research and teaching excellence from the Society for Information Management, NEEDS, Decision Sciences Institute, American Society for Engineering Education, American Society for Mechanical Engineering, International Network for Engineering Education & Research, Computer World, Campus Technology, and the Project Management Institute. He is certified as a Project Management Professional. He is the managing editor of the Journal of STEM Education: Innovations and Research and Director of the Geospatial Research and Applications Center.</p> </aug>
Header DbId: eric
DbLabel: ERIC
An: EJ1043427
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Identifying Characteristics of Dissemination Success Using an Expert Panel
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Bourrie%2C+David+M%2E%22">Bourrie, David M.</searchLink><br /><searchLink fieldCode="AR" term="%22Cegielski%2C+Casey+G%2E%22">Cegielski, Casey G.</searchLink><br /><searchLink fieldCode="AR" term="%22Jones-Farmer%2C+L%2E+Allison%22">Jones-Farmer, L. Allison</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Decision+Sciences+Journal+of+Innovative+Education%22"><i>Decision Sciences Journal of Innovative Education</i></searchLink>. Oct 2014 12(4):357-380.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 24
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2014
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: National Science Foundation
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: 1140542
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Expertise%22">Expertise</searchLink><br /><searchLink fieldCode="DE" term="%22Adoption+%28Ideas%29%22">Adoption (Ideas)</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Innovation%22">Educational Innovation</searchLink><br /><searchLink fieldCode="DE" term="%22Communication+%28Thought+Transfer%29%22">Communication (Thought Transfer)</searchLink><br /><searchLink fieldCode="DE" term="%22Delphi+Technique%22">Delphi Technique</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Characteristics%22">Student Characteristics</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Characteristics%22">Teacher Characteristics</searchLink><br /><searchLink fieldCode="DE" term="%22Administrator+Characteristics%22">Administrator Characteristics</searchLink><br /><searchLink fieldCode="DE" term="%22Information+Dissemination%22">Information Dissemination</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink><br /><searchLink fieldCode="DE" term="%22College+Faculty%22">College Faculty</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/dsji.12049
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1540-4595
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Although considerable work has been done to develop new educational innovations, few have found widespread acceptance in the classroom. To improve the likelihood of adoption of educational innovations, researchers need to understand why some innovations are adopted and routinely used, while others are not. An initial aspect of the diffusion of innovations, as defined in the classical sociological literature, involves the communication of ideas and concepts related to innovations between individuals. This article presents an expert panel's answer to the following question: "What are the most important characteristics that relate to the dissemination of educational innovations?" As dissemination is a critical facet of the diffusion of an innovation, 45 researchers who received technology and engineering grants from the National Science Foundation (NSF) participated in a Delphi study designed to address this research question. In three rounds, the experts identified and ranked 11 characteristics of educational innovations, 6 characteristics of students, 13 characteristics of faculty members, and 5 characteristics of administrators that can relate to the successful dissemination of educational innovations. The results of this study led to the formation of a Characteristics of Dissemination Success (CODS) framework. This framework offers useful guidance for educational innovators seeking a better understanding of the influences on the dissemination of educational innovations.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2014
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1043427
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1043427
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/dsji.12049
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 24
        StartPage: 357
    Subjects:
      – SubjectFull: Expertise
        Type: general
      – SubjectFull: Adoption (Ideas)
        Type: general
      – SubjectFull: Educational Innovation
        Type: general
      – SubjectFull: Communication (Thought Transfer)
        Type: general
      – SubjectFull: Delphi Technique
        Type: general
      – SubjectFull: Student Characteristics
        Type: general
      – SubjectFull: Teacher Characteristics
        Type: general
      – SubjectFull: Administrator Characteristics
        Type: general
      – SubjectFull: Information Dissemination
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: College Faculty
        Type: general
    Titles:
      – TitleFull: Identifying Characteristics of Dissemination Success Using an Expert Panel
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Bourrie, David M.
      – PersonEntity:
          Name:
            NameFull: Cegielski, Casey G.
      – PersonEntity:
          Name:
            NameFull: Jones-Farmer, L. Allison
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 10
              Type: published
              Y: 2014
          Identifiers:
            – Type: issn-print
              Value: 1540-4595
          Numbering:
            – Type: volume
              Value: 12
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Decision Sciences Journal of Innovative Education
              Type: main
ResultId 1