View in EDS HTML Full Text PDF Full Text

Pedagogical Approaches for E-Assessment with Authentication and Authorship Verification in Higher Education

Saved in:
Bibliographic Details
Title: Pedagogical Approaches for E-Assessment with Authentication and Authorship Verification in Higher Education
Language: English
Authors: Okada, Alexandra (ORCID 0000-0003-1572-5605), Noguera, Ingrid, Alexieva, Lyubka (ORCID 0000-0002-7877-5792), Rozeva, Anna (ORCID 0000-0001-6069-5826), Kocdar, Serpil (ORCID 0000-0001-9099-6312), Brouns, Francis (ORCID 0000-0002-6240-2684), Ladonlahti, Tarja (ORCID 0000-0002-4315-1947), Whitelock, Denise (ORCID 0000-0002-9897-4975), Guerrero-Roldán, Ana-Elena (ORCID 0000-0001-7073-7233)
Source: British Journal of Educational Technology. Nov 2019 50(6):3264-3282.
Availability: Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Peer Reviewed: Y
Page Count: 19
Publication Date: 2019
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Audits (Verification), Plagiarism, Cheating, Internet, Electronic Learning, College Faculty, Trust (Psychology), Computer Assisted Testing, Foreign Countries, Information Security, Privacy, Teacher Attitudes
Geographic Terms: United Kingdom, Spain, Netherlands, Bulgaria, Finland, Turkey
DOI: 10.1111/bjet.12733
ISSN: 0007-1013
Abstract: Checking the identity of students and authorship of their online submissions is a major concern in Higher Education due to the increasing amount of plagiarism and cheating using the Internet. The literature on the effects of e-authentication systems for teaching staff is very limited because it is a novel procedure for them. A considerable gap is to understand teaching staff' views regarding the use of e-authentication instruments and how they impact trust in e-assessment. This mixed-method study examines the concerns and practices of 108 teaching staff who used the TeSLA--Adaptive Trust-based e-Assessment System in six countries: the UK, Spain, the Netherlands, Bulgaria, Finland and Turkey. The findings revealed some technological, organisational and pedagogical issues related to accessibility, security, privacy and e-assessment design and feedback. Recommendations are to provide a FAQ and an audit report with results, to raise awareness about data security and privacy, to develop policies and guidelines about fraud detection and prevention, e-assessment best practices and course team support.
Abstractor: As Provided
Entry Date: 2019
Accession Number: EJ1232103
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFGLcN4Z1_dh7GE2Nh4aye5AAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDEOPzL0YsPZhfvMWtgIBEICBmqHb_M64lnDFmCd9Zq6IK9kYp6_xgLk9s-VwAGi72IjIpjP5GYBpX05ey_hKDd81ZbhtfHBAVVIZhs_oijlYmD9POcuxFGflNhVvlsQVlDCB6T-XD1Aar5COUsUbE4hjdKp0SX3ZMIqp_bAw2IDdCEeRqpSVbqZda1RA3hJLjZmYCnbzsWRv4xprSBAg57CXZvMLCXPT992136Y=
Text:
  Availability: 1
  Value: <anid>AN0139230518;58i01nov.19;2019Oct23.02:46;v2.2.500</anid> <title id="AN0139230518-1">Pedagogical approaches for e‐assessment with authentication and authorship verification in Higher Education </title> <p>Checking the identity of students and authorship of their online submissions is a major concern in Higher Education due to the increasing amount of plagiarism and cheating using the Internet. The literature on the effects of e‐authentication systems for teaching staff is very limited because it is a novel procedure for them. A considerable gap is to understand teaching staff' views regarding the use of e‐authentication instruments and how they impact trust in e‐assessment. This mixed‐method study examines the concerns and practices of 108 teaching staff who used the TeSLA—Adaptive Trust‐based e‐Assessment System in six countries: the UK, Spain, the Netherlands, Bulgaria, Finland and Turkey. The findings revealed some technological, organisational and pedagogical issues related to accessibility, security, privacy and e‐assessment design and feedback. Recommendations are to provide a FAQ and an audit report with results, to raise awareness about data security and privacy, to develop policies and guidelines about fraud detection and prevention, e‐assessment best practices and course team support.</p> <p></p> <ulist> <item> Checking the identity of students and authorship of their online submissions is a major concern in Higher Education.</item> <p></p> <item> A growing body of literature recommended the use of security mechanisms to identify students and detect illegitimate behaviours in e‐assessment.</item> <p></p> <item> The relation between pedagogical approaches and e‐assessment supported by e‐authentication and authorship verification is under‐explored.</item> <p></p> <item> Teaching‐learning process with the support of e‐authentication and authorship verification instruments integrated at the course design stage.</item> <p></p> <item> E‐assessment activities' characteristics identified to support face recognition, voice recognition, keystroke dynamics, forensic analysis and plagiarism detection.</item> <p></p> <item> Seven recommendations to help teaching staff with e‐assessment supported by e‐authentication and authorship verification.</item> <p></p> <item> Increasing the awareness of data security and privacy among all end‐users will be important to enhance the trust of the e‐assessment system with e‐authentication and authorship verification.</item> <p></p> <item> Universities must develop and share best practices for verifying fraud supported by the e‐assessment system based on combined e‐authentication and authorship verification instruments.</item> <p></p> <item> Universities should support course teams to plan useful activities and assessment tasks with e‐authentication and authorship verification instruments.</item> </ulist> <p>Practitioner NotesWhat is already known about this topic What this paper adds Implications for practice and/or policy</p> <hd id="AN0139230518-2">Introduction</hd> <p>One of the main concerns for the adoption of e‐assessment is to ensure that the person who performs the assessment is the correct claimant (authentication) and to demonstrate that the work performed is original (authorship) (Noguera, Guerrero‐Roldán, & Rodríguez, [<reflink idref="bib34" id="ref1">34</reflink>]; Okada, Mendonca, & Scott, [<reflink idref="bib35" id="ref2">35</reflink>]; Okada, Whitelock <emph>et al</emph>. [<reflink idref="bib37" id="ref3">37</reflink>]). A growing body of literature has explored and recommended the use of security mechanisms to identify students and detect illegitimate behaviours in e‐assessment (Harmon, Lambrinos, & Buffolino, [<reflink idref="bib20" id="ref4">20</reflink>]; Osman, Salim, & Abuobieda, [<reflink idref="bib38" id="ref5">38</reflink>]; Simon <emph>et al</emph>., [<reflink idref="bib45" id="ref6">45</reflink>]; Watson & Sottile, [<reflink idref="bib51" id="ref7">51</reflink>]).</p> <p>According to European research on the impact of policies for plagiarism in Higher Education (IPPHEAE, [<reflink idref="bib22" id="ref8">22</reflink>]), internet usage is considered as one of the catalysts for cheating. Moreover, no simple solutions have been found to tackle this problem (Bermingham, Watson, & Jones, [<reflink idref="bib6" id="ref9">6</reflink>]; Usoof, Hudson, & Lindgren, [<reflink idref="bib49" id="ref10">49</reflink>]). In order to reduce academic malpractice in online programmes, the authentication of student work through the use of digital identities has become increasingly important for universities that offer online and blended courses (Ardid, Gomez‐Tejedor, Meseguer‐Duenas, Riera, & Vidaurre, [<reflink idref="bib4" id="ref11">4</reflink>]; Chew, Ding, & Rowell, [<reflink idref="bib9" id="ref12">9</reflink>]). E‐assessment systems are perceived as secure and appropriate when the instruments successfully identify and authenticate the examinee (Apampa, Wills, & Argles, [<reflink idref="bib2" id="ref13">2</reflink>]; Gao, [<reflink idref="bib17" id="ref14">17</reflink>]; Karim & Shukur, [<reflink idref="bib27" id="ref15">27</reflink>]).</p> <p>This paper focuses on the use of these types of e‐authentication and authorship instruments by seven pilot institutions involved in the EU‐funded <emph>TeSLA</emph> project—<emph>Adaptive Trust‐based e‐Assessment System for Learning</emph> (<ulink href="http://tesla-project.eu">http://tesla-project.eu</ulink>). The TeSLA system combines the instruments developed by several institutions and companies that are part of the consortium. Biometric instruments are used to authenticate students, while the instruments that analyse text are employed in verifying authorship. Some textual analysis instruments like writing style (Forensic) analysis can also be used for authentication purposes. The instruments are described in detail below (Knuth, [<reflink idref="bib28" id="ref16">28</reflink>]). Apart from plagiarism detection, these instruments require a sample to be created separately by each student as a learner model to be compared with their e‐assessment activities.</p> <hd id="AN0139230518-3">Biometrics</hd> <p></p> <ulist> <item> <emph>Facial Recognition (FR):</emph> compares the face and facial expressions using images (minimum resolution of 640 px × 480 px) and videos of at least 10 seconds with the learner model.</item> <p></p> <item> <emph>Voice Recognition (VR): </emph>compares voice structures with the learner model. The set of speech samples must have a minimum resolution of 16 kHz.</item> <p></p> <item> <emph>Keystroke Dynamics (KD):</emph> compares the rhythm and speed of typing when using the keyboard with the learner model. At least 30 samples have to be collected that must contain dwells and flights, which must be extracted from 125 consecutive pressed keys.</item> <p></p> <item> <emph>Face presentation attack detection (FPAD):</emph> detects presentation attacks to the face recognition instrument. It expects a real person in front of the camera and not an image. The student must be close to the camera with a minimum distance of 50 pixels between the centre of the eyes.</item> <p></p> <item> <emph>Voice presentation attack detection (VPAD):</emph> detects attacks in the voice presentation. It expects a real person talking and not an audio file. The student's uncompressed recorded audio file must have a minimum resolution of 16 kHz.</item> </ulist> <hd id="AN0139230518-4">Textual analysis</hd> <p></p> <ulist> <item> <emph>Plagiarism detection (PL):</emph> detects similarities (word‐for‐word copies) between a given set of text documents created by students using text matching. The instrument supports common text, word‐processor and PDF formats. This instrument does not compare the given set with external content on the internet.</item> <p></p> <item> <emph>Forensic Analysis (FA):</emph> compares the personal writing style to the learner model. The user model is updated over time with submission of new documents.</item> </ulist> <p>The TeSLA instruments (Figure) can be integrated into any Institutional Virtual Learning Environment (VLE). The learner and teacher user interfaces are implemented via an LTI plugin. Three different versions of the TeSLA system have been tested in the project. The current study reports on the second version that was implemented during the spring semester of 2016/2017 academic year.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/58I/01nov19/bjet12733-fig-0001.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="bjet12733-fig-0001.jpg" title="TeSLA architecture (Knuth, [28])" /> </p> <p></p> <p>This study is based on Responsible Research and Innovation (RRI) approach, which refers to the transparent and interactive process for aligning scientific innovation to the values, needs and expectations of the society (EC, [<reflink idref="bib12" id="ref17">12</reflink>]; Owen, [<reflink idref="bib39" id="ref18">39</reflink>]; Von Schomberg, [<reflink idref="bib50" id="ref19">50</reflink>]). This work is part of a research programme whose aim is to examine how technology‐enhanced assessment (TEA) can address the needs and expectations of educational communities during the development of the TeSLA system. This particular RRI study focuses on teachers' perceptions of e‐assessment with e‐authentication and authorship verification. It analyses teaching staff concerns and practices with assessment activities combined with e‐authentication of students' identity and authorship verification, including students with Special Educational Needs and Disabilities (SEND). Findings on teachers' acceptability and desirability of the innovation process including its sustainability based on pedagogical approaches will be of interest for higher education institutions, course developers, pedagogical teams and policy makers.</p> <hd id="AN0139230518-6">Opportunities afforded by TEA</hd> <p>A recent research review (Timmis, Broadfoot, Sutherland, & Oldfield, [<reflink idref="bib47" id="ref20">47</reflink>]) presents seven features of TEA approaches. It highlights the critical role of TEA, which offers many potentially creative opportunities for innovation and for rethinking assessment purposes. Digital media provide students with new opportunities for representing knowledge and skills. Students can benefit from new ways to record achievement, such as, open badges and awards from gaming environments (Law, [<reflink idref="bib30" id="ref21">30</reflink>]). However, there are also various concerns, such as ethical issues and the implications for general data protection (GPDR) and ownership of making, mixing and publishing media online and consent about how such data should be collected, used and stored. Another issue is social exclusion, for example, students might not feel comfortable with the assessment environment when they do not feel safe enough in case of failure and overly concerned with the consequences.</p> <p>There is a lack of literature related to pedagogical recommendations for e‐assessment with instruments for e‐authentication and authorship verification. In the TeSLA project, the process of integrating these instruments starts at the course design stage (Figure).</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/58I/01nov19/bjet12733-fig-0002.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="bjet12733-fig-0002.jpg" title="Teaching‐learning process with the support of e‐authentication and authoring instruments" /> </p> <p></p> <p>The teacher redesigns, or creates new e‐assessment activities, where the security instruments are enabled. Two types of activities are necessary: enrolment and real. The <emph>enrolment activities</emph> are used to create a learner model (a biometric model) that is later used as a reference for user authentication in subsequent real activities. They are activities with non‐grading purpose that serve for user registration. The <emph>real e‐assessment activities</emph> aim to authenticate students and validate the authorship of their work and may have grading purpose. The enrolment activities have to be performed prior to the real e‐assessment activities.</p> <p>The most widespread classification of learning activities is Bloom's taxonomy which differentiates between knowledge, comprehension, application, analysis, synthesis and evaluation skills (Bloom, Engelhart, Furst, Hill, & Krathwohl, [<reflink idref="bib7" id="ref22">7</reflink>]). Several authors have redefined these categories by incorporating new activities and tools related to digital resources (Carrington, [<reflink idref="bib8" id="ref23">8</reflink>]; Churches, [<reflink idref="bib10" id="ref24">10</reflink>]; Schrock, [<reflink idref="bib44" id="ref25">44</reflink>]).</p> <p>The specific classifications regarding e‐assessment activities include questions (eg, closed, open, multiple‐choice, matching, ordering), e‐portfolios, essays, online discussions, concept maps, personal response systems, badging, online role playing or scenario‐based activities (Guàrdia, Crisp, & Alsina, [<reflink idref="bib18" id="ref26">18</reflink>]; Manoharan, [<reflink idref="bib31" id="ref27">31</reflink>]; Stödberg, [<reflink idref="bib46" id="ref28">46</reflink>]; Trumbull & Lash, [<reflink idref="bib48" id="ref29">48</reflink>]). A recent study proposes a new classification of e‐assessment activities organised into five competences: (<reflink idref="bib1" id="ref30">1</reflink>) ability to search for, process and analyse information, (<reflink idref="bib2" id="ref31">2</reflink>) ability to apply acquired knowledge, (<reflink idref="bib3" id="ref32">3</reflink>) ability to use language to communicate successfully, (<reflink idref="bib4" id="ref33">4</reflink>) ability to create learning products in diverse formats and (<reflink idref="bib5" id="ref34">5</reflink>) ability to apply knowledge in real or simulated scenarios (Guerrero‐Roldán & Noguera, [<reflink idref="bib19" id="ref35">19</reflink>]). Following the approach of classifying the activities based on the abilities they help to develop, in the context of the TeSLA project, e‐assessment activities are organised into three categories related to what types of skills are being tested:</p> <p></p> <ulist> <item> <emph>Selection:</emph> students select an answer (eg, a correct answer from a list, a match or an adequate next step in a procedure).</item> <p></p> <item> <emph>Creation</emph>: students create an answer or product (eg, an answer to an open question, a report, a table or an image).</item> <p></p> <item> <emph>Performance:</emph> students perform/enact/demonstrate attainment of learning outcomes (eg, giving a presentation, taking part in a role play, game or simulation, carrying out a laboratory practical or an internship).</item> </ulist> <p>These activities can be delivered in various formats. For instance, a text may be typed or generated using speech to text software, a performance may be delivered "real‐time" or submitted as a video and/or voice recording.</p> <hd id="AN0139230518-8">Methodology</hd> <p>The TeSLA system is being designed and refined following a cyclical process of improvement based on the quantitative and qualitative data collected and analysed with mixed‐methods from three sets of pilots. This paper focuses on the experience and data gathered from the second pilot that took place between February and June 2017 within the context of seven universities (including face‐to‐face, blended and online modes of delivery) in six countries. These universities were: Anadolu University (AU) in Turkey, the University of Jyväskylä (JYU) in Finland, Open Universiteit (OUNL) in the Netherlands, Open University (OU‐UK) in the United Kingdom, the University of Sofia (SU) and (Technical University of Sofia) TUS both in Bulgaria and Open University of Catalonia (UOC) in Catalunya.</p> <p>To understand the various affordances and issues raised above and to contribute to the refinement of the TeSLA e‐authentication and authoring instruments, we investigated teaching staff' views about the TeSLA system with respect to the following research questions:</p> <p></p> <ulist> <item> (RQ1) What are the pedagogical approaches used for implementing the TeSLA e‐authentication and authoring instruments for e‐assessment?</item> <p></p> <item> (RQ2) How accurate are TeSLA instruments in identifying plagiarism and cheating?</item> <p></p> <item> (RQ3) What are the teaching staff opinions about the use of the TeSLA e‐authentication and authoring instruments for e‐assessment?</item> <p></p> <item> (RQ4) What are the teaching staff recommendations for the use of the TeSLA instruments with students?</item> </ulist> <hd id="AN0139230518-9">Context and participants</hd> <p>The participants in this study were teaching staff of undergraduate and postgraduate courses from a range of fields of study. Table illustrates the characteristics of a total of 113 courses from the seven pilot institutions participating in the pilot.</p> <p>Characteristics of the courses participating in the pilot</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Characteristics</th><th align="left">Values</th><th align="center">%</th></tr></thead><tbody><tr><td align="left">Mode of delivery</td><td align="left">Blended</td><td align="left">48</td></tr><tr><td align="left" /><td align="left">Online and distance</td><td align="left">52</td></tr><tr><td align="left">Language of the course</td><td align="left">Bulgarian</td><td align="left">15</td></tr><tr><td align="left" /><td align="left">Catalan and Spanish</td><td align="left">14</td></tr><tr><td align="left" /><td align="left">English</td><td align="left">28</td></tr><tr><td align="left" /><td align="left">Finnish</td><td align="left">19</td></tr><tr><td align="left" /><td align="left">Turkish</td><td align="left">20</td></tr><tr><td align="left">Course level</td><td align="left">Continuing professional development (CPD)</td><td align="left">1</td></tr><tr><td align="left" /><td align="left">Post‐graduate</td><td align="left">11</td></tr><tr><td align="left" /><td align="left">Undergraduate</td><td align="left">71</td></tr><tr><td align="left" /><td align="left">Undergraduate & CPD</td><td align="left">16</td></tr><tr><td align="left" /><td align="left">Other or missing</td><td align="left">1</td></tr><tr><td align="left">Field of study</td><td align="left">Arts and Humanities</td><td align="left">10</td></tr><tr><td align="left" /><td align="left">Social and Legal Sciences</td><td align="left">66</td></tr><tr><td align="left" /><td align="left">Sciences</td><td align="left">6</td></tr><tr><td align="left" /><td align="left">Health sciences</td><td align="left">2</td></tr><tr><td align="left" /><td align="left">Engineering and architecture</td><td align="left">15</td></tr><tr><td align="left" /><td align="left">Other or missing</td><td align="left">1</td></tr></tbody></table> </ephtml> </p> <p>The five TeSLA instruments (FR, VR, KD, FA, PL) were employed in these courses. The choice of instruments was based on the assessment contexts and feasibility of use according to the assessment activity. Table illustrates the number of students using the TeSLA instruments and Table shows the total set of samples gathered and analysed by the TeSLA system during the enrolment and assessment activities.</p> <p>Number of students who used each instrument per pilot institution</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Instruments</th><th align="center">Institutions</th><th align="left">Total</th></tr><tr><th align="left" /><th align="left">AU</th><th align="left">JYU</th><th align="left">OUNL</th><th align="left">OUUK</th><th align="left">SU</th><th align="left">TUS</th><th align="left">UOC</th></tr></thead><tbody><tr><td align="left">(FR)Face recognition</td><td align="left">1443</td><td align="left">10</td><td align="left">14</td><td align="left">–</td><td align="left">262</td><td align="left">402</td><td align="left">503</td><td align="left">2634</td></tr><tr><td align="left">(VR)Voice recognition</td><td align="left">175</td><td align="left">10</td><td align="left">8</td><td align="left">–</td><td align="left">63</td><td align="left">200</td><td align="left">437</td><td align="left">893</td></tr><tr><td align="left">(KD)Keystroke dynamics</td><td align="left">191</td><td align="left">87</td><td align="left">7</td><td align="left">382</td><td align="left">61</td><td align="left">249</td><td align="left">500</td><td align="left">1095</td></tr><tr><td align="left">(FA)Forensic analysis</td><td align="left">72</td><td align="left">103</td><td align="left">32</td><td align="left">–</td><td align="left">125</td><td align="left">47</td><td align="left">486</td><td align="left">865</td></tr><tr><td align="left">(PL)Plagiarism detection</td><td align="left">70</td><td align="left">80</td><td align="left">132</td><td align="left">266</td><td align="left">125</td><td align="left">89</td><td align="left">455</td><td align="left">951</td></tr><tr><td align="left">Total of students</td><td align="left">1951</td><td align="left">280</td><td align="left">159</td><td align="left">648</td><td align="left">505</td><td align="left">506</td><td align="left">882</td><td align="left">4931</td></tr></tbody></table> </ephtml> </p> <p>Samples from seven institutions analysed by the TeSLA system</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left" /><th align="left">Audio</th><th align="left">Text document</th><th align="center">Keystroke</th><th align="center">Video</th></tr></thead><tbody><tr><td align="left">Valid enrolment samples</td><td align="left">10,830</td><td align="left">10,228</td><td align="left">19,021</td><td align="left">6281</td></tr><tr><td align="left">Valid verification samples from assessment activities</td><td align="left">4159</td><td align="left">993</td><td align="left">7843</td><td align="left">3894</td></tr></tbody></table> </ephtml> </p> <hd id="AN0139230518-10">The procedures for data collection</hd> <p>Teaching staff completed the following steps during the course of the TeSLA pilot:</p> <p></p> <ulist> <item> <emph>Course design:</emph> design and implement an e‐assessment with instruments for students. For instance, typing or choosing answers in quizzes or online text submissions (FA, KD and PL), performing a presentation (FR, VR), creating artefacts (FR, VR, KD, FA and PL) or uploading documents in assignment (PL, FA).</item> <p></p> <item> <emph>Enrolment</emph>: provide guidelines for students to create a baseline for the instruments used during the real e‐assessment activities. This procedure provides initial data for the system to compare against during the e‐authentication and authorship checking.</item> <p></p> <item> <emph>Pre‐questionnaire</emph>: complete a 20‐question questionnaire about their demographics, views about plagiarism and cheating, their opinions about e‐authentication and authorship checking instruments, data security, data privacy and trust.</item> <p></p> <item> <emph>Post‐questionnaire:</emph> complete a 15‐question post‐questionnaire about their experience with the TeSLA system after using it with students, with similar questions of the pre‐questionnaire.</item> <p></p> <item> <emph>Focus group:</emph> attend 40‐minute session online or face‐to‐face to provide detailed views.</item> </ulist> <p>One representative from each pilot institution was in regular communication with teaching staff, to support them and gather data about the courses and the pilot progress recorded in an online database.</p> <p>A representative sample of 108 teaching staff in terms of gender, age and occupation participated in this study (Table). Response rate for the pre‐pilot questionnaire was 84% (<emph>N</emph> = 78) and 45% (<emph>N</emph> = 42) for the post‐pilot questionnaire. This questionnaire included a question for willingness to participate in the focus group. From these, 35 interviewees participated in representative focus group sessions organised by the seven pilot countries (Table).</p> <p>Teaching staff demographics from respondents to pre‐pilot questionnaire (N = 78)</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Items</th><th align="left">Values</th><th align="left"><italic>N</italic></th><th align="left">%</th></tr></thead><tbody><tr><td align="left">Gender</td><td align="left">Prefer not to say</td><td align="left">3</td><td align="left">4</td></tr><tr><td align="left" /><td align="left">Female</td><td align="left">45</td><td align="left">58</td></tr><tr><td align="left" /><td align="left">Male</td><td align="left">30</td><td align="left">38</td></tr><tr><td align="left">Age</td><td align="left">Prefer not to say</td><td align="left">2</td><td align="left">3</td></tr><tr><td align="left" /><td align="left">From 31 to 40 years old</td><td align="left">25</td><td align="left">32</td></tr><tr><td align="left" /><td align="left">From 41 to 50 years old</td><td align="left">33</td><td align="left">42</td></tr><tr><td align="left" /><td align="left">51 years or over</td><td align="left">18</td><td align="left">23</td></tr><tr><td align="left">Occupation</td><td align="left">Course designer</td><td align="left">37</td><td align="left">47</td></tr><tr><td align="left" /><td align="left">Students' assessor</td><td align="left">31</td><td align="left">40</td></tr><tr><td align="left" /><td align="left">Course instructor</td><td align="left">69</td><td align="left">88</td></tr><tr><td align="left" /><td align="left">Other</td><td align="left">2</td><td align="left">3</td></tr></tbody></table> </ephtml> </p> <p>Teaching staff participants in focus groups (N = 35)</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left" /><th align="left" /><th align="left">AU</th><th align="left">JYU</th><th align="left">OUNL</th><th align="left">OUUK</th><th align="left">SU</th><th align="left">TUS</th><th align="left">UOC</th></tr></thead><tbody><tr><td align="left">Total</td><td align="left">Teaching staff</td><td align="left">4</td><td align="left">5</td><td align="left">4</td><td align="left">4</td><td align="left">7</td><td align="left">7</td><td align="left">4</td></tr><tr><td align="left">Gender</td><td align="left">Female</td><td align="left">2</td><td align="left">4</td><td align="left">2</td><td align="left">2</td><td align="left">3</td><td align="left">5</td><td align="left">2</td></tr><tr><td align="left" /><td align="left">Male</td><td align="left">2</td><td align="left">1</td><td align="left">2</td><td align="left">2</td><td align="left">4</td><td align="left">3</td><td align="left">2</td></tr><tr><td align="left">Role</td><td align="left">Course designer</td><td align="left">1</td><td align="left">5</td><td align="left">4</td><td align="left">2</td><td align="left">6</td><td align="left">8</td><td align="left">3</td></tr><tr><td align="left" /><td align="left">Course Instructor</td><td align="left">2</td><td align="left">5</td><td align="left">4</td><td align="left">3</td><td align="left">6</td><td align="left">8</td><td align="left">1</td></tr><tr><td align="left" /><td align="left">Students assessor</td><td align="left">0</td><td align="left">5</td><td align="left">4</td><td align="left">3</td><td align="left">6</td><td align="left">8</td><td align="left">0</td></tr><tr><td align="left" /><td align="left">Other</td><td align="left">1</td><td align="left">0</td><td align="left">0</td><td align="left">0</td><td align="left">1</td><td align="left">0</td><td align="left">0</td></tr></tbody></table> </ephtml> </p> <hd id="AN0139230518-11">Data analysis</hd> <p>In order to develop a unified approach to planning, implementing and reporting the outcome of the focus group sessions across the seven countries, co‐authors developed the focus group guidance and a template to support cross‐national data analysis of qualitative data from the interviewees and quantitative data from questionnaires' respondents as well. From a pre‐pilot study at the beginning of the project the categories Technical, Organisational and Pedagogical emerged, comprising eight themes (system interface, TeSLA instruments, real‐time feedback, usability, accessibility, security and privacy, fraud detection and pedagogy and assessment) (Figure).</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/58I/01nov19/bjet12733-fig-0003.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="bjet12733-fig-0003.jpg" title="Categories for data analysis of e‐assessment with e‐authentication instruments" /> </p> <p></p> <p>A deductive approach was implemented for data analysis to identify pattern matching (Hyde, [<reflink idref="bib21" id="ref36">21</reflink>]). The research questions and conceptual categories were used to group the data and detect the main findings across the seven institutions. Questionnaires' results were analysed based on the number of respondents and presented as a percentage.</p> <hd id="AN0139230518-13">Findings</hd> <p></p> <hd id="AN0139230518-14">Pedagogical approaches for teaching staff interested in TeSLA (RQ1)</hd> <p>The underlying TeSLA educational assessment model describes an "assessment" as an assembly of one or more "activities" designed to measure "learning outcomes" with the aim to establish (collect evidence of) a person's competences (cognitive and non‐cognitive traits) at a particular moment in time. The response to an activity ("activity response") can be of various kinds (ie, the learner has to select, create or perform). An assessor applies "assessment criterion" defined in connection with an activity to the activity response of a learner or a group of learners. The response format comprises: mouse click, text, sound, image or programming code. The assessor determines the result (eg, mark and feedback).</p> <p>To answer the first question of this study, the various types of e‐assessment activities descriptions used in the pilot have been categorised according to this model. Based on the literature and the assessment scenarios of the pilot courses, Table presents a set of e‐assessment activities that can be used for e‐assessment. Activities are grouped according to type of response and Bloom revised taxonomy (Krathwohl, [<reflink idref="bib29" id="ref37">29</reflink>]). For each activity, the applicable instrument is indicated.</p> <p>Potential E‐assessment activities in conjunction with face recognition, voice recognition, keystroke dynamics, forensic analysis and plagiarism detection</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">TeSLA</th><th align="left">Bloom taxonomy</th><th align="left">Online activities</th><th align="left">FR</th><th align="left">VR</th><th align="left">KD</th><th align="left">FA</th><th align="left">PL</th></tr></thead><tbody><tr><td>Selection</td><td>Recall facts, concepts</td><td>Complete quiz‐based exams</td><td>✔</td><td /><td /><td /><td /></tr><tr><td>Complete questionnaires with multiple choice and students can use voice</td><td>✔</td><td>✔</td><td /><td /><td /></tr><tr><td align="left">Performance</td><td align="left">Understand, explain ideas or meanings</td><td align="left">Complete questionnaire with open questions (250 characters)</td><td align="left">✔</td><td align="left" /><td align="left">✔</td><td align="left">✔</td><td align="left">✔</td></tr><tr><td align="left">Participate in a discussion forum, blog notes, or learning diary</td><td align="left">✔</td><td align="left" /><td align="left">✔</td><td align="left">✔</td><td align="left">✔</td></tr><tr><td align="left">Participate in a web conference with a discussion chat</td><td align="left">✔</td><td align="left">✔</td><td align="left">✔</td><td align="left" /><td align="left" /></tr><tr><td align="left">Apply, use knowledge in new situations or for inquiry‐based learning or problem solving</td><td align="left">Give an oral presentation</td><td align="left">✔</td><td align="left">✔</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">Participate in a game or simulation task (with voice)</td><td align="left">✔</td><td align="left">✔</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">Take part in a role‐play (with voice) </td><td align="left">✔</td><td align="left">✔</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">Analyse (compare, connect ideas), evaluate (justify decisions)</td><td align="left">Carry out a practice in a laboratory (with voice explanation)</td><td align="left">✔</td><td align="left">✔</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">Execute data analysis using Excel or software (with voice explanation)</td><td align="left">✔</td><td align="left">✔</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">Solve mathematical problems (using text and/or voice explanation)</td><td align="left">✔</td><td align="left">✔</td><td align="left">✔</td><td align="left" /><td align="left" /></tr><tr><td align="left">Creation</td><td align="left">Create Produce original work</td><td align="left">Develop a multimedia presentation a multimedia presentation or video clip (with voice slides)</td><td align="left">✔</td><td align="left">✔</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">Elaborate a poster or infographic about a research topic (with voice)</td><td align="left">✔</td><td align="left">✔</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">Develop programming code</td><td align="left">✔</td><td align="left" /><td align="left">✔</td><td align="left" /><td align="left" /></tr><tr><td align="left">Elaborate academic paper (essay) or report about a research theme or final written assignment </td><td align="left">✔</td><td align="left" /><td align="left">✔</td><td align="left">✔</td><td align="left">✔</td></tr></tbody></table> </ephtml> </p> <p>Altogether a full range of pedagogical activities linked to Bloom's taxonomy could be tested with the TeSLA instruments but not all authentication tools could be applied to all types of activities. Careful selection was required. Table presents the characteristics of the e‐assessment activities in the courses.</p> <p>Characteristics of e‐assessment activities (N = 108)</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Characteristics</th><th align="left">Values</th><th align="left">%</th></tr></thead><tbody><tr><td align="left">Assessment purpose</td><td align="left">Formative</td><td align="left">62</td></tr><tr><td align="left" /><td align="left">Summative</td><td align="left">23</td></tr><tr><td align="left" /><td align="left">Diagnostic</td><td align="left">4</td></tr><tr><td align="left" /><td align="left">Other or missing</td><td align="left">10</td></tr><tr><td align="left">Location</td><td align="left">At home</td><td align="left">85</td></tr><tr><td align="left" /><td align="left">At university</td><td align="left">15</td></tr><tr><td align="left">Monitoring</td><td align="left">Supervised</td><td align="left">15</td></tr><tr><td align="left" /><td align="left">Unsupervised</td><td align="left">85</td></tr><tr><td align="left">Individual or collaborative work</td><td align="left">Individual</td><td align="left">93</td></tr><tr><td align="left" /><td align="left">Collaborative</td><td align="left">6</td></tr><tr><td align="left" /><td align="left">Individual and collaborative</td><td align="left">1</td></tr><tr><td align="left">Response type</td><td align="left">Select answer</td><td align="left">17</td></tr><tr><td align="left" /><td align="left">Create answer or product</td><td align="left">73</td></tr><tr><td align="left" /><td align="left">Perform/enact/demonstrate</td><td align="left">10</td></tr><tr><td align="left">Response format</td><td align="left">Mouse click</td><td align="left">14</td></tr><tr><td align="left" /><td align="left">Text</td><td align="left">61</td></tr><tr><td align="left" /><td align="left">Sound (oral input)</td><td align="left">8</td></tr><tr><td align="left" /><td align="left">Image (picture, video)</td><td align="left">17</td></tr><tr><td align="left" /><td align="left">Programming code</td><td align="left">0</td></tr></tbody></table> </ephtml> </p> <p>In general, the e‐assessment activities represent a balanced sample in terms of assessment purposes: formative (62%), summative (23%) and diagnostic (4%). The majority of activities have been designed to be conducted at home (85%), in an unsupervised way (85%) and individually (93%). The response type most used has been the creation of an answer or product (73%) in text format (61%).</p> <hd id="AN0139230518-15">TeSLA Accuracy in detecting plagiarism and cheating (RQ2)</hd> <p>A large amount of data from 4931 students (Table) was gathered during the pilot to examine the accuracy of the TeSLA instruments. The accuracy value was computed for each instrument as the fraction of samples classified correctly as "true positive" and "true negative" over the total samples used in the tests (including non‐processed samples). Findings (Table) show that some instruments, like PL (plagiarism), whose data (text documents) could be easily interpreted by computers, performed well. Instruments, that have to process more complex data, such as video and audio, showed a lower rate of accuracy. Therefore, teaching staff will need to invest more time into checking result of these instruments including the cases of false positive and false negatives.</p> <p>Accuracy of TeSLA system instruments based on the samples of seven institutions</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Instrument</th><th align="center">Licit sample correctly identified as licit (TP) (%)</th><th align="center">Fraudulent sample correctly identified as fraudulent (TN) (%)</th><th align="center">Fraudulent sample incorrectly identified as licit (FP) (%)</th><th align="center">Licit sample incorrectly identified as fraudulent (FN) (%)</th><th align="left">Accuracy</th></tr></thead><tbody><tr><td align="left">FR</td><td align="char" char=".">4.80</td><td align="char" char=".">47.83</td><td align="char" char=".">0.46</td><td align="char" char=".">44.27</td><td align="char" char=".">54</td></tr><tr><td align="left">PL</td><td align="char" char=".">50.00</td><td align="char" char=".">45.00</td><td align="char" char=".">5.00</td><td align="char" char=".">0.00</td><td align="char" char=".">95</td></tr><tr><td align="left">FA</td><td align="char" char=".">30.00</td><td align="char" char=".">50.00</td><td align="char" char=".">0.00</td><td align="char" char=".">20.00</td><td align="char" char=".">80</td></tr><tr><td align="left">KD</td><td align="char" char=".">40.00</td><td align="char" char=".">50.00</td><td align="char" char=".">0.00</td><td align="char" char=".">10.00</td><td align="char" char=".">90</td></tr><tr><td align="left">VR</td><td align="char" char=".">0.00</td><td align="char" char=".">48.61</td><td align="char" char=".">0.00</td><td align="char" char=".">50.00</td><td align="char" char=".">49</td></tr></tbody></table> </ephtml> </p> <p>Accuracy data were not yet available to teaching staff during the pilot. However, the latest version of TeSLA system indicates results as illicit, not sure or licit and offers an audit data report to assist teachers in interpreting the instrument results.</p> <hd id="AN0139230518-16">Teaching staff opinions about the use of TeSLA (RQ3)</hd> <p>Data from the teaching staff post‐questionnaire show that the majority of partner universities' staff considered e‐authentication relevant and agreed that "their university is working to ensure the quality of the assessment process" (AU, JYU, SU and TUS 100%, UOC 91% and OUUK 75%). They also "trust an assessment system in which all assessment occurs online" (JYU 100%, SU 83%, AU and OUUK 75%, UOC 69% and TUS 50%).</p> <p>In addition, respondents indicated that the main advantages of e‐assessment with e‐authentication based on educators' views were:</p> <p></p> <ulist> <item> "To save time in commuting" (all institutions: 50%–100%).</item> <p></p> <item> "To prove originality of the students' work" (SU: 83%, OUUK: 75%, JYU: 60%, AU: 50%, TUS: 50%).</item> <p></p> <item> "To avoid face‐to‐face assessment" (SU: 67%, UOC: 66%, AU: 50%).</item> <p></p> <item> "To better adapt assessment to students' needs" (OUUK: 75%, AU: 50%).</item> </ulist> <p>Based on the pre‐pilot questionnaire data, most of staff members from the universities confirmed that students occasionally cheat in their courses (OUUK: 75%, UOC: 56%, JYU: 56%, OUNL: 50% and SU: 50%). The majority of respondents agreed (more than 90% from all universities) that "It is cheating if a student copy/pastes information from a website in a work developed by him/her without citing the original source." However, these respondents mentioned that they were "not so sure about students who plagiarise when they work together and submit similar work" (SU: 50%, TUS: 43%, UOC: 26%). They also confirmed that "There are clear and specific rules to follow at their institution to deal with cheating" AU (99%), UOC (93%), JYU (89%), OUUK (75%), TUS (71%), SU (50%), OUNL (50%).</p> <p>However, a significant number of respondents (more than 50%) found that the key disadvantages of e‐assessment were:</p> <p></p> <ulist> <item> "Work overload in performing the assessment activities" (OUUK: 75%, SU: 67%, UOC: 44%).</item> <p></p> <item> "To spend time learning to use new technologies" (OUUK: 75%, JYU: 60%, AU: 50%).</item> <p></p> <item> "Time overload in assessing the activities," particularly (AU: 50%).</item> </ulist> <p>In addition, data about respondents' views also confirmed a few issues about security and privacy, such as students' resistance in sharing personal data. For example:</p> <p></p> <ulist> <item> "<emph>Some students were very critical about sharing this kind of personal data.</emph>" JYU</item> <p></p> <item> "<emph>Some students, who used face recognition tool, were asking about who will watch their videos and what the aim of it is.</emph>" SU</item> <p></p> <item> "<emph>Some students asked various questions about data privacy and security by email.</emph>" OUUK</item> </ulist> <hd id="AN0139230518-17">Teaching staff recommendations about the use of TeSLA (RQ3)</hd> <p>Table presents solutions or recommendations for each of the eight themes for technical needs, organisational views and pedagogical issues as voiced by interviewees during the focus group.</p> <p>Recommendations based on focus group results</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Categories</th><th align="left">Themes</th><th align="left">Solution</th><th align="left">Remark</th></tr></thead><tbody><tr><td align="left">Technical needs</td><td align="left">1. System Interface</td><td align="left">Guidance</td><td align="left">More guidance from the system to combine instruments and analyse outcomes</td></tr><tr><td align="left" /><td align="left">2. TeSLA Instruments</td><td align="left">Familiarity</td><td align="left">Knowledge about the instruments (including technical requirements) early to plan activities</td></tr><tr><td align="left" /><td align="left">3. Feedback</td><td align="left">Audit report</td><td align="left">Audit Reports that can be accessed quickly after the activity including results</td></tr><tr><td align="left" /><td align="left">4. Usability</td><td align="left">New devices</td><td align="left">More flexible options for mobile devices</td></tr><tr><td align="left">Organisational issues</td><td align="left">5. Accessibility</td><td align="left">Inclusive tasks</td><td align="left">Adapting different instruments and tasks according to students' special needs</td></tr><tr><td align="left" /><td align="left" /><td align="left">SEND assistance</td><td align="left">Some SEND students had difficulties with the system based on their disability</td></tr><tr><td align="left" /><td align="left">6. Security and privacy</td><td align="left">Flexibility of e‐assessment</td><td align="left">TeSLA might support the shift from human invigilated exams to system invigilated exams</td></tr><tr><td align="left" /><td align="left" /><td align="left">Privacy concerns</td><td align="left">Various students felt comfortable by sharing their data but some of them had concerns about their privacy and security</td></tr><tr><td align="left" /><td align="left">7. Fraud, prevention and trust</td><td align="left">Cheating prevention</td><td align="left">An opportunity to limit cheating in the university education</td></tr><tr><td align="left" /><td align="left" /><td align="left">Fair assessment and accreditation</td><td align="left">Convenience and "fair assessment and accreditation of students based on their merits"</td></tr><tr><td align="left" /><td align="left" /><td align="left">Reliable system</td><td align="left">Reliable system will increase the confidence of the institution</td></tr><tr><td align="left">Pedagogical aspects</td><td align="left">8. Assessment pedagogy</td><td align="left">Course design changes</td><td align="left">Change in course design required to use TeSLA effectively</td></tr><tr><td align="left" /><td align="left" /><td align="left">New types of e‐assessment</td><td align="left">Opportunities of "new kinds of assessments"</td></tr><tr><td align="left" /><td align="left" /><td align="left">Variety and regularity</td><td align="left">More open‐ended questions, and variety of types of assessment needed</td></tr></tbody></table> </ephtml> </p> <p>In terms of technical needs, most of interviewees considered TeSLA easy‐to‐use and user‐friendly system. However, they would like to obtain more guidance from the system to combine instruments and analyse outcomes in order to use the instruments more effectively.</p> <p></p> <ulist> <item> "Teachers need to get to know the instruments early enough to be able to plan useful and meaningful activities" (JYU).</item> <p></p> <item> It might be more "effective when at least two instruments are combined" (SU).</item> </ulist> <p>The familiarity with instrument and the technical requirements will help teaching staff to support their students.</p> <p></p> <ulist> <item> "Students who worked with older versions of Office2003 and used Internet Explorer browser had problems with TeSLA" (SU).</item> <p></p> <item> "We ask students to attach charts and images, I am not sure if the authorship can be checked in such cases" (UOC).</item> </ulist> <p>Interviewees mentioned that they would like also reports that can be accessed quickly after the activity is completed. They would like to be notified when:</p> <p></p> <ulist> <item> "Are several faces detected in the camera frame?" (TUS).</item> <p></p> <item> "Cases where the same person performs differently in the enrolment and in other activities" (UOC).</item> <p></p> <item> "What are the procedures when the system does not recognise a student?" (OUUK).</item> </ulist> <p>Interviewees also mentioned about mobile devices. Teaching staff will not recommend taking an assessment using mobile phones because the screen is very small. However, if TeSLA can work in mobile devices (such as tablets) they think that this would be very useful because <emph>"students use their voice, camera and fingerprint authentication frequently on their mobile devices</emph>" (OUUK).</p> <p>In terms of organisational views, some interviewees presented positive comments related to the system that enabled inclusive e‐assessment tasks for <emph>special educational needs and disable</emph> students (SEND).</p> <p></p> <ulist> <item> "SEND students felt a little bit intimidated showing their face and been recorded. However, this system will be very helpful for adapting the assessment to their needs" (UOC).</item> <p></p> <item> "Students can use other programs and browsers/tabs on their computer while completing the exam, their actions during the assessment are not observed" (SU).</item> </ulist> <p>The students also shared a few concerns related to accessibility. For example, e‐assessment with e‐authentication will require technical assistance and more alternatives for SEND as described below.</p> <p></p> <ulist> <item> "Low fine motor students had problems with keystroke dynamics" (OUUK).</item> <p></p> <item> "Students with dyslexia who stammer were not comfortable with voice recognition" (SU).</item> </ulist> <p>Interviewees also commented about security and privacy. TeSLA enables flexibility of e‐assessment "<emph>A person can shift comfortably from the human‐managed exams to the system‐managed exams</emph>" (AU). They mentioned that "<emph>most of the students did not point privacy issues as they participated in the pilot on a voluntary basi</emph>s" (UOC). However, some educators indicated that "<emph>There were some students who did not want to share their information</emph>" (AU). They also shared that "<emph>Students are nowadays more aware of the privacy and security issues than earlier, and that they can and will question these matters</emph>" (JYU).</p> <p></p> <ulist> <item> "Our students found positive to be informed about Data Protection including consent form, but some of them presented various questions, such as: what data is sent when TeSLA is active? How is the data sent? What protocol is used? Is the data transmission secure? Will they be forced to use this regardless of their data privacy?" (OUUK).</item> </ulist> <p>Their comments related to fraud, prevention and trust, revealed their views in terms of cheating prevention: They consider that "<emph>e‐authentication will increase the trustworthiness of e‐learning however fraud would still be possible</emph>" (SU), and fair assessment and accreditation: "<emph>teachers considered that the system will make students feel more secure for those students that do not cheat</emph>" (UOC). Interviewees perceived TeSLA as "<emph>reliable system that fits to contemporary living and recent trends in higher education.</emph>" (SU), which "<emph>will increase the confidence of the institution</emph>" (TUS). Teachers see that "<emph>there is a lot of possibilities with the use of TeSLA, especially if it provides reliable results</emph>" (OUNL). Interviewees also suggested to use different systems for plagiarism detection; "<emph>Combining the use of TeSLA</emph><emph>PL with other systems such as Turnitin</emph><emph>and CopyCatch</emph><emph>will help course teams identify external and intrinsic plagiarism</emph>"<emph> (OUUK).</emph></p> <p>In terms of pedagogical issues, interviewees mentioned that e‐authentication requires some changes about existent course design, for example, "<emph>more open‐ended questions OUNL</emph>" Interviewees considered that TeSLA system will increase the opportunities of "<emph>new kinds of assessments</emph>" (JYU). For that variety and regularity will be necessary, that means, "<emph>variety of types of assessment</emph>" (AU) and "regular assessment <emph>that creates extra chances to demonstrate what has been learned and done during the course</emph>"<emph> (OUNL).</emph></p> <hd id="AN0139230518-18">Discussion</hd> <p>The teaching staff used diverse pedagogical approaches with a range of scenarios and types of e‐assessment, which matched the authentication tools chosen and plagiarism detection (JISC, [<reflink idref="bib24" id="ref38">24</reflink>]; Kakkonen & Mozgovoy, [<reflink idref="bib25" id="ref39">25</reflink>]; Wulff, [<reflink idref="bib53" id="ref40">53</reflink>]). All instruments opened up new types of both formative and summative assessments (Trumbull & Lash, [<reflink idref="bib48" id="ref41">48</reflink>]; Whitelock, [<reflink idref="bib52" id="ref42">52</reflink>]; Okada <emph>et al</emph>., 2018). Nevertheless, the response type and response format constituted the basis for the selection of instruments. Depending on the response type of e‐assessment activity, some instruments were more suitable than others. The revised Bloom's taxonomy (Bloom <emph>et al</emph>., [<reflink idref="bib7" id="ref43">7</reflink>]) has assisted with the description and communication of adapted pedagogical activities for e‐assessment with e‐authentication.</p> <p>Partners applied a student‐centred pedagogical approach (Baneres, Baró, & Guerrero‐Roldán, [<reflink idref="bib5" id="ref44">5</reflink>]; ESG, [<reflink idref="bib15" id="ref45">15</reflink>]) for the e‐assessment activities using TeSLA instruments. The assessments were designed to allow the students to reveal their skills and understanding of the different knowledge domains to the best of their abilities. The teachers also wanted to increase the students' trust in the various authentication tools and the system as a whole. This is because the consent forms complied with EU regulations (EC, [<reflink idref="bib11" id="ref46">11</reflink>]; EMHE, [<reflink idref="bib14" id="ref47">14</reflink>]). These forms were communicated as simply as possible so that students could understand that their personal data was protected according to GDPR (Albrecht, [<reflink idref="bib1" id="ref48">1</reflink>]). Any information from their TeSLA usage would be applied within their own University's Policy & Procedures if cheating was detected.</p> <p>Pilot coordinators play an important role to communicate needs and technical issues to the technology developers and keep course teams and students—including special educational needs students—well informed and supported on security and privacy as well as on fraud detection (QAA, [<reflink idref="bib43" id="ref49">43</reflink>]), prevention and trust (Mellar, Peytcheva‐Forsyth, Kocdar, Karadeniz, & Yovkova, [<reflink idref="bib32" id="ref50">32</reflink>]; Okada <emph>et al</emph>., 2018; Park, [<reflink idref="bib40" id="ref51">40</reflink>]).</p> <p>Participants described a positive acceptance of TeSLA instruments and consider that it will increase students' awareness of cheating and plagiarism and the trustworthiness of e‐assessment, but it will not eliminate the possibility of fraud<emph>. </emph>They also provided some suggestions to support the trust in the system and instruments including ways to face problems in using these forms of technological assistance. Seven recommendations were formulated to support teaching staff with the TeSLA system:</p> <hd id="AN0139230518-19">Technical needs</hd> <p></p> <ulist> <item> <bold> Technical FAQ </bold> that provides information about the system will be useful for teaching staff to avoid or deal with problems faced by them or their students.</item> <p></p> <item> <bold> Prompt _B_audit report</bold> about the results of each instrument with guidelines for the university staff will be essential for them to check and interpret results and ensure the quality of the e‐assessment.</item> </ulist> <hd id="AN0139230518-20">Organisational issues</hd> <p></p> <ulist> <item> <bold> 3 _B_Data security and privacy awareness</bold> for teaching staff and students will be useful through more information about the e‐authentication and authorship verification instruments to promote trust in the system.</item> <p></p> <item> <bold> 4 Universities must develop and share _B_e‐authentication and authorship verification policies</bold> for verifying and dealing with fraud. This should include legal and ethical recommendations to deal with problems and measures to prevent academic malpractices and for quality assurance.</item> <p></p> <item> <bold> 5 Local guidelines such as a _B_manual about the instruments</bold> with instructions should include information on data protection and privacy; and fraud detection and prevention.</item> </ulist> <hd id="AN0139230518-21">Pedagogical aspects</hd> <p></p> <ulist> <item> <bold> 6 Pedagogical teams must discuss and share _B_best practices of e‐assessment</bold> that can be combined with TeSLA instruments.</item> <p></p> <item> <bold> 7 Universities should offer _B_course team support</bold> for academic staff to plan useful activities and e‐assessment tasks with TeSLA system.</item> </ulist> <hd id="AN0139230518-22">Final remarks</hd> <p>The e‐assessment activities were designed to fit the TeSLA instruments, but like any pilot study technical difficulties arose. The RRI approach (EC, [<reflink idref="bib12" id="ref52">12</reflink>]; Von Schomberg, [<reflink idref="bib50" id="ref53">50</reflink>]) enabled the interaction between technology innovators with teaching staff and their views about their students' opinions to better align the TeSLA system development with the priorities and concerns of the end‐users (Okada <emph>et al</emph>., 2018). Issues such as the improvement of the instruments' reliability and its accuracy must be dealt with for the next round of testing. Additionally, views and concerns of teaching staff about the prompt reporting of outcomes and the instruments prevention audio and video attack (VPAD and FPAD) indicate the need for more assistance in determining student identity. This is addressed in the final version of TeSLA. Furthermore, a study with other plagiarism detection systems, such as Turnitin Urkund, SafeAssign, CopyCatch and TeSLA was implemented (Edwards <emph>et al</emph>., [<reflink idref="bib13" id="ref54">13</reflink>]). This study revealed that these systems can be used by the same institution for complimentary checks. Different ways of checking plagiarism will be helpful for teaching staff to ensure trust in e‐assessment.</p> <p>Good academic practice in (online) teaching should recognise academic malpractice such as external and intrinsic plagiarism (Appelgren Heyman <emph>et al</emph>., [<reflink idref="bib3" id="ref55">3</reflink>]; Ferrell, [<reflink idref="bib16" id="ref56">16</reflink>]; Potthast, Eiselt, Barrón Cedeño, Stein, & Rosso, [<reflink idref="bib42" id="ref57">42</reflink>]) including impersonation and presentation attack (Nikisins, Mohammadi, Anjos, & Marcel, [<reflink idref="bib33" id="ref58">33</reflink>]), which has become more common. All these forms of cheating and plagiarism should be addressed by academic staff (Ivanova <emph>et al</emph>., [<reflink idref="bib23" id="ref59">23</reflink>]; Kambourakis & Damopoulos [<reflink idref="bib26" id="ref60">26</reflink>]; Pell, [<reflink idref="bib41" id="ref61">41</reflink>]) with pedagogical strategies for e‐assessment supported by the effective use of technologies, such as the TesLA system.</p> <hd id="AN0139230518-23">Acknowledgements</hd> <p>We are grateful to the <emph>TeSLA</emph> consortium. This work is supported by the European Commission (H2020‐ICT‐2015/H2020‐ICT‐2015), Number 688520.</p> <hd id="AN0139230518-24">Statements on open data, ethics and conflict of interest</hd> <p>According to the consent forms presented to and agreed by participants, data can't be made available to third parties.</p> <p>Ethical approvals were gained from the hosting institution.</p> <p>No conflict of interest has been declared.</p> <ref id="AN0139230518-25"> <title> References </title> <blist> <bibl id="bib1" idref="ref30" type="bt">1</bibl> <bibtext> Albrecht, J. P. (2016). How the GDPR will change the world. European Data Protection Law Review, 2, 287 – 289.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref13" type="bt">2</bibl> <bibtext> Apampa, K., Wills, G., & Argles, D. (2010). User security issues in summative e‐assessment. International Journal of Digital Society, 1 (2), 135 – 147.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref32" type="bt">3</bibl> <bibtext> Appelgren Heyman, F., Olofsson, M., Hansson, H., Moberg, J., & Olsson, U. (2012). Can we rely on text originality check systems? Evaluation of three systems used in Higher Education and suggestion of a new methodological test approach. Paper presented at the proceedings of the 5th International Plagiarism Conference, Sage Gateshead, Newcastle upon Tyne, United Kingdom.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref11" type="bt">4</bibl> <bibtext> Ardid, M., Gomez‐Tejedor, J., Meseguer‐Duenas, J., Riera, J., & Vidaurre, A. (2015). Online exams for blended assessment. Study of different application methodologies. Computers & Education, 81, 296 – 303. https://doi.org/10.1016/j.compedu.2014.10.010</bibtext> </blist> <blist> <bibl id="bib5" idref="ref34" type="bt">5</bibl> <bibtext> Baneres, D., Baró, X., Guerrero-Roldán, A. E., & Rodriguez, M. E. (2016). Adaptive e-assessment system: A general approach. International Journal of Emerging Technologies in Learning (iJET), 11 (7), 16 – 23.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref9" type="bt">6</bibl> <bibtext> Bermingham, V., Watson, S., & Jones, M. (2010). Plagiarism in UK law schools: Is there a postcode lottery? Assessment & Evaluation in HE, 35 (1), 1 – 14.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref22" type="bt">7</bibl> <bibtext> Bloom, B. S. (Ed.). Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives, handbook I: The cognitive domain. New York : David McKay Co Inc.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref23" type="bt">8</bibl> <bibtext> Carrington, A. (2016). The pedagogy wheel V5.0. – It's not about the apps, it's about the pedagogy. Retrieved from https://designingoutcomes.com/assets/PadWheelV5/PW_ENG_V5.0_Android_SCREEN.pdf</bibtext> </blist> <blist> <bibl id="bib9" idref="ref12" type="bt">9</bibl> <bibtext> Chew, E., Ding, S. L., & Rowell, G. (2015). Changing attitudes in learning and assessment: Cast‐off 'plagiarism detection' and cast‐on self‐service assessment for learning. Innovations in Education and Teaching International, 52 (5), 454 – 463.</bibtext> </blist> <blist> <bibtext> Churches, A. (2008). Bloom's digital taxonomy. Retrieved from <ulink href="http://edorigami.wikispaces.com/file/view/Bloom%27s+quicksheets.pdf/296456574/Bloom%27s%20quicksheets.pdf">http://edorigami.wikispaces.com/file/view/Bloom%27s+quicksheets.pdf/296456574/Bloom%27s%20quicksheets.pdf</ulink></bibtext> </blist> <blist> <bibtext> EC. (2012). Assessment of key competences in initial education and training: Policy guidance. Commission staff working document. European Commission. Retrieved from <ulink href="http://eose.org/wp-content/uploads/2014/03/Assessment-of-Key-Competences-ininitial-education-and-training.pdf">http://eose.org/wp-content/uploads/2014/03/Assessment-of-Key-Competences-ininitial-education-and-training.pdf</ulink></bibtext> </blist> <blist> <bibtext> EC. (2017). Responsible research and innovation. European Commission. Retrieved from https://ec.europa.eu/pro gram mes/horizon2020/en/h2020-section/responsible-research-innovation</bibtext> </blist> <blist> <bibtext> Edwards, C., Whitelock, D., Brouns, F., Rodríguez, M. E., Okada, A., Baneres, D., & Holmes, W. (2018). An embedded approach to plagiarism detection using the TeSLA e‐authentication system. In: The 2018 International Technology Enhanced Assessment Conference (TEA 2018, 10–11 Dec 2018, Amsterdam, Netherlands)</bibtext> </blist> <blist> <bibtext> EMHE. (2012). Making the most of our potential: Consolidating the European Higher Education area. Retrieved from https://media.ehea.info/file/2012_Bucharest/67/3/Bucharest_Communique_2012_610673.pdf</bibtext> </blist> <blist> <bibtext> ESG. (2015). Standards and guidelines for quality assurance in the European Higher Education area Brussels, Belgium. Retrieved from <ulink href="http://www.enqa.eu/wp-content/uploads/2015/11/ESG%5f2015.pdf">http://www.enqa.eu/wp-content/uploads/2015/11/ESG%5f2015.pdf</ulink></bibtext> </blist> <blist> <bibtext> Ferrell, G. (2014). Electronic management of assessment (EMA): A landscape review. Retrieved from <ulink href="http://repository.jisc.ac.uk/5599/1/EMA%5fREPORT.pdf">http://repository.jisc.ac.uk/5599/1/EMA%5fREPORT.pdf</ulink></bibtext> </blist> <blist> <bibtext> Gao, Q. (2012). Biometric authentication to prevent e‐cheating. Instructional Technology, 9 (2), 3 – 13.</bibtext> </blist> <blist> <bibtext> Guàrdia, L., Crisp, G., & Alsina, I. (2017). Trends and challenges of e‐assessment to enhance student learning in Higher Education. In E. Cano & G. Ion (Eds.), Innovative practices for higher education assessment and measurement (pp. 36 – 56). Hershey, PA : IGI Global.</bibtext> </blist> <blist> <bibtext> Guerrero‐Roldán, A.‐E., & Noguera, I. (2018). A model for aligning assessment with competences and learning activities in online courses. The Internet and Higher Education, 38, 36 – 46. https://doi.org/10.1016/j.iheduc.2018.04.005</bibtext> </blist> <blist> <bibtext> Harmon, O., Lambrinos, J., & Buffolino, J. (2010). Assessment design and cheating risk in online instruction. Online Journal of Distance Learning Administration, 13 (3). Retrieved from https://<ulink href="http://www.learntechlib.org/p/52616/">www.learntechlib.org/p/52616/</ulink></bibtext> </blist> <blist> <bibtext> Hyde, K. F. (2000). Recognising deductive processes in qualitative research. Qualitative Market Research: An International Journal, 3 (2), 82 – 90.</bibtext> </blist> <blist> <bibtext> IPPHEAE. (2013). Impact of policies for plagiarism in Higher Education. Retrieved from <ulink href="http://plagiarism.cz/ippheae/">http://plagiarism.cz/ippheae/</ulink></bibtext> </blist> <blist> <bibtext> Ivanova, M., Rozeva, A., & Durcheva, M. (2016). Towards e‐assessment models in engineering education: Problems and solutions. Advances in Web‐based Learning ICWL, 10013, 178 – 181. Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-47440-3%5f20. https://doi.org/10.1007/978-3-319-47440-3_20</bibtext> </blist> <blist> <bibtext> JISC. (2010). Effective assessment in a digital age. Retrieved from https://<ulink href="http://www.dkit.ie/system/files/JISC%20Effective%20Assessment%20in%20a%20digital%20age.pdf">www.dkit.ie/system/files/JISC%20Effective%20Assessment%20in%20a%20digital%20age.pdf</ulink></bibtext> </blist> <blist> <bibtext> Kakkonen, T., & Mozgovoy, M. (2008). An evaluation of web plagiarism detection systems for student essays. In Proceedings of the 16th International Conference on Computers in Education. Retrieved from https://<ulink href="http://www.researchgate.net/publication/220017622%5fAn%5fEvaluation%5fof%5fWeb%5fPlagiarism%5fDetection%5fSystems%5ffor%5fStudent%5fEssays">www.researchgate.net/publication/220017622%5fAn%5fEvaluation%5fof%5fWeb%5fPlagiarism%5fDetection%5fSystems%5ffor%5fStudent%5fEssays</ulink></bibtext> </blist> <blist> <bibtext> Kambourakis, G., & Damopoulos, D. (2013). A competent post‐authentication and non‐repudiation biometric‐based scheme for m‐learning. Proceedings of the 10th IASTED International Conference on Web‐based Education, 821 – 827. Elsevier, Innsbruck, Austria.</bibtext> </blist> <blist> <bibtext> Karim, N. A., & Shukur, Z. (2016). Proposed features of an online examination interface design and its optimal values. Computers in Human Behavior, 64 (1), 414 – 422.</bibtext> </blist> <blist> <bibtext> Knuth, M. (2016). D5.3 – Instruments technical description and development scheduling. TeSLA adaptive trust‐based e‐assessment. Retrieved from <ulink href="http://tesla-project.eu/wp-content/uploads/2017/06/D5.3-Instruments-technical-description-and-developmenet-scheduling.pdf">http://tesla-project.eu/wp-content/uploads/2017/06/D5.3-Instruments-technical-description-and-developmenet-scheduling.pdf</ulink></bibtext> </blist> <blist> <bibtext> Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory into practice, 41 (4), 212 – 218.</bibtext> </blist> <blist> <bibtext> Law, P. (2016). Digital badging at The Open University: Recognition for informal learning. Open Learning: The Journal of Open, Distance and e‐Learning, 30 (3), 221 – 234.</bibtext> </blist> <blist> <bibtext> Manoharan, S. (2016). Personalized assessment as a means to mitigate plagiarism. IEEE Transactions on Education, 60 (2), 112 – 119. https://doi.org/10.1109/TE.2016.2604210</bibtext> </blist> <blist> <bibtext> Mellar, H., Peytcheva‐Forsyth, R., Kocdar, S., Karadeniz, A., & Yovkova, S. (2018). Addressing cheating in e‐assessment using student authentication and authorship checking systems: Teachers' perspectives. International Journal for Educational Integrity, 14 (2), 1 – 21. https://doi.org/10.1007/s40979-018-0025-x</bibtext> </blist> <blist> <bibtext> Nikisins, O., Mohammadi, A., Anjos, A., & Marcel, S. (2018). On effectiveness of anomaly detection approaches against unseen presentation attacks in face anti‐spoofing. Paper presented at the proceedings of the11th IAPR International Conference on Biometrics (ICB). Retrieved from https://ieeexp lor e.ieee.org/document/8411206. https://doi.org/10.1109/ICB2018.2018.00022</bibtext> </blist> <blist> <bibtext> Noguera, I., Guerrero‐Roldán, A. E., & Rodríguez, M. E. (2017) Assuring authorship and authentication across the e‐assessment process. In D. Joosten‐ten Brinke & M. Laanpere (Eds.), Technology enhanced assessment. Communications in computer and information science (pp. 86 – 92). Cham : Springer International Publishing. https://doi.org/10.1007/978-3-319-57744-9_8</bibtext> </blist> <blist> <bibtext> Okada, A., Mendonca, M., & Scott, P. (2015). Effective web videoconferencing for proctoring online oral exams: A case study at scale in Brazil. Open Praxis Journal of OECD, 7 (3), 227 – 242. Retrieved from https://openpraxis.org/index.php/OpenPraxis/article/view/215</bibtext> </blist> <blist> <bibtext> Okada, A., Rocha, K., Fuchter, S., Zucchi, S., & Wortley, D. (2018). Formative assessment of inquiry skills for responsible research and innovation using 3D virtual reality glasses and face recognition. In The 2018 International Technology Enhanced Assessment Conference (TEA 2018, 10–11 Dec 2018, Amsterdam, Netherlands).</bibtext> </blist> <blist> <bibtext> Okada, A., Whitelock, D., Holmes, W., & Edwards, C. (2018). e‐Authentication for online assessment: A mixed‐method study. British Journal of Educational Technology. Retried from https://onlinelibrary.wiley.com/doi/full/10.1111/bjet.12608. https://doi.org/10.1111/bjet.12608</bibtext> </blist> <blist> <bibtext> Osman, A. H., Salim, N., & Abuobieda, A. (2012). Survey of text plagiarism detection. Computer Engineering and Applications Journal, 1 (1), 37 – 45.</bibtext> </blist> <blist> <bibtext> Owen, R. (2015). Responsible research and innovation: Options for research and innovation policy in the EU. Retrieved from https://ec.europa.eu/research/innovation-union/pdf/expert-groups/Responsible_Research_and_Innovation.pdf</bibtext> </blist> <blist> <bibtext> Park, C. (2004). Rebels without a cause: Towards an institutional framework for dealing with student plagiarism. Journal of Further and Higher Education, 28 (3), 291 – 306.</bibtext> </blist> <blist> <bibtext> Pell, D. (2018). That's cheating: The (Online) academic cheating 'Epidemic' and what we should do about it. In J. Baxter, G. Callaghan, & J. McAvoy (Eds.), Creativity and critique in online learning – Exploring and examining innovations in online pedagogy (pp. 123–147). London : Palgrave Macmillan Ltd.</bibtext> </blist> <blist> <bibtext> Potthast, M., Eiselt, A., Barrón Cedeño, L. A., Stein, B., & Rosso, P. (2011). Overview of the 3rd international competition on plagiarism detection. In CEUR workshop proceedings. 1177. <ulink href="http://hdl.handle.net/10251/46639">http://hdl.handle.net/10251/46639</ulink></bibtext> </blist> <blist> <bibtext> QAA. (2016). Plagiarism in Higher Education – Custom essay writing services: An exploration and next steps for the UK Higher Education sector. Retrieved from <ulink href="http://dera.ioe.ac.uk/27679/">http://dera.ioe.ac.uk/27679/</ulink></bibtext> </blist> <blist> <bibtext> Schrock, K. (2013). Resources to support the SAMR Model. Retrieved from Kathy Schrock's guide to everything, <ulink href="http://www.schrockguide.net/samr.html">http://www.schrockguide.net/samr.html</ulink></bibtext> </blist> <blist> <bibtext> Simon, B., Sheard, J., Carbone, A., & Johnson, C. (2013). Academic integrity: differences between computing assessments and essays. Proceedings of the 13th Koli Calling International Conference on Computing Education Research, 23 – 32, ACM. where</bibtext> </blist> <blist> <bibtext> Stödberg, U. (2012). A research review of e‐assessment. Assessment & Evaluation in Higher Education, 37 (5), 591 – 604. https://doi.org/10.1080/02602938.2011.557496</bibtext> </blist> <blist> <bibtext> Timmis, S., Broadfoot, P., Sutherland, R., & Oldfield, A. (2016). Rethinking assessment in a digital age: Opportunities, challenges and risks. British Educational Research Journal, 42 (3), 454 – 476.</bibtext> </blist> <blist> <bibtext> Trumbull, E., & Lash, A. (2013). Understanding formative assessment: Insights from learning theory and measurement theory. San Francisco : WestEd. Retrieved from https://<ulink href="http://www.wested.org/online%5fpubs/resource1307.pdf">www.wested.org/online%5fpubs/resource1307.pdf</ulink></bibtext> </blist> <blist> <bibtext> Usoof, H., Hudson, B., & Lindgren, E. (2014). Plagiarism: Catalysts and not so simple solutions. In K. Sullivan, P. Czigler, & J. Sullivan Hellgren (Eds.), Cases on professional distance education degree programs and practices: Successes, challenges and issues. Advances in mobile and distance learning (AMDL) (pp. 49 – 85). Hershey PA, USA : IGI Global.</bibtext> </blist> <blist> <bibtext> Von Schomberg, R. (2011). Prospects for technology assessment in a framework of responsible research and innovation' (pp. 39 – 61). Springer. Retrieved from https://link.springer.com/chapter/10.1007/978-3-531-93468-6_2</bibtext> </blist> <blist> <bibtext> Watson, G., & Sottile, J. (2010). Cheating in the digital age: Do students cheat more in online courses? Online Journal of Distance Learning Administration, 13 (1). Retrieved from https://<ulink href="http://www.researchgate.net/publication/234703387%5fCheating%5fin%5fthe%5fDigital%5fAge%5fDo%5fStudents%5fCheat%5fMore%5fin%5fOnline%5fCourses">www.researchgate.net/publication/234703387%5fCheating%5fin%5fthe%5fDigital%5fAge%5fDo%5fStudents%5fCheat%5fMore%5fin%5fOnline%5fCourses</ulink></bibtext> </blist> <blist> <bibtext> Whitelock, D. (2011). Activating assessment for learning: Are we on the way with Web 2.0? In M. J. W. Lee & C. McLough (Eds.), Web 2.0‐based‐e‐learning: Applying social informatics for tertiary teaching (pp. 319 – 342). Hershey, PA : IGI Global.</bibtext> </blist> <blist> <bibtext> Wulff, D. W. (2008). On the Utility of Plagiarism Detection Software. Paper presented at the proceedings of the 3rd International Plagiarism Conference in Newcastle, England. Retrieved from <ulink href="http://plagiat.htw-berlin.de/softwareutility/">http://plagiat.htw-berlin.de/softwareutility/</ulink></bibtext> </blist> </ref> <aug> <p>By Alexandra Okada; Ingrid Noguera; Lyubka Alexieva; Anna Rozeva; Serpil Kocdar; Francis Brouns; Tarja Ladonlahti; Denise Whitelock and Ana‐Elena Guerrero‐Roldán</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author; Author; Author</p> <p></p> <p>Alexandra Okada is a research fellow on technology‐enhanced learning in the IET Open University (OUUK) and an honorary lecturer at the Open University Brazil.</p> <p>Ingrid Noguera is a research fellow on e‐learning in the Open University of Catalonia (UOC). She has also been an associate professor in Educational Sciences at the University of Barcelona (UB).</p> <p>Lyubka Alexieva is an assistant professor of ICT in Education and Mathematics Education in Sofia University (SU)</p> <p>Anna Rozeva is an associate professor in Mathematics and Informatics in the Technical University of Sofia</p> <p>Serpil Kocdar is an assistant Professor at the College of Open Education of Anadolu University (AU)</p> <p>Francis Brouns is an assistant professor at the Welten Institute, Research centre for learning, teaching and technology of the Open Universiteit of the Netherlands</p> <p>Tarja Ladonlahti is a lecturer in Special Education in the University of Jyväskylä (JYU)</p> <p>Denise Whitelock is a professor of Technology Enhanced Assessment in the IET (OUUK)</p> <p>Ana‐Elena Guerrero‐Roldán is an associate professor at Educational Science department at Open University of Catalonia (UOC)</p> </aug> <nolink nlid="nl1" bibid="bib34" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib35" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib37" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib20" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib38" firstref="ref5"></nolink> <nolink nlid="nl6" bibid="bib45" firstref="ref6"></nolink> <nolink nlid="nl7" bibid="bib51" firstref="ref7"></nolink> <nolink nlid="nl8" bibid="bib22" firstref="ref8"></nolink> <nolink nlid="nl9" bibid="bib49" firstref="ref10"></nolink> <nolink nlid="nl10" bibid="bib17" firstref="ref14"></nolink> <nolink nlid="nl11" bibid="bib27" firstref="ref15"></nolink> <nolink nlid="nl12" bibid="bib28" firstref="ref16"></nolink> <nolink nlid="nl13" bibid="bib12" firstref="ref17"></nolink> <nolink nlid="nl14" bibid="bib39" firstref="ref18"></nolink> <nolink nlid="nl15" bibid="bib50" firstref="ref19"></nolink> <nolink nlid="nl16" bibid="bib47" firstref="ref20"></nolink> <nolink nlid="nl17" bibid="bib30" firstref="ref21"></nolink> <nolink nlid="nl18" bibid="bib10" firstref="ref24"></nolink> <nolink nlid="nl19" bibid="bib44" firstref="ref25"></nolink> <nolink nlid="nl20" bibid="bib18" firstref="ref26"></nolink> <nolink nlid="nl21" bibid="bib31" firstref="ref27"></nolink> <nolink nlid="nl22" bibid="bib46" firstref="ref28"></nolink> <nolink nlid="nl23" bibid="bib48" firstref="ref29"></nolink> <nolink nlid="nl24" bibid="bib19" firstref="ref35"></nolink> <nolink nlid="nl25" bibid="bib21" firstref="ref36"></nolink> <nolink nlid="nl26" bibid="bib29" firstref="ref37"></nolink> <nolink nlid="nl27" bibid="bib24" firstref="ref38"></nolink> <nolink nlid="nl28" bibid="bib25" firstref="ref39"></nolink> <nolink nlid="nl29" bibid="bib53" firstref="ref40"></nolink> <nolink nlid="nl30" bibid="bib52" firstref="ref42"></nolink> <nolink nlid="nl31" bibid="bib15" firstref="ref45"></nolink> <nolink nlid="nl32" bibid="bib11" firstref="ref46"></nolink> <nolink nlid="nl33" bibid="bib14" firstref="ref47"></nolink> <nolink nlid="nl34" bibid="bib43" firstref="ref49"></nolink> <nolink nlid="nl35" bibid="bib32" firstref="ref50"></nolink> <nolink nlid="nl36" bibid="bib40" firstref="ref51"></nolink> <nolink nlid="nl37" bibid="bib13" firstref="ref54"></nolink> <nolink nlid="nl38" bibid="bib16" firstref="ref56"></nolink> <nolink nlid="nl39" bibid="bib42" firstref="ref57"></nolink> <nolink nlid="nl40" bibid="bib33" firstref="ref58"></nolink> <nolink nlid="nl41" bibid="bib23" firstref="ref59"></nolink> <nolink nlid="nl42" bibid="bib26" firstref="ref60"></nolink> <nolink nlid="nl43" bibid="bib41" firstref="ref61"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1232103
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Pedagogical Approaches for E-Assessment with Authentication and Authorship Verification in Higher Education
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Okada%2C+Alexandra%22">Okada, Alexandra</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-1572-5605">0000-0003-1572-5605</externalLink>)<br /><searchLink fieldCode="AR" term="%22Noguera%2C+Ingrid%22">Noguera, Ingrid</searchLink><br /><searchLink fieldCode="AR" term="%22Alexieva%2C+Lyubka%22">Alexieva, Lyubka</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-7877-5792">0000-0002-7877-5792</externalLink>)<br /><searchLink fieldCode="AR" term="%22Rozeva%2C+Anna%22">Rozeva, Anna</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-6069-5826">0000-0001-6069-5826</externalLink>)<br /><searchLink fieldCode="AR" term="%22Kocdar%2C+Serpil%22">Kocdar, Serpil</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-9099-6312">0000-0001-9099-6312</externalLink>)<br /><searchLink fieldCode="AR" term="%22Brouns%2C+Francis%22">Brouns, Francis</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-6240-2684">0000-0002-6240-2684</externalLink>)<br /><searchLink fieldCode="AR" term="%22Ladonlahti%2C+Tarja%22">Ladonlahti, Tarja</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-4315-1947">0000-0002-4315-1947</externalLink>)<br /><searchLink fieldCode="AR" term="%22Whitelock%2C+Denise%22">Whitelock, Denise</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-9897-4975">0000-0002-9897-4975</externalLink>)<br /><searchLink fieldCode="AR" term="%22Guerrero-Roldán%2C+Ana-Elena%22">Guerrero-Roldán, Ana-Elena</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-7073-7233">0000-0001-7073-7233</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22British+Journal+of+Educational+Technology%22"><i>British Journal of Educational Technology</i></searchLink>. Nov 2019 50(6):3264-3282.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 19
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2019
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Audits+%28Verification%29%22">Audits (Verification)</searchLink><br /><searchLink fieldCode="DE" term="%22Plagiarism%22">Plagiarism</searchLink><br /><searchLink fieldCode="DE" term="%22Cheating%22">Cheating</searchLink><br /><searchLink fieldCode="DE" term="%22Internet%22">Internet</searchLink><br /><searchLink fieldCode="DE" term="%22Electronic+Learning%22">Electronic Learning</searchLink><br /><searchLink fieldCode="DE" term="%22College+Faculty%22">College Faculty</searchLink><br /><searchLink fieldCode="DE" term="%22Trust+%28Psychology%29%22">Trust (Psychology)</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Information+Security%22">Information Security</searchLink><br /><searchLink fieldCode="DE" term="%22Privacy%22">Privacy</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Attitudes%22">Teacher Attitudes</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22United+Kingdom%22">United Kingdom</searchLink><br /><searchLink fieldCode="DE" term="%22Spain%22">Spain</searchLink><br /><searchLink fieldCode="DE" term="%22Netherlands%22">Netherlands</searchLink><br /><searchLink fieldCode="DE" term="%22Bulgaria%22">Bulgaria</searchLink><br /><searchLink fieldCode="DE" term="%22Finland%22">Finland</searchLink><br /><searchLink fieldCode="DE" term="%22Turkey%22">Turkey</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/bjet.12733
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0007-1013
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Checking the identity of students and authorship of their online submissions is a major concern in Higher Education due to the increasing amount of plagiarism and cheating using the Internet. The literature on the effects of e-authentication systems for teaching staff is very limited because it is a novel procedure for them. A considerable gap is to understand teaching staff' views regarding the use of e-authentication instruments and how they impact trust in e-assessment. This mixed-method study examines the concerns and practices of 108 teaching staff who used the TeSLA--Adaptive Trust-based e-Assessment System in six countries: the UK, Spain, the Netherlands, Bulgaria, Finland and Turkey. The findings revealed some technological, organisational and pedagogical issues related to accessibility, security, privacy and e-assessment design and feedback. Recommendations are to provide a FAQ and an audit report with results, to raise awareness about data security and privacy, to develop policies and guidelines about fraud detection and prevention, e-assessment best practices and course team support.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2019
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1232103
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1232103
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/bjet.12733
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 19
        StartPage: 3264
    Subjects:
      – SubjectFull: Audits (Verification)
        Type: general
      – SubjectFull: Plagiarism
        Type: general
      – SubjectFull: Cheating
        Type: general
      – SubjectFull: Internet
        Type: general
      – SubjectFull: Electronic Learning
        Type: general
      – SubjectFull: College Faculty
        Type: general
      – SubjectFull: Trust (Psychology)
        Type: general
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Information Security
        Type: general
      – SubjectFull: Privacy
        Type: general
      – SubjectFull: Teacher Attitudes
        Type: general
      – SubjectFull: United Kingdom
        Type: general
      – SubjectFull: Spain
        Type: general
      – SubjectFull: Netherlands
        Type: general
      – SubjectFull: Bulgaria
        Type: general
      – SubjectFull: Finland
        Type: general
      – SubjectFull: Turkey
        Type: general
    Titles:
      – TitleFull: Pedagogical Approaches for E-Assessment with Authentication and Authorship Verification in Higher Education
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Okada, Alexandra
      – PersonEntity:
          Name:
            NameFull: Noguera, Ingrid
      – PersonEntity:
          Name:
            NameFull: Alexieva, Lyubka
      – PersonEntity:
          Name:
            NameFull: Rozeva, Anna
      – PersonEntity:
          Name:
            NameFull: Kocdar, Serpil
      – PersonEntity:
          Name:
            NameFull: Brouns, Francis
      – PersonEntity:
          Name:
            NameFull: Ladonlahti, Tarja
      – PersonEntity:
          Name:
            NameFull: Whitelock, Denise
      – PersonEntity:
          Name:
            NameFull: Guerrero-Roldán, Ana-Elena
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 11
              Type: published
              Y: 2019
          Identifiers:
            – Type: issn-print
              Value: 0007-1013
          Numbering:
            – Type: volume
              Value: 50
            – Type: issue
              Value: 6
          Titles:
            – TitleFull: British Journal of Educational Technology
              Type: main
ResultId 1