Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100.
Saved in:
| Title: | Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100. |
|---|---|
| Authors: | Patterson, Fiona1,2 (AUTHOR) f.patterson@workpsychologygroup.com, Zibarras, Lara3 (AUTHOR), Ashworth, Vicki1 (AUTHOR) |
| Source: | Medical Teacher. Jan2016, Vol. 38 Issue 1, p3-17. 15p. |
| Subject Terms: | *Professional education, *Adults, *Research, *Theory, Judgment (Psychology), Medical practice, Study & teaching of medicine |
| Abstract: | Why use SJTs?Traditionally, selection into medical education professions has focused primarily upon academic ability alone. This approach has been questioned more recently, as although academic attainment predicts performance early in training, research shows it has less predictive power for demonstrating competence in postgraduate clinical practice. Such evidence, coupled with an increasing focus on individuals working in healthcare roles displaying the core values of compassionate care, benevolence and respect, illustrates that individuals should be selected on attributes other than academic ability alone. Moreover, there are mounting calls to widen access to medicine, to ensure that selection methods do not unfairly disadvantage individuals from specific groups (e.g. regarding ethnicity or socio-economic status), so that the future workforce adequately represents society as a whole. These drivers necessitate a method of assessment that allows individuals to be selected on important non-academic attributes that are desirable in healthcare professionals, in a fair, reliable and valid way. What are SJTs?Situational judgement tests (SJTs) are tests used to assess individuals’ reactions to a number of hypothetical role-relevant scenarios, which reflect situations candidates are likely to encounter in the target role. These scenarios are based on a detailed analysis of the role and should be developed in collaboration with subject matter experts, in order to accurately assess the key attributes that are associated with competent performance. From a theoretical perspective, SJTs are believed to measure prosocial Implicit Trait Policies (ITPs), which are shaped by socialisation processes that teach the utility of expressing certain traits in different settings such as agreeable expressions (e.g. helping others in need), or disagreeable actions (e.g. advancing ones own interest at others, expense). Are SJTs reliable, valid and fair?Several studies, including good quality meta-analytic and longitudinal research, consistently show that SJTs used in many different occupational groups are reliable and valid. Although there is over 40 years of research evidence available on SJTs, it is only within the past 10 years that SJTs have been used for recruitment into medicine. Specifically, evidence consistently shows that SJTs used in medical selection have good reliability, and predict performance across a range of medical professions, including performance in general practice, in early years (foundation training as a junior doctor) and for medical school admissions. In addition, SJTs have been found to have significant added value (incremental validity) over and above other selection methods such as knowledge tests, measures of cognitive ability, personality tests and application forms. Regarding differential attainment, generally SJTs have been found to have lower adverse impact compared to other selection methods, such as cognitive ability tests. SJTs have the benefit of being appropriate both for use in selection where candidates are novices (i.e. have no prior role experience or knowledge such as in medical school admissions) as well as settings where candidates have substantial job knowledge and specific experience (as in postgraduate recruitment for more senior roles). An SJT specification (e.g. scenario content, response instructions and format) may differ depending on the level of job knowledge required. Research consistently shows that SJTs are usually found to be positively received by candidates compared to other selection tests such as cognitive ability and personality tests. Practically, SJTs are difficult to design effectively, and significant expertise is required to build a reliable and valid SJT. Once designed however, SJTs are cost efficient to administer to large numbers of candidates compared to other tests of non-academic attributes (e.g. personal statements, structured interviews), as they are standardised and can be computer-delivered and machine-marked. [ABSTRACT FROM PUBLISHER] |
| Copyright of Medical Teacher is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 111969286 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Patterson%2C+Fiona%22">Patterson, Fiona</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> f.patterson@workpsychologygroup.com</i><br /><searchLink fieldCode="AR" term="%22Zibarras%2C+Lara%22">Zibarras, Lara</searchLink><relatesTo>3</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Ashworth%2C+Vicki%22">Ashworth, Vicki</searchLink><relatesTo>1</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Medical+Teacher%22">Medical Teacher</searchLink>. Jan2016, Vol. 38 Issue 1, p3-17. 15p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Professional+education%22">Professional education</searchLink><br />*<searchLink fieldCode="DE" term="%22Adults%22">Adults</searchLink><br />*<searchLink fieldCode="DE" term="%22Research%22">Research</searchLink><br />*<searchLink fieldCode="DE" term="%22Theory%22">Theory</searchLink><br /><searchLink fieldCode="DE" term="%22Judgment+%28Psychology%29%22">Judgment (Psychology)</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+practice%22">Medical practice</searchLink><br /><searchLink fieldCode="DE" term="%22Study+%26+teaching+of+medicine%22">Study & teaching of medicine</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Why use SJTs?Traditionally, selection into medical education professions has focused primarily upon academic ability alone. This approach has been questioned more recently, as although academic attainment predicts performance early in training, research shows it has less predictive power for demonstrating competence in postgraduate clinical practice. Such evidence, coupled with an increasing focus on individuals working in healthcare roles displaying the core values of compassionate care, benevolence and respect, illustrates that individuals should be selected on attributes other than academic ability alone. Moreover, there are mounting calls to widen access to medicine, to ensure that selection methods do not unfairly disadvantage individuals from specific groups (e.g. regarding ethnicity or socio-economic status), so that the future workforce adequately represents society as a whole. These drivers necessitate a method of assessment that allows individuals to be selected on important non-academic attributes that are desirable in healthcare professionals, in a fair, reliable and valid way. What are SJTs?Situational judgement tests (SJTs) are tests used to assess individuals’ reactions to a number of hypothetical role-relevant scenarios, which reflect situations candidates are likely to encounter in the target role. These scenarios are based on a detailed analysis of the role and should be developed in collaboration with subject matter experts, in order to accurately assess the key attributes that are associated with competent performance. From a theoretical perspective, SJTs are believed to measure prosocial Implicit Trait Policies (ITPs), which are shaped by socialisation processes that teach the utility of expressing certain traits in different settings such as agreeable expressions (e.g. helping others in need), or disagreeable actions (e.g. advancing ones own interest at others, expense). Are SJTs reliable, valid and fair?Several studies, including good quality meta-analytic and longitudinal research, consistently show that SJTs used in many different occupational groups are reliable and valid. Although there is over 40 years of research evidence available on SJTs, it is only within the past 10 years that SJTs have been used for recruitment into medicine. Specifically, evidence consistently shows that SJTs used in medical selection have good reliability, and predict performance across a range of medical professions, including performance in general practice, in early years (foundation training as a junior doctor) and for medical school admissions. In addition, SJTs have been found to have significant added value (incremental validity) over and above other selection methods such as knowledge tests, measures of cognitive ability, personality tests and application forms. Regarding differential attainment, generally SJTs have been found to have lower adverse impact compared to other selection methods, such as cognitive ability tests. SJTs have the benefit of being appropriate both for use in selection where candidates are novices (i.e. have no prior role experience or knowledge such as in medical school admissions) as well as settings where candidates have substantial job knowledge and specific experience (as in postgraduate recruitment for more senior roles). An SJT specification (e.g. scenario content, response instructions and format) may differ depending on the level of job knowledge required. Research consistently shows that SJTs are usually found to be positively received by candidates compared to other selection tests such as cognitive ability and personality tests. Practically, SJTs are difficult to design effectively, and significant expertise is required to build a reliable and valid SJT. Once designed however, SJTs are cost efficient to administer to large numbers of candidates compared to other tests of non-academic attributes (e.g. personal statements, structured interviews), as they are standardised and can be computer-delivered and machine-marked. [ABSTRACT FROM PUBLISHER] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Medical Teacher is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=111969286 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.3109/0142159X.2015.1072619 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 15 StartPage: 3 Subjects: – SubjectFull: Professional education Type: general – SubjectFull: Adults Type: general – SubjectFull: Research Type: general – SubjectFull: Theory Type: general – SubjectFull: Judgment (Psychology) Type: general – SubjectFull: Medical practice Type: general – SubjectFull: Study & teaching of medicine Type: general Titles: – TitleFull: Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Patterson, Fiona – PersonEntity: Name: NameFull: Zibarras, Lara – PersonEntity: Name: NameFull: Ashworth, Vicki IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 01 Text: Jan2016 Type: published Y: 2016 Identifiers: – Type: issn-print Value: 0142159X Numbering: – Type: volume Value: 38 – Type: issue Value: 1 Titles: – TitleFull: Medical Teacher Type: main |
| ResultId | 1 |
