View in EDS HTML Full Text PDF Full Text

Longitudinal Measurement Invariance of CCAPS-34 Scores with a Large University Sample

Saved in:
Bibliographic Details
Title: Longitudinal Measurement Invariance of CCAPS-34 Scores with a Large University Sample
Language: English
Authors: Khalid Stetkevych, Martin F. Sherman, Julie Sriken, Bradley T. Erford, Heather L. Smith, Adriana Kipper-Smith, Frances Niarhos
Source: Measurement and Evaluation in Counseling and Development. 2024 57(3):276-286.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 11
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Symptoms (Individual Disorders), Psychological Patterns, Counseling Services, Scores, Universities, College Students, Longitudinal Studies, Measures (Individuals), Private Colleges, Research Universities, Guidance Centers, Measurement, Sample Size
DOI: 10.1080/07481756.2023.2257206
ISSN: 0748-1756
1947-6302
Abstract: Objective: Counseling Center Assessment of Psychological Symptoms (CCAPS-34) scores were studied for longitudinal bias-free construct evidence. Method: A sample of 4,696 university students referred to a university counseling center were assessed twice for evidence of longitudinal measurement invariance. Results: Adequate or marginal longitudinal measurement invariance (LMI) of all subscales except for Eating Concerns was confirmed. Coefficients alpha and omega suggested that the internal consistencies for both initial and subsequent administrations were largely adequate for screening level test scores ([greater than or equal to] 0.80), except for the Hostility and Alcohol Use subscales ([greater than or equal to] 0.73). Second session effect sizes (Cohen's d) were small but detectable and statistically significant for most subscales (-0.02-0.17). Conclusions: Counselors can use most CCAPS-34 subscale scores for tracking psychological symptoms and constructs over time with confidence. The Generalized Anxiety, Social Anxiety, Academic Concerns, and Alcohol Use subscales can be used with great confidence. Some confidence is warranted from the LMI evidence when using the Depression and Hostility subscales, but caution in warranted when using the Eating Concerns subscale.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1428561
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFHGEnceVnIIV1TnFk0k5AQAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDFjopJc7wIuH6QxRNgIBEICBmyvQCi7ixf3slz-vBo5o-6BNWlcnDyF7YhCTFw0sxgwwAuhSr06AylvkigoDATI43A6ytT5GKVNNuDbHvDiqqZEvxlk-QirTQRkIGQcpzIGiNE3KuX-PTZVfMpCY56BWHctZWFfYEGTkVvn1h2RziZ1AAySbjD5nfri7owsug0NLLH4eTuuaXhRoxr6Rt6Y7zM7TPIRtyHDqNb9n
Text:
  Availability: 1
  Value: <anid>AN0177943418;mev01jul.24;2024Jun20.03:28;v2.2.500</anid> <title id="AN0177943418-1">Longitudinal Measurement Invariance of CCAPS-34 Scores with a Large University Sample </title> <p>Counseling Center Assessment of Psychological Symptoms (CCAPS-34) scores were studied for longitudinal bias-free construct evidence. A sample of 4,696 university students referred to a university counseling center were assessed twice for evidence of longitudinal measurement invariance. Adequate or marginal longitudinal measurement invariance (LMI) of all subscales except for Eating Concerns was confirmed. Coefficients alpha and omega suggested that the internal consistencies for both initial and subsequent administrations were largely adequate for screening level test scores (≥.80), except for the Hostility and Alcohol Use subscales (≥.73). Second session effect sizes (Cohen's d) were small but detectable and statistically significant for most subscales (-0.02–.17). Counselors can use most CCAPS-34 subscale scores for tracking psychological symptoms and constructs over time with confidence. The Generalized Anxiety, Social Anxiety, Academic Concerns, and Alcohol Use subscales can be used with great confidence. Some confidence is warranted from the LMI evidence when using the Depression and Hostility subscales, but caution in warranted when using the Eating Concerns subscale.</p> <p>Longitudinal measurement invariance (LMI) helps test users understand whether different groups view the construct being measured as equivalent over time. Counselors can use most CCAPS-34 subscale scores for tracking psychological symptoms and constructs over time with confidence.</p> <p>Keywords: CCAPS-34; Counseling Center Assessment of Psychological Symptoms; measurement invariance; confirmatory factor analysis; university students</p> <p>The Counseling Center Assessment of Psychological Symptoms (CCAPS; Center for Collegiate Mental Health (CCMH), [<reflink idref="bib2" id="ref1">2</reflink>]) is a commonly used assessment designed for university counseling centers to assess college student psychological symptoms and distress. While the psychometric properties of the CCAPS are well studied and supported (Sherman et al., [<reflink idref="bib18" id="ref2">18</reflink>]), and several studies have explored measurement invariance of CCAPS-34 scores on some demographic comparisons (Cline et al., [<reflink idref="bib4" id="ref3">4</reflink>]; Ghosh et al., [<reflink idref="bib11" id="ref4">11</reflink>]; Sherman et al., [<reflink idref="bib18" id="ref5">18</reflink>]; Yoon et al., [<reflink idref="bib23" id="ref6">23</reflink>]), research on longitudinal measurement invariance (LMI) is still needed to best understand the interpretive accuracy of the CCAPS-34 over time. The present study looks to research and establish the outcomes of LMI testing for repeated administrations of the CCAPS-34 scores.</p> <p>Measurement invariance (MI) helps test users understand whether different groups of respondents view the construct being measured as equivalent, for example, when women and men view the meaning of a question or series of questions differently. Longitudinal measurement invariance (LMI) helps test users understand whether different groups of respondents view the construct being measured as equivalent over time or repeated measurements. That is, is the measured construct viewed consistently by respondents over two, three or more exposures to the same construct item set?</p> <p>When exploring measurement invariance (MI), confirmatory factor analysis (CFA) is used to test whether a factor model maintains its structure across groups, thus confirming that the same construct is being measured across groups or scores across time periods (Millsap, [<reflink idref="bib14" id="ref7">14</reflink>]; Vandenberg & Lance, [<reflink idref="bib22" id="ref8">22</reflink>]). As U.S. colleges become increasingly diverse and involved in student mental health, it become increasingly important to confirm that scores on measures like the CCAPS-34 show adequate MI.</p> <p>Several studies of the MI of the CCAPS-34 scores have been conducted. In a population similar to the current study, Sherman et al. ([<reflink idref="bib18" id="ref9">18</reflink>]) showed negligible model degradation across ethnic groups, demonstrating support for measurement invariance. MI for the CCAPS-34 has also been found across levels of care. Smith et al. (in press) demonstrated MI across gender showing equivalence between scores of men and women, as well as across various racial groups in U.S. university students. Yoon et al. ([<reflink idref="bib23" id="ref10">23</reflink>]) found that CCAPS-34 scores showed MI between student outpatient and inpatient samples treated in a psychiatric hospital. Ghosh et al. ([<reflink idref="bib11" id="ref11">11</reflink>]) compared military and nonmilitary samples of university students and established MI. And Cline et al. ([<reflink idref="bib4" id="ref12">4</reflink>]) reported significant differences between Native Hawaiian and other Pacific Islander (NHPI) participants and Asian and other groups. All together, these studies provided foundational evidence that counselors and researchers can use the CCAPS-34 with confidence across these common ethnic groups, an especially important result given that the CCAPS is used by more than 650 U.S. universities (CCMH, [<reflink idref="bib2" id="ref13">2</reflink>]).</p> <p>In spite of this research and positive evidence, there has not been any research to date into the LMI of CCAPS-34 scores. Because the CCAP-34 is used as a repeated measure in treatment at baseline and subsequent sessions, the CCAPS-34 must demonstrate that its constructs are temporally stable and that the scores, factors, and model are consistent over repeated administrations. LMI allows clinicians and researchers to confidently interpret the development of a measure's constructs over time (Millsap & Cham, [<reflink idref="bib15" id="ref14">15</reflink>]). LMI seeks to confirm that any changes over time in the assessment item arises exclusively from changes in the underlying factor, such that a shift in scores is in fact representative of a shift in the construct of interest. As noted by Roesch et al. ([<reflink idref="bib17" id="ref15">17</reflink>]), LMI needs to be established on psychosocial measures before they can be used to make or support causal statements. The CCAPS-34 is meant to do just that as it has established psychometric properties along subscales that represent common areas of psychosocial distress: Depression, Generalized Anxiety, Social Anxiety, Academic Distress, Eating Concerns, Hostility, and Alcohol Use. And the CCAPS-34 is used to promote measurement-based therapeutic outcomes in many U.S. university counseling centers. Therefore, it is imperative to confirm evidence of LMI for CCAPS-34 scores so counselors can better surmise their effectiveness in treating psychosocial distress as clinicians.</p> <p>One study of the CCAPS-34 did use a multi-level CFA (MFA) over 10 or more observations to explore in-person versus between person variability. McAleavey et al. ([<reflink idref="bib13" id="ref16">13</reflink>]) found that MFA largely recovered the CCAPS-34 factor structure at the within person level but not at the between person level. This indicated that the narrow factors (e.g. depression, social anxiety) appeared to change over time, but not the broad factors (e.g. academic distress, hostility). Such studies point to the importance of construct consistency when instruments are used repeatedly. Thus, additional MI testing is needed to confirm longitudinal MI in scores on the CCAPS-34 to allow confident use across diverse clientele over time.</p> <p>The purpose of the current study was to fill this gap in research by assessing the LMI of the CCAPS-34's subscale scores using participants from a large U.S. university's counseling center. The specific research question was as follows: Is measurement invariance confirmed over time for scores on the CCAPS-34 subscales?</p> <hd id="AN0177943418-2">Method</hd> <p></p> <hd id="AN0177943418-3">Inclusion and Exclusion Criteria and Sampling Procedures</hd> <p>The study was approved by the Vanderbilt University institutional review board. The data were collected by the university counseling center at a large private Research I university in the southern United States. All students who visited the university counseling center from August 2015 to December 2020 and completed the CACAPS-34 at least twice were included in the study.</p> <hd id="AN0177943418-4">Participant Characteristics</hd> <p>Participants included 4,696 students who completed initial and at least one follow-up protocol from an initial sample of 9,452 students. The remainder of the participants (<emph>n</emph><bold></bold>=<bold></bold>4,756) completed the CCAPS-34 only once. Participants attended treatment sessions at the university counseling center of their large private southern U.S. university from August 2015 to December 2020. While the time frame for visits to counseling center was over 5 years, most students visited for only 1–3 sessions. Few visited for more than 10 sessions. To maximize sample size to power the analysis we selected scores from sessions 1 and 2 for analysis. Participants were administered the CCAPS-34 prior to initial and subsequent treatment sessions at the university counseling center. Self-identified demographic characteristics can be found in the Follow-up Sample column of Table 1. Due to complications from separate storage housings, sex was not a part of all available data. While sex cannot be used for analyses, a similar proportion can be inferred from available data, in which self-identified sex groups broke down as 38% male, 62% female, and less than 0.5% nonbinary or transgender. While 9,452 unique students visited the counseling center over the four-year period, only 4,696 students returned to complete a second CCAPS-34 protocol. Thus, the total participant sample for the analyses that follow was 4,696.</p> <p>Table 1. Self-identification of Demographic Characteristics for the Initial and Second Administration Samples.</p> <p> <ephtml> <table><thead><tr><td>Race</td><td>Initial Sample</td><td>Follow-up Sample</td></tr></thead><tbody valign="top"><tr><td>Asian</td><td char=".">889 (9.4%)</td><td char=".">434 (9.2%)</td></tr><tr><td>Black</td><td char=".">1,046 (11.1%)</td><td char=".">601 (12.8%)</td></tr><tr><td>Latinx</td><td char=".">767 (8.1%)</td><td char=".">398 (8.5%)</td></tr><tr><td>White</td><td char=".">4,868(51.5%)</td><td char=".">2,251 (47.9%)</td></tr><tr><td>Multiracial</td><td char=".">385 (4.1%)</td><td char=".">211 (4.5%)</td></tr><tr><td>Middle East/South Asian</td><td char=".">62 (0.7%)</td><td char=".">14 (0.3%)</td></tr><tr><td>Other</td><td char=".">43 (0.4%)</td><td char=".">16 (0.3%)</td></tr><tr><td>International</td><td char=".">408 (4.3%)</td><td char=".">278 (5.9%)</td></tr><tr><td>Missing</td><td char=".">984 (10.4%)</td><td char=".">493 (10.5%)</td></tr><tr><td>Total</td><td char=".">9,452</td><td char=".">4,696</td></tr></tbody></table> </ephtml> </p> <hd id="AN0177943418-5">Measurement of Constructs</hd> <p>The CCAPS-34 (CCMH, [<reflink idref="bib2" id="ref17">2</reflink>]) was used in this study. The CCAPS-34 is composed of 34 items and seven subscales: Depression, Generalized Anxiety, Social Anxiety, Academic Distress, Eating Concerns, Hostility, and Alcohol Use (see Table 2 for item-subscale assignments). The CCAPS-34 can be administered repeatedly at any time throughout treatment. The administration takes approximately three to five minutes to complete. Interpretation is done by subscale raw score comparisons and is criterion referenced, even though no raw score cut offs are available for clinical significance. Participant scores are compared over time and the changes in raw scores supply clinicians with valuable information for informing treatment, evaluating treatment effects, and interpreting patterns of distress. Sherman et al. ([<reflink idref="bib18" id="ref18">18</reflink>]) comprehensive review of its psychometric properties demonstrate the CCAPS-34 has shown robust reliability and validity of scores adequate for use as a screening and outcome assessment. This study's score reliability and validity evidence of CCAPS-34 scores is presented in the Results section and Table 2.</p> <p>Table 2. Coefficients Alpha and Omega for Scores from the First and Second Administrations of the CCAPS-34 in the Current Study.</p> <p> <ephtml> <table><thead><tr><td /><td>1st Administration (a, ὠ)</td><td>2nd Administration (a, ὠ)</td><td>CCAPS-34 Items</td></tr></thead><tbody valign="top"><tr><td>Depression</td><td char=".">.873,.879</td><td char=".">.879,.886</td><td char=".">4, 5, 11, 12, 21, 25</td></tr><tr><td>Generalized Anxiety</td><td char=".">.807,.808</td><td char=".">.826,.827</td><td char=".">2, 7, 9, 10, 15, 17</td></tr><tr><td>Social Anxiety</td><td char=".">.818,.812</td><td char=".">.824,.821</td><td char=".">1, 19R, 22, 24, 26</td></tr><tr><td>Academic Distress</td><td char=".">.827,.837</td><td char=".">.837,.844</td><td char=".">8R, 28, 30, 33</td></tr><tr><td>Eating Concerns</td><td char=".">.893,.893</td><td char=".">.905,.905</td><td char=".">3, 6, 13</td></tr><tr><td>Hostility</td><td char=".">.729,.757</td><td char=".">.735,.765</td><td char=".">18, 20, 29, 32, 34 deleted 23</td></tr><tr><td>Alcohol Use</td><td>.768,.777</td><td>.776,.781</td><td>14, 16, 31 deleted 27</td></tr></tbody></table> </ephtml> </p> <p>1 <emph>Note</emph>. First administration <emph>n</emph> = 4,690; second administration <emph>n</emph> = 4,690.</p> <hd id="AN0177943418-6">Data Collection</hd> <p>The Vanderbilt University counseling center collected the data for this study during the participants' course of treatment. The CCAPS-34 was electronically administered <emph>via</emph> smartphone or tablet before the participants' first intake session, and, when applicable, similarly readministered approximately every two weeks during follow-up sessions. Students provided demographic information such as self-identified sex and race to the counseling center. Inclusion criteria required that students be at least 18 years old and complete the CCAPS-34 at least twice. The data was de-identified by counseling center staff prior to transferring it to the research team. No other instruments were administered.</p> <hd id="AN0177943418-7">Analytic Plan</hd> <p></p> <hd id="AN0177943418-8">Data Diagnostics</hd> <p>Any cases with missing data from either administration of the CCAPS-34 were eliminated from the data base. Because the CCAPS-34 was administered electronically at intake prior to the first session of a self-referral, and again at the beginning of the next session, students were motivated to comply, and few blank answers were evident (<1%).</p> <hd id="AN0177943418-9">Statistical Power and Precision</hd> <p>The final sample size of 4,696 far exceeded the minimum case to items recommendations of 10:1 (Dimitrov, [<reflink idref="bib9" id="ref19">9</reflink>]), in this instance <emph>n</emph><bold></bold>=<bold></bold>340 for a 34-item inventory. Thus, sufficient power existed for the ensuing analyses to ensure reasonable precision.</p> <hd id="AN0177943418-10">Primary Analysis</hd> <p>While the primary analysis was of longitudinal measurement invariance (LMI), additional analyses included internal structure validity, internal consistency, and paired <emph>t</emph>-tests between initial and second administrations to determine short-term (about two-week) treatment gains and expected effect sizes. LMI and internal structure validity (CFA) were examined using <emph>Mplus</emph> version 8.9 (Muthen & Muthen, [<reflink idref="bib16" id="ref20">16</reflink>]). The CFA results were compared using Dimitrov's ([<reflink idref="bib8" id="ref21">8</reflink>], [<reflink idref="bib9" id="ref22">9</reflink>]) guidelines for determining adequate model fit: comparative fit index (<emph>CFI</emph>) or Tucker-Lewis Index (<emph>TLI</emph>) of at least.90 (≥.95 indicates excellent fit); standardized root mean square residual (<emph>SRMR</emph>) ≤.08; and root mean square error of approximation (<emph>RMSEA</emph>) of ≤.07 (≤.05 indicates excellent fit). Each CCAPS-34 subscale measurement model was fit to the entire initial data set, then subsequently used for LMI among the subscales. Lack of measurement invariance was interpreted through Chen's ([<reflink idref="bib3" id="ref23">3</reflink>]) criteria (increases in <emph>RMSEA</emph> or <emph>SRMR</emph> >.015 or a decrease in <emph>CFI</emph> >.01).</p> <hd id="AN0177943418-11">Invariance Testing</hd> <p>We conducted single-group CFAs on each subscale to determine the baseline models for first and second CCAPS-34 administrations (see Table 4). These CFAs yielded initial evidence for MI as both administration groups fit the data adequately to well across all reported indexes except <emph>RMSEA</emph>. We continued with the next three tests in the hierarchical sequence of evaluating measurement invariance (configural, metric, and scalar tests) because they are the most frequently examined and reported tests of measurement invariance and the remaining tests of measurement invariance proposed by Vandenberg ([<reflink idref="bib21" id="ref24">21</reflink>]) become more akin to structural invariance (Han et al., [<reflink idref="bib12" id="ref25">12</reflink>]). See Table 5 to find the metric and scalar testing deltas reported below.</p> <p>Table 4. Longitudinal CFAs at Time 1 and Time 2 for Each CCAPS-34 Subscale (<emph>n</emph> = 4,696).</p> <p> <ephtml> <table><thead><tr><td>Admin</td><td>Subscale</td><td>χ<sup>2</sup>(df)</td><td>RMSEA[90CI]</td><td>CFI</td><td>SRMR</td><td /></tr></thead><tbody valign="top"><tr><td>1</td><td>Depression</td><td char=".">524.1(9)</td><td char=".">.110[.103,.119]</td><td char=".">.986</td><td char=".">.025</td><td /></tr><tr><td>2</td><td>Depression</td><td>478.7(9)</td><td>.105[.098,.114]</td><td>.990</td><td>.021</td><td /></tr><tr><td>1</td><td>General Anxiety</td><td char=".">565.2(9)</td><td char=".">.115[.107,.123]</td><td char=".">.972</td><td char=".">.032</td><td /></tr><tr><td>2</td><td>General Anxiety</td><td>599.7(9)</td><td>.118[.110,.126]</td><td>.976</td><td>.030</td><td /></tr><tr><td>1</td><td>Social Anxiety</td><td char=".">1,534.3(5)</td><td char=".">.255[.245,.266]</td><td char=".">.928</td><td char=".">.053</td><td /></tr><tr><td>2</td><td>Social Distress</td><td>1,717.8(5)</td><td>.270[.260,.281]</td><td>.931</td><td>.054</td><td /></tr><tr><td>1</td><td>Academic Distress</td><td char=".">265.6(2)</td><td char=".">.168[.151,.185]</td><td char=".">.987</td><td char=".">.021</td><td /></tr><tr><td>2</td><td>Academic Distress</td><td>252.7(2)</td><td>.163[.147,.181]</td><td>.989</td><td>.019</td><td /></tr><tr><td>1</td><td>Eating Concerns</td><td char=".">0.0(0)</td><td char=".">.000[.000,.000]</td><td char=".">1.00</td><td char=".">.000</td><td /></tr><tr><td>2</td><td>Eating Concerns</td><td>0.0(0)</td><td>.000[.000,.000]</td><td>1.00</td><td>.000</td><td /></tr><tr><td>1</td><td>Hostility</td><td char=".">674.8(9)</td><td char=".">.126[.118,.134]</td><td char=".">.979</td><td char=".">.053</td><td /></tr><tr><td>2</td><td>Hostility</td><td>720.6(9)</td><td>.130[.122,.138]</td><td>.979</td><td>.055</td><td /></tr><tr><td>1</td><td>Alcohol</td><td char=".">419.8(2)</td><td char=".">.211[.194,.228]</td><td char=".">.988</td><td char=".">.032</td><td /></tr><tr><td>2</td><td>Alcohol</td><td>458.9(2)</td><td>.221[.204,.238]</td><td>.987</td><td>.033</td><td /></tr></tbody></table> </ephtml> </p> <p>Table 5. Longitudinal Invariance Testing by CCAPS-34 Subscale.</p> <p> <ephtml> <table><thead><tr><td>Invariance Test</td><td>χ<sup>2</sup>(<italic>df</italic>)</td><td><italic>CFI</italic></td><td>Δ<italic>CFI</italic></td><td><italic>RMSEA</italic>[CI 90%]</td><td>ΔRMSEA</td><td><italic>SRMR</italic></td><td>ΔSRMR</td><td><italic>Models Compared</italic></td><td><italic>χ<sup>2</sup>(df)[ρ]</italic></td></tr></thead><tbody valign="top"><tr><td><bold>Depression Longitudinal Time 1 v Time 2</bold></td></tr><tr><td>Configural</td><td char=".">819.74(56)*</td><td char=".">.991</td><td /><td char=".">.054[.051,.057]</td><td /><td char=".">.028</td><td /><td /><td /></tr><tr><td>Metric</td><td char=".">816.82(61)*</td><td char=".">.991</td><td char=".">.000</td><td char=".">.051[.048,.055]</td><td char=".">−.003</td><td char=".">.028</td><td char=".">.000</td><td>Metric vs. Configural</td><td char=".">44.32(5)[<.001]</td></tr><tr><td>Scalar</td><td char=".">1782.37(78)*</td><td char=".">.980</td><td char=".">−.011</td><td char=".">.068[.065,.071]</td><td char=".">+.017</td><td char=".">.036</td><td char=".">+.008</td><td>Scalar vs. Metric</td><td char=".">1559.80(17)[<.001]</td></tr><tr><td><bold>Generalized Anxiety Longitudinal Time 1 v Time 2</bold></td></tr><tr><td>Configural</td><td char=".">1391.72(56)*</td><td char=".">.977</td><td /><td char=".">.071[.068,.075]</td><td /><td char=".">.035</td><td /><td /><td /></tr><tr><td>Metric</td><td char=".">1268.92(61)*</td><td char=".">.979</td><td char=".">+.002</td><td char=".">.065[.062,.068]</td><td char=".">−.006</td><td char=".">.036</td><td char=".">+.001</td><td>Metric vs. Configural</td><td char=".">28.90(5)[<.001]</td></tr><tr><td>Scalar</td><td char=".">1396.20(77)*</td><td char=".">.977</td><td char=".">−.002</td><td char=".">.060[.058,.063]</td><td char=".">−.005</td><td char=".">.036</td><td char=".">.000</td><td>Scalar vs. Metric</td><td char=".">90.10(16)[<.001]</td></tr><tr><td><bold>Social Anxiety Longitudinal Time 1 v Time 2</bold></td></tr><tr><td>Configural</td><td char=".">4888.32(37)*</td><td char=".">.933</td><td /><td char=".">.167[.163,.171]</td><td /><td char=".">.065</td><td /><td /><td /></tr><tr><td>Metric</td><td char=".">4497.08(41)*</td><td char=".">.939</td><td char=".">+.006</td><td char=".">.152[.148,.156]</td><td char=".">−.015</td><td char=".">.065</td><td char=".">.000</td><td>Metric vs. Configural</td><td char=".">32.04(4)[<.001]</td></tr><tr><td>Scalar</td><td char=".">4951.30(54)*</td><td char=".">.933</td><td char=".">−.006</td><td char=".">.139[.136,.142]</td><td char=".">−.013</td><td char=".">.065</td><td char=".">.000</td><td>Scalar vs. Metric</td><td char=".">125.81(13)[<.001]</td></tr><tr><td><bold>Academic Concerns Longitudinal Time 1 v Time 2</bold></td></tr><tr><td>Configural</td><td char=".">500.46(22)*</td><td char=".">.991</td><td /><td char=".">.068[.063,.073]</td><td /><td char=".">.028</td><td /><td /><td /></tr><tr><td>Metric</td><td char=".">484.77(25)*</td><td char=".">.991</td><td char=".">.000</td><td char=".">.063[.058,.068]</td><td char=".">−.005</td><td char=".">.029</td><td char=".">+.001</td><td>Metric vs. Configural</td><td char=".">6.95(3)[.07]</td></tr><tr><td>Scalar</td><td char=".">578.81(35)*</td><td char=".">.989</td><td char=".">−.002</td><td char=".">.058[.053,.062]</td><td char=".">−.005</td><td char=".">.029</td><td char=".">.000</td><td>Scalar vs. Metric</td><td char=".">75.97(10)[<.001]</td></tr><tr><td><bold>Eating Concerns Longitudinal Time 1 v Time 2</bold> (major problem with item 13)</td></tr><tr><td>Configural</td><td char=".">8309.40(11)*</td><td char=".">.907</td><td /><td char=".">.401[.394,.408]</td><td /><td char=".">.065</td><td /><td /><td /></tr><tr><td>Metric</td><td char=".">7370.66(13)*</td><td char=".">.918</td><td char=".">+.011</td><td char=".">.347[.341,.354]</td><td char=".">−.054</td><td char=".">.065</td><td char=".">.000</td><td>Metric vs. Configural</td><td char=".">32.82(2)[<.001]</td></tr><tr><td>Scalar</td><td char=".">7499.16(20)*</td><td char=".">.916</td><td char=".">−.002</td><td char=".">.282[.277,.288]</td><td char=".">−.065</td><td char=".">.065</td><td char=".">.000</td><td>Scalar vs. Metric</td><td char=".">10.58(7)[.16]</td></tr><tr><td><bold>Hostility Longitudinal Time 1 v Time 2 (removed item 23)</bold></td></tr><tr><td>Configural</td><td char=".">1269.19(37)*</td><td char=".">.960</td><td /><td char=".">.084[.080,.088]</td><td /><td char=".">.050</td><td /><td /><td /></tr><tr><td>Metric</td><td char=".">1135.81(41)*</td><td char=".">.964</td><td char=".">+.004</td><td char=".">.075[.072,.079]</td><td char=".">−.009</td><td char=".">.050</td><td char=".">.000</td><td>Metric vs. Configural</td><td char=".">10.23(4)[.04]</td></tr><tr><td>Scalar</td><td char=".">1665.90(54)*</td><td char=".">.947</td><td char=".">−.017</td><td char=".">.080[.076,.083]</td><td char=".">+.005</td><td char=".">.052</td><td char=".">+.002</td><td>Scalar vs. Metric</td><td char=".">620.93(13)[<.001]</td></tr><tr><td><bold>Alcohol Longitudinal Time 1 v Time 2 (removed item 27)</bold></td></tr><tr><td>Configural</td><td char=".">81.89(11)*</td><td char=".">.998</td><td /><td char=".">.037[.030,.045]</td><td /><td char=".">.015</td><td /><td /><td /></tr><tr><td>Metric</td><td char=".">104.15(13)*</td><td char=".">.997</td><td char=".">−.001</td><td char=".">.039[.032,.046]</td><td char=".">+.002</td><td char=".">.016</td><td char=".">+.001</td><td>Metric vs. Configural</td><td char=".">27.40(2)[<.001]</td></tr><tr><td>Scalar</td><td char=".">118.88(19)*</td><td char=".">.997</td><td char=".">.000</td><td char=".">.033[.028,.039]</td><td char=".">−.006</td><td char=".">.016</td><td char=".">.000</td><td>Scalar vs. Metric</td><td char=".">10.10(6)[.12]</td></tr></tbody></table> </ephtml> </p> <p>2 <emph>Note</emph>. χ<sups>2</sups> = Chi-squared test statistic; <emph>df</emph> = degrees of freedom; * = significance at <emph>p</emph> <.05; <emph>CFI</emph> = comparative fit index; <emph>RMSEA</emph> = root mean square error of approximation; CI90 = 90% confidence interval; <emph>SRMR</emph> = standardized root mean square residual. In the Eating Concerns Longitudinal Time 1 v Time 2 analysis, item 13 created a major problem in deltas, but was retained because this subscale has only 3 items. In the Hostility Longitudinal Time 1 v Time 2 analysis, we removed item 23. In the Alcohol Longitudinal Time 1 v Time 2 analysis, we removed item 27.</p> <p>We used a multi-group CFA to compare the structure of the CCAPS-34 scores across time, beginning by testing configural invariance. This involves testing for differences in the CCAPS-34 subscale structure between administrations while item intercepts and factor loadings are unconstrained, allowing them to vary freely for each group. Testing metric invariance requires that factor structures are similar across administrations (Han et al., [<reflink idref="bib12" id="ref26">12</reflink>]). Once configural invariance was confirmed, we proceeded with metric invariance testing to examine to what extent factors loaded equivalently across group scores by constraining factor loadings to be equivalent while the item intercepts continue to vary freely. Once metric invariance was established, examining scalar invariance may proceed by constraining item intercepts to ascertain whether both factor loadings and item intercepts are equivalent across participant groups' scores.</p> <hd id="AN0177943418-12">Internal Consistency and Difference tests</hd> <p>This study used coefficient alpha (α) and omega (ὠ) to estimate internal consistency, and a standard of ≥.80 to indicate that a score's internal consistency was reliable enough for screening purposes (Erford, [<reflink idref="bib10" id="ref27">10</reflink>]). This study also used Cohen's ([<reflink idref="bib5" id="ref28">5</reflink>]) <emph>d</emph> effect size guidelines (0.2 indicates a small effect, 0.5 a medium effect, and 0.8 a large effect) to interpret the effects of the dependent <emph>t</emph>-test (see Table 3).</p> <p>Table 3. Paired t-tests between Initial Administration versus Second Administration.</p> <p> <ephtml> <table><thead><tr><td /><td>Time 1 <italic>M(SD)</italic></td><td>Time 2 <italic>M(SD)</italic></td><td><italic>t</italic>-test</td><td><italic>df</italic></td><td><italic>p</italic>-value (2-tailed)</td><td>Cohen's <italic>d</italic></td></tr></thead><tbody valign="top"><tr><td>Depression</td><td char=".">9.20 (5.70)</td><td char=".">8.60 (5.65)</td><td char=".">9.47</td><td char=".">4695</td><td char="."><.001</td><td char=".">0.138</td></tr><tr><td>General Anxiety</td><td char=".">11.43 (5.54)</td><td char=".">10.82 (5.56)</td><td char=".">10.18</td><td char=".">4695</td><td char="."><.001</td><td char=".">0.149</td></tr><tr><td>Social Anxiety</td><td char=".">10.07 (4.68)</td><td char=".">9.73 (4.65)</td><td char=".">8.10</td><td char=".">4695</td><td char="."><.001</td><td char=".">0.118</td></tr><tr><td>Academic Distress</td><td char=".">7.42 (4.19)</td><td char=".">7.49 (4.20)</td><td char=".">−1.43</td><td char=".">4695</td><td char=".">.154</td><td char=".">−0.021</td></tr><tr><td>Eating Concerns</td><td char=".">3.06 (3.39)</td><td char=".">2.92 (3.33)</td><td char=".">4.19</td><td char=".">4695</td><td char="."><.001</td><td char=".">0.059</td></tr><tr><td>Hostility</td><td char=".">2.83 (3.11)</td><td char=".">2.43 (2.90)</td><td char=".">11.52</td><td char=".">4695</td><td char="."><.001</td><td char=".">0.168</td></tr><tr><td>Alcohol Use</td><td char=".">1.64 (2.38)</td><td char=".">1.42 (2.22)</td><td char=".">8.82</td><td char=".">4695</td><td char="."><.001</td><td char=".">0.129</td></tr></tbody></table> </ephtml> </p> <hd id="AN0177943418-13">Results</hd> <p></p> <hd id="AN0177943418-14">Internal Structure Validity</hd> <p>We used <emph>Mplus 8.9</emph> (Muthen & Muthen, [<reflink idref="bib16" id="ref29">16</reflink>]) confirmatory factor analysis (CFA) procedures to compare the responses on the initial and follow-up administrations on the seven CCAPS-34 subscale models, resulting in: (<emph>CFI</emph>) of.928–.988 on initial administration and.931–.990 on follow-up (excluding Eating Concerns); (<emph>RMSEA</emph>) of.110[<emph>CI90:</emph>.103,.119] to.255[<emph>CI90:</emph>.245,.266] and.105[<emph>CI90:</emph>.098,.114] to.270[<emph>CI90:</emph>.260,.281] (excluding Eating Concerns); and <emph>SRMR</emph> of.021–.053 and.019–.055 (excluding Eating Concerns). Given the aforementioned guidelines (Dimitrov, [<reflink idref="bib8" id="ref30">8</reflink>], [<reflink idref="bib9" id="ref31">9</reflink>]), the CFIs were adequate to excellent for nearly all subscales except for Social Anxiety, whose CFIs were adequate on both administrations [<emph>CFI</emph> ≥.90 (≥.95 indicates excellent fit)]. <emph>RMSEA</emph>s were inadequate, with all but Eating Concerns falling above.07. Lastly, <emph>SRMR</emph>s all fell within the adequate range (≤.08). See Table 4 for summary data on CFAs. RMSEAs are sensitive to model misfit as sample sizes increase beyond 500 (Tennant & Pallant, [<reflink idref="bib20" id="ref32">20</reflink>]).</p> <hd id="AN0177943418-15">Longitudinal Measurement Invariance Analyses</hd> <p>LMI tests were applied to each subscale, as summarized in Table 5.</p> <hd id="AN0177943418-16">Invariance Testing for the Depression Subscale</hd> <p>We examined the LMI for the CCAPS-34's Depression subscale following the aforementioned steps (see Table 5). Configural invariance tests showed adequate model fit given the established criteria, with excellent <emph>CFI</emph> delta scores. Testing metric invariance showed an equally good model fit with no model degradation (Δ<emph>CFI</emph> =.000, <emph>ΔRMSEA</emph><bold></bold>=<bold></bold>−0.003, <emph>ΔSRMR</emph> =.000). Testing scalar invariance showed good model fit, with nominal model degradation (<emph>ΔCFI</emph><bold></bold>=<bold></bold>−0.011, <emph>ΔRMSEA</emph> = +.017, <emph>ΔSRMR</emph> = +.008). Thus, marginal LMI was confirmed for the Depression subscale scores.</p> <hd id="AN0177943418-17">Invariance Testing for the Generalized Anxiety Subscale</hd> <p>When testing configural invariance, the scores for the CCAPS-34 Anxiety subscale showed excellent model <emph>CFI</emph> and <emph>SRMR</emph> scores, with an <emph>RMSEA</emph> just.001 higher than the recommended interpretive guidelines (see Table 5). Testing the metric invariance showed excellent model fit and no model degradation (<emph>ΔCFI</emph> = +.002, <emph>ΔRMSEA</emph><bold></bold>=<bold></bold>−0.006, <emph>ΔSRMR</emph> = +.001). Testing the scalar invariance showed similarly excellent model fit with no model degradation (<emph>ΔCFI</emph><bold></bold>=<bold></bold>−0.002, <emph>ΔRMSEA</emph><bold></bold>=<bold></bold>−0.005, <emph>ΔSRMR</emph> =.000). Thus, LMI was confirmed for the Generalized Anxiety subscale scores.</p> <hd id="AN0177943418-18">Invariance Testing for the Social Anxiety Subscale</hd> <p>The Social Anxiety subscale showed adequate model fit on <emph>CFI</emph> and <emph>SRMR</emph> scores but fell outside of the interpretive guidelines on its RMSEA score (see Table 5). The subscale also showed similar model fit on its metric invariance testing with no model degradation <emph>(ΔCFI</emph> = +.006, <emph>ΔRMSEA</emph><bold></bold>=<bold></bold>−0.015, <emph>ΔSRMR</emph> =.000). Scalar invariance testing showed similar model fit with no model degradation (<emph>ΔCFI</emph><bold></bold>=<bold></bold>−0.006, <emph>ΔRMSEA</emph><bold></bold>=<bold></bold>−0.013, <emph>ΔSRMR</emph> =.000). Thus, LMI was confirmed for the Social Anxiety subscale scores.</p> <hd id="AN0177943418-19">Invariance Testing for the Academic Concerns Subscale</hd> <p>When testing configural invariance, the Academic Concerns subscale showed excellent model fit (see Table 5). Metric invariance testing showed excellent model fit and no model degradation (<emph>ΔCFI</emph> =.000, <emph>ΔRMSEA</emph><bold></bold>=<bold></bold>−0.005, <emph>ΔSRMR</emph> = +.001). Testing scalar invariance showed similarly excellent model fit with no model degradation (<emph>ΔCFI</emph><bold></bold>=<bold></bold>−0.002, <emph>ΔRMSEA</emph><bold></bold>=<bold></bold>−0.005, <emph>ΔSRMR</emph> =.000). Thus, LMI was confirmed for the Academic Concerns subscale scores.</p> <hd id="AN0177943418-20">Invariance Testing for the Eating Concerns Subscale</hd> <p>Configural invariance testing showed adequate model fit on <emph>CFI</emph> and <emph>SRMR</emph> but had a problematic <emph>RMSEA</emph> value (see Table 5). Metric invariance testing showed similar model fit and marginal to substantial model degradation (<emph>ΔCFI</emph> = +.011, <emph>ΔRMSEA</emph><bold></bold>=<bold></bold>−0.054, <emph>ΔSRMR</emph> =.00), while scalar invariance testing showed similar model fit and no model degradation on the <emph>ΔCFI</emph> or <emph>ΔSRMR,</emph> but significant degradation on the <emph>ΔRMSEA</emph> (<emph>ΔCFI</emph><bold></bold>=<bold></bold>−0.002, <emph>ΔRMSEA</emph><bold></bold>=<bold></bold>−0.065, <emph>ΔSRMR</emph> =.000). Model degradation was evident given Chen's ([<reflink idref="bib3" id="ref33">3</reflink>]) criteria; thus, a workable level of LMI could not confidently be established for the Eating Concerns subscale due to its erratic <emph>RMSEA</emph> scores.</p> <hd id="AN0177943418-21">Invariance Testing for the Hostility Subscale</hd> <p>Configural invariance testing showed excellent <emph>CFI</emph> and <emph>SRMR</emph> values and a marginal <emph>RMSEA</emph> score (see Table 5). Metric invariance testing showed a similar model fit with no model degradation (<emph>ΔCFI</emph> = +.004, <emph>ΔRMSEA</emph><bold></bold>=<bold></bold>−0.009, <emph>ΔSRMR</emph> =.000). Testing for scalar invariance showed excellent <emph>CFI</emph> and <emph>SRMR</emph> and a similarly marginal <emph>RMSEA</emph>, with nominal levels of model degradation for <emph>ΔCFI</emph><bold></bold>=<bold></bold>−0.017, and no model degradation according for <emph>ΔRMSEA</emph> = +.005 and <emph>ΔSRMR</emph> = +.002). Thus, LMI was marginal for the Hostility subscale scores.</p> <hd id="AN0177943418-22">Invariance Testing for the Alcohol Use Subscale</hd> <p>The Alcohol Use subscale scores underwent configural invariance testing, showing excellent model fit (see Table 5). The metric invariance testing showed excellent model fit and no model degradation (ΔCFI = −0.001, ΔRMSEA = +.002, ΔSRMR = +.001). The scalar invariance testing also showed excellent model fit and no model degradation (ΔCFI =.000, ΔRMSEA = −0.006, ΔSRMR =.000). Thus, LMI was confirmed for the Alcohol Use subscale scores.</p> <hd id="AN0177943418-23">Reliability: Internal Consistency</hd> <p>We obtained participant responses' coefficients α and ω for the seven subscales during the first and second CCAPS-34 administrations (see Table 2). As noted by Deng and Chan ([<reflink idref="bib7" id="ref34">7</reflink>]), obtaining coefficient ω helps to compensate for α 's potential for underestimating true reliability. Internal consistencies were largely adequate for screening level test scores across both administrations and coefficients, though Hostility and Alcohol Use subscales were the only subscale αs or ωs to fall below these guidelines (>.80; Erford, [<reflink idref="bib10" id="ref35">10</reflink>]; see Table 2). CCAPS-34 subscale alphas and omegas were highly similar, ranging from.729 to.893 and.757 to.893, respectively. Coefficients for the second administration were slightly higher, with subscale alphas and omegas respectively ranging from.735 to.905 and.765 to.905.</p> <hd id="AN0177943418-24">Tests of Difference Analysis</hd> <p>Table 3 summarizes interesting results from conducting dependent <emph>t</emph>-tests on the first and second CCAPS-34 administration means. The results of this additional analysis indicate most of the subscales approached significant improvement over a single session. While Academic Distress and Eating Concerns had insignificant effect sizes (-0.021,.059, respectively), the Cohen's <emph>d</emph> for the other subscales ranged from 0.12–0.17 (Cohen, [<reflink idref="bib5" id="ref36">5</reflink>]; 0.2 = small, 0.5 = medium, 0.8 = large). All Cohen's <emph>d</emph>s were <.20, the criterion for a small effect of treatment, embedded in the context that there was no control condition to reduce extraneous confounds.</p> <hd id="AN0177943418-25">Discussion</hd> <p></p> <hd id="AN0177943418-26">Implications for Counseling Practice</hd> <p>The research question guiding this study was: Is longitudinal measurement invariance (LMI) confirmed over time for scores on the CCAPS-34 subscales (LMI)? The answer on this data set of college students was yes for the Generalized Anxiety, Social Anxiety, Academic Concerns, and Alcohol Use subscales; no for the Eating Concerns subscale; and marginal LMI for the Depression and Hostility subscale scores. Internal consistency of scores was adequate (≥.80) for all subscales except the Hostility and Alcohol Use subscale scores. As the first study to assess LMI of the CCAPS-34, these results help create a more complete picture of the psychometric score validity and stability of this popular measure.</p> <p>The CCAPS-34 scores provided some evidence of LMI for all subscales except the Eating Concerns subscale. This study provides additional evidence that the CCAPS-34 is a well-designed instrument that mostly measures its intended constructs over time allowing counselors to interpret CCAPS-34 scores with confidence across time. When building upon previous research, evidence is amassing to support CCAPS-34's psychometric properties and structural integrity across demographic groups (Cline et al., [<reflink idref="bib4" id="ref37">4</reflink>]; Sherman et al., [<reflink idref="bib18" id="ref38">18</reflink>]; Smith et al., in press), severity levels (Yoon et al., [<reflink idref="bib23" id="ref39">23</reflink>]), and now, longitudinally over time.</p> <p>As statistical tests become more sophisticated it is perhaps even more important to reevaluate popular, well-studied assessments like the CCAPS-34 to ensure that their constructs retain structural relevance and their scores have remained clinically useful. In assessing LMI, this study provided evidence to practicing counselors that one of their commonly used tools is functioning appropriately across time. Similarly, this study also provided evidence that most of the CCAPS-34 subscales assess the same construct across time and that the constructs have the same theoretical structures across time. Counselors can use most CCAPS-34 subscale scores for tracking psychological symptoms and constructs over time with confidence. The Generalized Anxiety, Social Anxiety, Academic Concerns, and Alcohol Use subscales can be used with great confidence. Some confidence is warranted from the LMI evidence when using the Depression and Hostility subscales, but caution in warranted when using the Eating Concerns subscale, perhaps because only three items comprise the subscale.</p> <p>Understanding the strengths and weaknesses of the assessment instruments used allows practicing counselors to remain true to their professional values in respecting human diversity and minimizing bias. These results are particularly important for counseling practice because not only are recommendations for measurement-based care (Bickman et al., [<reflink idref="bib1" id="ref40">1</reflink>]) and outcome monitoring (Erford, [<reflink idref="bib10" id="ref41">10</reflink>]) increasing, but trials have supported that routine feedback during treatment shows improved treatment outcomes compared to treatment protocols without such measurement-based feedback (Bickman et al., [<reflink idref="bib1" id="ref42">1</reflink>]). Thus, helping clients understand the progress they are making session to session, helps create efficacy and personal responsibility for change, leading to greater therapeutic gains.</p> <hd id="AN0177943418-27">Study Limitations and Implications for Future Counseling Research</hd> <p>Response bias remains a threat to score reliability and validity of all self-report survey-type instruments, the CCAPS-34 included. Social desirability can affect a university student's response either consciously or unconsciously. While this study did not administer any additional assessments, an assessment like the Marlowe-Crown Social Desirability Scale (Crowne & Marlowe, [<reflink idref="bib6" id="ref43">6</reflink>]) administered concurrently with the CCAPS-34 may help to control for confounds in responses, such as social desirability, acquiescence, positive impression management, and negative impression management. Even though confidentiality separates university counseling centers from the universities in which they are housed, the association of these two entities could increase the likelihood of response bias in students. Validity scales could be a valuable addition to future revisions of the CCAPS-34. Additionally, future research could also use other-observer ratings to assess and correct for method variance effects that may arise from using only self-reports of client symptoms.</p> <p>While the effect sizes between Time 1 and Time 2 were interesting and possibly indicative of early psychosocial improvements, it is unlikely that the students were being treated for all of these measured conditions at once. It is plausible that statistical regression accounts for at least some of these small but significant changes. Confirmation of LMI is important, although more test-retest reliability studies are needed to further study consistency of CCAPS-34 subscale scores over time. That said, with fair LMI established, CCAPS-34 becomes a more legitimate tool for measuring the causality of treatment efficacy and psychosocial gains in university counseling centers.</p> <p>Some errors likely occurred along the way as well, leading to limitations in what we could confidently report. Data storage issues led to unavailable self-identification of gender. The database for the first half of this study was housed in two different storage locations at the participating university. This was corrected for the second half of the sample from which we can reasonably infer an approximation of the overall gender proportions. On the statistical side, while our focus was the subscales of the CCAPS-34, analyzing the LMI of the total scale became unwieldy. We were unable to extract any valuable information from the total scale due to extraneous errors stemming from model complexity.</p> <p>Vandenberg ([<reflink idref="bib21" id="ref44">21</reflink>]) pointed out important shortcomings needed to address the validity and applicability of the analytical procedures involved in measurement invariance. He specifically pointed to the need to remain up to date regarding the sensitivity involved in the configural and metric invariance so that researchers can have better confidence that what they are detecting aligns with what they had intended. Measurement invariance procedures may also be susceptible to contextual artifacts, such as how scores from community college or technical institute, comprehensive, or public 4-year university samples might differ from our large private 4-year university sample.</p> <hd id="AN0177943418-28">Conclusion</hd> <p>We failed to disconfirm LMI by showing that factor loadings and latent constructs remained approximately equivalent across time on all subscales except Eating Concerns. Coefficients alpha and omega suggested that the internal consistencies for both initial and subsequent administrations were largely adequate for screening level test scores (≥.80), except for the Hostility and Alcohol Use subscales (≥.73). In sum, most CCAPS-34 subscales yield adequately reliable and valid scores for screening purposes and can confidently be used for tracking psychological symptoms and constructs over time.</p> <hd id="AN0177943418-29">Disclosure statement</hd> <p>No potential conflict of interest was reported by the authors.</p> <ref id="AN0177943418-30"> <title> References </title> <blist> <bibl id="bib1" idref="ref40" type="bt">1</bibl> <bibtext> Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62 (12), 1423 – 1429. https://doi.org/10.1176/appi.ps.002052011</bibtext> </blist> <blist> <bibl id="bib2" idref="ref1" type="bt">2</bibl> <bibtext> Center for Collegiate Mental Health (CCMH). (2015). CCAPS user manual. Author.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref23" type="bt">3</bibl> <bibtext> Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14 (3), 464 – 504. https://doi.org/10.1080/10705510701301834</bibtext> </blist> <blist> <bibl id="bib4" idref="ref3" type="bt">4</bibl> <bibtext> Cline, J. I., Larsen, R. A., Griner, D., Beecher, M. E., Allen, G. E. K., Lee, C.-T., Lefrandt, J. B., & Worthen, V. (2022). Ethnoracial comparisons in anxiety and depression outcomes among Native Hawaiian and Pacific Islander college students. Journal of Multicultural Counseling and Development, 1 – 10. https://doi.org/10.1002/jmcd.12254</bibtext> </blist> <blist> <bibl id="bib5" idref="ref28" type="bt">5</bibl> <bibtext> Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Academic Press.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref43" type="bt">6</bibl> <bibtext> Crowne, D. P., & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24 (4), 349 – 354. https://doi.org/10.1037/h0047358</bibtext> </blist> <blist> <bibl id="bib7" idref="ref34" type="bt">7</bibl> <bibtext> Deng, L., & Chan, W. (2017). Testing the difference between reliability coefficients alpha and omega. Educational and Psychological Measurement, 77 (2), 185 – 203. https://doi.org/10.1177/0013164416658325</bibtext> </blist> <blist> <bibl id="bib8" idref="ref21" type="bt">8</bibl> <bibtext> Dimitrov, D. M. (2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43 (2), 121 – 149. https://doi.org/10.1177/0748175610373459</bibtext> </blist> <blist> <bibl id="bib9" idref="ref19" type="bt">9</bibl> <bibtext> Dimitrov, D. M. (2011). Statistical methods for validation of assessment scale data in counseling and related fields. American Counseling Association.</bibtext> </blist> <blist> <bibtext> Erford, B. T. (2021). Assessment for counselors. Pearson.</bibtext> </blist> <blist> <bibtext> Ghosh, A., Niileksela, C. R., Parham, A., & Janis, R. A. (2021). Investigating factorial invariance of the counseling center assessment of psychological symptoms – 34 (CCAPS-34) with military and nonmilitary students. Measurement and Evaluation in Counseling and Development, 54 (1), 42 – 55. https://doi.org/10.1080/07481756.2020.1745649</bibtext> </blist> <blist> <bibtext> Han, K., Colarelli, S. M., & Weed, N. C. (2019). Methodological and statistical advances in the consideration of cultural diversity in assessment: A critical review of group classification and measurement invariance testing. Psychological Assessment, 31 (12), 1481 – 1496. https://doi.org/10.1037/pas0000731</bibtext> </blist> <blist> <bibtext> McAleavey, A. A., Castonguay, L. G., Hayes, J. A., & Locke, B. D. (2020). Multilevel versus single-level factor analysis: Differentiating within-person and between-person variability using the CCAPS-34. Journal of Consulting and Clinical Psychology, 88 (10), 907 – 922. https://doi.org/10.1037/ccp0000529</bibtext> </blist> <blist> <bibtext> Millsap, R. (2011). Statistical approaches to measurement invariance. Routledge.</bibtext> </blist> <blist> <bibtext> Millsap, R. E., & Cham, H. (2012). Investigating factorial invariance in longitudinal data. In B. Laursen, T. D. Little, & N. A. Card (Eds.), Handbook of developmental research methods (pp. 109 – 126). The Guilford Press.</bibtext> </blist> <blist> <bibtext> Muthen, L. K., & Muthen, B. O. (2023). Mplus user's guide: Statistical analysis with latent variables (version 8.9). Muthen & Muthen.</bibtext> </blist> <blist> <bibtext> Roesch, S. C., Norman, G. J., Merz, E. L., Sallis, J. F., & Patrick, K. (2013). Longitudinal measurement invariance of psychosocial measures in physical activity research: An application to adolescent data. Journal of Applied Social Psychology, 43 (4), 721 – 729. https://doi.org/10.1111/jasp.12000</bibtext> </blist> <blist> <bibtext> Sherman, M. F., Sriken, J., Erford, B. T., Smith, H. L., MacInerney, E., Niarhos, F., & Kipper-Smith, A. (2021). Psychometric analysis of CCAPS-34 scores with a large university sample. Measurement and Evaluation in Counseling and Development, 54 (4), 219 – 232. https://doi.org/10.1080/07481756.2020.1843971</bibtext> </blist> <blist> <bibtext> Smith, H. A., Sherman, M. F., Sriken, J., Erford, B. T., Kipper-Smith, A., & Niarhos, F. (in press) Testing for measurement invariance of CCAPS-34 scores including self-identified gender and race with a large university sample. Journal of Mental Health Counseling.</bibtext> </blist> <blist> <bibtext> Tennant, A., & Pallant, J. F. (2012). The root mean square error of approximation (RMSEA) as a Supplementary statistic to determine fit to the Rasch model with large sample sizes. Rasch Measurement Transactions, 25 (4), 1348 – 1349.</bibtext> </blist> <blist> <bibtext> Vandenberg, R. J. (2002). Toward a further understanding of and improvement in measurement invariance methods and procedures. Organizational Research Methods, 5 (2), 139 – 158. https://doi.org/10.1177/1094428102005002001</bibtext> </blist> <blist> <bibtext> Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations of organizational research. Organizational Research Methods, 3 (1), 4 – 70. https://doi.org/10.1177/109442810031002</bibtext> </blist> <blist> <bibtext> Yoon, S., Nevins, C. M., Peckham, A. D., & Pinder-Amaker, S. L. (2022). Factor structure of the counseling center assessment of psychological symptoms (CCAPS): Comparisons across college students in inpatient and outpatient settings. Psychiatry Research, 310, 114464. https://doi.org/10.1016/j.psychres.2022.114464</bibtext> </blist> </ref> <aug> <p>By Khalid Stetkevych; Martin F. Sherman; Julie Sriken; Bradley T. Erford; Heather L. Smith; Adriana Kipper-Smith and Frances Niarhos</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author</p> <p></p> <p>Khalid Stetkevych , M.Ed., is a graduate of the human development counseling program in the Department of Human and Organiztional Development in the Peabody College of Education and Human Development at Vanderbilt University.</p> <p>Julie Sriken , Ph.D., is a graduate of the community research and action program in the Department of Human and Organizational Development in the Peabody College of Education and Human Development at Vanderbilt University, and currently an assistant professor at Regis University.</p> <p>Bradley T. Erford , Ph.D., is professor and director of the human development counseling program in the Department of Human and Organizational Development in the Peabody College of Education and Human Development at Vanderbilt University.</p> <p>Martin F. Sherman is professor emeritus in the Psychology Department at Loyola University Maryland.</p> <p>Heather L. Smith , Ph.D., is an associate professor in the Counseling and Guidance Department at New Mexico Highlands University.</p> <p>Adriana Kipper-Smith , Ph.D., is a counselor at the Vanderbilt University Medical College and in private practice in the Nashville, TN area.</p> <p>Frances Niarhos , Ph.D., is administrative clinical staff at the University Counseling Center at Vanderbilt University.</p> </aug> <nolink nlid="nl1" bibid="bib18" firstref="ref2"></nolink> <nolink nlid="nl2" bibid="bib11" firstref="ref4"></nolink> <nolink nlid="nl3" bibid="bib23" firstref="ref6"></nolink> <nolink nlid="nl4" bibid="bib14" firstref="ref7"></nolink> <nolink nlid="nl5" bibid="bib22" firstref="ref8"></nolink> <nolink nlid="nl6" bibid="bib15" firstref="ref14"></nolink> <nolink nlid="nl7" bibid="bib17" firstref="ref15"></nolink> <nolink nlid="nl8" bibid="bib13" firstref="ref16"></nolink> <nolink nlid="nl9" bibid="bib16" firstref="ref20"></nolink> <nolink nlid="nl10" bibid="bib21" firstref="ref24"></nolink> <nolink nlid="nl11" bibid="bib12" firstref="ref25"></nolink> <nolink nlid="nl12" bibid="bib10" firstref="ref27"></nolink> <nolink nlid="nl13" bibid="bib20" firstref="ref32"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1428561
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Longitudinal Measurement Invariance of CCAPS-34 Scores with a Large University Sample
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Khalid+Stetkevych%22">Khalid Stetkevych</searchLink><br /><searchLink fieldCode="AR" term="%22Martin+F%2E+Sherman%22">Martin F. Sherman</searchLink><br /><searchLink fieldCode="AR" term="%22Julie+Sriken%22">Julie Sriken</searchLink><br /><searchLink fieldCode="AR" term="%22Bradley+T%2E+Erford%22">Bradley T. Erford</searchLink><br /><searchLink fieldCode="AR" term="%22Heather+L%2E+Smith%22">Heather L. Smith</searchLink><br /><searchLink fieldCode="AR" term="%22Adriana+Kipper-Smith%22">Adriana Kipper-Smith</searchLink><br /><searchLink fieldCode="AR" term="%22Frances+Niarhos%22">Frances Niarhos</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Measurement+and+Evaluation+in+Counseling+and+Development%22"><i>Measurement and Evaluation in Counseling and Development</i></searchLink>. 2024 57(3):276-286.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 11
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Symptoms+%28Individual+Disorders%29%22">Symptoms (Individual Disorders)</searchLink><br /><searchLink fieldCode="DE" term="%22Psychological+Patterns%22">Psychological Patterns</searchLink><br /><searchLink fieldCode="DE" term="%22Counseling+Services%22">Counseling Services</searchLink><br /><searchLink fieldCode="DE" term="%22Scores%22">Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Universities%22">Universities</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink><br /><searchLink fieldCode="DE" term="%22Longitudinal+Studies%22">Longitudinal Studies</searchLink><br /><searchLink fieldCode="DE" term="%22Measures+%28Individuals%29%22">Measures (Individuals)</searchLink><br /><searchLink fieldCode="DE" term="%22Private+Colleges%22">Private Colleges</searchLink><br /><searchLink fieldCode="DE" term="%22Research+Universities%22">Research Universities</searchLink><br /><searchLink fieldCode="DE" term="%22Guidance+Centers%22">Guidance Centers</searchLink><br /><searchLink fieldCode="DE" term="%22Measurement%22">Measurement</searchLink><br /><searchLink fieldCode="DE" term="%22Sample+Size%22">Sample Size</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/07481756.2023.2257206
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0748-1756<br />1947-6302
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Objective: Counseling Center Assessment of Psychological Symptoms (CCAPS-34) scores were studied for longitudinal bias-free construct evidence. Method: A sample of 4,696 university students referred to a university counseling center were assessed twice for evidence of longitudinal measurement invariance. Results: Adequate or marginal longitudinal measurement invariance (LMI) of all subscales except for Eating Concerns was confirmed. Coefficients alpha and omega suggested that the internal consistencies for both initial and subsequent administrations were largely adequate for screening level test scores ([greater than or equal to] 0.80), except for the Hostility and Alcohol Use subscales ([greater than or equal to] 0.73). Second session effect sizes (Cohen's d) were small but detectable and statistically significant for most subscales (-0.02-0.17). Conclusions: Counselors can use most CCAPS-34 subscale scores for tracking psychological symptoms and constructs over time with confidence. The Generalized Anxiety, Social Anxiety, Academic Concerns, and Alcohol Use subscales can be used with great confidence. Some confidence is warranted from the LMI evidence when using the Depression and Hostility subscales, but caution in warranted when using the Eating Concerns subscale.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1428561
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1428561
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/07481756.2023.2257206
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
        StartPage: 276
    Subjects:
      – SubjectFull: Symptoms (Individual Disorders)
        Type: general
      – SubjectFull: Psychological Patterns
        Type: general
      – SubjectFull: Counseling Services
        Type: general
      – SubjectFull: Scores
        Type: general
      – SubjectFull: Universities
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: Longitudinal Studies
        Type: general
      – SubjectFull: Measures (Individuals)
        Type: general
      – SubjectFull: Private Colleges
        Type: general
      – SubjectFull: Research Universities
        Type: general
      – SubjectFull: Guidance Centers
        Type: general
      – SubjectFull: Measurement
        Type: general
      – SubjectFull: Sample Size
        Type: general
    Titles:
      – TitleFull: Longitudinal Measurement Invariance of CCAPS-34 Scores with a Large University Sample
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Khalid Stetkevych
      – PersonEntity:
          Name:
            NameFull: Martin F. Sherman
      – PersonEntity:
          Name:
            NameFull: Julie Sriken
      – PersonEntity:
          Name:
            NameFull: Bradley T. Erford
      – PersonEntity:
          Name:
            NameFull: Heather L. Smith
      – PersonEntity:
          Name:
            NameFull: Adriana Kipper-Smith
      – PersonEntity:
          Name:
            NameFull: Frances Niarhos
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 0748-1756
            – Type: issn-electronic
              Value: 1947-6302
          Numbering:
            – Type: volume
              Value: 57
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Measurement and Evaluation in Counseling and Development
              Type: main
ResultId 1