View in EDS HTML Full Text PDF Full Text

Multilevel Factor Mixture Modeling: A Tutorial for Multilevel Constructs

Saved in:
Bibliographic Details
Title: Multilevel Factor Mixture Modeling: A Tutorial for Multilevel Constructs
Language: English
Authors: Chunhua Cao (ORCID 0000-0003-3084-598X), Yan Wang (ORCID 0000-0003-2237-8816), Eunsook Kim (ORCID 0000-0003-1054-1735)
Source: Structural Equation Modeling: A Multidisciplinary Journal. 2025 32(1):155-171.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 17
Publication Date: 2025
Document Type: Journal Articles
Reports - Descriptive
Descriptors: Hierarchical Linear Modeling, Factor Analysis, Nonparametric Statistics, Statistical Analysis, Models, Measurement, Equations (Mathematics), Data
DOI: 10.1080/10705511.2024.2332257
ISSN: 1070-5511
1532-8007
Abstract: Multilevel factor mixture modeling (FMM) is a hybrid of multilevel confirmatory factor analysis (CFA) and multilevel latent class analysis (LCA). It allows researchers to examine population heterogeneity at the within level, between level, or both levels. This tutorial focuses on explicating the model specification of multilevel FMM that considers the conceptualization of multilevel constructs. Empirical data sets are used to demonstrate the applications of multilevel FMM for within-level constructs, between-level constructs, and within- and between-level constructs. Detailed model specifications of integrating latent classes into multilevel constructs are provided. For modeling the heterogeneity at the between level, parametric and nonparametric approaches are compared both conceptually and substantively using demonstration data. The interpretations of results using multilevel FMM are also provided. The tutorial is concluded with a discussion of some important aspects of applying multilevel FMM.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1457243
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwF57y2_0grmhsY32NWVwu0hAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDBJWLzeJDQjCwykRswIBEICBmuQ0v74oUvGPoRWHO_zmjDzoDx7SwsOzkszNgdf9kcsMN7swD_BUjIz3c29p-hK8kBUcdlFVVHlo6LxVwX2vL8dxqEpKIfZd1HnVmgmyHRPIfAKuB7IDR5AH95PLMZv5kDFjzJI6ak8MCIV8QHbRCKd-6SAMWu77CqqLXAUuBmHr1J65uAKXc7mpLJB8NfE11cV2hE6dSRGtKZg=
Text:
  Availability: 1
  Value: <anid>AN0182192617;7mz01jan.25;2025Jan15.04:19;v2.2.500</anid> <title id="AN0182192617-1">Multilevel Factor Mixture Modeling: A Tutorial for Multilevel Constructs </title> <p>Multilevel factor mixture modeling (FMM) is a hybrid of multilevel confirmatory factor analysis (CFA) and multilevel latent class analysis (LCA). It allows researchers to examine population heterogeneity at the within level, between level, or both levels. This tutorial focuses on explicating the model specification of multilevel FMM that considers the conceptualization of multilevel constructs. Empirical data sets are used to demonstrate the applications of multilevel FMM for within-level constructs, between-level constructs, and within- and between-level constructs. Detailed model specifications of integrating latent classes into multilevel constructs are provided. For modeling the heterogeneity at the between level, parametric and nonparametric approaches are compared both conceptually and substantively using demonstration data. The interpretations of results using multilevel FMM are also provided. The tutorial is concluded with a discussion of some important aspects of applying multilevel FMM.</p> <p>Keywords: Heterogeneity; multilevel constructs; multilevel FMM; nonparametric approach; parametric approach</p> <p>Factor mixture modeling (FMM) is an important statistical method used by applied researchers in educational, psychological, and other social and behavioral sciences to investigate the unobserved heterogeneity in the population. FMM incorporates both continuous latent variables (factors) and categorical latent variables (latent classes) into the modeling, combining common factor model and latent class analysis (LCA; Lubke & Muthén, [<reflink idref="bib32" id="ref1">32</reflink>]). Example empirical studies using FMM included anxiety sensitivity (Allan et al., [<reflink idref="bib2" id="ref2">2</reflink>]; Bernstein et al., [<reflink idref="bib7" id="ref3">7</reflink>]), aggressive behavior among children (Kim & Muthén, [<reflink idref="bib25" id="ref4">25</reflink>]), social desirability (Leite & Cooper, [<reflink idref="bib27" id="ref5">27</reflink>]), and autism spectrum disorder among children (Georgiades et al., [<reflink idref="bib15" id="ref6">15</reflink>]). For example, Leite and Cooper ([<reflink idref="bib27" id="ref7">27</reflink>]) used factor mixture models and identified two latent classes: respondents (8.47%) who were more likely to respond in a socially desirable way and respondents (91.53%) who were less likely to respond in a socially desirable way.</p> <p>The studies mentioned above assumed that data were a sample consisting of independent observations, and there was no clustering effect. However, multilevel data structure (Raudenbush & Bryk, [<reflink idref="bib43" id="ref8">43</reflink>]) in which observations are not independent but clustered is very common in social and behavioral sciences, such as students nested within schools, children nested within families, employees nested within workplaces, and patients nested within clinics. When data are nested, FMM can be extended to multilevel FMM, which is a hybrid of multilevel common factor model and multilevel LCA model. Multilevel FMM allows researchers to examine the presence of heterogeneity at the within level, the between level, or both levels. In this way, the individuals and/or clusters that are more homogeneous in terms of the parameter of interest can be grouped in the same latent class.</p> <p>Identifying and understanding unobserved grouping of clusters as well as individuals are helpful for multilevel intervention, which is an example of applications of multilevel FMM. For example, the latent class membership at multiple levels may serve to aid the implementation of multilevel intervention, which is designed to exert multiple levels of influence simultaneously or in close succession, such as individuals, organizations, and environments (LeNoble & Hudson, [<reflink idref="bib28" id="ref9">28</reflink>]; Paskett et al., [<reflink idref="bib42" id="ref10">42</reflink>]). Multilevel FMM with the probabilities of individuals or clusters belonging to different latent classes renders the differential multilevel interventions possible. For instance, De Angelis et al. ([<reflink idref="bib4" id="ref11">4</reflink>]) proposed to implement multilevel intervention to address the impact on mental health issues caused by COVID-19. The individual level intervention focuses on employees at higher risk for developing mental health and well-being issues. The organizational level intervention focuses on eliminating stressor in the environment. Multilevel FMM can be used to classify the individuals into different latent classes based on different probabilities of risk for mental health, and to classify the organizations into different latent classes based on the proportions of employees at risk of developing mental health problems.</p> <p>Research methodologists have discussed multilevel FMM (e.g., Asparouhov & Muthén, [<reflink idref="bib5" id="ref12">5</reflink>]; Vermunt, [<reflink idref="bib49" id="ref13">49</reflink>]) and investigated its performance in terms of testing latent mean differences (e.g., Allua et al., [<reflink idref="bib3" id="ref14">3</reflink>]; Son & Hong, [<reflink idref="bib47" id="ref15">47</reflink>]) and testing measurement invariance (Kim et al., [<reflink idref="bib19" id="ref16">19</reflink>]; Kim et al., [<reflink idref="bib18" id="ref17">18</reflink>]). The results from these studies showed the efficacy of multilevel FMM in testing latent factor mean differences and measurement invariance. Despite the promising results from simulation studies using multilevel FMM, this modeling strategy has not been adopted extensively by applied researchers in empirical studies. In the systematic review of FMM applications, Kim et al. ([<reflink idref="bib24" id="ref18">24</reflink>]) found none of them used multilevel FMM even when data showed a nested structure (36% of 76 applications). Part of the reason for the infrequent use of multilevel FMM may be the complexity of the modeling and the interpretation of its results. Therefore, this current study aims to provide a comprehensive and accessible tutorial for multilevel FMM in which the conceptualization of the construct of interest is taken into account.</p> <p>In multilevel FMM, like in multilevel confirmatory factor analysis (CFA) models, the constructs of interest can exist at the within level, the between level, or both within and between levels (Geldhof et al., [<reflink idref="bib14" id="ref19">14</reflink>]; Kim et al., [<reflink idref="bib19" id="ref20">19</reflink>]; Stapleton et al., [<reflink idref="bib46" id="ref21">46</reflink>]) in terms of CFA model specification. When data are collected in nested settings (e.g., when individuals are nested within organizations), the substantive consideration should be taken to conceptualize the constructs of interest at different levels. Some constructs are conceptually meaningful at the within level and individuals are the units of interest, for example, psychiatry disorder (e.g., Dun et al., [<reflink idref="bib11" id="ref22">11</reflink>]). Other constructs may be conceptually sensible at the between level and the clusters are the units of interest, for example, school or workplace climate, community risk factors, and work-family culture. These between-level constructs are often obtained through the collection of individuals' responses to the observed items. Finally, some constructs can be conceptualized at both the within level and between level, for example, individual performance within teams as well as team performance that is aggregated across individuals within teams. In this situation, the constructs need to be modeled at both within and between levels. Multilevel CFA model should be built accordingly as suggested by the literature about multilevel constructs (Kim et al., [<reflink idref="bib19" id="ref23">19</reflink>]; Stapleton et al., [<reflink idref="bib46" id="ref24">46</reflink>]). However, there has been no tutorial that aims to provide modeling and interpreting details of the multilevel FMM, especially for empirical researchers: how the multilevel LCA can be integrated with the multilevel CFA. In other words, there has been no explicit demonstration about how to integrate latent classes into multilevel constructs. Thus, there is an urgent need to systematically demonstrate and advocate the application of multilevel FMM through elaborating on the model specification and results interpretation of multilevel constructs in multilevel FMM.</p> <hd id="AN0182192617-2">1.1. Purpose of the Tutorial</hd> <p>In the context of multilevel mixture modeling, previous research illustrated the application of multilevel latent profile analysis (LPA; Mäkikangas et al., [<reflink idref="bib35" id="ref25">35</reflink>]) and multilevel LCA (Henry & Muthén, [<reflink idref="bib16" id="ref26">16</reflink>]) using empirical data. However, to the best of our knowledge, no previous studies illustrated and demonstrated the application of multilevel FMM with different scenarios of multilevel construct measured by indicators. The model specification of multilevel FMM is more complex than multilevel LPA and multilevel LCA because of the presence of multilevel latent constructs as well as multilevel latent class variables. Moreover, there could be more than one latent class variable in the scenario of between-level constructs, which adds complexity on top of multilevel LPA and multilevel LCA. Therefore, this tutorial purports to explicate multilevel FMM by taking into account different types of multilevel constructs and demonstrate the applications of these models using empirical data sets.</p> <p>To be more specific, we present three different scenarios of multilevel constructs that can be modeled using multilevel FMM. First, we demonstrate model specifications of multilevel FMM for within-level constructs: math self-efficacy using Trends in International Mathematics and Science Study (TIMSS) 2019 South Korea Grade 8 data set. The second scenario is using multilevel FMM when the construct of interest is at the between-level: students' perceptions of their physics teacher's instructional quality using TIMSS 2019 France Grade 8 data. Third, multilevel FMM can be used to model multilevel constructs that are sensible at both the within and between levels: using Students Bullied at School scale as part of School Climate Scale of TIMSS 2019 Kuwait Grade 4 data. Model specifications will be explained to promote the application of multilevel FMM. We provide annotated M<emph>plus</emph> (Muthén & Muthén, 1998–[<reflink idref="bib39" id="ref27">39</reflink>]) syntax in the Appendix because in the systematic review of FMM, Kim et al. ([<reflink idref="bib24" id="ref28">24</reflink>]) found that over 90% of the applications they reviewed (n = 76) used <emph>Mplus</emph>. However, note that it is also possible to use other software programs such as Latent GOLD software and R packages (e.g., OpenMx) to conduct multilevel FMM analysis (Kim et al., [<reflink idref="bib24" id="ref29">24</reflink>]). Interested readers can refer to the user manual of Latent GOLD (e.g., Vermunt & Magidson, [<reflink idref="bib50" id="ref30">50</reflink>]) and R packages.</p> <p>The remainder of this tutorial is organized as follows. FMM is introduced first. Next, we discuss different types of multilevel constructs and multilevel latent classes. Then we elaborate on the model specifications of multilevel FMM in terms of multilevel constructs and latent class variables. We follow this with the illustration of different types of multilevel constructs in multilevel FMM using empirical data sets that have nested data structure. This article is concluded with a short discussion and recommendations for practitioners.</p> <hd id="AN0182192617-3">2. Factor Mixture Modeling</hd> <p>In single-level FMM with independent data, the linear relationship between the observed variable <emph>y</emph> and the underlying latent factor</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>η</mi></mrow></math> </ephtml> for individual <emph>i</emph> is modeled conditional on the categorial latent class variable, <emph>C</emph> (<emph>C</emph> = 1, 2, ..., <emph>c</emph>) (Asparouhov & Muthén, [<reflink idref="bib5" id="ref31">5</reflink>]; Lubke & Muthén, [<reflink idref="bib32" id="ref32">32</reflink>]):</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="true">[</mo><mrow><msub><mrow><mi>y</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow><mo stretchy="true">|</mo><msub><mrow><mi>C</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>=</mo><mi>c</mi><mo>]</mo><mo>=</mo><msub><mrow><mi>α</mi></mrow><mrow><mi>c</mi></mrow></msub><mo>+</mo><msub><mrow><mi mathvariant="normal">Λ</mi></mrow><mrow><mi>c</mi></mrow></msub><msub><mrow><mi>η</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>+</mo><msub><mrow><mi>ϵ</mi></mrow><mrow><mtext mathvariant="italic">ci</mtext></mrow></msub><mo>,</mo></math> </ephtml> (<reflink idref="bib1" id="ref33">1</reflink>)</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="true">[</mo><mrow><msub><mrow><mi>η</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow><mo stretchy="true">|</mo><msub><mrow><mi>C</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>=</mo><mi>c</mi><mo>]</mo><mo>=</mo><msub><mrow><mi>μ</mi></mrow><mrow><mi>c</mi></mrow></msub><mo>+</mo><msub><mrow><mi mathvariant="normal">B</mi></mrow><mrow><mi>c</mi></mrow></msub><msub><mrow><mi>η</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>+</mo><msub><mrow><mi>ζ</mi></mrow><mrow><mtext mathvariant="italic">ci</mtext></mrow></msub><mo>.</mo></math> </ephtml> (<reflink idref="bib2" id="ref34">2</reflink>)</p> <p>In Equation 1,</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="normal">Λ</mi></mrow><mrow><mi>c</mi></mrow></msub></math> </ephtml> represents the latent class specific factor loading matrix that links the observed variables y with the latent factors η, and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>α</mi></mrow><mrow><mi>c</mi></mrow></msub></math> </ephtml> and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>ϵ</mi></mrow><mrow><mtext mathvariant="italic">ci</mtext></mrow></msub></math> </ephtml> are class specific intercepts and residuals, respectively. In Equation 2,</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>μ</mi></mrow><mrow><mi>c</mi></mrow></msub></math> </ephtml> indicates the intercept of the latent variable η,</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="normal">B</mi></mrow><mrow><mi>c</mi></mrow></msub></math> </ephtml> shows the relationship of the latent variable to other exogenous latent variables, and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>ζ</mi></mrow><mrow><mtext mathvariant="italic">ci</mtext></mrow></msub></math> </ephtml> is the residual.</p> <p>To model the unordered latent class membership, a multinomial logistic regression (or logistic regression for a two-category latent class variable) can be used to model the log odds.</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><mtext mathvariant="normal">ln</mtext></mrow><mo /><mrow><mo stretchy="true">[</mo><mrow><mfrac><mrow><mi>P</mi><mo stretchy="true">(</mo><mrow><msub><mrow><mi>C</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>=</mo><mi>k</mi></mrow><mo>|</mo><mrow><msub><mrow><mi>X</mi></mrow><mrow><mtext mathvariant="italic">ik</mtext></mrow></msub></mrow><mo stretchy="true">)</mo></mrow><mrow><mi>P</mi><mo stretchy="true">(</mo><mrow><msub><mrow><mi>C</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>=</mo><mi>r</mi></mrow><mo>|</mo><mrow><msub><mrow><mi>X</mi></mrow><mrow><mtext mathvariant="italic">ik</mtext></mrow></msub></mrow><mo stretchy="true">)</mo></mrow></mfrac></mrow><mo stretchy="true">]</mo></mrow></mrow><mo>=</mo><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>k</mi></mrow></msub><mo>+</mo><msub><mrow><mi mathvariant="normal">Γ</mi></mrow><mrow><mi>k</mi></mrow></msub><msub><mrow><mi>X</mi></mrow><mrow><mtext mathvariant="italic">ik</mtext></mrow></msub><mo>,</mo></math> </ephtml> (<reflink idref="bib3" id="ref35">3</reflink>)</p> <p>where the log odds ratio of belonging to a specific latent class <emph>k</emph> over a reference class <emph>r</emph> is a linear function of the covariates,</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>X</mi></mrow><mrow><mtext mathvariant="italic">ik</mtext></mrow></msub><mo>.</mo></math> </ephtml> </p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>k</mi></mrow></msub></math> </ephtml> is the intercept, and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="normal">Γ</mi></mrow><mrow><mi>k</mi></mrow></msub></math> </ephtml> is the regression coefficients of the covariates.</p> <hd id="AN0182192617-4">3. Multilevel Constructs</hd> <p>When data are collected in multilevel settings, the construct of interest might be conceptualized at different levels depending on the ultimate level of interest in the measurement and the intended use of the construct (Bliese, [<reflink idref="bib8" id="ref36">8</reflink>]; Stapleton et al., [<reflink idref="bib46" id="ref37">46</reflink>]). We present three different types of multilevel constructs in multilevel factor analysis model: within-level constructs, between-level constructs, and within- and between-level constructs. Their conceptualization is introduced in this section. Note that these three types of multilevel constructs might not clearly align with multilevel constructs defined and discussed in the multilevel CFA literature (e.g., Bliese, [<reflink idref="bib8" id="ref38">8</reflink>]; Kozlowski & Klein, [<reflink idref="bib26" id="ref39">26</reflink>]; Lüdtke et al., [<reflink idref="bib31" id="ref40">31</reflink>]; Stapleton et al., [<reflink idref="bib46" id="ref41">46</reflink>]). It should also be noted that there is no consensus on multilevel constructs and their terms across studies (e.g., shared construct, configural construct, reflective construct, formative construct, composition, compilation, etc.). Our propositions are based on at which level (within and/or between) applied researchers build their factor models based on their conceptualization of multilevel constructs. That is, we focus on multilevel FMM specification to demonstrate the specification of latent class variables in addition to latent factors at within and between levels.</p> <hd id="AN0182192617-5">3.1. Within-Level Constructs</hd> <p>The first scenario of multilevel CFA is the within-level construct when the individuals are the units of interest. Within-level constructs are expected to reflect individual variability and there is no assumption that a between-level construct exists. The within-level construct in multilevel data structure should not be influenced by contextual effect. For example, a measure of students' lactose intolerance (Stapleton et al., [<reflink idref="bib46" id="ref42">46</reflink>]) is a within-level construct and the observed cluster dependency, if any, is spurious. The intended future use of the measure also plays a critical role in determining if the construct in multilevel context is a within-level construct (Stapleton et al., [<reflink idref="bib46" id="ref43">46</reflink>]). If the researcher intends to use the measures to investigate the variability of the individuals in a cluster (no substantial variability across clusters), then it is more appropriate to treat the measures as within-level constructs. We adopt the latter definition of within-level constructs for multilevel FMM. Thus, the primary focus of multilevel FMM is on the classification of individuals although clusters can be categorized as we illustrate in this tutorial. Examples of within-level constructs that researchers would model at the within level given unsubstantial between-level variances include psychiatric disorder and personality traits.</p> <hd id="AN0182192617-6">3.2. Between-Level Constructs</hd> <p>The second scenario of multilevel construct is the between-level construct, typically estimated using information from individuals' responses within each cluster. Bliese ([<reflink idref="bib8" id="ref44">8</reflink>]) used the term "bottom-up process" to describe the between-level constructs that are aggregated using the within-level data. The between-level constructs may be pertinent to reflective (composition or shared) between-level constructs (Bliese, [<reflink idref="bib8" id="ref45">8</reflink>]; Kozlowski & Klein, [<reflink idref="bib26" id="ref46">26</reflink>]; Lüdtke et al., [<reflink idref="bib31" id="ref47">31</reflink>]; Stapleton et al., [<reflink idref="bib46" id="ref48">46</reflink>]). The individual-level measures are designed to provide reflective indicators to the between-level constructs, which are assumed to cause the within-level indicators (the structural relationship arrows go from the between-level construct to the within-level indicators). Individuals' score on the observed indicators in the same cluster are assumed to be isomorphic or interchangeable (Bliese, [<reflink idref="bib8" id="ref49">8</reflink>]; Lüdtke et al., [<reflink idref="bib31" id="ref50">31</reflink>]; Stapleton et al., [<reflink idref="bib46" id="ref51">46</reflink>]). Note that Bliese ([<reflink idref="bib8" id="ref52">8</reflink>]) pointed out that true isomorphism was quite rare, and partially isomorphic situation, or "fuzzy" composition processes were more common in empirical research areas. Examples of reflective between-level constructs include individual students' ratings of the classroom or school safety, or of instructor's or coach's instructional quality. The within-level construct measured by the indicators is of no interest to the researchers. The wording of designing items measuring between-level constructs is very important. For example, the statement "My teacher makes mathematics fun" is more a between-level construct than the statement "I think math is fun."</p> <p>As noted earlier, although shared or reflective constructs can be modeled with both within- and between-level latent factors (as within- and between-level constructs), our proposition of between-level constructs is more based on the intended use of measures by applied researchers and reflects research settings in which applied researchers build a CFA model at the between level with a saturated within-level model. Accordingly, the focus of specifying latent class variables is on the classification of clusters not individuals. For example, although for a theoretically between-level construct (teacher's instructional quality) a within-level construct (student perceptions of teacher's instructional quality) may be inevitable when students rate their teachers, researchers may not be interested in modeling individual perceptions of the between construct but focus on modeling teacher's instructional quality only and their associated latent classes. Then, they may build a saturated model at the within level to avoid model misspecification (due to ignoring the within-level factor of student perceptions) and build the between factor model only, which was observed in the systematic review of multilevel factor analysis (Kim et al., [<reflink idref="bib19" id="ref53">19</reflink>]). We illustrate how to build multilevel FMM in this case.</p> <hd id="AN0182192617-7">3.3. Within- and Between-Level Constructs</hd> <p>Third, the construct of interest exists at both the within level and between level. In specifying multilevel FMM, applied researchers may consider within- and between-level constructs for multilevel constructs that are termed configural (formative or compilation constructs) (Bliese, [<reflink idref="bib8" id="ref54">8</reflink>]; Kozlowski & Klein, [<reflink idref="bib26" id="ref55">26</reflink>]; Lüdtke et al., [<reflink idref="bib31" id="ref56">31</reflink>]; Stapleton et al., [<reflink idref="bib46" id="ref57">46</reflink>]) or contextual constructs (Marsh et al., [<reflink idref="bib36" id="ref58">36</reflink>]). In contrary to reflective constructs, the scores for different individuals in the same cluster are not interchangeable in configural constructs. It is very likely that the within-level and between-level aggregated variables reflect the same construct (Lüdtke et al., [<reflink idref="bib31" id="ref59">31</reflink>]). We introduce two types of within- and between-level constructs. In the first type, the between-level construct is simply a mere mirror of within-level construct. Because the constructs are the same across levels conceptually, how within-level items are related to the within-level factor (within variance covariance matrix) is assumed to be identical to how between-level items (aggregated item values) are related to the between-level factor (between variance covariance matrix). That is, we need to impose cross-level invariance (the pattern coefficients or factor loadings at within level = those at between level). An example of this type of multilevel constructs includes measuring English literacy of students who are nested in classroom (Mehta & Neale, [<reflink idref="bib38" id="ref60">38</reflink>]). The latent factor of English literacy can be conceptualized at both the student and classroom levels. This type of construct in multilevel context can be modeled at both within and between levels, and accordingly both clustering of individuals based on the within-level construct and clustering of organizations based on the between-level construct can be investigated.</p> <p>In the second type of within- and between-level constructs, the concept of constructs at the between level is different from the constructs that are conceptualized at the within level (e.g., bullying at the within level and school climate at the between level). Based on Bliese ([<reflink idref="bib8" id="ref61">8</reflink>]), the aggregated between-level latent variable measures some phenomenon that is not evident at the within level.</p> <hd id="AN0182192617-8">4. Multilevel FMM Model Specification for Multilevel Constructs</hd> <p>The model specification of the multilevel factor model and multilevel latent class variable in multilevel FMM varies considerably depending on the type of multilevel constructs and the choice of how to model the heterogeneity of individuals or clusters or both. Compared to single-level FMM, in which the observations are independent and individuals are grouped into latent classes based on their probability of belonging to a specific latent class, a complication arises when it comes to the modeling of the latent classes in multilevel FMM. That is, latent classes in multilevel FMM can occur at within, between, or both within and between levels, depending on the different types of multilevel constructs and the hypothesis of modeling the variability of the within-level latent classes at the between level. This section details the model specification in multilevel FMM for within-level constructs, between-level constructs, and within- and between-level constructs with the assistance of figures and equations. Table 1 summarizes the within-level and between-level models of different types of multilevel constructs. For within-level constructs, the within-level factors and latent classes are specified, and there are two different approaches to modeling the within-level probabilities at the between level: using a between-level latent class variable to group clusters based on within-level latent class probabilities (nonparametric), and specifying within-level latent class probabilities as a random effect at the between level (parametric). For between-level construct, there is no within-level factor model or latent class variable but between-level latent factor and latent class variable. For within- and between-level construct, both within-level and between-level factor models are specified, and there are nonparametric and parametric approaches to modeling the within-level latent class probabilities. Also, there can be another between-level latent class variable defined by the between-level factor model. The detailed model specification of each type of multilevel constructs are as follows.</p> <p>Table 1. Summary of multilevel FMM specifications by type of multilevel construct.</p> <p> <ephtml> <table><thead><tr><td>Construct</td><td>Within-level model</td><td>Between-level model</td></tr><tr><td>Latent factor</td><td>Latent class</td><td>Latent factor</td><td>Latent class based on within-level latent class probabilities (non-parametric)</td><td>Random effect of within-level latent class probabilities (parametric)</td><td>Latent class based on between-level factor</td></tr></thead><tbody valign="top"><tr><td>Within Construct</td><td>X</td><td>X</td><td /><td>X</td><td /><td /></tr><tr><td>X</td><td>X</td><td /><td /><td>X</td><td /></tr><tr><td>Between Construct</td><td /><td /><td>X</td><td /><td /><td>X</td></tr><tr><td>Within- and Between Construct</td><td>X</td><td>X</td><td>X</td><td>X</td><td /><td /></tr><tr><td>X</td><td>X</td><td>X</td><td /><td>X</td><td /></tr><tr><td>X</td><td>X</td><td>X</td><td>X</td><td /><td>X</td></tr><tr><td>X</td><td>X</td><td>X</td><td /><td>X</td><td>X</td></tr></tbody></table> </ephtml> </p> <p>1 <emph>Note</emph>. X indicates the presence of the component for model specification.</p> <hd id="AN0182192617-9">4.1. Multilevel FMM Model Specification for Within-Level Constructs</hd> <p>To model the within-level construct in multilevel context, the CFA measurement model, in which the observed items are specified to load on their respective latent factors, is only specified at the within level as shown in Equation 4, and there is no measurement model at the between level.</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="true">[</mo><mrow><msub><mrow><mi>y</mi></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub></mrow><mo stretchy="true">|</mo><msub><mrow><mi>C</mi></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub><mo>=</mo><mi>c</mi><mo>]</mo><mo>=</mo><msub><mrow><mi>α</mi></mrow><mrow><mi>c</mi></mrow></msub><mo>+</mo><msub><mrow><mi mathvariant="normal">Λ</mi></mrow><mrow><mi>c</mi></mrow></msub><msub><mrow><mi>η</mi></mrow><mrow><mtext mathvariant="italic">cij</mtext></mrow></msub><mo>+</mo><msub><mrow><mi>ϵ</mi></mrow><mrow><mtext mathvariant="italic">cij</mtext></mrow></msub><mo>,</mo></math> </ephtml> (<reflink idref="bib4" id="ref62">4</reflink>)</p> <p>where the observed indicator</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>y</mi></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub></math> </ephtml> of person <emph>i</emph> in cluster <emph>j</emph> (<emph>j</emph> = 1, 2, ..., <emph>J</emph>) is a linear function of the latent class specific intercept, factor loadings (</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>α</mi></mrow><mrow><mi>c</mi></mrow></msub></math> </ephtml> and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="normal">Λ</mi></mrow><mrow><mi>c</mi></mrow></msub></math> </ephtml> ), and latent factor</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>η</mi></mrow><mrow><mtext mathvariant="italic">cij</mtext></mrow></msub><mo>,</mo></math> </ephtml> and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>ϵ</mi></mrow><mrow><mtext mathvariant="italic">cij</mtext></mrow></msub></math> </ephtml> denotes the residuals.</p> <p>The within-level latent class variable (CW) is defined by the parameter(s) of the within-level CFA model, for example, factor means, factor variances and covariances, item intercepts, and loadings. The latent class membership of each participant is unknown beforehand, and multilevel FMM predicts the probability of belonging to each within-level latent class <emph>k</emph> for each individual using multinomial regression as shown in Equation 5.</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><mtext mathvariant="normal">ln</mtext></mrow><mo /><mrow><mo stretchy="true">[</mo><mrow><mfrac><mrow><mi>P</mi><mo stretchy="true">(</mo><mrow><msub><mrow><mtext mathvariant="italic">CW</mtext></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub><mo>=</mo><mi>k</mi></mrow><mo>|</mo><mrow><msub><mrow><mi>X</mi></mrow><mrow><mtext mathvariant="italic">ijk</mtext></mrow></msub></mrow><mo stretchy="true">)</mo></mrow><mrow><mi>P</mi><mo stretchy="true">(</mo><mrow><msub><mrow><mtext mathvariant="italic">CW</mtext></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub><mo>=</mo><mi>r</mi></mrow><mo>|</mo><mrow><msub><mrow><mi>X</mi></mrow><mrow><mtext mathvariant="italic">ijk</mtext></mrow></msub></mrow><mo stretchy="true">)</mo></mrow></mfrac></mrow><mo stretchy="true">]</mo></mrow></mrow><mo>=</mo><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub><mo>+</mo><msub><mrow><mi mathvariant="normal">Γ</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub><msub><mrow><mi>X</mi></mrow><mrow><mtext mathvariant="italic">ijk</mtext></mrow></msub><mo>,</mo></math> </ephtml> (<reflink idref="bib5" id="ref63">5</reflink>)</p> <p>where</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub></math> </ephtml> represents the latent class specific intercept of cluster <emph>j</emph>, and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="normal">Γ</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub></math> </ephtml> denotes the regression coefficient of the covariate</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>X</mi></mrow><mrow><mtext mathvariant="italic">ijk</mtext></mrow></msub><mo>.</mo></math> </ephtml> </p> <p>The class probability of each participant, as well as latent class specific measurement parameters were estimated during the model estimation. For example, if the within-level factor mean is specified to vary across latent classes, the estimate of the within-level factor mean is computed for each latent class. Also, all the participants are grouped in their specific latent class based on the calculated probability of class membership. For simplicity in this tutorial, we do not employ any covariates (</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>X</mi></mrow><mrow><mtext mathvariant="italic">ijk</mtext></mrow></msub></math> </ephtml> ) to predict latent class membership. Moreover, we assume that the random probabilities or log odds (</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub></math> </ephtml> ) is the only random effect (varying across <emph>J</emph> clusters) that will be further modeled at the between level as explained below.</p> <p>The log odds of belonging to the within-level class <emph>k</emph> over the within-level reference class <emph>r</emph> (</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub></math> </ephtml> in Equation 5 or CW in Figure 1) varies across clusters as noted by the subscript <emph>j</emph> and this random effect is modeled at the between level. There are two approaches to specifying a random effect of within-level probabilities at the between level: parametric and nonparametric.</p> <p>Graph: Figure 1. Multilevel FMM for within-level constructs (parametric).</p> <hd id="AN0182192617-10">4.1.1. Parametric Approach</hd> <p>As illustrated in the studies about multilevel LCA model (Finch & French, [<reflink idref="bib12" id="ref64">12</reflink>]; Henry & Muthén, [<reflink idref="bib16" id="ref65">16</reflink>]; Vermunt, [<reflink idref="bib48" id="ref66">48</reflink>]) and multilevel LPA models (Mäkikangas et al., [<reflink idref="bib35" id="ref67">35</reflink>]),</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub></math> </ephtml> can be modeled as a continuous random effect at the between level, which is termed as the parametric approach as shown in Figure 1. Note that when <emph>cw</emph> represents the number of within-level latent classes, there are <emph>cw</emph> − 1 random probabilities (</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub><mo>;</mo></math> </ephtml> </p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mi>k</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mo>...</mo><mo>,</mo><mtext mathvariant="italic">cw</mtext><mo>−</mo><mn>1</mn></math> </ephtml> ). For example, if there are three within-level latent classes (1 to 3 with class 3 as a reference class), two random probabilities (probability of belonging to class 1 over class 3; probability of belonging to class 2 over class 3) are specified at the between level. For the</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>k</mi></mrow></math> </ephtml> th random log odds (or probability),</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub><mo>=</mo><msub><mrow><mi>γ</mi></mrow><mrow><mn>0</mn><mi>k</mi></mrow></msub><mo>+</mo><msub><mrow><mi>γ</mi></mrow><mrow><mn>1</mn><mi>k</mi></mrow></msub><msub><mrow><mi>w</mi></mrow><mrow><mi>j</mi></mrow></msub><mo>+</mo><msub><mrow><mi mathvariant="bold-italic">u</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub><mo>,</mo><mo /><msub><mrow><mi mathvariant="bold-italic">u</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub><mo>∼</mo><mi>N</mi><mo>(</mo><mn>0</mn><mo>,</mo><mi mathvariant="bold">Ω</mi><mo>)</mo></math> </ephtml> (<reflink idref="bib6" id="ref68">6</reflink>)</p> <p>where</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="normal" /><msub><mrow><mi>γ</mi></mrow><mrow><mn>0</mn><mi>k</mi></mrow></msub></math> </ephtml> is the intercept of</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub></math> </ephtml> and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>γ</mi></mrow><mrow><mn>1</mn><mi>k</mi></mrow></msub></math> </ephtml> is the regression coefficient of the between-level covariate</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>w</mi></mrow><mrow><mi>j</mi></mrow></msub><mo>.</mo></math> </ephtml> For simplicity in this tutorial, we do not employ any between-level covariates (</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>w</mi></mrow><mrow><mi>j</mi></mrow></msub></math> </ephtml> ), and the mean of</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub></math> </ephtml> will be estimated at the between level. The residuals</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold-italic">u</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub></math> </ephtml> are assumed to be normally distributed with variances and covariances (among <emph>cw</emph> − 1 random log odds)</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi mathvariant="bold">Ω</mi></mrow><mo>.</mo></math> </ephtml> </p> <p>This multilevel FMM is illustrated in Figure 1. According to Figure 1, in the within-level model the filled dot represents the random effect of the within-level latent classes across clusters. There are <emph>cw</emph> − 1 random log odds (</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub></math> </ephtml> ), where <emph>cw</emph> equals the number of within-level latent classes, which are specified at the between level as illustrated with ovals (e.g., C#1, C#2, ..., C#(<emph>cw</emph>-1)) in Figure 1. When there are three or more latent classes, the <emph>cw</emph> − 1 random probabilities are correlated with one another. There is no between-level latent class variable. Note that there could be covariates at both the within and between level that predicts the within-level latent class variable and the random means at the between level, respectively, and they are omitted in the Figure 1 for simplicity. As shown by Equation 6, the between-level predictors (i.e.,</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>w</mi></mrow><mrow><mi>j</mi></mrow></msub></math> </ephtml> ) can be easily incorporated into this model to predict the random probability,</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mtext mathvariant="italic">jk</mtext></mrow></msub><mo>.</mo></math> </ephtml> For example, researchers can use a school characteristics variable (e.g., school size) to predict the proportions of students classified into different within-latent classes of emotional behaviors for the between-level clusters (i.e., schools).</p> <hd id="AN0182192617-11">4.1.2. Nonparametric Approach</hd> <p>The parametric approach assumes that random log odds are normally distributed, while the nonparametric approach assumes a multinomial distribution (Vermunt, [<reflink idref="bib49" id="ref69">49</reflink>]). The nonparametric approach to specifying the between-level differences in probabilities of students in different within-level latent classes is to use the between-level latent class variable. To be more specific, the between-level units (e.g., schools or clinics) can be classified into latent classes based on the within-level random probabilities, and this is called the nonparametric approach. For example, some schools have higher proportion of at-risk latent class of emotional behaviors students, while others have lower proportion of at-risk latent class. Using a between-level latent class variable, the schools with similar proportions of at-risk latent class students are classified into the same between-level latent class. Therefore, the between-level latent classes may be defined by two latent classes: one that represents schools where students have higher probability of belonging to the at-risk latent class and one that represents schools where students have a higher probability of belonging to the low-risk latent class. At the between-level model of multilevel FMM, the within-level random effect is specified to regress on the between-level latent class variable.</p> <p>For the nonparametric approach as shown in Figure 2, there are two latent class variables in this multilevel FMM model: the within-level latent classes variable, CW, and the between-level latent class variable, CB. The <emph>cw</emph> − 1 random probabilities for the within-level latent class solutions are used as indicators of the between-level latent class model. Different between-level latent classes have varying distributions of the random probabilities (the probability of belonging to a particular within-level latent class). The within-level latent class solution can be defined as:</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mi>P</mi><mo stretchy="true">(</mo><mrow><msub><mrow><mtext mathvariant="italic">CW</mtext></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub><mo>=</mo><mi>l</mi><mo stretchy="false">|</mo></mrow><mrow><msub><mrow><mtext mathvariant="italic">CB</mtext></mrow><mrow><mi>j</mi></mrow></msub><mo>=</mo><mi>m</mi></mrow><mo stretchy="true">)</mo><mo>=</mo><mfrac><mrow><mtext mathvariant="normal">exp</mtext><mo>⁡</mo><mo>(</mo><msub><mrow><mi>γ</mi></mrow><mrow><mtext mathvariant="italic">lm</mtext></mrow></msub><mo>)</mo></mrow><mrow><mrow><munderover><mo stretchy="false">∑</mo><mrow><mi>r</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>K</mi></mrow></munderover><mrow><mtext mathvariant="normal">exp</mtext><mo>⁡</mo><mo>(</mo><msub><mrow><mi>γ</mi></mrow><mrow><mtext mathvariant="italic">rm</mtext></mrow></msub><mo>)</mo></mrow></mrow></mrow></mfrac><mo>,</mo></math> </ephtml> (<reflink idref="bib7" id="ref70">7</reflink>)</p> <p>where</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mtext mathvariant="italic">CB</mtext></mrow><mrow><mi>j</mi></mrow></msub></math> </ephtml> represents cluster <emph>j</emph>'s score on the between-level latent class variable, <emph>m</emph> denotes a particular between-level latent class or mixture,</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>γ</mi></mrow><mrow><mtext mathvariant="italic">lm</mtext></mrow></msub></math> </ephtml> represents the intercept term of the linear predictor of the logit model for the latent class probabilities.</p> <p>Graph: Figure 2. Multilevel FMM for within-level constructs (nonparametric).</p> <hd id="AN0182192617-12">4.2. Multilevel FMM Model Specification for Between-Level Constructs</hd> <p>Given that the characteristics of the clusters are of interest when researchers work with between-level constructs, we propose the corresponding specification of multilevel FMM of the between-level constructs as shown in Figure 3. That is, a CFA model is included at the between level: the between-level latent factor is measured by the five items that are aggregated at the between level denoted in ovals. The between-level latent class variable CB is specified to predict the between-level latent factor score. That is, CB is defined by the between-level latent factor scores. Note that CB can also be identified based on the heterogeneity in other measurement parameters in the between-level CFA model (e.g., item intercepts, factor loadings). At the within level, the five observed items denoted in squares are correlated with each other to create a saturated model. With <emph>j</emph> denoting the cluster and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi></mrow></math> </ephtml> referring to the cluster's latent class membership (</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi></mrow></math> </ephtml> = 1, 2, ..., <emph>G</emph>), the between-level model can be expressed as:</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>Y</mi></mrow><mrow><mtext mathvariant="italic">jg</mtext></mrow></msub><mo>=</mo><msub><mrow><mi>ν</mi></mrow><mrow><mi>g</mi></mrow></msub><mo>+</mo><msub><mrow><mi mathvariant="normal">Λ</mi></mrow><mrow><mi>g</mi></mrow></msub><msub><mrow><mi>η</mi></mrow><mrow><mtext mathvariant="italic">jg</mtext></mrow></msub><mo>+</mo><msub><mrow><mi>ϵ</mi></mrow><mrow><mtext mathvariant="italic">jg</mtext></mrow></msub><mo>,</mo></math> </ephtml> (<reflink idref="bib8" id="ref71">8</reflink>)</p> <p>where</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>Y</mi></mrow><mrow><mtext mathvariant="italic">jg</mtext></mrow></msub></math> </ephtml> is the aggregated response for cluster <emph>j</emph> conditional on the latent class membership (i.e., ovals for Y1–Y5 at the between level in Figure 3).</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>ν</mi></mrow><mrow><mi>g</mi></mrow></msub></math> </ephtml> and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="normal">Λ</mi></mrow><mrow><mi>g</mi></mrow></msub></math> </ephtml> represent class-specific intercepts and factor loadings, respectively.</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>η</mi></mrow><mrow><mtext mathvariant="italic">jg</mtext></mrow></msub></math> </ephtml> and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>ϵ</mi></mrow><mrow><mtext mathvariant="italic">jg</mtext></mrow></msub></math> </ephtml> refer to the factor score and residual for a given cluster conditional on the latent class membership.</p> <p>Graph: Figure 3. Multilevel FMM for between-level constructs.</p> <p>The log odds of a particular cluster belonging to a</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi></mrow></math> </ephtml> latent class can be expressed as</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><mtext mathvariant="normal">ln</mtext></mrow><mo /><mrow><mo stretchy="true">[</mo><mrow><mfrac><mrow><mi>P</mi><mo stretchy="true">(</mo><mrow><msub><mrow><mtext mathvariant="italic">CB</mtext></mrow><mrow><mi>j</mi></mrow></msub><mo>=</mo><mi>g</mi></mrow><mrow><mo stretchy="false">|</mo><msub><mrow><mi>X</mi></mrow><mrow><mtext mathvariant="italic">jg</mtext></mrow></msub></mrow><mo stretchy="true">)</mo></mrow><mrow><mi>P</mi><mo stretchy="true">(</mo><mrow><msub><mrow><mtext mathvariant="italic">CB</mtext></mrow><mrow><mi>j</mi></mrow></msub><mo>=</mo><mi>r</mi></mrow><mrow><mo stretchy="false">|</mo><msub><mrow><mi>X</mi></mrow><mrow><mtext mathvariant="italic">jg</mtext></mrow></msub></mrow><mo stretchy="true">)</mo></mrow></mfrac></mrow><mo stretchy="true">]</mo></mrow></mrow><mo>=</mo><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>g</mi></mrow></msub><mo>+</mo><msub><mrow><mi mathvariant="normal">Γ</mi></mrow><mrow><mi>g</mi></mrow></msub><msub><mrow><mi>X</mi></mrow><mrow><mtext mathvariant="italic">jg</mtext></mrow></msub><mo>,</mo></math> </ephtml> (<reflink idref="bib9" id="ref72">9</reflink>)</p> <p>where the log odds of the probability of belonging to a specific between-level latent class</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi></mrow></math> </ephtml> over a reference class <emph>r</emph> is a function of the covariates</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>X</mi></mrow><mrow><mtext mathvariant="italic">jg</mtext></mrow></msub><mo>.</mo></math> </ephtml> </p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>g</mi></mrow></msub></math> </ephtml> is the intercept, and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="normal">Γ</mi></mrow><mrow><mi>g</mi></mrow></msub></math> </ephtml> is the regression coefficients of the covariates. For simplicity, covariates are not included in Figure 3.</p> <hd id="AN0182192617-13">4.3. Multilevel FMM Model Specification for Within- and Between-Level Constructs</hd> <p>When the constructs are conceptualized at both the within and between levels, a CFA model is specified at both the within and between levels. As discussed extensively in the multilevel CFA literature, the within- and between-level constructs can be specified in different ways depending on how the constructs are conceptualized and accordingly latent classes can be examined differently. We discuss model specification of different types of within- and between-level constructs and their corresponding strategies of classification to classify organizations as well as individuals (classification of units of analysis at different levels of data).</p> <p>For the within- and between-level constructs that measure the same phenomenon at both levels, we need to impose cross-level loading invariance. In terms of latent classes, we can examine latent classes at the within level with within factor (individual classification) and latent classes based on the between factor (organization classification). Thus, we need to specify two types of latent classes (within-level latent classes or CW and between-level latent classes or CB) each associated with measurement model at each level as shown in Figure 4. The within-level latent class variable CW can be defined by the factor scores of the within-level common factor model. At the between-level there can be one or more latent class variables. One latent class variable (CB1) can be defined by the differences in the parameter values of the between-level CFA model. In addition, the variability of the within-level random probability can be modeled using either the parametric method without specifying another between-level latent class variable or the nonparametric method by specifying an additional between-level latent class variable defined by the within-level random log odds (CB2) as shown in Figure 4. The nonparametric model in Figure 4 renders it not only possible to categorizes individuals into different latent classes of CW, but also possible to categorize the between-level units into between-level latent classes defined by the between-level factor score, CB1, and between-level latent classes defined by the within-level random probabilities, CB2. The incorporation of both CB1 and CB2 for within- and between-level construct is the unique feature of multilevel FMM. The within- and between-level CFA model as shown in Figure 4 can be expressed as:</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>y</mi></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub><mo>=</mo><mi>α</mi><mo>+</mo><msub><mrow><mi mathvariant="normal">Λ</mi></mrow><mrow><mi>W</mi></mrow></msub><msub><mrow><mi>η</mi></mrow><mrow><mtext mathvariant="italic">Wij</mtext></mrow></msub><mo>+</mo><msub><mrow><mi>ε</mi></mrow><mrow><mtext mathvariant="italic">Wij</mtext></mrow></msub><mo>+</mo><msub><mrow><mi mathvariant="normal">Λ</mi></mrow><mrow><mi>B</mi></mrow></msub><msub><mrow><mi>η</mi></mrow><mrow><mtext mathvariant="italic">Bj</mtext></mrow></msub><mo>+</mo><msub><mrow><mi>ε</mi></mrow><mrow><mtext mathvariant="italic">Bj</mtext></mrow></msub></math> </ephtml> (<reflink idref="bib10" id="ref73">10</reflink>)</p> <p>where</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>α</mi></mrow></math> </ephtml> denotes the intercept;</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="normal">Λ</mi></mrow><mrow><mi>W</mi></mrow></msub></math> </ephtml> and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="normal">Λ</mi></mrow><mrow><mi>B</mi></mrow></msub></math> </ephtml> correspond to within- and between-cluster factor loadings;</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>η</mi></mrow><mrow><mtext mathvariant="italic">Wij</mtext></mrow></msub></math> </ephtml> and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>η</mi></mrow><mrow><mtext mathvariant="italic">Bj</mtext></mrow></msub></math> </ephtml> represent within- and between-cluster factors;</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>ε</mi></mrow><mrow><mtext mathvariant="italic">Wij</mtext></mrow></msub></math> </ephtml> and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>ε</mi></mrow><mrow><mtext mathvariant="italic">Bj</mtext></mrow></msub></math> </ephtml> indicate the residuals of the within and between levels, respectively. The residuals at each level are assumed to be normally distributed with a mean of zero and not correlated with the factors or residuals at the other level.</p> <p>Graph: Figure 4. Multilevel FMM for within- and between-level constructs with nonparametric approach.</p> <p>For the within- and between-level constructs that measure different phenomena at the within and between levels, cross-level factor loadings invariance may not be assumed although cross-level invariance ensures metric invariance across clusters and facilitates interpretations (e.g., Jak et al., [<reflink idref="bib17" id="ref74">17</reflink>]). Thus, the latent classes that emerge from each construct will not be comparable conceptually, not to mention different units are classified at each level (individuals classified vs. clusters classified). In this case, three latent class variables (one within-level latent class variable and two between-level latent class variables) can be specified. To be more specific, one within-level latent class variable that is defined by factor mean difference among latent classes, one between-level latent class variable is based on between-level factor mean difference, and the other between-level latent class variable is based on the random probabilities of the within-level latent class variable. Note that the second between-level latent class variable is a nonparametric method as discussed above. Instead of explicitly modeling between-level latent classes based on random probabilities, the random probabilities can be estimated as a random effect that varies across clusters, which is termed the parametric approach as discussed above.</p> <hd id="AN0182192617-14">5. Model Selection Criteria</hd> <p>To determine the number of latent classes underlying the data in mixture modeling, researchers rely on fit indices to select the best fitting model among a series of plausible models that specify different numbers of latent classes. The most frequently used fit indices are the information criterion (ICs), including Akaike Information Criterion (AIC; Akaike, [<reflink idref="bib1" id="ref75">1</reflink>]), Bayesian Information Criterion (BIC; Schwarz, [<reflink idref="bib44" id="ref76">44</reflink>]), and sample size adjusted BIC (SaBIC; Sclove, [<reflink idref="bib45" id="ref77">45</reflink>]). Also, because BIC tends to select the more parsimonious model, or underestimate the number of latent classes while AIC tends to overestimate the number of latent classes, AIC3 (Bozdogan, [<reflink idref="bib10" id="ref78">10</reflink>]) has been recommended as a compromise. Thus, AIC3 is also included for model comparison. Some previous studies have examined the performance of fit indices in detecting the mixture model with the correct number of latent classes (e.g., Gao et al., [<reflink idref="bib13" id="ref79">13</reflink>]; Nylund et al., [<reflink idref="bib41" id="ref80">41</reflink>]); however, there has been no unanimous agreement on the most effective fit indices in model selection. The AIC, AIC3, BIC, and SaBIC are defined respectively as:</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mtext mathvariant="italic">AIC</mtext><mo>=</mo><mo>–</mo><mn>2</mn><mrow><mrow><mtext mathvariant="normal">log</mtext></mrow><mo /><mrow><mi>L</mi></mrow></mrow><mo>+</mo><mn>2</mn><mi>p</mi><mo>,</mo></math> </ephtml> (<reflink idref="bib11" id="ref81">11</reflink>)</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mtext mathvariant="italic">AIC</mtext><mn>3</mn><mo>=</mo><mo>–</mo><mn>2</mn><mrow><mrow><mtext mathvariant="normal">log</mtext></mrow><mo /><mrow><mi>L</mi></mrow></mrow><mo>+</mo><mn>3</mn><mi>p</mi><mo>,</mo></math> </ephtml> (<reflink idref="bib12" id="ref82">12</reflink>)</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mtext mathvariant="italic">BIC</mtext><mo>=</mo><mo>–</mo><mn>2</mn><mrow><mrow><mtext mathvariant="normal">log</mtext></mrow><mo /><mrow><mi>L</mi></mrow></mrow><mo>+</mo><mi>p</mi><mi mathvariant="normal">*</mi><mtext mathvariant="normal">ln</mtext><mo stretchy="true">(</mo><mrow><mi>n</mi></mrow><mo stretchy="true">)</mo><mo>,</mo></math> </ephtml> (<reflink idref="bib13" id="ref83">13</reflink>)</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mtext mathvariant="italic">SaBIC</mtext><mo>=</mo><mo>–</mo><mn>2</mn><mrow><mrow><mtext mathvariant="normal">log</mtext></mrow><mo /><mrow><mi>L</mi></mrow></mrow><mo>+</mo><mi>p</mi><mi mathvariant="normal">*</mi><mtext mathvariant="normal">ln</mtext><mo stretchy="true">(</mo><mrow><mfrac><mrow><mi>n</mi><mo>+</mo><mn>2</mn></mrow><mrow><mn>24</mn></mrow></mfrac></mrow><mo stretchy="true">)</mo><mo>,</mo></math> </ephtml> (<reflink idref="bib14" id="ref84">14</reflink>)</p> <p>where log<emph>L</emph> is the log likelihood of a fitted model, <emph>p</emph> is the number of free parameters, and <emph>n</emph> is the sample size. ICs can be used to compare the fit of mixture modeling, with a lower value associated with a better model. Compared to AIC, BIC imposes a harsher penalty for the number of parameters estimated in the models, favoring the simpler model (e.g., Vrieze, [<reflink idref="bib51" id="ref85">51</reflink>]). SaBIC imposes a less harsh penalty for model complexity compared to BIC by including an adjusted term of sample size.</p> <p>In multilevel FMM, when the constructs of interest are at the between level or at both within- and between-level, the corresponding ICs (i.e., BIC and SaBIC, because they have sample size in their computation) with the number of clusters (NC) as a sample size, that is, BIC(NC) and SaBIC(NC) can also be calculated to select the best fitting models in multilevel mixture modeling suggested by previous studies (Kim & Wang, [<reflink idref="bib22" id="ref86">22</reflink>]; Lukočienė et al., [<reflink idref="bib30" id="ref87">30</reflink>]). BIC(NC) and SaBIC (NC) performed comparably to other ICs (Kim & Wang, [<reflink idref="bib22" id="ref88">22</reflink>]) or outperformed BIC and SaBIC in the context of multilevel mixture models (Lukočienė et al., [<reflink idref="bib30" id="ref89">30</reflink>]).</p> <p>In addition to ICs, log likelihood-based tests are available for model selection, including the Lo-Mendell-Rubin (LMR) test and adjusted LMR test (aLMR; Lo et al., [<reflink idref="bib29" id="ref90">29</reflink>]), and bootstrap likelihood ratio test (BLRT; McLachlan & Peel, [<reflink idref="bib37" id="ref91">37</reflink>]). All these tests compare the fit of a <emph>k</emph>-class model and a (<emph>k</emph> − 1)-class model and a significant result (i.e., <emph>p</emph> <.05) suggests that the <emph>k</emph>-class model has significantly better fit. Note that these tests for model selection in multilevel FMM is not available when there is more than one latent class variable (e.g., when there is a latent class variable at each level). Entropy has often been reported as a measure of classification quality. Entropy ranges from 0 to 1 and higher values indicate better classification quality.</p> <hd id="AN0182192617-15">6. Illustrations of Multilevel FMM Using Empirical Data Sets</hd> <p>The illustrations using empirical data sets are organized based on different types of multilevel constructs: the within-level constructs, between-level constructs, and within- and between-level constructs. Note that the datasets have been selected for demonstration purposes only. Applied researchers may choose to specify their multilevel FMM models illustrated in this tutorial based on their specific research questions and the ultimate goal of the use of the results.</p> <hd id="AN0182192617-16">6.1. Using Multilevel FMM to Illustrate Within-Level Constructs</hd> <p></p> <hd id="AN0182192617-17">6.1.1 Participants and Variables</hd> <p>The demonstration data for the within-level construct is the TIMSS 2019 data, specifically, the items of 8<sups>th</sups> grade students' mathematics self-efficacy in South Korea. The data set consisted of 168 South Korean schools in the TIMSS 2019 survey. The total sample size of 8<sups>th</sups> graders in Korea was 3,815. The average number of participating students per school was 22.71 (min = 5; max = 31). The four items of students' perception of mathematics self-efficacy (BSBM19A, BSBM19D, BSBM19F, BSBM19G) were used for demonstrating the within-level construct in multilevel context using multilevel FMM. Note that these variables were treated as continuous in data analysis of multilevel FMM. Because categorical data analytic procedures may produce unstable and improper solutions when the fitted model is overly complex like multilevel FMM (Lubke & Neale, [<reflink idref="bib34" id="ref92">34</reflink>]; Marsh et al., [<reflink idref="bib36" id="ref93">36</reflink>]), treating variables with four response categories as continuous is justifiable (Beauducel & Herzberg, [<reflink idref="bib6" id="ref94">6</reflink>]; Nagengast & Marsh, [<reflink idref="bib40" id="ref95">40</reflink>]). The list of four items and their descriptive statistics are available in Table 2. The intraclass correlation coefficients (ICCs) were.029,.023,.040, and.040 for the four items, respectively (design effects from 1.5 to 1.8).</p> <p>Table 2. Descriptive statistics of the four items about students' self-efficacy of mathematics.</p> <p> <ephtml> <table><thead><tr><td>Item</td><td>Mean</td><td>SD</td></tr></thead><tbody valign="top"><tr><td>I usually do well in mathematics</td><td>2.51</td><td>0.92</td></tr><tr><td>I learn quickly in mathematics</td><td>2.43</td><td>0.84</td></tr><tr><td>I am good at working out difficult mathematics problems</td><td>2.75</td><td>0.86</td></tr><tr><td>My teacher tells me I am good at mathematics</td><td>2.69</td><td>0.87</td></tr></tbody></table> </ephtml> </p> <p>2 <emph>Note.</emph> SD = standard deviation.</p> <hd id="AN0182192617-18">6.1.2 Fitted Models and Results</hd> <p>The four items were specified to measure the latent factor of self-efficacy at the within level only. For the nonparametric method of modeling within-level construct using multilevel FMM, there were two latent class variables: the within-level latent classes variable, CW, and the between-level latent class variable, CB. The within-level latent class variable CW was defined by the within-level factor mean difference between the within-level latent classes. The between-level latent class was expressed as the different probabilities of belonging to the within-level latent classes. A series of models with different number of latent classes were fitted. To be more specific, five nonparametric models with different number of latent classes for the between-level latent class and within-level latent class were run: 1 CW and 1 CB, 2 CW and 1 CB, 2 CW and 2 CB, 2 CW and 3 CB, and 3 CW and 1 CB. Note that when three within-level latent classes (CW) were specified, one of the latent classes had a proportion of 0, thus, models with more than three within-level latent classes were not explored.</p> <p>Based on AIC3, BIC, and SaBIC, the model with 2 CW and 2 CB fitted the data best. The model comparison results using the criteria of fit indices are in Table 3. The model results of the best fitted model showed that for the two within-level latent classes, Class 1 had a factor mean of 1.16, while Class 2 had a factor mean of −0.13. Thus, Class 1 was the higher self-efficacy students, and Class 2 was the lower self-efficacy students. The probabilities of within-level latent Class 1 of the higher math self-efficacy and Class 2 of the lower self-efficacy were 72% (the red and green lines combined) and 28% (the blue and purple lines combined), respectively, as shown in Figure 5. The probabilities for the two between-level latent classes were 53% and 47%, respectively, as shown in Figure 6. CB#1 (47%) was the high self-efficacy schools that comprised students who mostly had high mathematics self-efficacy. CB#2 (53%) may be regarded as the mixed self-efficacy schools, with almost half higher self-efficacy students and half lower self-efficacy students. Figure 5 shows the two between-level latent classes based on within-level latent class probabilities. The red line (27.9)% represented CB#1 CW#1 (higher self-efficacy schools and higher self-efficacy students), the blue line (19.0%) CB#1 CW#2 (higher self-efficacy schools and lower self-efficacy students), the green line (44.3%) CB#2 CW#1 (mixed self-efficacy schools and higher self-efficacy students), and the purple line (8.8%) CB#2 CW#2 (mixed self-efficacy schools and lower self-efficacy students). Note that there are only two distinctive lines for CW#1 and CW#2 because only within classes are determined based on item response patterns. In other words, CB#1 CW#1 and CB2#1 CW#1 share the same response patterns at the within level and CB#1 and CB#2 are determined based on within-level latent class probabilities but not based on response patterns.</p> <p>PHOTO (COLOR): Figure 5. The proportion for the latent classes based on posterior probabilities using nonparametric multilevel FMM for within-level constructs.</p> <p>PHOTO (COLOR): Figure 6. The proportion for the between-level latent classes based on posterior probabilities using nonparametric multilevel FMM for within-level constructs.</p> <p>Table 3. Summary of model fit comparisons for multilevel FMM with within-level constructs.</p> <p> <ephtml> <table><thead><tr><td>CW</td><td>CB</td><td>p</td><td>AIC</td><td>AIC3</td><td>BIC</td><td>saBIC</td><td>Entropy</td><td>Class Proportions</td></tr></thead><tbody valign="top"><tr><td>1</td><td>1</td><td>12</td><td>29647</td><td>29659</td><td char=".">29721</td><td char=".">29683</td><td>NA</td><td>1.00</td></tr><tr><td>2</td><td>1</td><td>15</td><td>29579</td><td>29594</td><td char=".">29673.</td><td char=".">29625</td><td char=".">.51</td><td>.72/.28</td></tr><tr><td>2</td><td>2</td><td>17</td><td>29546</td><td><bold>29564</bold></td><td char="."><bold>29653.</bold></td><td char="."><bold>29599</bold></td><td char=".">.46</td><td>.28/19/.44/.09</td></tr><tr><td>2</td><td>3</td><td>19</td><td><bold>29544</bold></td><td>29563</td><td char=".">29662</td><td char=".">29602</td><td char=".">.58</td><td>.53/.17/.12/.11/.07/0</td></tr><tr><td>3</td><td>1</td><td>17</td><td>29583</td><td>29600</td><td char=".">29689</td><td char=".">29636</td><td char=".">.53</td><td>.19/.09/.72</td></tr></tbody></table> </ephtml> </p> <p>3 <emph>Note.</emph> CW = within-level latent class; CB = between-level latent class; p = number of free parameters.</p> <p>The parametric method allows the within-level latent class mean to vary across the clusters instead of specifying a discrete between-level latent class as in the nonparametric method. The results of the three within-level latent class model showed the best model fit with the lowest AIC, BIC, and SaBIC compared to the models with 1, 2, and 4 latent classes. The results showed that within-level Class 1 had a factor mean of zero, accounting for 19.65% of the sample. Class 2 had a higher factor mean of 2.21, accounting for 19.72% of the sample. Class 3 had a factor mean of 1.20, accounting for 60.63% of the sample. At the between level, the log odds of belonging to within-level Class 1 and Class 2 over the reference Class 3 had a variance of.30 and.31 across the clusters. The plot of sample means of the parametric method was presented in Figure 7. Note that the produced numbers of latent classes in the nonparametric approach (two within-level latent classes and two between-level latent classes) and in the parametric approach (3 within-level latent classes) were different, which will discussed further in the Discussion section.</p> <p>PHOTO (COLOR): Figure 7. The proportion for the latent classes based on posterior probabilities using parametric multilevel FMM for within-level constructs.</p> <hd id="AN0182192617-19">6.2. Using Multilevel FMM to Illustrate Between-Level Constructs</hd> <p></p> <hd id="AN0182192617-20">6.2.1. Participants and Variables</hd> <p>To demonstrate the modeling of between-level constructs using multilevel FMM, we used TIMSS 2019 data, specifically, the items of 8<sups>th</sups> grade students' perceptions of their physics teacher's instructional quality in France. The data set consisted of 176 8<sups>th</sups> grade classrooms in the TIMSS 2019 survey. The total sample size of 8<sups>th</sups> graders in France was 3,440. The average number of participating students per classroom was 20.44 (min = 8; max = 28). The seven items of students' perception of their physics teacher (BSBP39A - BSBP39G) were used for demonstrating the between-level constructs in multilevel context using multilevel FMM. These items have four response categories (agree a lot, agree a little, disagree a little, and disagree a lot). Higher scores on each item indicated more positive perception of their physics teacher. Note that these variables were treated as continuous in data analysis of multilevel FMM. The list of seven items and their descriptive statistics are available in Table 4. The ICCs were.107,.261,.254,.250,.201,.141, and.191 for the seven items, respectively (design effects ranging from 3.0 to 6.0).</p> <p>Table 4. Descriptive statistics of the seven items about students' perception of their physics teacher.</p> <p> <ephtml> <table><thead><tr><td>Item</td><td>Mean</td><td>SD</td></tr></thead><tbody valign="top"><tr><td>I know what my teacher expects me to do</td><td>2.21</td><td>0.93</td></tr><tr><td>My teacher is easy to understand</td><td>2.20</td><td>0.95</td></tr><tr><td>My teacher has clear answers to my questions</td><td>2.12</td><td>0.94</td></tr><tr><td>My teacher is good at explaining physics</td><td>2.06</td><td>0.91</td></tr><tr><td>My teacher does a variety of things to helps us learn</td><td>2.14</td><td>0.91</td></tr><tr><td>My teacher links new lessons to what I already know</td><td>2.30</td><td>0.92</td></tr><tr><td>My teacher explains a topic again when we don't understand</td><td>1.97</td><td>0.88</td></tr></tbody></table> </ephtml> </p> <p>4 <emph>Note.</emph> SD = standard deviation.</p> <hd id="AN0182192617-21">6.2.2. Fitted Models and Results</hd> <p>We fitted multilevel FMM with</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi></mrow></math> </ephtml> = 1, 2, and 3. Because one class in the</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi></mrow></math> </ephtml> = 3 model had zero proportion, we therefore did not increase the number of classes to 4 or more. Although latent classes can be defined based on any of the between-level CFA parameters (see Equation 8), we separated classes by factor means for demonstration purposes because factor means are often of major interest to applied researchers in the interpretations of latent class solutions. However, other class-specific parameters (e.g., intercepts and factor loadings) can be allowed for in the model. Note that, the factor mean of the last class was fixed to be zero by default and thus factor means of other classes were estimated as the difference in factor mean when compared with the reference class. Model fit was compared using AIC, AIC3, BIC, SaBIC, BIC(CN), SaBIC(CN), LMR, aLMR, and BLRT as discussed earlier.</p> <p>As can be seen from Table 5, all the fit indices and BLRT consistently supported the 2-class multilevel FMM, whereas LMR and aLMR selected the 1-class model. None of the model selection criteria supported the 3-class model and this model had one class with proportion zero, so we did not consider this solution as substantively meaningful. Therefore, the 2-class model was chosen as the best-fitting model. The entropy of this model is.84, which can be interpreted as relatively high classification accuracy. The dominant class (80%) was characterized by.65 points lower factor mean than the other class (20%), and were thus labeled as <emph>less effective instruction</emph> class and <emph>more effective instruction</emph> class, respectively. This statistically significant factor mean difference between classes thus revealed the heterogeneity in instructional quality across classrooms: a relatively small proportion of physics teachers were perceived to have higher instructional quality than the rest of physics teachers. Note that this finding can be further investigated with predictors or distal outcomes of latent class membership but is beyond the scope of this study.</p> <p>Table 5. Summary of model fit comparisons for multilevel FMM with between-level constructs.</p> <p> <ephtml> <table><thead><tr><td>K</td><td>p</td><td>AIC</td><td>AIC3</td><td>BIC(N)</td><td>saBIC(N)</td><td>BIC(CN)</td><td>saBIC(CN)</td><td>Entropy</td><td>LMR</td><td>aLMR</td><td>BLRT</td><td>Class Proportions</td></tr></thead><tbody valign="top"><tr><td>1</td><td>42</td><td>44183</td><td>44225</td><td>44441</td><td>44307</td><td>44316</td><td>44183</td><td>—</td><td>—</td><td>—</td><td>—</td><td>1.00</td></tr><tr><td>2</td><td>44</td><td><bold>44161</bold></td><td><bold>44205</bold></td><td><bold>44432</bold></td><td><bold>44292</bold></td><td><bold>44301</bold></td><td><bold>44161</bold></td><td>.84</td><td>.1569</td><td>.1742</td><td><.001</td><td>.80/.20</td></tr><tr><td>3</td><td>46</td><td>44165</td><td>44211</td><td>44448</td><td>44301</td><td>44311</td><td>44165</td><td>.66</td><td>.3802</td><td>.3858</td><td>.3333</td><td>.79/.21/.00</td></tr></tbody></table> </ephtml> </p> <p>5 <emph>Note</emph>. LMR = Lo-Mendell-Rubin test; aLMR = adjusted Lo-Mendell-Rubin test; BLRT = bootstrap likelihood ratio test; K = number of between-level latent classes; p = number of free parameters.</p> <hd id="AN0182192617-22">6.3. Using Multilevel FMM to Illustrate Within- and Between-Level Constructs</hd> <p></p> <hd id="AN0182192617-23">6.3.1. Participants and Variables</hd> <p>To illustrate the modeling of the multilevel level constructs that exist at both within and between level, the Students Bullied at School (SBS) scale in TIMSS 2019 of Kuwait Grade 4 was used. The scale is part of the school climate measure surveying "During this year, how often have any of the following things happened to you at school?." The scale consisted of six items: (<reflink idref="bib1" id="ref96">1</reflink>) "I was made fun of or called names"; (<reflink idref="bib2" id="ref97">2</reflink>) "I was left out of games or activities by other students"; (<reflink idref="bib3" id="ref98">3</reflink>) "Someone spread lies about me"; (<reflink idref="bib4" id="ref99">4</reflink>) "Something was stolen from me"; (<reflink idref="bib5" id="ref100">5</reflink>) "I was hit or hurt by other student(s) (e.g., shoving, hitting, kicking)"; and (<reflink idref="bib6" id="ref101">6</reflink>) "I was made to do things I didn't want to do by other students." Responses were frequencies of experiencing the bullying behaviors at school, ranging from 1 (Never) to 4 (At least once a week). The data set used was for 4th graders in Kuwait, which comprised 3,680 students (level-1 unit) and 164 schools (level-2 unit). The average number of students per school was 22 (min = 8; max = 46). The item intraclass correlation (ICC) of the six items were.105,.108,.117,.080,.113, and.097, respectively (design effects from 2.7 to 3.5). The descriptive statistics of the six items are available in Table 6.</p> <p>Table 6. Descriptive statistics of the six items about students' perception of their physics teacher.</p> <p> <ephtml> <table><thead><tr><td>Item</td><td>Mean</td><td>SD</td></tr></thead><tbody valign="top"><tr><td>I was made fun of or called names</td><td>2.08</td><td>1.26</td></tr><tr><td>I was left out of games or activities by other students</td><td>1.95</td><td>1.23</td></tr><tr><td>Someone spread lies about me</td><td>1.89</td><td>1.18</td></tr><tr><td>Something was stolen from me</td><td>1.83</td><td>1.16</td></tr><tr><td>I was hit or hurt by other student(s) (e.g., shoving, hitting, kicking)</td><td>1.94</td><td>1.98</td></tr><tr><td>I was made to do things I didn't want to do by other students</td><td>1.68</td><td>1.11</td></tr></tbody></table> </ephtml> </p> <p>6 <emph>Note.</emph> SD = standard deviation.</p> <hd id="AN0182192617-24">6.3.2. Fitted Models and Results</hd> <p>We specified the within-level factor (named bullying) with the six items of the SBS scale. Similarly, the between-level factor (named school climate) was specified with the corresponding item means at the between level. Factor loadings are specified to be equal for the within and between levels to have cross-level invariance as suggested by Jak et al. ([<reflink idref="bib17" id="ref102">17</reflink>]).</p> <p>The sources of heterogeneity can be multiple: latent classes at the within level due to heterogeneity in the within-level construct (bullying) across the within-level unit of analysis (students), latent classes at the between level due to heterogeneity in the between-level construct (school climate) across the between-level unit of analysis (e.g., schools), and latent classes at the between level due to random within-level classes across schools (varying proportions of within-level classes across schools). Thus, in addition to the multilevel measurement models of within- and between-constructs (bullying and school climate, respectively at within and between levels), we specified three types of latent classes: within latent classes, between latent classes based on school climate, and between latent class based on random probabilities of within-level classes. We increased the number of latent classes by one for each type of latent class sequentially until an additional class resulted in nonconvergence, empty cells, or deteriorated model fit. Across latent classes, we assumed measurement invariance with measurement parameters constrained equal, but factor means and variances and class proportions were allowed to be different across classes. Note that when one of the between-level latent classes (CB1 or CB2) was specified to have only one class, the model actually had only one type of between-level latent class variable. For example, when CB2 was specified to be one and CB1 was specified to be two or above, the model had only one between-level latent class variable of CB1 because CB2 had the value one without any variation (no CB2). Thus, we could compare the model with both CB1 and CB2 to the model with only CB1 or only CB2. The results are summarized in Table 7.</p> <p>Table 7. Summary of model fit comparisons for multilevel FMM with within- and between-level constructs.</p> <p> <ephtml> <table><thead><tr><td>CW</td><td>CB1</td><td>CB2</td><td>p</td><td>AIC</td><td>AIC3</td><td>BIC</td><td>BIC(NC)</td><td>SaBIC</td><td>SaBIC(NC)</td><td>Entropy</td><td>Class Proportions</td></tr></thead><tbody valign="top"><tr><td>1</td><td>1</td><td>1</td><td>24</td><td>64129</td><td char=".">64153</td><td char=".">64278</td><td char=".">64203</td><td char=".">64202</td><td char=".">64127</td><td>NA</td><td>1</td></tr><tr><td>1</td><td>2</td><td>1</td><td>26</td><td>64132</td><td char=".">64158</td><td char=".">64294</td><td char=".">64213</td><td char=".">64211</td><td char=".">64131</td><td>0.54</td><td>.81/.19</td></tr><tr><td>1</td><td>3</td><td>1</td><td>28</td><td>64137</td><td char=".">64165</td><td char=".">64310</td><td char=".">64223</td><td char=".">64221</td><td char=".">64135</td><td>0.56</td><td>.78/.12/.10</td></tr><tr><td>2</td><td>1</td><td>1</td><td>27</td><td>60078</td><td char=".">60105</td><td char=".">60245</td><td char=".">60161</td><td char=".">60159</td><td char=".">60076</td><td>1</td><td>.78/.22</td></tr><tr><td>2</td><td>2</td><td>1</td><td>31</td><td>60086</td><td char=".">60117</td><td char=".">60278</td><td char=".">60182</td><td char=".">60180</td><td char=".">60083</td><td>1</td><td>.78/.22 (CB1 empty)</td></tr><tr><td>2</td><td>2</td><td>2</td><td>30</td><td>61556</td><td char=".">61586</td><td char=".">61742</td><td char=".">61649</td><td char=".">61647</td><td char=".">61554</td><td>0.88</td><td>.47/.07/.30/.16 (CB2 empty)</td></tr><tr><td>2</td><td>3</td><td>1</td><td>30</td><td>61666</td><td char=".">61696</td><td char=".">61853</td><td char=".">61759</td><td char=".">61757</td><td char=".">61664</td><td>0.66</td><td>.57/.16/.202/.06/.005/.003</td></tr><tr><td>3</td><td>2</td><td>1</td><td>30</td><td><bold>59914</bold></td><td char="."><bold>59944</bold></td><td char="."><bold>60100</bold></td><td char="."><bold>60007</bold></td><td char="."><bold>60005</bold></td><td char="."><bold>59912</bold></td><td>0.83</td><td>.62/.13/.16/.06/.01/.02</td></tr><tr><td>3</td><td>3</td><td>1</td><td>32</td><td>59918</td><td char=".">59950</td><td char=".">60116</td><td char=".">60017</td><td char=".">60015</td><td char=".">59915</td><td>0.61</td><td>.44/.08/.06/.22/.08/.07/.02/.008/.009</td></tr></tbody></table> </ephtml> </p> <p>7 <emph>Note</emph>. CW = within-level latent class; CB1 = between-level latent class based on the between-level construct; CB2 = between-level latent class based on random class proportions; p = number of parameters.</p> <p>As shown in Table 7, a model with two between classes and three within classes was selected based on the AIC, AIC3, BIC, SaBIC, BIC(NC), and SaBIC(NC). There were no between-level classes based on random within probabilities. The entropy value was reasonable, but the proportion of one between class was only 9% (<emph>n</emph> = 336) at the between level and thus the three within classes in this between class were small (6%, 1%, and 2%). Although some cell proportions are very small and may not be practically meaningful, we selected this model for illustration purposes. Note that some models with number of latent classes falling between the simplest model and the most complex model was not included in Table 7 because of nonconvergence, for example, the model with 2 CW, 1 CB1, and 2 CB2.</p> <p>The identified latent classes (two between × three within = six classes) are illustrated in Figure 8 based on their observed means of six items. We first named the two between classes as safe schools (91%) and less-safe schools (9%) and the three within classes as low risk (68%), medium risk (14%), and high risk (18%) as a victim of bullying. The majority (62%) belonged to the low-risk class within safe schools. Even in the safe schools there were students who experienced bullying with medium and high risks. However, bullying experiences were even higher in the less-safe schools. Thus, the high-risk students in the less safe schools (1.5%) reported the highest bullying experiences (the brown line in the graph). Note that some of the cells had a small percentage of students, and it might not be practically meaningful to applied researchers. In the context of school safety and school climate, given the big sample, a smaller percentage still included a large number of students, and it might be meaningful for school counselors and administrators. Through the multilevel FMM with within- and between- constructs of SBS scale we could identify the schools of higher bullying climate in addition to identifying students who had been bullied, which could allow educational researchers and practitioners to investigate school characteristics associated with higher bullying climate and develop a school-level intervention on the identified schools as well as to conduct such investigation and intervention at the student level. However, of note is that in this example, within school heterogeneity (e.g., high risk vs. low risk within safe schools) was much larger than between school heterogeneity (e.g., high risk in less safe schools vs. high risk in safe schools) as illustrated in Figure 8, which could indicate the importance of student-level interventions.</p> <p>PHOTO (COLOR): Figure 8. The proportion for the between and within-level latent classes based on posterior probabilities using nonparametric multilevel FMM for within- and between-level constructs.</p> <p>In this example, there were no between-level classes based on random within probabilities. This is possibly because latent classes based on random within probabilities and those based on the between-level construct might not be very different. In other words, the class of safe schools might match the class of schools with lower proportions of students at high risk; the class of less safe schools might match the class of schools with higher proportions of students at high risk. We speculate that the two types of classes could often match in reality although they can be distinctive.</p> <hd id="AN0182192617-25">7. Discussion</hd> <p>This tutorial introduced different types of multilevel constructs along with their corresponding methods of modeling the multilevel latent classes in multilevel FMM. Model specification of different scenarios of multilevel FMM was detailed. Also, three empirical data sets were utilized to demonstrate the applications of multilevel FMM that applied researchers may encounter in real world research. In the demonstrations, one-factor models were used for the three scenarios of multilevel constructs. The indicators that measure the same construct tended to behave similarly without showing latent classes of very different shapes as shown in Figures 5, 7, and 8. However, in applied research, there might be multi-factor models and the shapes of mixtures could be very different from each other. For example, one latent class might be high in Factor 1 score and low in Factor 2 score, and another latent class might be in low in Factor 1 score and High in Factor 2 score. There are some aspects of multilevel FMM that we need to pay attention to when we apply it to empirical research studies.</p> <hd id="AN0182192617-26">7.1. Sample Size Requirements</hd> <p>In the demonstration data sets used in this article, the sample size, including individuals in each cluster and the number of clusters, was quite large. For the within-level construct example, the total sample size was 3,815 students in 168 schools, 3,440 students in 176 classrooms for between-level construct example, and 3,680 students in 164 schools in the within- and between-level construct. There was no convergence issue in all the demonstration examples. However, applied researchers should be cognizant of several caveats related to sample sizes when conducting multilevel FMMs. First, prior to using multilevel FMMs, sufficient sample sizes for reasonable performance of the multilevel FMMs need to be considered at two levels: individuals and clusters, given that parameter estimation occurs at both levels. Second, given that multilevel FMMs are still emerging, there is a lack of guidelines on the sample sizes needed for good performance of the multilevel FMMs. Future methodological studies are warranted to examine the required sample sizes for accurate class enumeration and parameter recovery. A particular focus of future studies can be the impact of multilevel FMM specifications on sample size requirements. As demonstrated in this study, multilevel FMM specifications vary depending on the level of the constructs and the type of heterogeneity that is of interest to applied researchers. For instance, constructs that are considered meaningful at both levels would entail a more complicated multilevel FMM specification than constructs that are only meaningful at one level, which might indicate that sample size requirements will be higher for the former. Likewise, for constructs that are meaningful at both levels, the required sample sizes might be larger when three types of heterogeneity are modeled simultaneously: within-level latent classes and between-level latent classes defined by factor means and within-level latent class compositions, compared with other multilevel FMMs that only aim to identify within-level latent classes. Wang et al. ([<reflink idref="bib53" id="ref103">53</reflink>]) have conducted a Monte Carlo simulation study to examine sample size requirements of FMM and found that class separation had substantial influence on sample sizes needed to achieve adequate class enumeration and parameter recovery. Moreover, including covariates that explain latent class membership was found to reduce sample size requirements as it contributes to class separation (Lubke & Muthén, [<reflink idref="bib33" id="ref104">33</reflink>]). However, there have been no suggested guidelines in the literature about sample size requirement to achieve the correct enumeration of latent classes and unbiased estimate of model parameters for different scenarios of modeling the latent classes in multilevel FMM. A Monte Carlo simulation study could be conducted in <emph>Mplus</emph> to examine the performance of the multilevel FMM with different sample sizes. In this way, the applied researchers can evaluate their data at hand to have a general expectation of the performance of the class enumeration and parameter estimate. This could be one of the directions for future research in multilevel FMM.</p> <hd id="AN0182192617-27">7.2. Parametric and Nonparametric Approach</hd> <p>For modeling the variability of the probabilities of belonging to within-level latent classes at the between level, the researchers can choose either the parametric or nonparametric approach. For the parametric approach, the probabilities are assumed to be continuous and vary across the clusters, and it is computationally more demanding (Henry & Muthén, [<reflink idref="bib16" id="ref105">16</reflink>]). On the other hand, the nonparametric approach does not impose the assumption of the probabilities being continuous, instead, it categories the clusters into different between-level latent classes based on their within-level probabilities. The choice between the between-level parametric and nonparametric approach might depend on the research questions and the intended use of the results. It is advisable for applied researchers to provide the purpose of the selected model in the theory and method sections to justify the choice. For example, the information of between-level latent classes using nonparametric method can be of important interest to policy makers in education, who wants to examine the clustering of schools based on students' self-efficacy and administer differentiated intervention to assist the schools. On the other hand, if the researcher is interested in the variation of the within-level latent classes probabilities instead of the grouping of the between-level units, the parametric method might be more appropriate.</p> <p>In our demonstration of the within-level construct, the parametric approach produced three within-level latent classes while the nonparametric approach produced two within-level latent classes and two between-level latent classes. Of note is that the numbers of within-level latent classes selected by the model selection criteria were different in the two approaches. We investigated further to understand this discrepancy. First, the comparison of the ICs of the two approaches supported the parametric approach with three within-level latent classes although the differences in ICs were very small. Next, we conducted single-level FMM with adjusted standard errors for data dependency (using TYPE = COMPLEX in M<emph>plus</emph>) and checked the number of latent classes. Because the construct we tested was assumed to be a within-level construct, this single-level approach was considered appropriate to class enumeration. Relatedly, because between-level latent classes were defined based on the probabilities of within-level latent classes, how to model between-level latent classes (parametric or nonparametric) should not affect the results at the within level theoretically. It should be noted that the class proportions were almost identical at the within level in both parametric and nonparametric approaches if the same number of classes (i.e., two classes) were specified. Thus, the single-level analysis with adjusted standard errors seemed reasonable to inform the number of within-level latent classes. That is, researchers may take a two-step approach: (<reflink idref="bib1" id="ref106">1</reflink>) determine the number of classes using the single-level FMM with adjusted standard errors, and (<reflink idref="bib2" id="ref107">2</reflink>) conduct multilevel FMM given the number of classes to determine the number of between-level classes (nonparametric) or model the variability of within-level class probabilities (parametric). With the demonstration data, the model selection criteria of the single-level model selected two classes. However, future research is needed to unravel the behaviors of multilevel FMM in parametric and nonparametric approaches and provide guidelines for applied researchers.</p> <hd id="AN0182192617-28">7.3. Model Selection Criteria</hd> <p>The implementation of multilevel FMM involves running a series of competing models with different numbers of latent classes. The process of class enumeration is to select the model with a particular number of latent classes that best fits the data based on a combination of criteria, such as AIC, AIC3, BIC, and SaBIC. The ideal situation is that the ICs agree to each other and consistently select a particular model. However, sometimes if not often, the ICs do not unanimously agree to select a specific model, and the researcher needs to decide on the best criteria. There has been no common acceptance of the best criteria for determining on the number of latent classes in FMM, especially in multilevel FMM. In multilevel FMM, the sample size used to compute BIC and SaBIC can be the number of clusters, or between-level units, and they can be referred to as BIC(NC) and SaBIC(NC), respectively. According to Kim and Wang ([<reflink idref="bib22" id="ref108">22</reflink>]), BIC(NC) and SaBIC (NC) were very comparable to BIC and SaBIC when multilevel FMM was utilized to investigate the sources of heterogeneity in measurement invariance testing. The performance of BIC(NC) and SaBIC(NC) remains unknown in multilevel FMM when they do not agree with their counterparts using the total sample size in their formulae.</p> <hd id="AN0182192617-29">7.4. Measurement Invariance</hd> <p>In FMM, heterogeneity is allowed across latent classes. Heterogeneity in parameter estimates can include not only structural parameters such as factor means and variances/covariances which indicates substantive differences across classes in the constructs of interest but also measurement parameters such as factor loadings, intercepts, and residual variances/covariances which indicates differences in interpretation of items and response tendencies across classes (namely, measurement noninvariance). In CFA, particularly for mean comparisons between groups (either observed or latent), measurement invariance is indispensable although there are discussions about partial invariance and approximate invariance which alleviate the strict assumption of exact invariance. In other words, for meaningful comparisons across groups in terms of the construct measured, the equivalence of measurement parameters is required. Similarly, the comparison of factor means across latent classes will be valid only if measurement invariance holds across latent classes. However, factor mean comparisons are not always of interest in FMM applications (Kim et al., [<reflink idref="bib24" id="ref109">24</reflink>]). When classification (that is, class enumeration) is of focal interest and thus mean comparisons are not conducted, measurement invariance may not be relevant. Moreover, FMM can be used to evaluate measurement invariance across unknown groups to detect any potential noninvariance (Kim et al., [<reflink idref="bib19" id="ref110">19</reflink>]; Lubke & Muthén, [<reflink idref="bib32" id="ref111">32</reflink>]).</p> <p>When the constructs are conceptualized as either within or between, the discussions above on MI based on single-level FMM will be applied without loss of generality because a measurement model is built in the either within or between level. For within- and between-level constructs, the issues of MI are more complex. As discussed earlier, when the within- and between- constructs are conceptually equivalent, MI in the measurement models across levels are required. Thus, cross-level invariance needs to be tested and imposed, which also ensures metric invariance across clusters (e.g., Jak et al., [<reflink idref="bib17" id="ref112">17</reflink>]). In addition, MI across latent classes can be evaluated. On the other hand, when the constructs at the within and between levels are not expected to be equivalent, cross-level invariance may not be a reasonable assumption and factor loadings are allowed to be different. Previous simulation studies evidenced the adequacy of multilevel FMM in detecting measurement noninvariance of between-level latent classes (Kim et al., [<reflink idref="bib19" id="ref113">19</reflink>]) and in detecting within-level group factorial invariance (Kim et al., [<reflink idref="bib20" id="ref114">20</reflink>]).</p> <p>The constraints on the parameters are directly related to model complexity in FMM and ML FMM. For example, allowing all measurement parameters to be class-specific (configural invariance) in addition to structural parameters could result in nonconvergence due to a large number of free parameters (Lubke & Muthén, [<reflink idref="bib33" id="ref115">33</reflink>]; Lubke & Neale, [<reflink idref="bib34" id="ref116">34</reflink>]). Estimation issues such as nonconvergence, local maxima, and unstable solutions are even more serious with categorical FMM in which item thresholds are estimated. To resolve nonconvergence and improve estimation, it is recommended to impose a certain level of invariance across latent classes unless theory suggests class-specific parameters. For example, unlike common practices in multiple group CFA in which the assumption of strict invariance is often relaxed and residual variances are allowed to be heterogenous, equality constraints on residual variances are recommended between latent classes (e.g., Lubke & Muthén, [<reflink idref="bib32" id="ref117">32</reflink>]). Kim et al. ([<reflink idref="bib23" id="ref118">23</reflink>]) examined the use of item parceling to alleviate estimation issues in FMM, but item parceling was found useful only with binary items. ML FMM is even more complex with consideration of multiple sources of variances in the nested data. Estimating certain effects as random by allowing them different across clusters could lead to nonconvergence and unreasonable solutions. Such estimation difficulty may indicate the absence of sizable heterogeneity across clusters (no random effect), and researchers may consider estimating them as fixed.</p> <p>Multilevel FMM is flexible in modeling the heterogeneity across latent classes as needed. The researchers should be aware of the level of heterogeneity allowed in the model because the interpretation of latent classes as well as latent factors will be different depending on the level of invariance or heterogeneity. For example, when measurement parameters are allowed to be class specific and thus the assumption of measurement invariance is violated, factor mean comparisons across latent classes may not be valid given different meanings of constructs across classes. Thus, researchers should clearly articulate the level of invariance and heterogeneity specified in the model with justifications.</p> <hd id="AN0182192617-30">7.5. ICC</hd> <p>As a common indicator of measuring the clustering effect of individuals nested in clusters, ICC needs to be examined before conducting multilevel modeling analysis to provide the rationale for this relatively complex modeling technique. In the TIMSS data for demonstration, indicator ICC ranged from.02 to.04, from.11 to.26, and from.08 to.12 for within-level constructs, between-level constructs, and within- and between-level constructs, respectively. Although these data sets were just one sample data set for each type of multilevel construct, these values of ICC did convey a message for the magnitude of ICC in different multilevel constructs (i.e., smallest ICC in within-level constructs, the largest ICC in between-level constructs). The ICC of the within-level may be low as shown in the demonstration example, and the intended use of the within-level construct does not require the specification of the between-level construct. However, it is still of interest to model the distributions of the within-level latent classes at the between level given the multilevel data structure. A relatively large ICC for the between-level constructs in our case may help with the stability of parameter estimates and latent class enumeration at the between level. Based on Bliese ([<reflink idref="bib8" id="ref119">8</reflink>]), the typical magnitude of ICC in multilevel factor analysis was between.05 and.20, and it was very rare to observe an ICC greater than.30. In methodological studies using multilevel SEM and multilevel FMM, ICC was typically manipulated between the range of.05 and around.30 (e.g., Kim et al., [<reflink idref="bib20" id="ref120">20</reflink>]; Kim & Wang, [<reflink idref="bib22" id="ref121">22</reflink>]; Lüdtke et al., [<reflink idref="bib31" id="ref122">31</reflink>]). Even when ICC was low, there was evidence that multilevel models are appropriate (Bliese et al., [<reflink idref="bib9" id="ref123">9</reflink>]). However, when ICC was large, the number of latent classes were overestimated (Kim & Wang, [<reflink idref="bib22" id="ref124">22</reflink>]). The impact of ICC on class enumeration in different types of multilevel constructs remains unknown and can be one of the future research directions.</p> <p>Furthermore, it is critical to check the ICC of the items, the magnitude of which combined with the data structure provides insights into the option for within-level, between-level, and within- and between-level constructs in the multilevel FMM context. For example, for illustration purposes, we treated teacher's instructional quality as the between-level construct when students rated their teachers, assuming researchers' focal interest is on the classification of teachers based on their factor scores. However, given the item ICC values around.20 which indicate substantial variations in student perceptions (about 80% of the total variance), it would be justifiable to build multilevel FMM with both within and between factors (e.g., student perceptions of teacher's instructional quality at the within level and teacher's instructional quality at the between level) and examine latent classes at both levels.</p> <p>In sum, multilevel FMM is a relatively complicated modeling approach in that it is comprised of multilevel factor model and multilevel latent class variables. To the best of our knowledge, no prior research has comprehensively and systematically introduced this modeling procedure of multilevel FMM. This tutorial purported to introduce and demonstrate multilevel FMM by incorporating latent class variables into multilevel constructs in different scenarios to investigate the heterogeneity of the population. The model specification and the demonstration of how to model within-level constructs, between-level constructs and within- and between-level multilevel constructs using empirical data sets can serve to aid empirical researchers in applying the multilevel FMM to their real-world research studies.</p> <ref id="AN0182192617-31"> <title> References </title> <blist> <bibl id="bib1" idref="ref33" type="bt">1</bibl> <bibtext> Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716 – 723. https://doi.org/10.1109/TAC.1974.1100705</bibtext> </blist> <blist> <bibl id="bib2" idref="ref2" type="bt">2</bibl> <bibtext> Allan, N. P., MacPherson, L., Young, K. C., Lejuez, C. W., & Schmidt, N. B. (2014). Examining the latent structure of anxiety sensitivity in adolescents using factor mixture modeling. Psychological Assessment, 26, 741 – 751. https://doi.org/10.1037/a0036744</bibtext> </blist> <blist> <bibl id="bib3" idref="ref14" type="bt">3</bibl> <bibtext> Allua, S., Stapleton, L. M., & Beretvas, S. N. (2008). Testing latent mean differences between observed and unobserved groups using multilevel factor mixture models. Educational and Psychological Measurement, 68, 357 – 378. https://doi.org/10.1177/0013164407312600</bibtext> </blist> <blist> <bibl id="bib4" idref="ref11" type="bt">4</bibl> <bibtext> De Angelis, M., Giusino, D., Nielsen, K., Aboagye, E., Christensen, M., Innstrand, S. T., Mazzetti, G., van den Heuvel, M., Sijbom, R. B. L., Pelzer, V., Chiesa, R., & Pietrantoni, L. (2020). H-WORK project: Multilevel interventions to promote mental health in SMEs and public workplaces. International Journal of Environmental Research and Public Health, 17, 8035. https://doi.org/10.3390/ijerph17218035</bibtext> </blist> <blist> <bibl id="bib5" idref="ref12" type="bt">5</bibl> <bibtext> Asparouhov, T., & Muthén, B. (2008). Multilevel mixture models. In G. R. Hancock & K. M. Samuelsen (Eds.), Advances in latent variable mixture modeling (pp. 27 – 51). Information Age.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref68" type="bt">6</bibl> <bibtext> Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling: A Multidisciplinary Journal, 13, 186 – 203. https://doi.org/10.1207/s15328007sem1302_2</bibtext> </blist> <blist> <bibl id="bib7" idref="ref3" type="bt">7</bibl> <bibtext> Bernstein, A., Stickle, T. R., & Schmidt, N. B. (2013). Factor mixture model of anxiety sensitivity and anxiety psychopathology vulnerability. Journal of Affective Disorders, 149, 406 – 417. https://doi.org/10.1016/j.jad.2012.11.024</bibtext> </blist> <blist> <bibl id="bib8" idref="ref36" type="bt">8</bibl> <bibtext> Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In K. J. Kline & S.W. J. Kozlowski (Eds.), Multilevel theory, research, and methods inorganizations: Foundations, extensions, and new directions (pp. 349 – 381). Jossey-Bass.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref72" type="bt">9</bibl> <bibtext> Bliese, P. D., Maltarich, M. A., & Hendricks, J. L. (2018). Back to basics with mixed-effects models: Nine takeaway points. Journal of Business and Psychology, 33, 1 – 23. https://doi.org/10.1007/s10869-017-9491-z</bibtext> </blist> <blist> <bibtext> Bozdogan, H. (1993). Choosing the number of component clusters in the mixture-model using a new informational complexity criterion of the inverse-Fisher information matrix. In O. Opitz, B. Lausen, & R. Klar (Eds.), Studies in classification, data analysis, and knowledge organization. Springer-Verlag.</bibtext> </blist> <blist> <bibtext> Dun, E. C., Masyn, K. E., Yudron, M., Jones, S. M., & Subramanian, S. V. (2014). Translating multilevel theory into multilevel research: Challenges and opportunities for understanding social determinants of psychiatric disorders. Social Psychiatry and Psychiatric Epidemiology, 49, 859 – 872. https://doi.org/10.1007/s00127-013-0809-5</bibtext> </blist> <blist> <bibtext> Finch, W. H., & French, B. F. (2014). Multilevel latent class analysis: Parametric and nonparametric models. The Journal of Experimental Education, 82, 307 – 333. https://doi.org/10.1080/00220973.2013.813361</bibtext> </blist> <blist> <bibtext> Gao, M., Leite, W., & Hu, J. (2018). The performance of model fit indices for class enumeration in multilevel factor mixture models. American Journal of Theoretical and Applied Statistics, 7, 222 – 228.</bibtext> </blist> <blist> <bibtext> Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19, 72 – 91. https://doi.org/10.1037/a0032138</bibtext> </blist> <blist> <bibtext> Georgiades, S., Szatmari, P., Boyle, M., Hanna, S., Duku, E., Zwaigenbaum, L., Bryson, S., Fombonne, E., Volden, J., Mirenda, P., Smith, I., Roberts, W., Vaillancourt, T., Waddell, C., Bennett, T., & Thompson, A. (2012). Investigating phenotypic heterogeneity in children with autism spectrum disorder: A factor mixture modeling approach. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 54, 206 – 215. https://doi.org/10.1111/j.1469-7610.2012.02588.x</bibtext> </blist> <blist> <bibtext> Henry, K. L., & Muthén, B. (2010). Multilevel latent class analysis: An application of adolescent smoking typologies with individual and contextual predictors. Structural Equation Modeling: a Multidisciplinary Journal, 17, 193 – 215. https://doi.org/10.1080/10705511003659342</bibtext> </blist> <blist> <bibtext> Jak, S., Oort, F. J., & Dolan, C. V. (2013). A test for cluster bias: Detecting violations of measurement invariance across clusters in multilevel data. Structural Equation Modeling: A Multidisciplinary Journal, 20, 265 – 282. https://doi.org/10.1080/10705511.2013.769392</bibtext> </blist> <blist> <bibtext> Kim, E. S., Cao, C., Wang, Y., & Nguyen, D. (2017). Measurement invariance testing with many groups: A comparison of five approaches. Structural Equation Modeling: A Multidisciplinary Journal, 24, 524 – 544. https://doi.org/10.1080/10705511.2017.1304822</bibtext> </blist> <blist> <bibtext> Kim, E. S., Joo, S.-H., Lee, P., Wang, Y., & Stark, S. (2016). Measurement invariance testing across between-level latent classes using multilevel factor mixture modeling. Structural Equation Modeling: A Multidisciplinary Journal, 23, 870 – 887. https://doi.org/10.1080/10705511.2016.1196108</bibtext> </blist> <blist> <bibtext> Kim, E. S., Yoon, M., Wen, Y., Luo, W., & Kwok, O. (2015). Within-level group factorial invariance with multilevel data: Multilevel factor mixture and multilevel MIMIC model. Structural Equation Modeling: A Multidisciplinary Journal, 22, 603 – 616. https://doi.org/10.1080/10705511.2014.938217</bibtext> </blist> <blist> <bibtext> Kim, E. S., Robert, F. D., Cao, C., & Ferron, J. M. (2016). Multilevel factor analysis: Reporting guidelines and a review of reporting practices. Multivariate Behavioral Research, 51, 881 – 898. https://doi.org/10.1080/00273171.2016.1228042</bibtext> </blist> <blist> <bibtext> Kim, E. S., & Wang, Y. (2018). Investigating sources of heterogeneity with 3-step multilevel factor mixture modeling: Beyond testing measurement invariance in cross-national studies. Structural Equation Modeling: A Multidisciplinary Journal, 26, 165 – 181. https://doi.org/10.1080/10705511.2018.1521284</bibtext> </blist> <blist> <bibtext> Kim, E. S., Nguyen, D., Liu, S., & Wang, Y. (2022). Mixed effect of item parceling on performance of factor mixture modeling. Structural Equation Modeling: A Multidisciplinary Journal, 29, 207 – 217. https://doi.org/10.1080/10705511.2021.1965483</bibtext> </blist> <blist> <bibtext> Kim, E. S., Wang, Y., & Hsu, H.-Y. (2023). A systematic review of and reflection on the applications of factor mixture modeling. Psychological Methods. Advance online publication. https://doi.org/10.1037/met0000630</bibtext> </blist> <blist> <bibtext> Kim, Y., & Muthén, B. O. (2009). Two-part factor mixture modeling: Application to an aggressive behavior measurement instrument. Structural Equation Modeling: a Multidisciplinary Journal, 16, 602 – 624. https://doi.org/10.1080/10705510903203516</bibtext> </blist> <blist> <bibtext> Kozlowski, S. W. J., & Klein, K. J. (2000). A multi-level approach to theory and research in organizations: Contextual, temporal, and emergent processes. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations extensions, and new directions (pp. 3 – 90). Jossey-Bass.</bibtext> </blist> <blist> <bibtext> Leite, W. L., & Cooper, L. A. (2010). Detecting social desirability bias using factor mixture models. Multivariate Behavioral Research, 45, 271 – 293. https://doi.org/10.1080/00273171003680245</bibtext> </blist> <blist> <bibtext> LeNoble, C. A., & Hudson, M. F. (2022). A multilevel approach for advancing organizational interventions. Industrial and Organizational Psychology, 15, 105 – 109. https://doi.org/10.1017/iop.2021.131</bibtext> </blist> <blist> <bibtext> Lo, Y., Mendell, N. R., & Rubin, D. B. (2001). Testing the number of components in a normal mixture. Biometrika, 88, 767 – 778. https://doi.org/10.1093/biomet/88.3.767</bibtext> </blist> <blist> <bibtext> Lukočienė, O., Varriale, R., & Vermunt, J. K. (2010). The simultaneous decision(s) about the number of lower- and higher-level classes in multilevel latent class analysis. Sociological Methodology, 40, 247 – 283. https://doi.org/10.1111/j.1467-9531.2010.01231.x</bibtext> </blist> <blist> <bibtext> Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13, 203 – 229. https://doi.org/10.1037/a0012869</bibtext> </blist> <blist> <bibtext> Lubke, G. H., & Muthén, B. (2005). Investigating population heterogeneity with factor mixture models. Psychological Methods, 10, 21 – 39. https://doi.org/10.1037/1082-989X.10.1.21</bibtext> </blist> <blist> <bibtext> Lubke, G. H., & Muthén, B. O. (2007). Performance of factor mixture models as a function of model size, covariate effects, and class-specific parameters. Structural Equation Modeling: A Multidisciplinary Journal, 14, 26 – 47. https://doi.org/10.1080/10705510709336735</bibtext> </blist> <blist> <bibtext> Lubke, G. H., & Neale, M. C. (2008). Distinguishing between latent classes and continuous factors with categorical outcomes: Class invariance of parameters of factor mix models. Multivariate Behavioral Research, 43, 592 – 620. https://doi.org/10.1080/00273170802490673</bibtext> </blist> <blist> <bibtext> Mäkikangas, A., Tolvanen, A., Aunola, K., Feldt, T., Mauno, S., & Kinnunen, U. (2018). Multilevel latent profile analysis with covariates: Identifying job characteristics profiles in hierarchical data as an example. Organizational Research Methods, 21, 931 – 954. https://doi.org/10.1177/1094428118760690</bibtext> </blist> <blist> <bibtext> Marsh, H. W., Lüdtke, O., Nagengast, B., Trautwein, U., Morin, A. J. S., Abduljabbar, A. S., & Köller, O. (2012). Classroom climate and contextual effects: Conceptual and methodological issues in the evaluation of group-level effects. Educational Psychologist, 47, 106 – 124. https://doi.org/10.1080/00461520.2012.670488</bibtext> </blist> <blist> <bibtext> McLachlan, G., & Peel, D. (2000). Finite mixture models. John Wiley.</bibtext> </blist> <blist> <bibtext> Mehta, P. D., & Neale, M. C. (2005). People are variables too: Multilevel structural equations modeling. Psychological Methods, 10, 259 – 284. https://doi.org/10.1037/1082-989X.10.3.259</bibtext> </blist> <blist> <bibtext> Muthén, L. K., & Muthén, B. O. (1998 –2021). Mplus user's guild. (8th ed.). Muthén & Muthén.</bibtext> </blist> <blist> <bibtext> Nagengast, B., & Marsh, H. W. (2013). Motivation and engagement in science around the globe: Testing measurement invariance with multigroup SEMs across 57 countries using PISA 2006. In L. Rutkowski, M. von Davier, & D. Rutkowski (Eds.), A handbook of international large-scale assessment (pp. 317 – 344). Chapman & Hall, CRC Press.</bibtext> </blist> <blist> <bibtext> Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal, 14, 535 – 569. https://doi.org/10.1080/10705510701575396</bibtext> </blist> <blist> <bibtext> Paskett, E., Thompson, B., Ammerman, A. S., Ortega, A. N., Marsteller, J., & Richardson, D. (2016). Multilevel Interventions to address health disparities shown promise in improving population health. Health Affairs (Project Hope), 35, 1429 – 1434. https://doi.org/10.1377/hlthaff.2015.1360</bibtext> </blist> <blist> <bibtext> Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models. (2nd ed.). Sage.</bibtext> </blist> <blist> <bibtext> Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461 – 464.</bibtext> </blist> <blist> <bibtext> Sclove, S. L. (1987). Application of model-selection criterion to some problems in multivariate analysis. Psychometrika, 52, 333 – 343. https://doi.org/10.1007/BF02294360</bibtext> </blist> <blist> <bibtext> Stapleton, L. M., Yang, J. S., & Hancock, G. R. (2016). Construct meaning in multilevel settings. Journal of Educational and Behavioral Statistics, 41, 481 – 520. https://doi.org/10.3102/1076998616646200</bibtext> </blist> <blist> <bibtext> Son, S., & Hong, S. (2021). Multiple group analysis in multilevel data across within-level groups: A comparison of multilevel factor mixture modeling and multilevel multiple-indicators multiple-causes modeling. Educational and Psychological Measurement, 81, 904 – 935. https://doi.org/10.1177/0013164420987899</bibtext> </blist> <blist> <bibtext> Vermunt, J. K. (2003). Multilevel latent class models. Sociological Methodology, 33, 213 – 239. https://doi.org/10.1111/j.0081-1750.2003.t01-1-00131.x</bibtext> </blist> <blist> <bibtext> Vermunt, J. K. (2008). Latent class and finite mixture models for multilevel data sets. Statistical Methods in Medical Research, 17, 33 – 51. https://doi.org/10.1177/0962280207081238</bibtext> </blist> <blist> <bibtext> Vermunt, J. K., & Magidson, J. (2005). Latent GOLD 4.0 [ computer program]. Statistical Innovations Inc.</bibtext> </blist> <blist> <bibtext> Vrieze, S. I. (2012). Model selection and psychological theory: A discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Psychological Methods, 17, 228 – 243. https://doi.org/10.1037/a0027127</bibtext> </blist> <blist> <bibtext> Wang, Y., Hsu, H.-Y., & Kim, E. (2021). Investigating the impact of covariate inclusion on sample size requirements of factor mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal, 28, 716 – 724. https://doi.org/10.1080/10705511.2021.1910036</bibtext> </blist> <blist> <bibtext> Wang, Y., Kim, E., Ferron, J. M., Dedrick, R. F., Tan, T. X., & Stark, S. (2021). Testing measurement invariance across unobserved groups: The role of covariates in factor mixture modeling. Educational and Psychological Measurement, 81, 61 – 89. https://doi.org/10.1177/0013164420925122</bibtext> </blist> </ref> <aug> <p>By Chunhua Cao; Yan Wang and Eunsook Kim</p> <p>Reported by Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib32" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib25" firstref="ref4"></nolink> <nolink nlid="nl3" bibid="bib27" firstref="ref5"></nolink> <nolink nlid="nl4" bibid="bib15" firstref="ref6"></nolink> <nolink nlid="nl5" bibid="bib43" firstref="ref8"></nolink> <nolink nlid="nl6" bibid="bib28" firstref="ref9"></nolink> <nolink nlid="nl7" bibid="bib42" firstref="ref10"></nolink> <nolink nlid="nl8" bibid="bib49" firstref="ref13"></nolink> <nolink nlid="nl9" bibid="bib47" firstref="ref15"></nolink> <nolink nlid="nl10" bibid="bib19" firstref="ref16"></nolink> <nolink nlid="nl11" bibid="bib18" firstref="ref17"></nolink> <nolink nlid="nl12" bibid="bib24" firstref="ref18"></nolink> <nolink nlid="nl13" bibid="bib14" firstref="ref19"></nolink> <nolink nlid="nl14" bibid="bib46" firstref="ref21"></nolink> <nolink nlid="nl15" bibid="bib11" firstref="ref22"></nolink> <nolink nlid="nl16" bibid="bib35" firstref="ref25"></nolink> <nolink nlid="nl17" bibid="bib16" firstref="ref26"></nolink> <nolink nlid="nl18" bibid="bib39" firstref="ref27"></nolink> <nolink nlid="nl19" bibid="bib50" firstref="ref30"></nolink> <nolink nlid="nl20" bibid="bib26" firstref="ref39"></nolink> <nolink nlid="nl21" bibid="bib31" firstref="ref40"></nolink> <nolink nlid="nl22" bibid="bib36" firstref="ref58"></nolink> <nolink nlid="nl23" bibid="bib38" firstref="ref60"></nolink> <nolink nlid="nl24" bibid="bib12" firstref="ref64"></nolink> <nolink nlid="nl25" bibid="bib48" firstref="ref66"></nolink> <nolink nlid="nl26" bibid="bib10" firstref="ref73"></nolink> <nolink nlid="nl27" bibid="bib17" firstref="ref74"></nolink> <nolink nlid="nl28" bibid="bib44" firstref="ref76"></nolink> <nolink nlid="nl29" bibid="bib45" firstref="ref77"></nolink> <nolink nlid="nl30" bibid="bib13" firstref="ref79"></nolink> <nolink nlid="nl31" bibid="bib41" firstref="ref80"></nolink> <nolink nlid="nl32" bibid="bib51" firstref="ref85"></nolink> <nolink nlid="nl33" bibid="bib22" firstref="ref86"></nolink> <nolink nlid="nl34" bibid="bib30" firstref="ref87"></nolink> <nolink nlid="nl35" bibid="bib29" firstref="ref90"></nolink> <nolink nlid="nl36" bibid="bib37" firstref="ref91"></nolink> <nolink nlid="nl37" bibid="bib34" firstref="ref92"></nolink> <nolink nlid="nl38" bibid="bib40" firstref="ref95"></nolink> <nolink nlid="nl39" bibid="bib53" firstref="ref103"></nolink> <nolink nlid="nl40" bibid="bib33" firstref="ref104"></nolink> <nolink nlid="nl41" bibid="bib20" firstref="ref114"></nolink> <nolink nlid="nl42" bibid="bib23" firstref="ref118"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1457243
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Multilevel Factor Mixture Modeling: A Tutorial for Multilevel Constructs
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Chunhua+Cao%22">Chunhua Cao</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3084-598X">0000-0003-3084-598X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Yan+Wang%22">Yan Wang</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-2237-8816">0000-0003-2237-8816</externalLink>)<br /><searchLink fieldCode="AR" term="%22Eunsook+Kim%22">Eunsook Kim</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-1054-1735">0000-0003-1054-1735</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Structural+Equation+Modeling%3A+A+Multidisciplinary+Journal%22"><i>Structural Equation Modeling: A Multidisciplinary Journal</i></searchLink>. 2025 32(1):155-171.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 17
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Descriptive
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Hierarchical+Linear+Modeling%22">Hierarchical Linear Modeling</searchLink><br /><searchLink fieldCode="DE" term="%22Factor+Analysis%22">Factor Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Nonparametric+Statistics%22">Nonparametric Statistics</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Analysis%22">Statistical Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Models%22">Models</searchLink><br /><searchLink fieldCode="DE" term="%22Measurement%22">Measurement</searchLink><br /><searchLink fieldCode="DE" term="%22Equations+%28Mathematics%29%22">Equations (Mathematics)</searchLink><br /><searchLink fieldCode="DE" term="%22Data%22">Data</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/10705511.2024.2332257
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1070-5511<br />1532-8007
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Multilevel factor mixture modeling (FMM) is a hybrid of multilevel confirmatory factor analysis (CFA) and multilevel latent class analysis (LCA). It allows researchers to examine population heterogeneity at the within level, between level, or both levels. This tutorial focuses on explicating the model specification of multilevel FMM that considers the conceptualization of multilevel constructs. Empirical data sets are used to demonstrate the applications of multilevel FMM for within-level constructs, between-level constructs, and within- and between-level constructs. Detailed model specifications of integrating latent classes into multilevel constructs are provided. For modeling the heterogeneity at the between level, parametric and nonparametric approaches are compared both conceptually and substantively using demonstration data. The interpretations of results using multilevel FMM are also provided. The tutorial is concluded with a discussion of some important aspects of applying multilevel FMM.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1457243
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1457243
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/10705511.2024.2332257
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 17
        StartPage: 155
    Subjects:
      – SubjectFull: Hierarchical Linear Modeling
        Type: general
      – SubjectFull: Factor Analysis
        Type: general
      – SubjectFull: Nonparametric Statistics
        Type: general
      – SubjectFull: Statistical Analysis
        Type: general
      – SubjectFull: Models
        Type: general
      – SubjectFull: Measurement
        Type: general
      – SubjectFull: Equations (Mathematics)
        Type: general
      – SubjectFull: Data
        Type: general
    Titles:
      – TitleFull: Multilevel Factor Mixture Modeling: A Tutorial for Multilevel Constructs
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Chunhua Cao
      – PersonEntity:
          Name:
            NameFull: Yan Wang
      – PersonEntity:
          Name:
            NameFull: Eunsook Kim
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 1070-5511
            – Type: issn-electronic
              Value: 1532-8007
          Numbering:
            – Type: volume
              Value: 32
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Structural Equation Modeling: A Multidisciplinary Journal
              Type: main
ResultId 1