Skip to main content
Cross Validated

Return to Answer

more metrics, references, clarity..
Source Link
pds
pds
  • 121
  • 4

  • In Omega a fitted hypothetical hierarchical structure of factors is tested for reliability. The user must define which hypothetical hierarchical structures to test for reliability.

  • In Cronbach's Alpha a unidimensional structure is evaluated for reliability; with predeclared assumptions, such as equal factor loadings (tau-equivalence) and only a single general factor.

  • In Omega a fitted hypothetical hierarchical structure of factors is tested for reliability. The user must define which hypothetical hierarchical structures to test for reliability.

psych's Omega implementation seems to allowsupport for most/all varietiesa comphrensive set of structuresfactor structure varieties (e.g. general/first-order, bifactor, hierarchical) which are evaluated after fitting via a factor analysis (FA) modelling procedure (e.g. CFA/SEM or, E-CFA/E-SEM, etc.). It seems to have many facilities to evaluate and simulate different structure types, and report the factor loadings in diagrammatic form and calculate fitness measurements (indices) of the FA models.

If the assumptions of Cronbach's Alpha or the assumptions of the user's hypothetical hierarchical factor model for McDonald's Omega functions are not held, then all will incorrectly report the reliability.

Therefore, confirm the Model Fit Before Proceeding:

To confirm a hierarchical factor model's assumptions is to measure its fitness to the data (see this answer, in particular [Hu & Bentler, 1999]).

  Therefore, each hypothetical structure (for Omega) should be evaluated through their Factor Analysis model fitness metrics, such as CFI/TLI (>.95 / >.9) and RMSEA (<.06 / <.08) [Husee [Hooper, 2008; Hu & Bentler, 1999] or answer).

Examples of Factor Analysis model fitness metrics:

  • [Hu & Bentler, 1999] (see this answer) identified absolute and relative measure of fit:
    • Absolute: e.g. RMSEA (<.06 / <.08) or SPMR (<.08)
    • Relative: e.g. CFI/TLI (>.95 / >.9)
  • Other metrics included in the Python FactorAnalyzer library:
    • Kaiser-Meyer-Olkin Criterion (>.6 / >.8)
    • Bartlett’s Sphericity (null hypothesis) test (Pvalue < critical alpha e.g. <0.05).

If the hypothetical structure model has an adequate fit of the data, then its reliability can be accepted and reported.

(This is how I understand it).

  • ($\omega_h$) and ($\omega_h$-inf) are estimates of reliability to a general factor.
  • ($\omega_t$) is the estimate for the overall hypothetical hierarchical model.
  • ValuesCoefficient values are dependable, only if the hypothetical factor analysis model loadings had a good fit.

(This is a rough-guide of how one can apply and interpret Omega, as I understand it.)

  • Hooper, D., Coughlan, J., & Mullen, M. R. (2008). Structural equation modeling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 53-60.
  • Hu, L., Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
  • McDonald, R.P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum.
  • Revelle W. (2023) Using R and the psych package to find ω. Department of Psychology, Northwestern University (June 29, 2023)

  • In Omega a fitted hypothetical hierarchical structure of factors is tested for reliability. The user must define which hypothetical hierarchical structures to test for reliability.

  • In Cronbach's Alpha a unidimensional structure is evaluated for reliability; with predeclared assumptions, such as equal factor loadings (tau-equivalence) and only a single general factor.

psych's Omega implementation seems to allow for most/all varieties of structures (e.g. general/first-order, bifactor, hierarchical) which are evaluated after fitting via a factor analysis modelling procedure (e.g. CFA/SEM or E-CFA/E-SEM). It seems to have many facilities to evaluate and simulate different structure types, and report the factor loadings in diagrammatic form.

If the assumptions of Cronbach's Alpha or the assumptions of the user's hypothetical hierarchical factor model for McDonald's Omega functions are not held, then all will incorrectly report the reliability.

To confirm a hierarchical factor model's assumptions is to measure its fitness to the data (see this answer, in particular [Hu & Bentler, 1999]).

  Therefore, each hypothetical structure (for Omega) should be evaluated through their Factor Analysis model fitness metrics, such as CFI/TLI (>.95 / >.9) and RMSEA (<.06 / <.08) [Hu & Bentler, 1999]. If the hypothetical structure model has an adequate fit of the data, then its reliability can be accepted and reported.

(This is how I understand it).

  • ($\omega_h$) and ($\omega_h$-inf) are estimates of reliability to a general factor.
  • ($\omega_t$) is the estimate for the overall hypothetical hierarchical model.
  • Values are dependable, only if the hypothetical factor analysis model had a good fit.
  • Hu, L., Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
  • McDonald, R.P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum.
  • Revelle W. (2023) Using R and the psych package to find ω. Department of Psychology, Northwestern University (June 29, 2023)

  • In Cronbach's Alpha a unidimensional structure is evaluated for reliability; with predeclared assumptions, such as equal factor loadings (tau-equivalence) and only a single general factor.

  • In Omega a fitted hypothetical hierarchical structure of factors is tested for reliability. The user must define which hypothetical hierarchical structures to test for reliability.

psych's Omega implementation seems to support for a comphrensive set of factor structure varieties (e.g. general/first-order, bifactor, hierarchical) which are evaluated after fitting via a factor analysis (FA) modelling procedure (e.g. CFA/SEM, E-CFA/E-SEM, etc.). It seems to have facilities to evaluate and simulate different structure types, report the factor loadings in diagrammatic form and calculate fitness measurements (indices) of the FA models.

If the assumptions of Cronbach's Alpha or the assumptions of the user's hypothetical hierarchical factor model for McDonald's Omega functions are not held, then all will incorrectly report the reliability.

Therefore, confirm the Model Fit Before Proceeding:

To confirm a hierarchical factor model's assumptions is to measure its fitness to the data. Therefore, each hypothetical structure (for Omega) should be evaluated through their Factor Analysis model fitness metrics (see [Hooper, 2008; Hu & Bentler 1999] or answer).

Examples of Factor Analysis model fitness metrics:

  • [Hu & Bentler, 1999] (see this answer) identified absolute and relative measure of fit:
    • Absolute: e.g. RMSEA (<.06 / <.08) or SPMR (<.08)
    • Relative: e.g. CFI/TLI (>.95 / >.9)
  • Other metrics included in the Python FactorAnalyzer library:
    • Kaiser-Meyer-Olkin Criterion (>.6 / >.8)
    • Bartlett’s Sphericity (null hypothesis) test (Pvalue < critical alpha e.g. <0.05).

If the hypothetical structure model has an adequate fit of the data, then its reliability can be accepted and reported.

  • ($\omega_h$) and ($\omega_h$-inf) are estimates of reliability to a general factor.
  • ($\omega_t$) is the estimate for the overall hypothetical hierarchical model.
  • Coefficient values are dependable, only if the hypothetical factor analysis model loadings had a good fit.

(This is a rough-guide of how one can apply and interpret Omega, as I understand it.)

  • Hooper, D., Coughlan, J., & Mullen, M. R. (2008). Structural equation modeling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 53-60.
  • Hu, L., Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
  • McDonald, R.P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum.
  • Revelle W. (2023) Using R and the psych package to find ω. Department of Psychology, Northwestern University (June 29, 2023)
Source Link
pds
pds
  • 121
  • 4

Interpretation:

Revelle's psych HowTo guide (page ~7) notes that 3 results are typically presented, and these are based on [McDonald, 1999]'s two omega functions:

  • Omega hierarchical ($\omega_h$) is an estimate of the general factor saturation of a test.
  • Omega hierarchical infinite ($\omega_h$-inf) is the omega for an infinite length test with a structure similar to the observed test.
    • (NB: I interpret this test as the same hypothetical hierarchical structure defined for $\omega_h$ and fitted via a Factor Analysis procedure, ie. E-CFA. In this case the fitting is not limited by a number of iterations, and therefore is likely to maximize the fit. I empirically note it tends to receive a higher coefficient value that $\omega_h$.)
  • Omega total ($\omega_t$) is an estimate of the total reliability of a test, including the general factor, group factors (i.e. first-order factors) and item indicators.

Assumptions / Prerequisites:

  • In Omega a fitted hypothetical hierarchical structure of factors is tested for reliability. The user must define which hypothetical hierarchical structures to test for reliability.

  • In Cronbach's Alpha a unidimensional structure is evaluated for reliability; with predeclared assumptions, such as equal factor loadings (tau-equivalence) and only a single general factor.

psych's Omega implementation seems to allow for most/all varieties of structures (e.g. general/first-order, bifactor, hierarchical) which are evaluated after fitting via a factor analysis modelling procedure (e.g. CFA/SEM or E-CFA/E-SEM). It seems to have many facilities to evaluate and simulate different structure types, and report the factor loadings in diagrammatic form.

If the assumptions of Cronbach's Alpha or the assumptions of the user's hypothetical hierarchical factor model for McDonald's Omega functions are not held, then all will incorrectly report the reliability.

To confirm a hierarchical factor model's assumptions is to measure its fitness to the data (see this answer, in particular [Hu & Bentler, 1999]).

Therefore, each hypothetical structure (for Omega) should be evaluated through their Factor Analysis model fitness metrics, such as CFI/TLI (>.95 / >.9) and RMSEA (<.06 / <.08) [Hu & Bentler, 1999]. If the hypothetical structure model has an adequate fit of the data, then its reliability can be accepted and reported.

(This is how I understand it).

Fundamental, Intepretation:

  • ($\omega_h$) and ($\omega_h$-inf) are estimates of reliability to a general factor.
  • ($\omega_t$) is the estimate for the overall hypothetical hierarchical model.
  • Values are dependable, only if the hypothetical factor analysis model had a good fit.

References:

  • Hu, L., Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
  • McDonald, R.P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum.
  • Revelle W. (2023) Using R and the psych package to find ω. Department of Psychology, Northwestern University (June 29, 2023)