2
$\begingroup$

As part of my thesis, I'm proving (or attempting to prove...) a few asymptotic results. Because these results depend on the condition number, I'd like to have some idea about the typical sizes of a condition numbers that crop up in social science research. That way, I can give some guidance about how large the sample size has to be before we reach the happy land of asymptopia.

I'd be happy for any guidance.

My very specific setup is as follows. For the standard Generalized Least Squares (GLS) model

$$Y = X\beta + e \quad \quad \quad e \sim N(0, V\sigma^2) $$

where $V$ is assumed to be known and positive definite, we define

$$ X^- = (X^\top X)^{-1} X^\top \quad \quad \quad U = (I-XX^-)V$$ and the condition number $\kappa$

$$ \kappa = \frac{ \lambda_{\text{max}} }{ \lambda_{\text{min}} } $$

where the $\lambda_\star$ values are the maximum and minimum eigenvalues of the matrix $U$.

Does anyone have pointers to references for the sizes of condition numbers in social science research? I don't even know where to look. Any pointers for either

  1. OLS estimators (used incorrectly in a GLS context as posed above)
  2. GLS estimators (correctly analyzed)
  3. REML/ML estimators where $V$ is estimated and then conditioned upon, or
  4. OLS fixed effect only models where $V$ is the identity matrix

would be most welcome!

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ It would appear that condition numbers are used here merely as a surrogate for the sample size, so why don't you just focus on sample sizes? I am baffled by two aspects of this question. The first concerns the nature of your asymptotics: what are you doing that assures the condition numbers won't grow without bound? The second is why a "typical" condition number in any field would even matter, since there is no general connection between it and the inferences one would like to draw. $\endgroup$
    – whuber ♦
    Commented Aug 9, 2013 at 14:29
  • $\begingroup$ @whuber Thanks for the comment. (1) My goal is understanding sample size via condition numbers. (2) In a simple 2-level hierarchical model, with indep. groups and equal correlation within groups, then bounding the size of the largest group bounds the condition number (e.g. adding more "schools", but not more "students per school"). (3) From my (limited) understanding, condition numbers are proxies for the expected numerical error. My naive assumption is that different fields will have orders of magnitude differences (e.g. IQ tests v. diameters of tree trunks). Do you believe that to be false? $\endgroup$
    – Nathan VanHoudnos
    Commented Aug 9, 2013 at 15:10
  • $\begingroup$ I have seen everything from 10 to 10,000. It might be true that in well designed experiments, the condition numbers will be close to 1. It is definitely true that there is no single "social science" number to talk about. $\endgroup$
    – StasK
    Commented Aug 9, 2013 at 15:48
  • 1
    $\begingroup$ @StasK I have seen infinite values but they didn't matter: they arise when there are collinearities among variables that don't affect the parameters of interest. That's the basis of my concerns about this approach: although it's true that high condition numbers (CN) create numerical instability (a practical issue) and large standard errors for some coefficients (a theoretical and practical issue), what matters is whether those inflated SEs are pertinent to the investigation objective. I therefore don't see how it would be possible to establish any meaningful kind of "typical" CN threshold. $\endgroup$
    – whuber ♦
    Commented Aug 9, 2013 at 16:15
  • $\begingroup$ @whuber, I agree that there is little in the ways of providing the single best number; I assumed that the OP has weeded out perfect multicollinearities. Also, there may be condition numbers for the raw data, but then you can start adding nonlinear terms (interactions; polynomials, splines, etc.) that would affect the CN actually encountered. For non-linear models like logit, the CNs on the parameter estimates would not be the same as the CN for the regressors. Finally, there are also multilevel models in which information set differ for different parameters, producing really weird CN patterns. $\endgroup$
    – StasK
    Commented Aug 9, 2013 at 18:17

1 Answer 1

Reset to default
0
$\begingroup$

To close the loop for those who don't want to follow the comment thread above:

No. People do not normally compile these things because they are specific to both the particular model chosen and the particular data onto which the model is fit. To talk of a "typical" number is not well posed.

If someone would like to post a more comprehensive answer, I will be happy to "unaccept" this answer and accept theirs instead.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.