1
$\begingroup$

The problem:

My study is on the impact of human factor on regional development.

My “subjects” are regions of the European Union. 276 exist in total under NUTS 2 classification. The idea is to use a sample of 16-20, so that I can also do a small case study on each of them and approach them qualitatively as well.

My independent variables are demographic statistics (e.g. % of people with university degrees, % regional GDP spent on R&D and so on). I have around 20 different potential IVs or predictors, although I suppose I can group them or skip some unnecessary ones and narrow them down to around 8 if necessary.

I have a single dependent variable which will be taken from some widely-accepted scale showing the level of regional development so that doesn’t need tampering with.

Obviously, the goal is to check the impact of the IVs (both separately and as a complete model) on the DV. A regression seems to be the right way to go.

My questions:

  1. Is it incorrect to attempt to check the impact of 20 predictors on such a small sample (16-20 regions)?
  2. If so, will things improve if I narrow down the predictors to 8 or so?
  3. What is the best way to narrow down or group the predictors into few important ones? Common sense? Factor analysis? Cronbach’s Alpha? A stepwise regression?
  4. If that is still not enough, should I also increase the sample size to, say, 50 or 100 regions (out of a total “population” of 276) and skip the qualitative part?
  5. Is a regression the best choice for the “main” test?
  6. Somebody mentioned an “artificial neural network” to me!? Isn’t that way too complicated for a relatively straightforward study like this one?
Improve this question
$\endgroup$
  • $\begingroup$ What is/are "human factor[s]"? $\endgroup$ – AdamO Dec 18 '15 at 19:37
  • $\begingroup$ That is an entirely different question, which I must also answer in time, but according to most sources "the human factor" from a regional development perspective, assessed quantitatively, mostly refers to indicators reflecting the quality of human capital in terms of education, knowledge, training etc. Hence, stuff like % people with university degrees, money spent on R&D, number of universities in the region, number of vocational training programmes etc. $\endgroup$ – George Dec 18 '15 at 19:47
1
$\begingroup$

  1. No it's not stupid to look at such analyses. Ecological comparisons are very important, but very limited in their implications. I would interpret findings very cautiously, especially the role of unmeasured confounding variables.

  2. Absolutely. If you are able to prespecify comparisons of key importance, this increases the power and credibility of any possible conclusions.

  3. Common sense. By leagues more than other approaches. The latter concern prediction but you are interested in inference.

  4. The choice of including other regions should be determined by the hypotheses you're trying to address. It depends if you're breaking existing regions down to granular levels or incorporating other regions not previously accounted for. Weighting may also be used to standardize some regions to an appropriate demographic composition like GDP (for economic) or population size.

  5. Simple regression models are identical approaches to t-tests and ANOVAs for continuous outcomes or contingency table test for logistic regression. In journal articles, I always say, "regression was performed" but in my thesis, I go into great depth about the type of regression, what was controlled for, and how the variables were coded. A thesis should be far more on the verbose side with how the model was fit. But yet, you should use regression... everything is regression!

  6. Way too complicated.

Improve this answer
$\endgroup$
  • $\begingroup$ Thanks, Adam! That was extremely fast and very helpful!!! How can I weigh the sample to ensure an appropriate demographic composition, like you said? $\endgroup$ – George Dec 18 '15 at 19:52
  • $\begingroup$ It depends on whom you're trying to make inference on. Andrew Gelman writes extensively on the subject. $\endgroup$ – AdamO Dec 18 '15 at 20:02
  • $\begingroup$ Thanks again! I will check the article, as well as his other work. $\endgroup$ – George Dec 18 '15 at 20:10
  • 2
    $\begingroup$ (-1) With all due respect to @AdamO, it is too soon to accept this (only) answer. There are areas in which people are very likely to disagree. First of all, part 1. Depending on your or your committee or audience's threshold for statistical significance and/or multicollinearity, three or even two predictors might be too many. Beyond that, you have many questions, and it seems your situation calls for extended readings or, if you are in a hurry, extended communication with at least one advisor/consultant/mentor. Good luck ~ $\endgroup$ – rolando2 Dec 18 '15 at 20:12
  • $\begingroup$ @rolando2 I wouldn't suggest fitting all regression variables in a single regression model, it may simply be a matter of inspecting several bivariate effects. We have used similar measures in exploratory analyses of clinical correlates of aging and dementia. $\endgroup$ – AdamO Dec 18 '15 at 20:14
1
$\begingroup$

I think the idea of conducting both quantitative / statistical analysies (regression models), and qualitative case-study analyses is a good one.

That said, you cannot fit a model with 20 predictors and 20 data. Such a model would be saturated1 and basically worthless. One strategy would be to fit 20 individual simple regression models (one independent variable each), but that isn't an ideal solution. The relationships amongst your independent predictors will lead you astray2. From a statistical point of view, you will get a better model if you use more data. If 276 regions of data are available, you would definitely do best to use all of them.

Now, you clearly won't be able to do 276 in-depth case studies, so how do you square that circle? My advice would be to do the qualitative analyses on a small subsample (say, 16-20, just to make up some numbers) of your full dataset. These could be chosen to pick out different combinations of independent variables (e.g.: high on var 1 & high on var 2; high on var 1 & low on var 2; low on var 1 & high on var 2; low on var 1 & low on var 2; etc.), and/or you could look at the residuals of your multiple regression model and examine regions that seem well explained vs. those that show different kinds of poor fit (e.g., much higher / lower development than expected).

(Having said the above, which I take to the the primary / underlying issue, I largely agree with @AdamO on his answers to your specific questions.)

1. See: Maximum number of independent variables that can be entered into a multiple regression equation,
    and: Rules of thumb for minimum sample size for multiple regression.
2. See: Is there a difference between 'controlling for' and 'ignoring' other variables in multiple regression?

Improve this answer
$\endgroup$
  • $\begingroup$ This is really good advice (and I appreciate the links too)! I think I can narrow down my predictors to around 8 and break them down into two subgroups to run two regression models (each with 4 or 5 predictors). Would that be better? I really like your other suggestion. Using the data from all 276 regions (some data will be missing, of course) and then doing the case studies on some examples, including those specific regions that defy the general trend. However, this will take much more work than I anticipated and some data might be unavailable. I will have to check if it’s possible. $\endgroup$ – George Dec 18 '15 at 20:54
  • $\begingroup$ @George, this is technical, & you may want to work w/ a statistical consultant. If the data are Missing At Random, then you can do a valid analysis with only those regions that aren't missing data. I would still not do 2 models, but using background knowledge to select variables is good. If you think there are some variables that naturally 'go together' in that they measure essentially the same thing, you could do a PCA & use the 1st PC as a single variable to represent the set. You could do separate PCAs for each coherent set & end up w/ only 2 vars in your model. $\endgroup$ – gung - Reinstate Monica Dec 18 '15 at 21:06
  • $\begingroup$ I haven’t checked the data for all 276 regions yet but some countries don’t seem to be providing all variables on a regional level, so it’s not missing at random. Or some data might be missing for specific regions. I will have to check this in detail. Or, can I avoid the problem of the missing data if I simply use PCA to end up with two IVs and regress those on a representative or random sample of 20 or 30 regions at most? Is PCA the best technique to do this? $\endgroup$ – George Dec 18 '15 at 21:19
  • $\begingroup$ @George, MAR is a technical term. It is possible your data are still MAR. If they are NMAR, you have much bigger problems to deal with. PCA won't necessarily get you out of the problem of missing data, & may or may not be the best technique to use; it's just the simplest. This is advanced stuff. You will probably need to work w/ a statistical consultant. Universities often have them available somewhere. $\endgroup$ – gung - Reinstate Monica Dec 18 '15 at 21:32
  • $\begingroup$ You’re right, this is too advanced for me. I should contact a consultant but I’m in Greece and the universities suffer from a lack of funds and poor organisation. There’s no consultant available, I have to make do. But you folks here have been really helpful! $\endgroup$ – George Dec 18 '15 at 22:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.