1
$\begingroup$

I have 14 observations.

There are 11 independent variables $(x_1,x_2,\dots,x_{11})$.

Dependent variables and all independent variables are continuous.

I am not sure which regression model we use in this case to predict y( dependent variable).

Any hints ?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ To my understanding, the size of the dependent and independent variables must match for regression. $\endgroup$
    – SmallChess
    Commented Mar 22, 2017 at 9:15
  • $\begingroup$ @StudentT, sample size is 14 for complete data. for ex: 11 are dependent variables with 14 observations value and 1 dependent variable with 14 observations $\endgroup$
    – e4e5
    Commented Mar 22, 2017 at 9:18
  • 1
    $\begingroup$ This question depends on your problem domain. If you already have a known model that the data's expected to fit, e.g. as in measurements, then it's sufficient to have one observation per independent variable. However if you have no idea what sort of models might fit the data, then regressions are basically guess-and-check, which is why big data analysts tend to split up their data sets into training data (which is regressed) and validation data (which validates the regression of the training data). [...] $\endgroup$
    – Nat
    Commented Mar 22, 2017 at 17:49
  • 1
    $\begingroup$ Then, there're also intermediate cases in which you have a general idea about what regressions might fit the data. Since there's such a range of possible answers given your problem domain, it'd probably be a good idea to state the problem background in the question. Otherwise, people who provide answers will respond with what they know - which is nothing - resulting in a training/validation approach. And you don't have enough observations to do that well, so that's unlikely to be productive. $\endgroup$
    – Nat
    Commented Mar 22, 2017 at 17:51
  • 1
    $\begingroup$ tl;dr- You should add your problem domain to the question statement. $\endgroup$
    – Nat
    Commented Mar 22, 2017 at 17:53

2 Answers 2

Reset to default
10
$\begingroup$

When you have only 14 observations and 11 independent variables, almost any model is likely to be overfit, even if you just choose among the 11 models with one variable, you are going to be biasing results.

You could still try LASSO to build the model, but it is likely to pick no variables at all (just the mean).

Another option is to present all 11 regressions so the reader can see what is going on.

In a comment, you suggest using backward and forward selection, but these have been proven to give wrong output.

Improve this answer
$\endgroup$
2
5
$\begingroup$

If your sample is this small, I would suggest you go through a methodical process of building different models with different sets of variables. Any domain knowledge you have will be useful as well.

In any case, you will find it hard to finally choose a model with confidence as you will not be able to validate with any accuracy.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Can you explain which regression model I have to use? i am thinking about backward-forward, can I use that ? $\endgroup$
    – e4e5
    Commented Mar 22, 2017 at 11:09
  • 5
    $\begingroup$ I think that any model with more than 1 IV here is likely to be overfit. $\endgroup$
    – Peter Flom
    Commented Mar 22, 2017 at 12:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.