3
$\begingroup$

I designed a model for my dissertation which has 7 predictors based on existing theories. After data selection, I got a sample size of 126 and I thought I was okay since the data is cross sectional. After running the regression I obtained the following result:

regression result

As you can see (i) the $R^2$ is -3.3% and p value for F test is .86 so I'm worried about the sample size being too small. (ii) all the coefficient has very large p value.

I can't find more samples as the available data is very limited and I don't have enough time to resign the dissertation. How should I interpret this result? Will conducting a failed experiment lead to a problem with the dissertation?

$\endgroup$
  • $\begingroup$ Could that be a duplicate? $\endgroup$ – Richard Hardy Sep 13 '17 at 5:14
  • $\begingroup$ Can you share some insight into what you are modelling? $\endgroup$ – Repmat Sep 13 '17 at 8:33
  • $\begingroup$ @RichardHardy Hello, none of the observations are identical. $\endgroup$ – Christina Sep 13 '17 at 8:33
  • $\begingroup$ @Repmat Of course. The model is for testing the relationship between the average annual return per investment and cross-border capital. The followings are the main variables for my hypotheses: CB - the amount of cross-border capital, CBS - the cross-border capital share. Firm - the no. of investor, CBFrim - no. of cross-border investor. $\endgroup$ – Christina Sep 13 '17 at 8:41
  • $\begingroup$ By duplicate I meant that a very similar question had been asked and answered before, which means you could go and find that answer and your problem would be solved. I did not locate the duplicate myself, but I suspect it may exist. We try to avoid duplicates here at Cross Validated. $\endgroup$ – Richard Hardy Sep 13 '17 at 8:54
0
$\begingroup$

Well, that didn't work. Try eliminating parameters with the least contributory p(CB)=0.951 the first to go. Eventually, at least the constant should become significant, which would at least demonstrate invariant data. The other thing to try would be to look at the distribution of the $y$-values. It is entirely possible that they are not nicely distributed. In case they aren't nice, a transformation of that data by something, maybe logs, reciprocals, squares, square roots, exponentiations may produce much better regressions. That is a long list of trials because sometimes the logs or other transforms of the independent parameters is needed as well, and, for best fitting, there may be a mixture of different transforms for $y$ and $x$'s that produce best results. What that means is that if there is a functional relationship, $f(x\text{'s})$ may take any form, e.g., $f(x \text{'s})=\text{sech}(x_1^2)\space e^{x_2^{-({x_3/2})^3 }}\dots$ Although finding a functional relationship is an art form, thinking about what is happening physically can eliminate lots of elbow grease. For example, if the data is proportional type, take logs. If the data has very long right tails, try reciprocation, etc.

$\endgroup$
  • $\begingroup$ Thank you, transforming the data is a good advice. However, I wonder if transforming the dependent variable means R2 is less meaningful? $\endgroup$ – Christina Sep 13 '17 at 9:11
  • $\begingroup$ No, it does not mean that. Consider, for example, that if the variables are proportional type, the log transform would increase the R$^2$ value, and that would not only be more appropriate, but more indicative of the interdependent variable covariance. $\endgroup$ – Carl Sep 13 '17 at 11:51
  • $\begingroup$ Thank you that makes sense. Some of the y-values are negative or zero. Could I add a fixed value to each of the y and make it at least larger than zero before I log it? Will it affect (I) the distribution of the logged values and (ii) the significance of each variables? Someone also suggested me to leave the zero and negative values as it is and only log the positive ones. $\endgroup$ – Christina Sep 13 '17 at 13:54
  • $\begingroup$ robjhyndman.com/hyndsight/transformations $\endgroup$ – Carl Sep 14 '17 at 19:40
2
$\begingroup$

It seems to me you can make a fairly strong conclusion: The outcome you are measuring is not linearly related to the eight independent variables you selected. You can never prove a negative, but there is no indication here that the model fits your data at all. That might be useful information.

$\endgroup$
  • 1
    $\begingroup$ Yes, I know fail experiment happens. If I could not fix it I would describe the characteristic of my samples and try to discuss why is my outcome so different from the others. However, given my sample is not the best, I would have to admit that I had underestimated the information availability and it is my fault that the data do not fit the model (or the other way around). Would that be a problem for my dissertation? $\endgroup$ – Christina Sep 13 '17 at 8:53
  • $\begingroup$ @Christina if there are other similar analyses available with different conclusions, could you do power calculations/simulations and see if you should be able to find anything in your sample? $\endgroup$ – juod Sep 13 '17 at 10:33
  • $\begingroup$ @Christina . I don't see why you call it a "failure". I don't know what you measured, or what hypothesis you are testing, but you seem to have a pretty strong conclusion. You'd want to look deeper (look at residuals, etc) to make sure the findings are solid, but at first glance, you have solid negative results. $\endgroup$ – Harvey Motulsky Sep 14 '17 at 14:16
0
$\begingroup$

If you have anomalies in your data it can create a downwards bias in your significance tests. Additionally if you have series that are highly cross-correlated this also can have an impact on your results. If you wish you can post your data and perhaps some further inspection can be accomplished.

$\endgroup$
0
$\begingroup$

Very small $R^2$ and very high $p$-values for the parameters suggest that your model is poor. But, for a moment, don't look at those values, look at the standard errors and think of your parameters in terms of confidence intervals. You would calculate the 95% confidence intervals, by taking, approximately, the parameter value $\pm$ two times the errors. Look at your errors, those intervals would be huge as compared to the point estimates of the parameters, so your parameters can be literally anything and cannot be trusted. (In fact, the $p$-values for the parameters are calculated by looking at size of the parameters as compared to the magnitude of errors.)

So you cannot make any conclusions based on this data and this model. However this doesn't mean that other model wouldn't be appropriate, but that's a different story.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.