0
$\begingroup$

I am researching the effect of parameters on energy consumption. To determine the effect of parameters, I want to use the R Studio's F-test. In this way I want to investigate if the model with the parameter (dummy categories) is significantly better than a model without the parameter. Here I run into a problem. My dataset has a very high skewness and kurtosis, resulting in fact that the linear regression assumptions are violated. I tried to solve this with a transformation (10log) of the dependent variable. The skewness is now lower (less than 1), but the kurtosis is still high (around 4, which was 30 without transformation). The assumptions now seem to be a bit better, but there are still fat tails in the qqplot. I was wondering if the test is reliable after the transformation? Or is the F test not affected by violated assumptions and is it possible to make the F-test reliable without transformation? And if the F-test isn't relaible with and without transformations, which other way can I use to test the significance of a parameter.

Thanks in advance:)

These are the regression outputs of the regression on the dependent variable (energy consumption) without transformations

enter image description here enter image description here enter image description here

After transformation (The F-statistic has become significant (at 0.05 and 0.01 now) enter image description here enter image description here enter image description here

Boxplot enter image description here

$\endgroup$
12
  • $\begingroup$ Welcome to Cross Validated! There is no assumption about normality of $y$. Further, if you transform $y$, you now have a different model, and inferences about the original data need not be legitimate. // This seems like an XY problem where you have problem X (effect of parameters on energy consumption) that you think you can solve using method Y (regression F-test). Then, when you have trouble implementing Y, you ask about Y instead of X. What exactly do you want to do with X? $\endgroup$
    – Dave
    Jun 9, 2022 at 14:03
  • $\begingroup$ Dear Dave, thank you for your reaction. My goal is to prove that my variables (for example the function type) have a significant effect on the energyconsumption (the dependent variable). So I just want to show that including dummies for the function types (office, sport etc.) will give a better model than a model without these variable (and I tought this could be done by looking at the F-statistic) $\endgroup$
    – Siebe
    Jun 9, 2022 at 20:19
  • $\begingroup$ And the problem is that I learned to check the assumptions before interpretating the results of the regression. However the normal residuals assumption is violated (I think) because the qqplot is not in line with the normal distribution. And also the residuals vs the fitted values plot is showing some sort of trend. Maybe I can send you pictures about the resisdual plots, but I can not find out how to do that. $\endgroup$
    – Siebe
    Jun 9, 2022 at 20:27
  • $\begingroup$ Please feel free to edit your original post to include additional information that would be valuable. $\endgroup$
    – Dave
    Jun 9, 2022 at 21:23
  • $\begingroup$ I added pictures about the residuals plots and the regression output. I thought I wasn't allowed to intepret the regression output when the assumptions were violated. Is that true or can I still say something abou the F-test? Or is there another way to show that including dummies for the function types (office, sport etc.) will give a better model than a model without this variable? $\endgroup$
    – Siebe
    Jun 10, 2022 at 6:57

1 Answer 1

1
$\begingroup$

Model assumptions are mathematical constructs and are never precisely fulfilled in reality. The relevant question is not whether assumptions are fulfilled or not (as they never are), but rather whether assumptions are violated in critical ways, i.e., ways that lead to misleading conclusions. Unfortunately assessing this is rather subtle. Roughly said, in your situation, extreme outliers and strong skewness are very problematic, as would be dependence between observations. Heterogeneity of variances can be a problem but has often rather mild influence. Data with untransformed $Y$ seem very problematic in this respect, so you were right to worry. Data with transformed $Y$ don't look perfect but much better.

I'd personally think that the model with transformed $Y$ is good enough to interpret the F-test despite model violations still being visible (but as I said before, the model is never true anyway). As I said, these things are subtle, so here are some considerations that back up this opinion (that is admittedly to some extent subjective, so if you publish it, reviewers may disagree):

  1. The p-value of the F-test is not borderline but small enough that it cannot be explained by rather mild assumption violations only.

  2. The effect of too large kurtosis is usually that there is loss of power, meaning that significances are still meaningful (but harder to obtain).

  3. There seems to be a mild heteroscedasticity issue, but I don't think it has much of an impact.

  4. Boxplots show fairly convincingly (at reasonably large sample size) that Kantoor values tend to be higher by and large.

But keep in mind that $R^2$ is very low at 7%, so your model doesn't explain much of the variation of the transformed $Y$.

I've written more generally about model assumptions here: Relevance of assumption of normality, ways to check and reading recommendations for non-statisticians

$\endgroup$
11
  • $\begingroup$ Thank you very much for this explanation. When I include extra variables, the r squared goed up to about 20% so I know this isn't much and I will point this out in my report. The thing I wasn't sure about is wheter I can interpret the F-statistic and the R squared, when the assumptions are violated. Now I know it's better to use the tranformed dependent variables, because the assumptions are less violated than in the model without transformation. Last thing I am wondering now, is wheter I can intepret the F-statistic and the R squared when I transform my dependent variable? $\endgroup$
    – Siebe
    Jun 10, 2022 at 10:11
  • $\begingroup$ Because on internet and in these comments, I see different answers regarding this question. My dependent variable has totally different values now and I am not sure if I can still point out the significance (or unsignificance) of explanatory variables when the dependent variable is transformed to a totally different scale $\endgroup$
    – Siebe
    Jun 10, 2022 at 10:13
  • $\begingroup$ @Siebe You need to interpret results with transformed variables making reference to the transformation. So you can say for example (see my elaboration in the answer) that there's evidence that the Kantoor object type tends to produce, on average, larger values of log(y) if log was the transformation you have used. $\endgroup$ Jun 10, 2022 at 10:16
  • $\begingroup$ Okay clear, but does that also say something about the Kantoor object type tending to produce on average larger values of y? Because otherwise this isn't very usefull if I want to argue wheter to include object type in a model to predict the actual (not transformed) energyconsumption, or is it? $\endgroup$
    – Siebe
    Jun 10, 2022 at 10:20
  • $\begingroup$ @Siebe This is another subtle issue. log is a monotonic transformation, so if values of log(y) tend to be larger, roughly said, also values of y tend to be larger. However, speaking of "on average", this does not necessarily hold, as outliers, kurtosis and skewness mean that the act of averaging in itself is problematic. Averages are critically affected by outliers. So I'd say if we're talking about what is going on on average (as such models do), the transformed scale is the more appropriate one to comment on. $\endgroup$ Jun 10, 2022 at 10:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.