# Interpreting a weak model proceeding from there

I originally posted this question and since then I've posted another more specific branch-off question.

I've followed the advice I received in response to my questions, and I've done some statistical analysis. Now I need help analyzing the results and determining my next move. Here's my original explanation of the situation:

My data is a list of records, each one representing an educational seminar event. I have a continuous variable that represents the revenue brought in by each seminar, which is the response variable in my regression. I also have a number of categorical variables which are acting as factors/IVs.

To add a little more detail, these factors include things like day of the week the seminar was held, topic, speaker, etc.

My main goal is to build a model that A) can help to explain what factors most influence revenue and how, and B) has some predictive power.

I performed a multiple regression in R, but much to my dismay, the adjusted $R^2$ value was a mere 0.2188. I know this doesn't mean the factors have no predictive power at all, but I'm wary of making any major strategic decisions based on such a poorly fit model--am I right to feel that way?

I also performed an ANOVA test, and certain factors seemed to be labelled as more significant than others.

My questions are:

1. How should I proceed given the weak fit of my model? Could it be that the data simply isn't helpful and that either random chance or factors I haven't considered are at play here?

2. Though ANOVA tells me which factors appear significant, it doesn't give me any idea of how they are significant. How can I determine what effect each significant factor has from a practical standpoint?

3. Are there any other tests that would help me understand how all these variables are related?

Bear in mind that I am new to R and rusty on statistical methods (I took an intro-level course a few years ago).

-

I would suggest you the following steps of further analysis.

1. Perform sensitivity analysis (global or possibly local if your data has complex behavior) in order to find out most important variables. I think this R package would be suitable for your needs. On this step you will have a sorted list of variables according to their impact on response variable.
2. Drop unimportant (it is generally your choice how many) variables from the further analysis. Just try several numbers of dropped variables in order to get the best results in next steps.
3. Build your model. Do not forget to use dummy variables for categorical factors! If your are not obliged to use linear model try polynomial or other nonlinear (that is of course if you have enough observation). Although multiple ridge regression is a good choice (but you need to carefully select the parameters).
4. Now you need to estimate accuracy of the model build on the previous steps. $R^2$ is one of the ways although not very illustrative. ANOVA accounts only for linear dependence. Another and more general way is to perform cross validation and calculate mean error of prediction.

Hope this helps! And good luck in your analysis ;)

EDIT: By the way about you last question. You can visualize internal relations in your regressors using e.g. matrix scatterplots. It is a graphical representation of correlation matrix. And if you see straight line somewhere it tells you that the corresponding variables are perfectly correlated and you can drop one of them. As I understand in your case all factors are categorical. So it is not straightforward how to perform correlation analysis. Although R provides some specific tools.

-

I think statement 1 is probably right. The total R square can be partitioned into contributions due to each of the factors. This shows in the ANOVA table (though the effect of the facotrs is influenced by the other factors in the model. You could also look at how big R square is by trying separate 1 factor ANOVAs. So keep what looks to be the most imprtant and see if you can think about other factors that could explain the response better.

-