1
$\begingroup$

I don't want to explain my real data, but I will make up an example that is equivalent.

Let's say people are judging the quality of paintings and can say "like" or "dislike". Paintings can have five colours in them: red, yellow, blue, green and/or orange. The paintings can be any size.

Count Approach

Let's say I categorize each square cm based on its colour (and that in this example each square cm can only be one colour). Because paintings can be of different sizes, the totals will differ across paintings. I create a model that is the number of square cms that are each colour.

So my model is: Liked = TotalRed + TotalYellow + TotalBlue + TotalGreen + TotalOrange

Analyzed like this, my results are something like this:

Term B
Intercept .8*
TotalRed -.2
TotalYellow .04
TotalBlue -.5
TotalGreen -.5*
TotalOrange -.3*

*= p < .05

Proportion Approach

Let's say for each painting I calculate the proportion of the painting that is each colour. This time I leave out blue because it is the least frequent and I want to avoid perfect redundancy among my predictors.

So my model is: Liked = PropRed + PropYellow + PropGreen + PropOrange

Analyzed like this, my results are something like this:

Term B
Intercept -.4
PropRed 1
PropYellow 1.5*
PropGreen -.01
PropOrange -.15

*= p < .05

Some predictors change signs, different predictors are significant, and my intercept changes.

How do I know which one is "more right"?

Edit:

To give some more detail, this is a logistic regression, with multiple predictors. I have about 450 observations.

$\endgroup$

1 Answer 1

0
$\begingroup$

While answering the question all kinds of critical questions about your method pop up (For instance, how did you compute the p-values, is it based on anova or on t-tests? How much data do you have? Why choose a linear regression? Are these reasonable/good models to start with? What is the point of the experiment? Etc)...

...but, answering the main question, generally one could compare the results based on some comparison of performance. Either based on some theoretical ideas about the distribution of the error (like computing AIC), or based on evaluation of performance with a holdout/test data set (like cross-validation/machine-learning).

Personally, I wouldn't attack this problem with a straightforward linear regression, and instead I would like to observe and explore the results in order to gain a better idea of a potential model and during such exploring work I would try to involve theories and background of previous knowledge from the field.

$\endgroup$
2
  • $\begingroup$ I should have mentioned this is a logistic regression with multiple predictors. Good idea, I will take a look at the AIC for the two models. $\endgroup$
    – Dave
    Feb 22, 2023 at 18:22
  • $\begingroup$ AIC is roughly the same for the two models. $\endgroup$
    – Dave
    Feb 22, 2023 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.