0
$\begingroup$

This text is from An Introduction to Statistical Learning with Applications in R(by • Gareth James • Daniela Witten • Trevor Hastie • Robert Tibshirani) List item Can anyone help me by making linear models with these different variable so that I can practically understand what is wrong with these

Suppose that we are trying to predict the medical condition of a patient in the emergency room on the basis of her symptoms. In this simplified example, there are three possible diagnoses: stroke, drug overdose, and epileptic seizure. We could consider encoding these values as a quantitative response variable, Y , as follows:

y=1 if stroke;

y= 2 if drug overdose;

y= 3 if epileptic seizure.

Using this coding, least squares could be used to fit a linear regression model to predict Y on the basis of a set of predictors X1,...,Xp. Unfortunately, this coding implies an ordering on the outcomes, putting drug overdose in between stroke and epileptic seizure, and insisting that the difference between stroke and drug overdose is the same as the difference between drug overdose and epileptic seizure. In practice there is no particular reason that this needs to be the case.

For instance, one could choose an equally reasonable coding,

y=1 if epileptic seizure;

y=2 if stroke;

y=3 if drug overdose.

which would imply a totally different relationship among the three conditions. Each of these coding would produce fundamentally different linear models that would ultimately lead to different sets of predictions on test observations.

please help me to understand this that

1)how to produce linear model using any of the relationship

2)how different relationship among the three condition produce different linear model

Improve this question
$\endgroup$
4
  • $\begingroup$ What is unclear for you in this example? $\endgroup$ – Tim♦ Apr 1 '16 at 11:41
  • 1
    $\begingroup$ Hint: what would it mean if your model returned prediction of 2.5 or 4..? $\endgroup$ – Tim♦ Apr 1 '16 at 11:44
  • $\begingroup$ @Tim I didn't understand how these coding produce different linear model and how to produce linear model using these $\endgroup$ – learner Apr 1 '16 at 12:16
  • $\begingroup$ @NickCox Now i update . Thanks to your suggestion. $\endgroup$ – learner Apr 1 '16 at 12:20
1
$\begingroup$

There are two viable models for this situation and neither of them is close to the linear model (ordinary least squares).

  1. Multinomial logistic model, which doesn't assume an ordering of $Y$ but which requires twice as many $\beta$ coefficients with a 3-level $Y$.
  2. Ordinal regression (e.g., proportional odds ordinal logistic model) which makes a strong ordering assumption about $Y$ (but no spacing assumptions) and only needs as many $\beta$s as the linear model, plus 2 intercepts for 3-level $Y$.

To me it is not useful to conceptualize this using a model for continuous $Y$.

I changed the original title of the question, which included the word classification. Classification has nothing to do with the issues at hand.

Improve this answer
$\endgroup$
6
  • $\begingroup$ They mentioned in the text that it is possible to build linear model using binary variable and i am curious to find out how .Just wanted to know from you is it possible to build linear model using binary variable 0 and 1 ? Please clear the confusion as you are an expert in this field $\endgroup$ – learner Apr 1 '16 at 12:51
  • 2
    $\begingroup$ Why would anyone be interested in a linear model for binary $Y$ when it would yield "probabilities" <0 or >1? To answer your question it is possible. Lots of things are possible that are terrible ideas. $\endgroup$ – Frank Harrell Apr 1 '16 at 12:56
  • $\begingroup$ Well I am bit curious to learn by going to the bottom of the subject. Helpful if you provide any link or way to create linear model using binary variable. $\endgroup$ – learner Apr 1 '16 at 12:58
  • $\begingroup$ What you want in terms of a "linear model using binary variable" is explained farther down the page in the text that you cite (page 130): the section on logistic regression. You use a transformation of the probabilities, the logit, as the outcome variable rather than raw probabilities. That avoids the inconsistencies that Frank Harrell points out in his comment. Some ways to handle multiple discrete outcomes are covered later in the same Chapter 4 of ISLR. Try the exercises at the end of the chapter. $\endgroup$ – EdM Apr 1 '16 at 13:53
  • $\begingroup$ @FrankHarrell "Classification has nothing to do with the issues at hand." I would love to hear your explanation about why this isn't a classification problem. $\endgroup$ – Dave Sep 12 '19 at 10:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.