5
$\begingroup$

I've done some searching on StackExchange and become more and more confused about using factors and ordered factors in R modeling.

I've encountered several circumstances:

  1. A variable is characters like A\B\C of different kinds with no order, such as race(white/black etc) and gender (female/male). I know then I have to use factors, and not ordered factor.

  2. When I meet some variable that records a series of stages, for example, tumor stage 1-4, acute kidney injury stage 1-4, should I use factors now? It seems that I should use ordered factors? Or is this simply a continuous numeric variable?

  3. Sometimes quantiles of a variable are calculated as a new variable, that is, values between 0~25% are categorized to 1, 25%~50% to 2, 50~75% to 3 and 75~100% to 4. The original varible is then removed from modeling, leaving only 1/2/3/4 representing Q1/Q2/Q3/Q4. This seems like the 2nd question above, so an ordered factor. Or, again, this is just a continuous numeric variable?
  4. And finally, in a glm() logit regression model as an example, should I make the Y variable which is already 0/1 as integers a factor or not? Same question goes for X variables. I saw both glm(y ~ factor(x1) + ...) and glm(y ~ x1 + ...) on StackExchange, which is really getting me confused.
$\endgroup$
  • 1
    $\begingroup$ On the terminology of quantiles: Historically quantiles are points on a scale defining values by associated cumulative probabilities, so that medians are defined by 50% of the distribution being below (and also above), the quartiles by 25% being one side and 75% the other side, and so on. The general term quantile is often attributed to M.G. Kendall, but is older. To the point now: quantiles are now often also used as a term for the bins these quantiles delimit. The ambiguity does not often bite hard, but watch out. So is the first quartile the value or the bin below it? $\endgroup$ – Nick Cox Dec 13 '18 at 18:08
  • $\begingroup$ @NickCox I am sorry I am not a native English speaker, I don't quite understand your question here. I only meant Q1-Q4 are encoded to 1-4 as a new variable if that's what you're asking. I've editted question accordingly. thanks, Nick. $\endgroup$ – Jun Jiang Dec 14 '18 at 2:17
  • $\begingroup$ I don't have a question. I was pointing out to you and to other readers that the term quantile is used in two different ways, to refer to values (points) and to refer to bins (classes, intervals). $\endgroup$ – Nick Cox Dec 14 '18 at 8:54
4
$\begingroup$

Q1 - yes you are correct

Q2 - you would usually choose to fit this as an ordered factor but you have the option of unordered if you wish to parameterise it in some unusual way.

Q3 - if the variable is inherently continuous then categorising it loses information and leads to fitting a model which assumes the effect is identical for all values within each category with a jump to a new value at the boundary. This is usually thought to be scientifically implausible. Other people do it but there is no reason for you to do it.

Q4 - this should make no difference, but why not try it and see what happens? You will not break R.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Re 4: I recall we saw a question, appearing earlier this month, that reported on different R behavior because in one case 0/1 variables were treated as numbers and in another case as factors. See stats.stackexchange.com/questions/380138. It has (at this time) two insightful answers. $\endgroup$ – whuber Dec 13 '18 at 17:45
  • $\begingroup$ Q3 @Frank Harrell has a different take. He may chime in here sooner or later. $\endgroup$ – Nick Cox Dec 13 '18 at 18:01
  • $\begingroup$ @whuber I was assuming the OP was using it on the left hand side whereas if I read it correctly the question you link to has it on the right hand side. $\endgroup$ – mdewey Dec 13 '18 at 18:07
  • $\begingroup$ @NickCox I think his view is that one can always use a continuous variable on the left hand side as an ordered categorical and use proportional odds but I do not think that implies it is a good idea to apply the sort of coarse categorisation to which the OP is referring. $\endgroup$ – mdewey Dec 13 '18 at 18:09
  • $\begingroup$ That's an important and helpful distinction. In fact Frank is one of the most articulate critics of binning continuous predictors $\endgroup$ – Nick Cox Dec 13 '18 at 18:11
1
$\begingroup$

Factor variables are used any time you have categorical data, where you wish to fit your model in a way that allows freedom for "effects" of those categories to vary arbitrarily. Ordered factors should be used when you have more than two categories that are ordinal in nature, and unordered factors should be used when you have only two categories, or any number of nominal categories. When you impose ordering of factors in a statistical model, this imposes a constraint that ensures that the magnitude and direction of the "effects" of those factors obey the proposed ordering. Thus, ordering of categories is equivalent to a constraint on the parameter space in the model.

Full understanding the use of factors and numerical variables requires you to understand the underlying theory of measurement scales. The theory of measurement classifies variables according to whether they has meaningful relations according to various binary relations and arithmetic operators. A common typology separates variables into nominal, ordinal, interval, and ratio measurements. If you would like to know more about this subject then I would recommend reading some literature on measurement theory.

  1. A variable is characters like A\B\C of different kinds with no order, such as race(white/black etc) and gender (female/male). I know then I have to use factors, and not ordered factor.

That is correct - race and gender would both be treated as nominal categories and so they should be treated as factors without a specification or ordering. (Technically there may be some rare applications where you would want to impose a parametric constraint that orders racial categories with respect to the magnitude and direction of "effects". In this case race would be treated an ordinal variable. This would only occur if there was a sound basis for imposing a parametric constraint on ordering, such as if race is associated with some other characteristic that justifies a constraint in the model.)

  1. When I meet some variable that records a series of stages, for example, tumor stage 1-4, acute kidney injury stage 1-4, should I use factors now? It seems that I should use ordered factors? Or is this simply a continuous numeric variable?

These kind of classifications are generally discrete categorisations of some underlying variables that give more detail on the particular characteristics. For example, tumor stage is a simple categorisation determined by staging systems that take account of the size and spread of the tumor. These stages should not be treated as numerical variables, since that kind of treatment imposes implicit constraints that treat the "intervals" between the stages as being of equivalent magnitude in the model. Such variables should generally be treated as ordered factors. In this case the ordering constrains the magnitude of the "effects" in the model to the assigned order, but does not constrain the magnitude of the change in effects between different factor levels.

When using these kinds of ordinal factor variables, it is also worth noting that they are commonly classified according to one or more underlying continuous variables. The categorical variable thus represents a loss of information compared to the underlying variables that determine the category. Hence, if the underlying continuous variables are part of the available data, you will usually get best results by using the underlying continuous variables in your model, and removing the factor variable entirely.

  1. Sometimes quantiles of a variable are calculated as a new variable, that is, values between 0~25% are categorized to 1, 25%~50% to 2, 50~75% to 3 and 75~100% to 4. The original variable is then removed from modeling, leaving only 1/2/3/4 representing Q1/Q2/Q3/Q4. This seems like the 2nd question above, so an ordered factor. Or, again, this is just a continuous numeric variable?

That would be an ordered factor, and again, it involves a loss of information relative to the original variable. If the original variable is available it is usually best to model it directly.

  1. And finally, in a glm() logit regression model as an example, should I make the Y variable which is already 0/1 as integers a factor or not? Same question goes for X variables. I saw both glm(y ~ factor(x1) + ...) and glm(y ~ x1 + ...) on StackExchange, which is really getting me confused.

When you use a factor variable in a statistical model, this variable is treated as a set of binary variables (with values 0 and 1) that indicate the different categories. One of the categories is treated as the baseline (represented by all indicators being 0) and the remaining categories are given an indicator variable. Thus, if you have a categorical variable with $k$ categories, that is treated as $k-1$ indicator variables in the model.

One consequence of this is that a binary categorical variable is treated as a single indicator variable, and those two things are essentially equivalent in mathematical terms. Hence, when you are using a binary numerical variable with only two possible outcomes (e.g., 0 and 1), mathematically that is treated the same as any binary factor variable.

In terms of actual syntax for putting variables into models in R, most of the modelling procedures will automatically deal with binary numeric variables and binary factor variables. For a logit regression the response variable y is binary, which is implicitly the same as a binary categorical variable. It is usually just treated as a numerical variable when fitting the model. If you put a binary factor variable into the glm function (either as the response or predictor) the function automatically assigns a corresponding binary indicator, so this does not cause any problem.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.