# Tag Info

7

As touched upon by @scortchi the reviewer was operating under the false impression that it is not possible to model nonlinear effects of predictors on the logit scale in the context of logistic regression. The original model was quick to assume linearity of all predictors. By relaxing the linearity assumption, using for example restricted cubic splines ...

6

To put the argument of @alecos-papadopoulos in graphical terms: Say his $a$ is -1 and his $b$ is 0.5 and $x$ ranges between 0 and 2. If we were to graph that relationship we could type in Stata: twoway function Pr = invlogit(-1+0.5*x), range(0 2) lwidth(medthick) That would result in the following graph: which looks pretty linear. However, if we were ...

5

To get a sense of what might be going on here, consider a simple logistic regression model with one continuous regressor $$P(Y=1\mid X) = \Lambda (g(x)) =\Big(1 + \exp\{g(x)\}\Big)^{-1} =\Big(1 + \exp\{a+bx\}\Big)^{-1}$$ The marginal effect is $$\frac {\partial P}{\partial X} = \Lambda (1-\Lambda)b$$ The 2nd-order Maclaurin series of $\Lambda$ (its ...

4

1: Are both correct? What is the difference? If you want to model the probability of occurrence based on the level of the predictor then you want to use logistic regression (a type of binomial GLM). For example, the probability of defaulting on a loan based on marital status. If you want to model the number of events based on the predictor level then ...

4

It seems to me that the reviewer was just looking for something to say. Before examining such features of the specification like the implied inflection point, there is a ton of assumptions that we have made, in order to arrive at an estimable model. All could be questioned and debated -the use of the logistic function itself being a possible primary target: ...

4

Without going into detail, cubic splines have advantages over linear splines, namely more properly reflecting smooth underlying relationships and being less sensitive to knot placement knots can be placed using subject matter knowledge or using the observed data density (e.g., put knots at fixed quantiles of a predictor) getting nonlinear main effects ...

4

As written, your question can't work, since y is a 0-1 variable and you're doing logistic regression. If you mean that the linear predictor had a nonlinear relationship with one of the independent variables, that is, $\eta = a + bf(x)$, say, for some nonlinear $f$ (with all other variables held constant), then you can write $x^* = f(x)$ and put $x^*$ in ...

3

You are right that logistic regression does not make any assumptions about the distribution of your independent variable. What will occur as a result of this is that you will have less power than if you had equal $n$s. However, reducing the $n$ in the Rich group will only lessen your power further. Rather, the idea is that if you had the same total $N$, ...

3

Another possibility are neural networks, if you use the cross-entropy as the cost functional with sigmoidal output units. That will provide you with the estimates you are looking for. Neural networks, as well as logistic regression, are discriminative classifiers, meaning that they attempt to maximize the conditional distribution on the training data. ...

3

SVM is closely related to logistic regression, and can be used to predict the probabilities as well based on the distance to the hyperplane (the score of each point). You do this by making score -> probability mapping some way, which is relatively easy as the problem is one-dimensional. One way is to fit an S-curve (e.g. the logistic curve, or its slope) to ...

3

There are a variety of ordinal regression models (see Agresti) but they rely on certain assumptions. When those assumptions are violated, the models may become incorrect. The most common assumption is that of proportional odds. Multinomial regression does not make this assumption and can therefore model odds that are not proportional. However, an ...

3

(1) There is an extensive literature on why one should prefer full models to restricted/parsimonious models. My understanding are few reasons to prefer the parsimonious model. However, larger models may not be feasible for many clinical applications. (2) As far as I know, Discrimination/Discrimination indexes aren’t (?should not be) used as a ...

2

One option is to use pseudo R-square measures for both models. A strong difference in pseudo R-square would suggest that the model fit strongly decreases by omitting V17. There are different kinds of Pseudo R-squares available. An overview can be found here, for example: http://www.ats.ucla.edu/stat/mult_pkg/faq/general/Psuedo_RSquareds.htm A popular ...

2

(1) Do you really need a smaller model? If not, you're set. (2) Can you honestly pre-specify your model? From your knowledge of the field can you choose a subset of predictors your interested in without using your knowledge of this dataset? If so, you're set. (2.5) If all your data valid? Assess this without looking at outcomes. (3) Consider using some form ...

2

The fact that you are using 4 out of 14 parameters implied that you used significance testing to select the variables. This is invalid. There are a number of other problems: Your total sample size is far too small for data splitting to be a reliable method You are seeking arbitrary cutoffs You are not using a proper accuracy score such as deviance, ...

2

For the typical low signal:noise ratio we see in most problems, a common rule of thumb is that you need about 15 times as many events and 15 times as many non-events as there are parameters that you entertain putting into the model. The rationale for that "rule" is that it results in a model performance metric that is likely to be as good or as bad in new ...

1

Having too many parameters compared to observations may lead to overfitting. Various adjustments or measures can be used to correct for this. AIC for example accounts for both the number of variables and the number of observations in your dataset and is probably most often used. AIC itself doesn't adjust the model, but serves as a tool to select the best ...

1

If the categories have order, one can use either nominal or ordinal. If the categories do not have order, one must use nominal. When there is order, one should perform model diagnostics to see which of ordinal and nominal is preferred. Although, the ordinal may have better interpretation. In general, for comparison among different models, one should ...

1

That should not happen. Two possibilities I can think off are maybe you have outliers, or maybe you have perfect prediction that messed up your logistic regression estimates. Since the LPM is just a linear model you can use all the diagnostics developed for that. I would use those to try to find influential observations. Perfect prediction is handeled ...

1

(1) This doesn't seem like a multinomial regression questions, but rather a "how to use R with a large dataset" question. There is nothing intrinsic about multinomial regression that restricts your number of observations. (2) I would use a commercial package. (3) Many others have used R successfully. I think this questions has already been addressed here a ...

1

There are many - and what works best depends on the data. There are also many ways to cheat - for example, you can perform probability calibration on the outputs of any classifier that gives some semblance of a score (i.e.: a dot product between the weight vector and the input). The most common example of this is called Platt's scaling. There is also the ...

1

If the DV is ordinal, as yours is, you should do ordinal logistic regression. However, ordinal logistic regression can also be hierarchical and multiple: Those terms refer to the number of independent variables and how they are entered into the regression. "Multiple" means there are more than one IV and "hierarchical" means they are entered into the ...

1

You can account for certain unobserved heterogeneity in panel, called correlated random effects, if you are willing to make certain assumptions about the correlation of the unobserved heterogeneity with the observed regressors. Let us say $y_{it}$ is your outcome of interest (perhaps a binary variable), $X_{it}$ are observable individual characteristics, ...

1

You don't select data to fit the model ordinarily; you select a model that is likely to fit the data. Have you read some standard texts on logistic regression? What do you mean by 'reduce the prediction'? Note that use of proportion classified correctly in this context will be very misleading, e.g., there are cases in which you get a higher classification ...

1

In addition to the comments by Antoine above, a conditional logistic regression is often used when the cases and controls are matched. A conditional logistic regression can account for the fact that there is a dependence between how the patients were selected. Another example of when conditional logistic regression would be helpful is in the context of a ...

1

I only like the area under the ROC curve ($c$-index) because it happens to be a concordance probability. $c$ is a building block of rank correlation coefficients. For example, Somers' $D_{xy} = 2\times (c - \frac{1}{2})$. For ordinal $Y$, $D_{xy}$ is an excellent measure of predictive discrimination, and the R rms package provides easy ways to get ...

Only top voted, non community-wiki answers of a minimum length are eligible