8

Welcome to CV. You have misunderstood the interpretation of the intercept. The intercept is the log odds (not the odds ratio) of the outcome when all the predictors are at 0 (not the marginal log odds, as you described). When the predictors are standardized, this corresponds to when all the raw predictors are at their mean. So, for an individual with average ...


7

As Noah says but just with formulas ... Consider logistic regression $$ \Pr(Y=1) = \frac{\exp(\beta_0 + \mathbf x^\top\beta)}{1+ \exp(\beta_0 + \mathbf x^\top\beta)}$$ and then offcourse $$ \Pr(Y=0) = 1- \Pr(Y=1)=1 - \frac{\exp(\beta_0 + \mathbf x^\top\beta)}{1+ \exp(\beta_0 + \mathbf x^\top\beta)} = \frac{1}{1+\exp(\beta_0 + \mathbf x^\top\beta)}$$ ...


4

My first question is: Why is it starting from this? Should not it start from $P(\mathbf{t|w,X)}?$ The statement is Bayes theorem. It is used to derive the posterior distribution from prior distribution $P(\mathbf{w)}$ and likelihood $P(\mathbf{t|w,X)}$. I don't know what do you mean by "starting" from the likelihood, but in Bayesian setting, you need ...


4

there's nothing to predict Certainly there is. And your logistic regression outputs probabilistic predictions. Which is a Very Good Thing. So what you need to do is to assess the quality of probabilistic predictions. (Out-of-bag, as always in cross-validation.) The tool to do so is scoring-rules, which map probabilistic predictions and corresponding ...


3

Yes, the p-value for the effect of a 10-unit change in the covariate (years in this case) is the same as that for the effect of a 1-unit change. Here's the demonstration: the p-value for the original question (effect of a 1-unit change) is $P(|Z|>\frac{\hat\beta}{\sigma_{\hat\beta}})$, i.e. the tail area of the Normal distribution beyond the ratio of (...


3

Since $Y_i$ is a binary variable, its distribution is the Bernoulli distribution: $$Y_i | \mathbf{x}, \mathbf{w} \sim \text{Bern} \Bigg( \text{Prob} = \frac{\exp(\mathbf{w}^\text{T} \mathbf{x})}{1 + \exp(\mathbf{w}^\text{T} \mathbf{x})} \Bigg).$$ One alternative way of looking at the logistic regression is to regard the observed response variable as a ...


3

You can do whatever you want to simulate data. It's your data. If you have values between 0 and 1, you can choose to treat them as probabilities and generate a Bernoulli (0/1) variable using them. You could do this regardless of how you got those probabilities. That said, the data-generating model does not correspond to a logistic regression model, so ...


3

There is no such assumption about predictors. This is an easy mistake to make, because of the Gauss-Markov theorem requiring uncorrelated errors. Another way to make this mistake is by confusing independence of predictors and independence of predictions. I've made this mistake. Probably most of us have made this mistake. Welcome to the club of learning ...


2

No transformation is necessary to achieve linearity, because that isn't required. You simply have a curvilinear relationship between this variable and the outcome. So add a squared version of this variable in addition to the untransformed version. If you need a test of this variable, you would perform a simultaneous test of both terms. It may help you ...


2

My recommendation: use age in years as is, without any binning, and represent it with regression splines. Or Try generalized additive models (GAMs.) See for instance Are both of these generalized additive models? (and search this site, many posts.) One post with simple R code is Violation of linearity assumption in Logistic Regression.


2

In your link, you have the cumulative distribution function for the logistic distribution as $$\frac{1}{1+e^{-\frac{x-\mu}{s}}}$$ while in your question you have $$\dfrac{\exp(w^TX)}{(1+\exp(w^TX))} \text{ which is } \dfrac{1}{1+e^{-w^TX}}$$ and these are essentially the same so long as $w^TX$ has mean $0$ and variance $1$. You can see that the first ...


2

First of all, there's nothing wrong with your model. You haven't told us anything about the nature of the data (which appear to have been invented for the sake of demonstration) or the purpose of your model, so I don't see any point in talking about "improving" the model. It sounds like you would benefit from a better understanding of how correlated ...


2

Firstly, this is a very strange model --- it is a binomial GLM with an identity link function, which is a strange link function to choose in this context. In any case, even taking that model as fixed, the explanation in the book seems quite strange to me. Re-arranging the regression equation, and taking the variance of both sides (treating the explanatory ...


1

If every disease is either chronic or persistent, then you can remove the disease variable. If there are other ways to have a disease, then you can have diseasechronic, diseasepersistent and diseaseother for each. This will avoid collinearity; it becomes a little more complex to look at the overall effet of disease.


1

I use the repeats as I have a small dataset (<200) and would like tighter bounds on my model performance for significance testing. Repetitions of $k$-fold cross validation allow you to measure model instability, and the uncertainty related to that source of variance will go down if you average repetitions. But it doesn't do anything about the actual ...


1

One possibility that you didn't mention is to use ridge regression. That has a conceptual relationship to principal components regression (PCR), but instead of making an all-or-none choice of which principal components to include it weights the principal components differentially to avoid overfitting. The result is a separate penalized ridge regression ...


1

I would use model fitting to assess the likelihood of x2 being a specific value within a given range: Let’s say that there are 3 samples (i=3) Every sample is 0.31, 0.49 or 0.84 Distribution of x = average of all the unique x combinations ([0.31, 0.31, 0.31], [0.31, 0.31, 0.49] etc until max of [0.84, 0.84, 0.84]) For each possible value of x, compare the ...


1

You can fit a multiple logistic regression. But your larger goal is to make a claim about causality, which is rarely easy to do with observational data. In surveys/questionnaires, for example, you need to worry about nonresponse bias, reverse causation, and human biases associated with people's perceptions of themselves (I'm sure there are others too, this ...


1

an alternative explanation is the margin odds are incorporated into your fitted values. The ML gradient equations (set to 0) are equal to the following constraints.... $$\sum_i p_i = \sum_i y_i$$ $$\sum_i x_{1i}p_i = \sum_i x_{1i}y_i$$ ... $$\sum_i x_{ki}p_i = \sum_i x_{ki}y_i$$ Where $p_i$ is the fitted probability, $y_i$ is the 0-1 indicator you are ...


1

You can use a spline of the independent variable. These allow very flexible fitting of almost any shape of relationship between the IV and the DV (or, in this case, the log odd of the DV).


1

It is not only possible that different sets of predictors are used in different folds, it very likely in my experience. It is however OK to average the error estimates from each fold. You want the instability of the predictor set to go into the error estimate: You are not estimating the expected error of a specific model, you are estimating the expected ...


1

Your y-axis is not actually a probability, it's a self reported score. You've fit a linear model and constructed 95% CIs using a normal probability model for the mean response. It's moot to make specific pairwise comparisons between household income-deciles. The most meaningful summary of the data is the slope of the trendline, an expected difference in ...


1

Here's what I think is producing the pattern you see. Models are approximations of $\Pr(Y=1|X) = E[Y|X]$ using data $Y, X$. Compared to unregularised logistic regression, machine learning models typically do a much better job of that, judged by mean squared error, log loss etc, when $X$ is high dimensional because they are not as likely to pick up spurious ...


1

As Tim wrote above the logistic regression gives you a prediction of the probability of each class. Concretely, you will get $$P(C_a|D)$$ and $$P(C_n|D)$$ where $C_a$ and $C_n$ are the anomaly class and normal class and $D$ is your data. According to Baye's Theorem: $$P(C_a|D)= \frac{P(D|C_a) P(C_a)}{P(D)}$$ $$P(C_n|D)= \frac{P(D|C_n) P(C_n)}{P(D)}$$ ...


1

I agree with Kjetil's answer, if you have age in years available. If age was already binned into 5 categories, then splines probably won't work well as there are very few choices for the knots. You could try optimal scaling. This is available in the R package optiscale and in SAS PROC TRANSREG and probably in some other packages as well. But binning ...


Only top voted, non community-wiki answers of a minimum length are eligible