# Tag Info

18

Controlling for something and ignoring something are not the same thing. Let's consider a universe in which only 3 variables exist: $Y$, $X_1$, and $X_2$. We want to build a regression model that predicts $Y$, and we are especially interested in its relationship with $X_1$. There are two basic possibilities. We could assess the relationship between ...

4

They are not ignored. If they were 'ignored' they would not be in the model. The estimate of the explanatory variable of interest is conditional on the other variables. The estimate is formed "in the context of" or "allowing for the impact of" the other variables in the model.

4

Yes, it had been tried (including by myself - I tried it with neural nets, with rather mixed success). The Relevance Vector Machine (RVM) does pretty much exactly that, and the regularisation parameters are tuned by maximising the marginal likelihood. The advantage of this is that it leads to a sparse model where uninformative attributes end up with large ...

2

Just some strayed thoughts. 1) You fit a multiple regression to examine the effect of a particular variable a worker in another department is interested in. The variable comes back insignificant, but your co-worker says that this is impossible as it is known to have an effect. What would you do? Many reasons could have caused this: The study is ...

2

Package segmented could help you: Given a linear regression model (of class "lm" or "glm"), segmented tries to estimate a new model having broken-line relationships with the variables specified in seg.Z. A segmented (or broken-line) relationship is defined by the slope parameters and the break-points where the linear relation changes. The number ...

2

Probably the best way to gain an understanding is through graphs and through example computations. You don't say how many other variables you have in the model, nor whether they are continuous or categorical or what, but.... Suppose you have a model: $Y = 12 + 3x_1^{\frac{1}{3}} + 2x_2$ You could calculate predicted Y at various typical levels of $x_1$ ...

2

This is a good question and one that hasn't really been answered in the literature. But I would rather emphasize (1) what is the original goal, (2) what is an acceptable final (unbiased) $R^2$, and (3) is our approach good in relationship to other possible modeling approaches, i.e., could someone using the same dataset do significantly better than us? ...

2

To avoid blackening the place, I will not use bold symbols -but the answer will be carried out in matrix form. Vectors are column vectors, a prime will denote the transpose. Let a linear regression model $$y = X_1b_1 + X_2b_2 + u_A \qquad [A]$$ The normal equations for the OLS estimator are \begin{align} ... 2 (1) Most stepwise selection procedures use the p-value from a z or t-score (2) Most variable selection methods - including stepwise - have been extended to glm including negative binomial models. So yes, you can use stepwise variable selection. (3) Almost all statisticians express concern about stepwise variable selection, but in many fields (the biomedical ... 2 Multiple linear regression coefficient and partial correlation are directly linked and have the same significance (p-value). Partial r is just another way of standardizing the coefficient, along with beta coefficient. So, if the DV is y and the IVs are x_1 and x_2 then\beta_{x_1} = \frac{r_{yx_1} - r_{yx_2}r_{x_1x_2} }{\sqrt{1-r_{x_1x_2}^2}} ...

1

You can plot the predicted value of Y from the equation at various levels of $X_1$, but you have to choose values of $X_2$ and $X_3$ for those lines. One choice would be the median value of each. Taking a step back: 1) Are you sure you want multiple regression on a time series? This can cause some problems (e.g. if Y and any of the X are both increasing ...

1

do I ALWAYS get a tighter confidence interval if I include more variables in my model? Yes, you do. Here's why: adding more variables reduces the SSE and thereby the variance of the model, on which your confidence and prediction intervals depend. This even happens (to a lesser extent) when the variables you are adding are completely independent of the ...

1

The skewness of the outcome variable (treated unconditionally on the other variables) will depend on the arrangement of the independent variables -- it might validly be anything. You shouldn't be trying to make the distribution of the outcome look like any particular thing. It's the error term the normal assumption is needed for. Normality of residuals ...

1

If the DV is ordinal, as yours is, you should do ordinal logistic regression. However, ordinal logistic regression can also be hierarchical and multiple: Those terms refer to the number of independent variables and how they are entered into the regression. "Multiple" means there are more than one IV and "hierarchical" means they are entered into the ...

1

1) This is a multiple regression. Presumably there are other variables. It may be the case that the specific variable in question has an effect on the response, but in the presence of the other variables its effect is not significant. I'd use partial least squares pairing the variable in question with the other variables to determine which other variables ...

1

(1) This doesn't quite make sense. "What would you do" is very general. Often variables that are known to be significant/related to the outcome are included in a regression model even if they are found not to be significant (to increase confidence in that you are providing unbiased estimates for the other coefficients). So one answer would be you keep the ...

1

As near as I can make out, you have 1000+ patients, one disease which the patients either have or do not have, and about 600 genetic markers, for which you have number of copies of that marker. What you have done with that is to run a polynomial regression of the number of repeats against the disease prevalence. You then have some kind of model (you don't ...

1

Are these the "not very nice" formulas that you found? (Note that $\hat{b}_1$ and $\hat{b}_2$ have the same denominator.) $\hat{b}_1 = (s_{zz}\,s_{xy} - s_{xz}\,s_{zy})\,/\,(s_{xx}\,s_{zz} - s_{xz}^2)$ $\hat{b}_2 = (s_{xx}\,s_{zy} - s_{xz}\,s_{xy})\,/\,(s_{xx}\,s_{zz} - s_{xz}^2)$ $\hat{b}_0 = \overline{y}$

Only top voted, non community-wiki answers of a minimum length are eligible