# Tag Info

19

Controlling for something and ignoring something are not the same thing. Let's consider a universe in which only 3 variables exist: $Y$, $X_1$, and $X_2$. We want to build a regression model that predicts $Y$, and we are especially interested in its relationship with $X_1$. There are two basic possibilities. We could assess the relationship between ...

5

Categorical variables can be represented several different ways in a regression model. The most common, by far, is reference cell coding. From your description (and my prior), I suspect that is what was used in your case. The standard statistical output will give you two tests. Let's say that A is the reference level, you will have a test of B vs. A, and ...

4

They are not ignored. If they were 'ignored' they would not be in the model. The estimate of the explanatory variable of interest is conditional on the other variables. The estimate is formed "in the context of" or "allowing for the impact of" the other variables in the model.

4

The question asks to test whether the slope equals $1$. This can most easily be conducted by regressing $\log(y) - \log(x)$ against $\log(x),$ but can also be done by post-processing the regression summary (when the original data might not be available, for instance). Analysis The model is $$\mathbb{E}[\log(y)] = \beta_0 + \beta_1\log(x).$$ Doubling ...

4

1: Are both correct? What is the difference? If you want to model the probability of occurrence based on the level of the predictor then you want to use logistic regression (a type of binomial GLM). For example, the probability of defaulting on a loan based on marital status. If you want to model the number of events based on the predictor level then ...

3

There are a variety of ordinal regression models (see Agresti) but they rely on certain assumptions. When those assumptions are violated, the models may become incorrect. The most common assumption is that of proportional odds. Multinomial regression does not make this assumption and can therefore model odds that are not proportional. However, an ...

3

Linear regression may be appropriate. You do not necessarily violate any of the following assumptions of linear regression: The relationship between dependent and independent variables are linear ($y$ is roughly twice that of $x_1$ so check). Errors terms are independent (you'll have to check this but there's no reason that $y$ always being more than or ...

3

The only real difference is in the regularisation that is applied. A regularised RBF network typically uses a penalty based on the squared norm of the weights. For the kernel version, the penalty is typically on the squared norm of the weights of the linear model implicitly constructed in the feature space induced by the kernel. The key practical ...

3

As written, your question can't work, since y is a 0-1 variable and you're doing logistic regression. If you mean that the linear predictor had a nonlinear relationship with one of the independent variables, that is, $\eta = a + bf(x)$, say, for some nonlinear $f$ (with all other variables held constant), then you can write $x^* = f(x)$ and put $x^*$ in ...

2

(1) Do you really need a smaller model? If not, you're set. (2) Can you honestly pre-specify your model? From your knowledge of the field can you choose a subset of predictors your interested in without using your knowledge of this dataset? If so, you're set. (2.5) If all your data valid? Assess this without looking at outcomes. (3) Consider using some form ...

2

The fact that you are using 4 out of 14 parameters implied that you used significance testing to select the variables. This is invalid. There are a number of other problems: Your total sample size is far too small for data splitting to be a reliable method You are seeking arbitrary cutoffs You are not using a proper accuracy score such as deviance, ...

2

The number of parameters (n+3, including the unknown error variance) exceeds the size of the sample (that's the "incidental parameters" situation). The two orthogonality restrictions that you mention reduce the number of unknowns by 2, no more. But if you are interested mainly in estimating the $\beta$'s, then these restrictions permit you (through a ...

1

You could replace your optimization penalties with their sample equivalents and minimize something like: $$Q(\beta, g) = \sum_i \left\{Y_i - \beta_0 - \beta_1 X_i - g(X_i)\right\}^2 \\ - c \sum_i (\hat g_i'')^2 - \lambda_1 \sum_i g_i - \lambda_2 \sum_i g_i X_i ,$$ Here $g_i = g(X_i)$, $\hat g_i'= (g_{i+1}-g_i)/(X_{i+1}-X_i)$ and $\hat g_i''= (\hat ... 1 The key difference is in the training criterion. A least squares training criterion is often used for regression as this gives (penalised) maximum likelihood estimation of the model parameters assuming Gaussian noise corrupting the response (target) variable. For classification problems it is common to use a cross-entropy training criterion, to give ... 1 Giving the author and title of the book would be helpful in deciphering the author's intention, since readers here might have read it. But based on this information, it would appear to simply be a teaching approach intended to simplify the problem for illustrative purposes. Rather than estimating both parameters at once, the author estimates them in turn ... 1 As you mention, the error are errors of prediction--the closer you are to the observed value with your regression equation, the smaller the error. Therefore, to reduce the error, you need to improve your prediction. To do so, you would add other predictors to your model that are related to your dependent variable. 1 The comment made by @user32164 still stands as I write: "highly correlated with a poor$R^2$" is contradictory. Regardless of what you consider as highly correlated, a high correlation means a high$R^2$. I am assuming that you measured color somehow so that it may fairly be used as a quantitative predictor in a regression model. Whether that's so is an ... 1 You have a case of data-dependent truncated regression model, which is easily handled by the R package truncreg. Here is some code that simulates such data, and then estimates the parameters of it: library(truncnorm) library(truncreg) truncPoints = rnorm(100) vBeta = c(1, 3, 6) mX = matrix(rnorm(100*3), nrow = 100) vY = rtruncnorm(100, a=truncPoints, ... 1 Assuming that the regressors are deterministic, and so they do not have a variance of their own (or alternatively that we are considering the variance of the fitted dependent variable conditional on the specific realizations of the regressors that comprise our sample of observations), we have (for a sample of$n$observations and for$k+1$regressors ... 1 Let$X_1$and$X_2$denote two random variables with variances$\sigma^2_{X1}$and$\sigma^2_{X2}$. Let$Y = X_1 + X_2$. The variance of$Y$is then equal to$\sigma^2_{X1} + \sigma^2_{X2} + \rho \sigma_{X1} \sigma_{X2}$, where$\rho$is the correlation between$X_1$and$X_2$. So, unless the two variables are uncorrelated, the sum of the variances will not ... 1 You can plot the predicted value of Y from the equation at various levels of$X_1$, but you have to choose values of$X_2$and$X_3\$ for those lines. One choice would be the median value of each. Taking a step back: 1) Are you sure you want multiple regression on a time series? This can cause some problems (e.g. if Y and any of the X are both increasing ...

1

One solution is actually given as an example in the book on the multcomp package, section 4.6: Bretz, F., Hothorn, T., & Westfall, P. H. (2011). Multiple comparisons using R. Boca Raton, FL: CRC Press. One only needs to slightly adapt your code (everything needs to be in one data.frame instead of floating around): require(multcomp) require(sandwich) ...

1

Regardless of what level you set as reference, the resulting model fit will be equivalent. You are probably interested in if there are there are mean differences in the substrate levels. After fitting the model you should run a Tukey Post-Hoc comparison to see which levels differ. TukeyHSD function in R. Again, it does not matter on what the reference ...

1

If the categories have order, one can use either nominal or ordinal. If the categories do not have order, one must use nominal. When there is order, one should perform model diagnostics to see which of ordinal and nominal is preferred. Although, the ordinal may have better interpretation. In general, for comparison among different models, one should ...

1

Interpretation of complex interactions is always tricky. The best way, I think, is to graph them. One way to start is with 3 graphs, one for each IV. For each of these graphs, one IV will be on the x-axis, and the the DV on the y-axis. Then make lines for the predicted value of the DV for each of several combinations of the other two IVs (e.g. quartiles). ...

1

Yes, ridge regression can be used as a classifier, just code the response labels as -1 and +1 and fit the regression model as normal. Allen's PRESS statistic (i.e. the leave-one-out estimate of the squared error) works fine as a model selection criterion (e.g. for selecting the ridge parameter). In my experience it works about as well as a linear support ...

1

if Y and X are both normally distributed, Y/X follows the Cauchy distribution, (which has several strange properties... for example, the mean is not a measure of central tendency in this distribution... and the properties of the Cauchy distribution might be what is messing up the power in your regressions).

1

While they work differently than your algorithm, I believe you'll find mob() and FTtree interesting. For Zeileis' mob see http://cran.r-project.org/web/packages/party/vignettes/MOB.pdf For FTtree,Gama's functional trees an implementation is available in Weka and thus RWeka. See http://cran.r-project.org/web/packages/RWeka/index.html for details.

Only top voted, non community-wiki answers of a minimum length are eligible