Tag Info

15

Methods of censored regression can handle data like this. They assume the residuals behave as in ordinary linear regression but have been modified so that (Left censoring): all values smaller than a low threshold, which is independent of the data, (but can vary from one case to the other) have not been quantified; and/or (Right censoring): all values ...

12

John Fox's book An R companion to applied regression is an excellent ressource on applied regression modelling with R. The package car which I use throughout in this answer is the accompanying package. The book also has as website with additional chapters. Transforming the response (aka dependent variable, outcome) Box-Cox transformations offer a ...

10

There exist a number of frequenly mentioned regressional effects which conceptually are different but share much in common when seen purely statistically (see e.g. this paper or Wikipedia articles): Mediator: IV which conveys effect (totally of partly) of another IV to the DV. Confounder: IV which constitutes or precludes, totally or partly, effect of ...

9

Well, first of, the dummy variable is interpreted as a change in intercept. That is, your coefficient $\beta_3$ gives you the difference in the intercept when $D=1$, i.e. when $D=1$, the intercept is $\beta_0 + \beta_3$. That interpretation doesn't change when adding the squared $x_1$. Now, the point of adding a squared to the series is that you assume ...

9

Let us first distinguish between perfect multi-collinearity (model matrix not of full rank, so that usual matrix inversions fail. Usually due to misspecification of the predictors) and non-perfect multi-collinearity (some of the predictors are correlated without leading to computational problems). This answer is about the second type, which occurs in almost ...

8

$$\mathbf X'\mathbf e = \mathbf X'(\mathbf y -\mathbf {\hat y})= \mathbf X'(\mathbf y -\mathbf X\hat \beta) =...$$ ADDENDUM $$=\mathbf X'\left(\mathbf y -\mathbf X (\mathbf X'\mathbf X)^{-1}\mathbf X' \mathbf y\right) =\mathbf X'\mathbf y -\mathbf X'\mathbf X (\mathbf X'\mathbf X)^{-1}\mathbf X' \mathbf y$$ $$\mathbf X'\mathbf y -\mathbf X' \mathbf y = ... 7 A good example of including square of variable comes from labor economics. If you assume y as wage (or log of wage) and x as an age, then including x^2 means that you are testing the quadratic relationship between an age and wage earning. Wage increases with the age as people become more experienced but at the higher age, wage starts to increase at ... 7 Evaluation of analytic adjustments to R-square @ttnphns referred me to the Yin and Fan (2001) article that compares different analytic methods of estimating R^2. As per my question they discriminate between two types of estimators. They use the following terminology: \rho^2: Estimator of the squared population multiple correlation coefficient ... 7 There are two points here: The passage recommends transforming IVs to linearity only when there is evidence of nonlinearity. Nonlinear relationships among IVs can also cause collinearity and, more centrally, may complicate other relationships. I am not sure I agree with the advice in the book, but it's not silly. Certainly very strong linear relationships ... 6 There are many things you could do to select variables from multiply imputed data, but not all yield appropriate estimates. See Wood et al (2008) Stat Med for a comparison of various possibilities. I have found the following two-step procedure useful in practice. Apply your preferred variable selection method independently to each of the m imputed data ... 6 "Dichotomous Predictor Variables", there are two ways to code dichotomous predictors: using the contrast 0,1 or the contrast 1,-1. This is factually wrong. There is no limit to the number of ways they can be coded. Those two are merely the most common (indeed between them, almost ubiquitous), and probably the easiest to deal with. I kind of ... 6 You should tell us more about the nature of your response (outcome, dependent) variable. From your first plot it is strongly positively skewed with many values near zero and some negative. From that it is possible, but not inevitable, that transformation would help you, but the most important question is whether transformation would make your data closer to ... 6 If all aspects of a test are specified in advance and its assumptions are met, you can safely conclude that the null hypothesis will be rejected erroneously at the frequency defined by the error level. If you conduct several tests (a “family” of tests), each of these tests is an additional occasion to commit this error. Each individual test might still have ... 6 The first thing you need to do is make a matrix, where each column is a word and each row is a document. The entries in this matrix will be counts of that word's frequency in the document. This is known as the bag of words model. You might consider removing words that occur in almost all documents as well as words that occur in almost no documents. It ... 5 Are the values always between 0 and 1? If so you might consider a beta distribution and beta regression. But make sure to think through the process that leads to your data. You could also do a 0 and 1 inflated model (0 inflated models are common, you would probably need to extend to 1 inflated by your self). The big difference is if those spikes ... 5 Consider s \neq t. The law of iterated expectations tells us that$$\begin{equation*} \mathbb{E}[u_su_t \mid y_{s-1}, y_{t-1}] = \mathbb{E}_{u_s}\left[\mathbb{E}[u_su_t \mid u_s, y_{s-1}, y_{t-1}]\right] = \mathbb{E}_{u_s}\left[u_s\mathbb{E}[u_t \mid u_s, y_{s-1}, y_{t-1}]\right]. \end{equation*}  Suppose that $s < t$, with the opposite case ...

5

Is that possible? Yes. Stepwise regression pursues to maximize overall (joint) prediction by the variables left in the model while attempting to minimize their number. Because variables usually intercorrelate, their relations are complex and the significance level of the variable in the model after removing some other variables from it can change ...

5

A parameter estimate in a regression model (e.g., $\hat\beta_i$) will change if a variable, $X_j$, is added to the model that is: correlated with that parameter's corresponding variable, $X_i$ (which was already in the model), and correlated with the response variable, $Y$ An estimated beta will not change when a new variable is added, if either of ...

5

You are apparently applying classification terminology to hypothesis testing. That's fine but couching the problem in the more traditional type I error/type II error/power terms might help relate this to the multiple testing literature. Hypothesis tests can be presented in this way: Null hypothesis (H0) is true H0 is ...

5

You seem to be confusing some things. 1) Regression does not require normally distributed data, it assumes normally distributed errors (which you approximate by residuals) 2) The plots you give don't give good evidence of normality or non-normality; try a quantile-normal plot 3) A single variable can't be linear or not linear; linearity is a quality of ...

5

Very nice work. I think this situation is a candidate for the proportional odds semiparametric ordinal logistic model. The lrm function in the R rms package will fit the model. For now you may want to round $Y$ to have only 100-200 levels. Soon a new version of rms will be released with a new function orm that efficiently allows for thousands of ...

5

Use dichotomous indicators (often referred to as dummy variables) to represent the items within this one question. For example, a variable called internet, and then a variable called newspaper, so on so forth. If a person picked both, they got a 1 in each. If a person only picked newspaper, then enter 0 for internet and then 1 for newspaper.

5

What the model consists of depends, partly, on what you're doing, and partly on whatever conventions for writing a model you're using. I'll give some examples, assuming you want simple linear regression. Typically a (statistical) model will give a distribution for the $y$ values, like so: (i) $y_i = \beta_0 + \beta_1 x_i + \varepsilon_i$; where ...

5

Here is another geometric view of suppression, but rather than being in the observation space as @ttnphns's example is, this one is in the variable space, the space where everyday scatterplots live. Consider a regression $\hat{y}_i=x_i+z_i$, that is, the intercept is 0 and both predictors have a partial slope of 1. Now, the predictors $x$ and $z$ may ...

4

In my experience, the term outliers doesn't make sense without the context of the application. That is, if you want to exclude data points from your data set, you should be able to give reasons why this or that data point is removed. These reasons may suggest appropriate filtering rules. Therefore I think that something like the "recommended R ...

4

If you have an exact linear relationship in your independent variables (a bit more common jargon than predictor variables) like $c=a+b$, then you cannot apply the regression purposefully. In other words, it is misspecified. A statistical software will usually come up with an error message here. Intuitively, there is no unique estimator as there is no room ...

4

No, I don't think you should be concerned about the R-squared directly. Here's an example. R squared must be increasing, but because of precision, you might not be seeing it. First generate some data: library(MASS) sigma <- matrix(c(1.0, 0.8, 0.8, 0.4, 0.8, 1.0, 0.7, 0.4, 0.8, 0.7, 1.0, 0.4, ...

4

I would think lme4 would be highly appropriate for this. Treat your huge categorical factor as a practical random effect. I won't go into the theoretical definitions. Alternatively, use sparse.model.matrix() from Matrix to build the design frame and then pass that into glmnet() from glmnet package. (lme4 naturally builds the sparse design matrix so you ...

4

The best way is by thinking about which interactions make sense, are interesting, are suggested by theory, etc. Stepwise is not something I'd recommend. The whole "automated process" is something I wouldn't recommend - after all, if two way interactions should be explored this way, why not three way? Or four? If you really have no theory to guide you, ...

4

The model that you wrote is (or appears to be) a mixed-effects model. In your specific case, with a count DV, it would usually be called a generalized linear mixed model (GLMM). These mixed models can handle the type of data that you mentioned. There are many variants of these mixed models, although many (not all) of the terms floating around are just ...

Only top voted, non community-wiki answers of a minimum length are eligible