# Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

24,048 questions
Filter by
Sorted by
Tagged with
5k views

### Are t tests of coefficients in multiple regression post hoc tests?

In multiple regression, if a global F test is significant, then are t tests (or Wald tests) for the coefficients considered to be multiple comparisons and post hoc tests and should they be adjusted?
41k views

### What intuitively is “bias”?

I'm struggling to grasp the concept of bias in the context of linear regression analysis. What is the mathematical definition of bias? What exactly is biased and why/how? Illustrative example?
2k views

### Something more “accurate” than linear regression?

At the moment I'm using linear regression of 4 series with: mod <- lm(x ~ y + z + v + 0) # I need zero intercept I'm using the linear regression to calculate ...
1k views

### Analysing relation between changes in two time series

In relation to the question below I have uploaded the detrended plot and the differenced plot to the following links (I tried to add the images to the post but I got a 'new user' error msg). If ...
25k views

### Proxy variables versus instrumental variables

Very short question. What exactly is the difference between an instrumental variable and a proxy variable when building a regression model?
2k views

### prcomp() vs lm() results in R [duplicate]

I have a simple matrix: [,1] [,2] [,3] [1,] 1 2 3 [2,] 4 5 6 [3,] 7 8 9 [4,] 10 11 12 I have to calculate linear regression ...
292 views

### In which setting would you expect model found by LARS to differ most from the model found by exhaustive search?

A bit more info; suppose that you know before hand how many variables to select and that you set the complexity penalty in the LARS procedure such as to have exactly that many variables with non 0 ...
4k views

### Summary of “Large p, Small n” results

Can anybody point me to a survey paper on "Large $p$, Small $n$" results? I am interested in how this problem manifests itself in different research contexts, e.g. regression, classification, ...
7k views

### How to analyze residuals of Poisson log-linear model?

I have bird count data and use classical poisson loglinear model, i.e. we have counts obs(i,j) - observed count for site i and ...
2k views

### Fitting a line to a log-log plot

I have some data that I'm playing around with; for simplicity, let's suppose the data contains information on number of posts a blogger has written vs. number of people who have subscribed to that ...
90 views

### Do non-significant correlates of a DV, which is significant, suggest anything about the effect of that DV?

Let's assume one has an analysis in which there are multiple correlated DVs (average correlation .46) being examined in separate univariate analyses (e.g. t-tests; insufficient df and frustration ...
71 views

### Maximizing/minimizing product of output and response

I need to optimize based on an objective function that is non-standard, as far as I know. If the predictors are $X$, output of the model is $\hat y$, and response is $y$, my objective is essentially: ...
1k views

### How do I test whether an extrapolated mean for a regression model differs from an observed mean?

Suppose task performance $y$ increases with trial number $x$ ($x = 1, 2, …, 10$), so that there is a practice effect. Let’s suppose the practice effect is linear. Subjects have a long break, and then ...
1k views

### What does X represent in this linear trendline?

I'm trying to extend this graph via the provided trendline formula. I'm inserting the formula into the next row and trying to assign X to be the date value, but it doesn't appear to be working. Do I ...
5k views

### Estimating linear regression with OLS vs. ML

Assume that I'm going to estimate a linear regression where I assume $u\sim N(0,\sigma^2)$. What is the benefit of OLS against ML estimation? I know that we need to know a distribution of $u$ when we ...
434 views

### Weighted average semi-parametric regression in R [closed]

Are there any packages that supports weighted average semi-parametric regressions in R? An example of such a regression is in the links below. I see that there is package GAM in R for Generalized ...
37k views

### Is $R^2$ useful or dangerous?

I was skimming through some lecture notes by Cosma Shalizi (in particular, section 2.1.1 of the second lecture), and was reminded that you can get very low $R^2$ even when you have a completely linear ...
610 views

### Assuming $u\sim N(0,\sigma^2)$ when y is highly skewed

does it make sense to assume $u\sim N(0,\sigma^2)$ when I know from a histogram that $y$ is highly skewed. Because from the assumption $u\sim N(0,\sigma^2)$ it follows that $y\sim N(x\beta,\sigma^2)$ ...
59k views

### Simple linear regression output interpretation

I have run a simple linear regression of the natural log of 2 variables to determine if they correlate. My output is this: ...
277 views

### Countering false positives in GWAS due to linkage disequilibrium

We know that due to LD, we can get significant p-values for markers near a causal marker (or a marker closes to the causal region) in GWAS studies. I've seen attempts looking at LD to do multiple ...
241 views

### References about the theory of linear regression or regression in general

I am looking for references about the theory of linear regression or regression in general. More specifically, I am interested in knowing under what circumstances an estimated regressor is going to ...
755 views

### What is the expected value of the sample variance under a linear regression with omitted variables of an AR(2) process?

Lately, I have been interested in phenomenons related to omission of variables. For example, it can be shown that the expected value of the sample variance under the inclusion of one variable $x_1$ ...
17k views

### The order of variables in ANOVA matters, doesn't it?

Am I correct to understand that the order in which variables are specified in a multifactorial ANOVA makes a difference but that the order does not matter when doing a multiple linear regression? So ...
6k views

### Support vector machines and regression

There's already been an excellent discussion on how support vector machines handle classification, but I'm very confused about how support vector machines generalize to regression. Anyone care to ...
5k views

### Looking at residuals vs. residual percentages

Suppose I fit a linear regression to some data (say, weight vs. height), and all the standard linear regression assumptions are satisfied (in particular, the data is homoscedastic). For example, here'...
152 views

### Analyzing treatment effect with possibly flawed control data

I've got some rather messy data from a natural experiment. A number of subjects were measured (the measurements were hopefully-Poisson distributed counts and associated offsets), placed on a ...
599 views

### How to predict shares?

Lets say I know what is the overall budget for some units and I want to predict share of budget each unit will get. I have historical data and could do regression analysis. Is it better to predict ...
34k views

### What's the difference between binomial regression and logistic regression?

I've always thought of logistic regression as simply a special case of binomial regression where the link function is the logistic function (instead of, say, a probit function). From reading the ...
3k views

### Understanding intercept in simple linear regression and why one variable is a predictor and the other is an outcome variable

I'm a new statistics student :) I have some questions about linear regression, i'm using R to do some tests. I have two simple lists, like: ...
50k views

### What is the correct way to test for significant differences between coefficients?

I'm hoping someone can help straighten out a point of confusion for me. Say I want to test whether 2 sets of regression coefficients are significantly different from each other, with the following set ...
922 views

### Offset needed in regression when response is continuous?

I know that for poisson regressions on count data that originate from different sampling "sizes", i.e. different volumes, areas etc, require an offset in order to adjust for the different sizes. ...
941 views

### Book for broad and conceptual overview of statistical methods

I am very interested about the potential of statistical analysis for simulation/forecasting/function estimation, etc. However, I don't know much about it and my mathematical knowledge is still quite ...
17k views

### Building a linear model for a ratio vs. percentage?

Suppose I want to build a model to predict some kind of ratio or percentage. For example, let's say I want to predict the number of boys vs. girls who will attend a party, and features of the party I ...
194 views

### What exactly is the name of the type of regression analysis where you try to see if the model is significant over *multiple* start/end values?

An example is here: http://www.reddit.com/r/askscience/comments/ine4x/regarding_the_recent_lapse_of_global_warming_in/c2554al I'm sure it's related to robust statistics. But I'm sure that there's a ...
3k views

### Standardized residuals vs. regular residuals

I've got an easy question concerning residual analysis. So when I compute a QQ-Plot with standardized residuals $\widehat{d}$ on the y-axis and I observe normal distributed standardized residuals, why ...
264k views

### When is R squared negative?

My understanding is that $R^2$ cannot be negative as it is the square of R. However I ran a simple linear regression in SPSS with a single independent variable and a dependent variable. My SPSS output ...
787 views

7k views

### How to test whether a regression coefficient is moderated by a grouping variable?

I have a regression done on two groups of the sample based on a moderating variable (say gender). I'm doing a simple test for the moderating effect by checking whether the significance of the ...
115 views

So... let's say I have data that look something like this... (as I look at it now, in the actual data the red line is about 20% shorter than the black (at the high end... but you get the idea) I've ...
3k views

### Should quantitative predictors be transformed to be normally distributed?

I am always struggling with normality testing for quantitative predictors (no factors) and transforming them to normality. If I am running a GLMM and my predictors are really non-normal, should I ...
123 views

### How to combine 2 different observations to improve state estimates?

Context Let $\mathbf{x}_i \in \mathbb{R}^{100}$ and $\mathbf{z}_i \in \mathbb{R}^{20}$ be input vectors with the same corresponding target $\mathbf{y}_i \in \mathbb{R}^{25}$. Using ridge regression we ...
12k views

### Formula for weighted simple linear regression

This wiki page Simple linear regression has formulas to calculate $\alpha$ and $\beta$. Could anyone tell me how to derive the formulas in weighted case?