Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
8
votes
2answers
5k views

Are t tests of coefficients in multiple regression post hoc tests?

In multiple regression, if a global F test is significant, then are t tests (or Wald tests) for the coefficients considered to be multiple comparisons and post hoc tests and should they be adjusted?
24
votes
3answers
41k views

What intuitively is “bias”?

I'm struggling to grasp the concept of bias in the context of linear regression analysis. What is the mathematical definition of bias? What exactly is biased and why/how? Illustrative example?
-1
votes
1answer
2k views

Something more “accurate” than linear regression?

At the moment I'm using linear regression of 4 series with: mod <- lm(x ~ y + z + v + 0) # I need zero intercept I'm using the linear regression to calculate ...
3
votes
2answers
1k views

Analysing relation between changes in two time series

In relation to the question below I have uploaded the detrended plot and the differenced plot to the following links (I tried to add the images to the post but I got a 'new user' error msg). If ...
8
votes
2answers
25k views

Proxy variables versus instrumental variables

Very short question. What exactly is the difference between an instrumental variable and a proxy variable when building a regression model?
2
votes
1answer
2k views

prcomp() vs lm() results in R [duplicate]

I have a simple matrix: [,1] [,2] [,3] [1,] 1 2 3 [2,] 4 5 6 [3,] 7 8 9 [4,] 10 11 12 I have to calculate linear regression ...
9
votes
2answers
292 views

In which setting would you expect model found by LARS to differ most from the model found by exhaustive search?

A bit more info; suppose that you know before hand how many variables to select and that you set the complexity penalty in the LARS procedure such as to have exactly that many variables with non 0 ...
20
votes
4answers
4k views

Summary of “Large p, Small n” results

Can anybody point me to a survey paper on "Large $p$, Small $n$" results? I am interested in how this problem manifests itself in different research contexts, e.g. regression, classification, ...
4
votes
3answers
7k views

How to analyze residuals of Poisson log-linear model?

I have bird count data and use classical poisson loglinear model, i.e. we have counts obs(i,j) - observed count for site i and ...
8
votes
1answer
2k views

Fitting a line to a log-log plot

I have some data that I'm playing around with; for simplicity, let's suppose the data contains information on number of posts a blogger has written vs. number of people who have subscribed to that ...
3
votes
2answers
90 views

Do non-significant correlates of a DV, which is significant, suggest anything about the effect of that DV?

Let's assume one has an analysis in which there are multiple correlated DVs (average correlation .46) being examined in separate univariate analyses (e.g. t-tests; insufficient df and frustration ...
2
votes
0answers
71 views

Maximizing/minimizing product of output and response

I need to optimize based on an objective function that is non-standard, as far as I know. If the predictors are $X$, output of the model is $\hat y$, and response is $y$, my objective is essentially: ...
4
votes
2answers
1k views

How do I test whether an extrapolated mean for a regression model differs from an observed mean?

Suppose task performance $y$ increases with trial number $x$ ($x = 1, 2, …, 10$), so that there is a practice effect. Let’s suppose the practice effect is linear. Subjects have a long break, and then ...
-1
votes
1answer
1k views

What does X represent in this linear trendline?

I'm trying to extend this graph via the provided trendline formula. I'm inserting the formula into the next row and trying to assign X to be the date value, but it doesn't appear to be working. Do I ...
8
votes
3answers
5k views

Estimating linear regression with OLS vs. ML

Assume that I'm going to estimate a linear regression where I assume $u\sim N(0,\sigma^2)$. What is the benefit of OLS against ML estimation? I know that we need to know a distribution of $u$ when we ...
2
votes
0answers
434 views

Weighted average semi-parametric regression in R [closed]

Are there any packages that supports weighted average semi-parametric regressions in R? An example of such a regression is in the links below. I see that there is package GAM in R for Generalized ...
260
votes
6answers
37k views

Is $R^2$ useful or dangerous?

I was skimming through some lecture notes by Cosma Shalizi (in particular, section 2.1.1 of the second lecture), and was reminded that you can get very low $R^2$ even when you have a completely linear ...
1
vote
1answer
610 views

Assuming $u\sim N(0,\sigma^2)$ when y is highly skewed

does it make sense to assume $u\sim N(0,\sigma^2)$ when I know from a histogram that $y$ is highly skewed. Because from the assumption $u\sim N(0,\sigma^2)$ it follows that $y\sim N(x\beta,\sigma^2)$ ...
20
votes
6answers
59k views

Simple linear regression output interpretation

I have run a simple linear regression of the natural log of 2 variables to determine if they correlate. My output is this: ...
2
votes
0answers
277 views

Countering false positives in GWAS due to linkage disequilibrium

We know that due to LD, we can get significant p-values for markers near a causal marker (or a marker closes to the causal region) in GWAS studies. I've seen attempts looking at LD to do multiple ...
3
votes
2answers
241 views

References about the theory of linear regression or regression in general

I am looking for references about the theory of linear regression or regression in general. More specifically, I am interested in knowing under what circumstances an estimated regressor is going to ...
7
votes
1answer
755 views

What is the expected value of the sample variance under a linear regression with omitted variables of an AR(2) process?

Lately, I have been interested in phenomenons related to omission of variables. For example, it can be shown that the expected value of the sample variance under the inclusion of one variable $x_1$ ...
23
votes
2answers
17k views

The order of variables in ANOVA matters, doesn't it?

Am I correct to understand that the order in which variables are specified in a multifactorial ANOVA makes a difference but that the order does not matter when doing a multiple linear regression? So ...
27
votes
2answers
6k views

Support vector machines and regression

There's already been an excellent discussion on how support vector machines handle classification, but I'm very confused about how support vector machines generalize to regression. Anyone care to ...
2
votes
1answer
5k views

Looking at residuals vs. residual percentages

Suppose I fit a linear regression to some data (say, weight vs. height), and all the standard linear regression assumptions are satisfied (in particular, the data is homoscedastic). For example, here'...
6
votes
2answers
152 views

Analyzing treatment effect with possibly flawed control data

I've got some rather messy data from a natural experiment. A number of subjects were measured (the measurements were hopefully-Poisson distributed counts and associated offsets), placed on a ...
1
vote
2answers
599 views

How to predict shares?

Lets say I know what is the overall budget for some units and I want to predict share of budget each unit will get. I have historical data and could do regression analysis. Is it better to predict ...
25
votes
2answers
34k views

What's the difference between binomial regression and logistic regression?

I've always thought of logistic regression as simply a special case of binomial regression where the link function is the logistic function (instead of, say, a probit function). From reading the ...
2
votes
2answers
3k views

Understanding intercept in simple linear regression and why one variable is a predictor and the other is an outcome variable

I'm a new statistics student :) I have some questions about linear regression, i'm using R to do some tests. I have two simple lists, like: ...
22
votes
2answers
50k views

What is the correct way to test for significant differences between coefficients?

I'm hoping someone can help straighten out a point of confusion for me. Say I want to test whether 2 sets of regression coefficients are significantly different from each other, with the following set ...
2
votes
1answer
922 views

Offset needed in regression when response is continuous?

I know that for poisson regressions on count data that originate from different sampling "sizes", i.e. different volumes, areas etc, require an offset in order to adjust for the different sizes. ...
12
votes
9answers
941 views

Book for broad and conceptual overview of statistical methods

I am very interested about the potential of statistical analysis for simulation/forecasting/function estimation, etc. However, I don't know much about it and my mathematical knowledge is still quite ...
22
votes
2answers
17k views

Building a linear model for a ratio vs. percentage?

Suppose I want to build a model to predict some kind of ratio or percentage. For example, let's say I want to predict the number of boys vs. girls who will attend a party, and features of the party I ...
2
votes
1answer
194 views

What exactly is the name of the type of regression analysis where you try to see if the model is significant over *multiple* start/end values?

An example is here: http://www.reddit.com/r/askscience/comments/ine4x/regarding_the_recent_lapse_of_global_warming_in/c2554al I'm sure it's related to robust statistics. But I'm sure that there's a ...
7
votes
3answers
3k views

Standardized residuals vs. regular residuals

I've got an easy question concerning residual analysis. So when I compute a QQ-Plot with standardized residuals $\widehat{d}$ on the y-axis and I observe normal distributed standardized residuals, why ...
129
votes
3answers
264k views

When is R squared negative?

My understanding is that $R^2$ cannot be negative as it is the square of R. However I ran a simple linear regression in SPSS with a single independent variable and a dependent variable. My SPSS output ...
0
votes
0answers
787 views

Standard deviation when estimating a Poisson regression using R

I'm interested in plotting the estimator of the standard deviation in a Poisson regression. The variance is $Var(y)=\phi⋅V(\mu)$ where $\phi=1$ and $V(\mu)=\mu$. So the variance should be $Var(y)=V(\...
1
vote
0answers
53 views

How to predict significant dominant regions of two sequence of numeric values by Hidden Markov Model?

I have few training on Hidden Markov Model. But, I intend to solve my problem by HMM. I would like to have your helps/directions to me. Here, I have two variables to define the 8 one-dimension space (...
22
votes
4answers
93k views

Maximum number of independent variables that can be entered into a multiple regression equation

What is the limit to the number of independent variables one may enter in a multiple regression equation? I have 10 predictors that I would like to examine in terms of their relative contribution to ...
6
votes
2answers
3k views

Estimating standard deviation in Poisson regression

I'm interested in an estimator of the standard deviation in a Poisson regression. So the variance is $$Var(y)=\phi\cdot V(\mu)$$ where $\phi=1$ and $V(\mu)=\mu$. So the variance should be $Var(y)=V(...
11
votes
2answers
7k views

How to test whether a regression coefficient is moderated by a grouping variable?

I have a regression done on two groups of the sample based on a moderating variable (say gender). I'm doing a simple test for the moderating effect by checking whether the significance of the ...
2
votes
1answer
115 views

Inferences about non-overlapping lines

So... let's say I have data that look something like this... (as I look at it now, in the actual data the red line is about 20% shorter than the black (at the high end... but you get the idea) I've ...
4
votes
1answer
3k views

Should quantitative predictors be transformed to be normally distributed?

I am always struggling with normality testing for quantitative predictors (no factors) and transforming them to normality. If I am running a GLMM and my predictors are really non-normal, should I ...
2
votes
1answer
123 views

How to combine 2 different observations to improve state estimates?

Context Let $\mathbf{x}_i \in \mathbb{R}^{100}$ and $\mathbf{z}_i \in \mathbb{R}^{20}$ be input vectors with the same corresponding target $\mathbf{y}_i \in \mathbb{R}^{25}$. Using ridge regression we ...
12
votes
2answers
12k views

Formula for weighted simple linear regression

This wiki page Simple linear regression has formulas to calculate $\alpha$ and $\beta$. Could anyone tell me how to derive the formulas in weighted case?
14
votes
2answers
2k views

Box-Jenkins model selection

The Box-Jenkins model selection procedure in time series analysis begins by looking at the autocorrelation and partial autocorrelation functions of the series. These plots can suggest the appropriate $...
32
votes
1answer
25k views

Equivalence between least squares and MLE in Gaussian model

I am new to Machine Learning, and am trying to learn it on my own. Recently I was reading through some lecture notes and had a basic question. Slide 13 says that "Least Square Estimate is same as ...
13
votes
3answers
7k views

Software package to solve L-infinity norm linear regression

Is there any software package to solve the linear regression with the objective of minimizing the L-infinity norm.
3
votes
1answer
1k views

Perform simple regression without raw data

I have a dataset that I can collect some quantities from, eg. sum,mean,variance... I want to perform a simple regression on column(x,y). According to Wikipedia, the closed form for $\alpha,\beta$ is \...
19
votes
1answer
17k views

Constrained linear regression through a specified point

I have a point (x,y) that I need a linear regressor to pass through given a data set (X,Y). How do I implement this in R?

1
471 472
473
474 475
481