Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
29
votes
1answer
19k views

Is interaction possible between two continuous variables?

All of my variables are continuous. There are no levels. Is it possible to even have interaction between the variables?
14
votes
1answer
3k views

LARS vs coordinate descent for the lasso

What are the pros and cons of using LARS [1] versus using coordinate descent for fitting L1-regularized linear regression? I am mainly interested in performance aspects (my problems tend to have ...
14
votes
2answers
6k views

Coordinate descent for the lasso or elastic net

Are there any good papers or books dealing with the use of coordinate descent for L1 (lasso) and/or elastic net regularization for linear regression problems?
18
votes
4answers
20k views

Can I simply remove one of two predictor variables that are highly linearly correlated?

Using Pearson's Correlation Coefficient, I have several variables that are highly correlated ($\rho = 0.978$ and $\rho = 0.989$ for 2 pairs of variables that are in my model). The reason some of the ...
42
votes
6answers
33k views

What are best practices in identifying interaction effects?

Other than literally testing each possible combination of variable(s) in a model (x1:x2 or x1*x2 ... xn-1 * xn). How do you ...
20
votes
1answer
13k views

Logistic Regression - Multicollinearity Concerns/Pitfalls

In Logistic Regression, is there a need to be as concerned about multicollinearity as you would be in straight up OLS regression? For example, with a logistic regression, where multicollinearity ...
47
votes
1answer
36k views

Regression: Transforming Variables

When transforming variables, do you have to use all of the same transformation? For example, can I pick and choose differently transformed variables, as in: Let, $x_1,x_2,x_3$ be age, length of ...
18
votes
5answers
13k views

How to add periodic component to linear regression model?

I have some cumulative frequency data. A line $y=ax+b$ looks like it fits the data extremely well, but there is cyclic/periodic wiggle in the line. I would like to estimate when the cumulative ...
2
votes
1answer
7k views

F-test for Lack-of-Fit in SPSS

Some googling revealed that doing the F-test for Lack-of-Fit in SPSS is not so trivial. It seems one has to “trick” SPSS to do that. See for example this. Can anybody suggest a better source of ...
40
votes
6answers
16k views

Least-angle regression vs. lasso

Least-angle regression and the lasso tend to produce very similar regularization paths (identical except when a coefficient crosses zero.) They both can be efficiently fit by virtually identical ...
5
votes
1answer
197 views

Toy regression question with latent variables

I originally asked this on a machine learning site, but one of the responses made me think that maybe this site is more suitable. Suppose you have two weighted coins, and every day you flip each one ...
17
votes
2answers
44k views

Interpreting the drop1 output in R

In R, the drop1command outputs something neat. These two commands should get you some output: example(step)#-> swiss ...
12
votes
1answer
786 views

Updating the lasso fit with new observations

I am fitting an L1-regularized linear regression to a very large dataset (with n>>p.) The variables are known in advance, but the observations arrive in small chunks. I would like to maintain the ...
5
votes
3answers
3k views

Java implementations of the lasso [closed]

Are there any open-source Java implementations of lasso or least angles regression? Pure Java code would be best, but clean implementations in other languages would also be of interest. I am already ...
3
votes
2answers
3k views

Explaining variation in a dependent variable based on a factorial experiment

I have run a factorial type test in a processing plant and have run a forward and backward step regression in R. How can I use the regression results and the anova created from the regression to ...
10
votes
1answer
4k views

In R, does “glmnet” fit an intercept?

I am fitting a linear model in R using glmnet. The original (non-regularized) model was fitted using lm and did not have a ...
2
votes
1answer
1k views

Regression on a triangular shaped region of points representing a symmetric relation

I plotted a set of about 200,000 points and got a triangular shaped region. The shape is roughly like the triangle made by the points $(1,0)$, $(0,1)$ and $(0,0)$. My points have the property that ...
65
votes
7answers
170k views

Regression with multiple dependent variables?

Is it possible to have a (multiple) regression equation with two or more dependent variables? Sure, you could run two separate regression equations, one for each DV, but that doesn't seem like it ...
1
vote
2answers
15k views

How to compute goodness of fit for a linear model in R

I have fit a linear model using the lm function in R... model <- lm(trans.baseline.CD4 ~ hiv$Julian.Date) ... and I would ...
4
votes
2answers
2k views

Validating a linear model with R, lm() [closed]

I've created a model (cue ominous music) in R based on previous months data using lm(). Now, I would like to see how well it predicts the current months data. For example, my model predicts sales ...
1
vote
4answers
615 views

Resources to learn about block bootstrap in time series analysis

Way back when, I used to work in finance, and I remember helping a coworker use some kind of block bootstrap. (I believe the application was: we had weekly data on some financial indicator X, along ...
6
votes
4answers
2k views

Small sample linear regression: Where to start

FULL DISCLOSURE: This is homework. I have been provided with a small data set (n=21) the data are messy, looking at it in a scatterplot matrix provides me with little to no insight. I've been ...
3
votes
1answer
3k views

How to test for parallelism for two linear models?

I'm taking a graduate course in regression analysis and I'm suck on a particular homework question that should be very simple to me! I have the following model: ...
53
votes
7answers
21k views

Intuitive explanation of the bias-variance tradeoff?

I am looking for an intuitive explanation of the bias-variance tradeoff, both in general and specifically in the context of linear regression.
88
votes
2answers
40k views

When to use regularization methods for regression?

In what circumstances should one consider using regularization methods (ridge, lasso or least angles regression) instead of OLS? In case this helps steer the discussion, my main interest is improving ...
2
votes
1answer
607 views

Significance of $r^2$ value

I know about $r^2$ tells you about the amount of variation that can be explained by the predictor variables. I have run a model in which the rsquare has value 0.3010 but has false positive rate of ...
9
votes
1answer
21k views

Regression Proof that the point of averages (x,y) lies on the estimated regression line

How do you show that the point of averages (x,y) lies on the estimated regression line?
6
votes
4answers
607 views

Using weighted regression to obtain fit lines for which I only have summary data

I'm working with a dataset for which I only have means, standard deviations, and sample sizes for different levels of a continuous predictor. E.G. Y X SD_Y N_Y 5 1 3 4 10 2 6 2 15 3 2 ...
11
votes
1answer
1k views

Difference between GLS and SUR

I've been reading some about Generalized Least Squares (GLS) and trying to tie it back to my basic econometric background. I recall in grad school using Seemingly Unrelated Regression (SUR) which ...
13
votes
3answers
13k views

Testing nonlinearity in logistic regression (or other forms of regression)

One of the assumption of logistic regression is the linearity in the logit. So once I got my model up and running I test for nonlinearity using Box-Tidwell test. One of my continuous predictors (X) ...
4
votes
1answer
150 views

What distribution to use to model changes in ratios?

I have some data corresponding to changes in a binomial variable - i.e, in month 1 there were n1 trials and k1 successes, and in month 2 there were n2 trials and k2 successes. Say I have M of these ...
16
votes
2answers
27k views

Explain model adjustment, in plain English

Reading about methods and results of statistical analysis, especially in epidemiology, I very often hear about adjustment or controlling of the models. How would you explain, to a non-statistician, ...
5
votes
4answers
3k views

Start time requirements or assumptions for survival analysis

We have prospective data from an observational registry and wish to consider the affects of a gene on time to cardiovascular events. The data includes standard data like age, gender, ... and also the ...
2
votes
2answers
5k views

Adding coefficients to obtain interaction effects - can I add standard errors?

I posted this question earlier and am rewriting it in hopes of getting some guidance. I am using a weighted regression (after propensity score matching) to obtain estimates of the effects of a ...
14
votes
3answers
9k views

Adding coefficients to obtain interaction effects - what to do with SEs?

I have a multivariate regression, which includes interactions. For example, to get the estimate of the treatment effect for the poorest quintile I need to add the coefficients from the treatment ...
2
votes
2answers
247 views

Should we regress x or use logistic regression on x>5000

Suppose we need to take an action on a population with income (x) more than $5,000. Income is not observed directly. Should we use logistic regression to estimate x, or should we use logistic ...
9
votes
4answers
4k views

Cox regression and time scale

Does X (hazard) variable in Cox proportional hazard regression analysis always have to be time? If not, could you provide an example, please? Can age of cancer patient be a hazard variable? If so, ...
23
votes
6answers
14k views

Dealing with correlated regressors

In a multiple linear regression with highly correlated regressors, what is the best strategy to use? Is it a legitimate approach to add the product of all the correlated regressors?
78
votes
9answers
57k views

Why is it possible to get significant F statistic (p<.001) but non-significant regressor t-tests?

In a multiple linear regression, why is it possible to have a highly significant F statistic (p<.001) but have very high p-values on all the regressor's t tests? In my model, there are 10 ...
8
votes
3answers
856 views

Doing regressions on samples from a very large file: are the means and SEs of the sample coefficients consistent estimators?

I have a fairly larege file 100M rows and 30 columns or so on which I would like to run multiple regressions. I have specialized code to run the regressions on the entire file, but what I would like ...
12
votes
3answers
15k views

Why use a lagged DV as an instrumental variable?

I have inherited some data analysis code that, not being an econometrician, I am struggling to understand. One model runs an instrumental variables regression with the following Stata command ...
4
votes
1answer
457 views

Using Covariance Estimator to Perform Linear Regression?

Suppose you had a method for estimating the population covariance of a vector-valued random variable given observations of that random variable, say $f(Z) \rightarrow C$, where the rows of $Z$ are ...
58
votes
5answers
40k views

Is adjusting p-values in a multiple regression for multiple comparisons a good idea?

Lets assume you are a social science researcher/econometrician trying to find relevant predictors of demand for a service. You have 2 outcome/dependent variables describing the demand (using the ...
1
vote
1answer
2k views

Cross-section and fixed effects models

Imagine 2 models: 1 for production, 1 for revenue, have cross-section and time dummies. Is it permissable to use CS in one and time in the other? How do I justify this? F-test for fixed effects is ...
8
votes
4answers
15k views

Linear model with constraints

I'm quite new to R and I have a following problem: I have a simple 2-factor linear model: ...
4
votes
1answer
336 views

Quantifying the degree of consistency of two fitted curves

I previously asked how to estimate the latent potential of a runner who ran the 100 metres each day for 200 days. Latent skill was defined as "the latent time it would take the individual to run if ...
7
votes
3answers
2k views

Regression analysis and parameter estimates with populations

I've seen a little bit here about the difference between statistical inference for random samples, and what happens when we actually have population data. Most arguments seem to suggest you "never ...
15
votes
3answers
2k views

OLS is BLUE. But what if I don't care about unbiasedness and linearity?

The Gauss-Markov theorem tells us that the OLS estimator is the best linear unbiased estimator for the linear regression model. But suppose I don't care about linearity and unbiasedness. Then is ...
4
votes
3answers
271 views

What is a good internet based source of information on Hierarchical Modeling?

I am talking about the regression method that measures the impact of several layers of independent variables upon a dependent variable.
14
votes
3answers
18k views

How to detect when a regression model is over-fit?

When you are the one doing the work, being aware of what you are doing you develop a sense of when you have over-fit the model. For one thing, you can track the trend or deterioration in the Adjusted ...