Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
13 votes
3 answers
6k views

Understanding SVM regression: objective function and "flatness"

SVMs for classification make intuitive sense to me: I understand how minimizing $||\theta||^2$ yields the maximum margin. However, I don't understand that objective in the context of regression. ...
user avatar
  • 3,051
16 votes
4 answers
7k views

Confidence intervals for regression parameters: Bayesian vs. classical

Given two arrays x and y, both of length n, I fit a model y = a + b*x and want to calculate a 95% confidence interval for the slope. This is (b - delta, b + delta) where b is found in the usual way ...
user avatar
  • 561
10 votes
2 answers
13k views

Knot selection for cubic regression splines [duplicate]

I was wondering if anybody had experience in how to set the knot points when using cubic regression splines. Some background: I have a response and predictor variable, and I want to determine the ...
user avatar
  • 7,719
6 votes
1 answer
2k views

Leave-one-out cross validation and boosted regression trees

Colleagues of mine recently presented a work where they calibrate boosted regression trees (BRT) models on small data sets ($n= 30$). They validated the models using leave-one-out cross validation (...
user avatar
7 votes
1 answer
5k views

How to tell if the slope of a line is 0 or there is just no relationship?

I am attempting to examine the change in slope between a predictor and response over two years. In year 1, it is definitely positive. (Linear regression, the 95% CI of the slope doesn't overlap 0). ...
user avatar
  • 935
3 votes
1 answer
182 views

When is it better to average observations at the same abscissa?

If you want to regress y on x, where multiple y's are observed at each x, is it ever better to instead take the mean at each x, and the use those means for the regression? Does it depend on the ...
user avatar
9 votes
4 answers
13k views

Partialling or regressing out a categorical variable?

Occasionally I see in literature that a categorical variable such as sex is “partialled” or “regressed” out in (fixed-effects or mixed-effects) regression analysis. I'm troubled with the following ...
user avatar
  • 2,406
11 votes
6 answers
11k views

Finding the change point in data from a piecewise linear function

Greetings, I'm performing research that will help determine the size of observed space and the time elapsed since the big bang. Hopefully you can help! I have data conforming to a piecewise linear ...
user avatar
4 votes
3 answers
984 views

Confidence interval of slope in linear regression

When computing a confidence interval of slope in linear regression, should you use the z- or t-statistic?
user avatar
3 votes
2 answers
3k views

Average effect of coefficients across multiple linear models?

I have several OLS models with robust s.e.'s that predict an outcome variable Y. For instance: Model 1: $Y=B_0 +B_1X_1$ Model 2: $Y=B_0 + B_1X_1 + B_2X_2$ Model 3: $Y=B_0 +B_1X_1 + B_2X_2 +...
user avatar
11 votes
3 answers
6k views

Comparing regression models on count data

I recently fit 4 multiple regression models for the same predictor/response data. Two of the models I fit with Poisson regression. ...
user avatar
5 votes
6 answers
23k views

Get the number of parameters of a linear model

Is there a way to get the number of parameters of a linear model like that? model <- lm(Y~X1+X2) I would like to get the number 3 somehow (intercept + X1 + X2)...
user avatar
  • 307
27 votes
4 answers
29k views

What if interaction wipes out my direct effects in regression?

In a regression, the interaction term wipes out both related direct effects. Do I drop the interaction or report the outcome? The interaction was not part of the original hypothesis.
user avatar
  • 273
5 votes
3 answers
9k views

No valid coefficients for NegBin regression

I am doing multiple regression with some data (5 predictors, 1 response). Since the response is discrete and non-negative, I thought I would try Poisson regression. However, the data are significantly ...
user avatar
10 votes
2 answers
4k views

How to compute the confidence intervals on regression coefficients in PLS?

The underlying model of PLS is that a given $n \times m$ matrix $X$ and $n$ vector $y$ are related by $$X = T P' + E,$$ $$y = T q' + f,$$ where $T$ is a latent $n \times k$ matrix, and $E, ...
user avatar
  • 10.4k
1 vote
3 answers
1k views

Which test to find out best concentration (the one having maximum effect)?

Hi my apologies first, I'm a biologist and not so good in statistics. In my study I'm studying the effect of concentration of feed on growth of a certain specimen. I have with me the different ...
user avatar
10 votes
4 answers
27k views

What are the assumptions for applying a Tobit regression model?

My (very basic) knowledge of the Tobit regression model isn't from a class, like I would prefer. Instead, I have picked up pieces of information here and there through several Internet searches. My ...
user avatar
  • 1,867
49 votes
3 answers
35k views

Why is there a difference between manually calculating a logistic regression 95% confidence interval, and using the confint() function in R?

Dear everyone - I've noticed something strange that I can't explain, can you? In summary: the manual approach to calculating a confidence interval in a logistic regression model, and the R function <...
user avatar
  • 5,618
29 votes
5 answers
92k views

What is the expected correlation between residual and the dependent variable?

In multiple linear regression, I can understand the correlations between residual and predictors are zero, but what is the expected correlation between residual and the criterion variable? Should it ...
user avatar
2 votes
1 answer
7k views

How to do meta-regression analysis with SPSS?

I have done a meta analysis and heterogeneity is too high. I am working with (even/Total) for experimental and control groups to calculate the Odds Ratio. I have done fixed-effect and random-effect ...
user avatar
6 votes
2 answers
293 views

Discerning between two different linear regression models in one sample

Suppose I observe a sample $(y_i,x_i)$, $i=1,...,n$. Suppose that I know the following: $y_i=\alpha_0+\alpha_1x_i+\varepsilon_i$, $i \in J\subset\{1,...,n\}$ $y_i=\beta_0+\beta_1x_i+\varepsilon_i$, ...
user avatar
  • 33.5k
2 votes
1 answer
218 views

What type of statistical analysis solves this problem?

I have database of 78706 resident incidents in aged care facilities (5 years of data). I want to to learn and implement a tool allowing analyzing these data using following attributes: Resident Date/...
user avatar
10 votes
3 answers
2k views

Resources for learning about spurious time series regression

"Spurious regression" (in the context of time series) and associated terms like unit root tests are something I've heard a lot about, but never understood. Why/when, intuitively, does it occur? (I ...
user avatar
  • 9,270
7 votes
4 answers
4k views

Determining trend significance in a time series

I have some time series data and want to test for the existence of and estimate the parameters of a linear trend in a dependent variable w.r.t. time, i.e. time is my independent variable. The time ...
user avatar
  • 7,485
274 votes
2 answers
215k views

Interpretation of R's lm() output

The help pages in R assume I know what those numbers mean, but I don't. I'm trying to really intuitively understand every number here. I will just post the output and comment on what I found out. ...
user avatar
2 votes
2 answers
2k views

How can I find the best linear combination of a set of matrices to approximate a target matrix?

I want to find $\theta$ such that $ \theta = argmin_{\theta} \left( \left|\left| Y - \sum_{i=1}^k \theta_i X_i \right|\right| \right) $ where $X_i$ and $Y$ are N x N matrices and $\theta$ is a ...
user avatar
4 votes
1 answer
220 views

In a linear regression whose components can also be broken down, is it better to do multi-layered regression, or flatten to final components?

Consider a series like CPI (inflation), which I know is composed of a series of component prices (e.g. meat prices, grain prices, non-food prices, etc.), which in turn are also composed of a series of ...
user avatar
  • 1,337
9 votes
2 answers
2k views

How to test if the slopes in the linear model are equal to a fixed value?

Suppose we have a simple linear regression model $Z = aX + bY$ and would like to test the null hypothesis $H_0: a=b=\frac{1}{2}$ against the general alternative. I think one can use the estimate of $...
user avatar
  • 1,251
6 votes
1 answer
644 views

Stochastic coordinate descent for $\ell_1$ regularization

I recently came across the following paper: "Stochastic Methods for $\ell_1$ Regularized Loss Minimization" by Shai Shalev-Shwartz and Ambuj Tewari, ICML 2009. In the paper, the authors propose a ...
user avatar
  • 5,381
11 votes
4 answers
6k views

Lasso fitting by coordinate descent: open-source implementations? [closed]

What open-source implementations -- in any language -- exist out there that can compute lasso regularisation paths for linear regression by coordinate descent? So far I am aware of: glmnet scikits....
user avatar
  • 5,381
30 votes
1 answer
22k views

Is interaction possible between two continuous variables?

All of my variables are continuous. There are no levels. Is it possible to even have interaction between the variables?
user avatar
14 votes
1 answer
4k views

LARS vs coordinate descent for the lasso

What are the pros and cons of using LARS [1] versus using coordinate descent for fitting L1-regularized linear regression? I am mainly interested in performance aspects (my problems tend to have ...
user avatar
  • 5,381
14 votes
2 answers
7k views

Coordinate descent for the lasso or elastic net

Are there any good papers or books dealing with the use of coordinate descent for L1 (lasso) and/or elastic net regularization for linear regression problems?
user avatar
  • 5,381
20 votes
4 answers
29k views

Can I simply remove one of two predictor variables that are highly linearly correlated?

Using Pearson's Correlation Coefficient, I have several variables that are highly correlated ($\rho = 0.978$ and $\rho = 0.989$ for 2 pairs of variables that are in my model). The reason some of the ...
user avatar
42 votes
6 answers
35k views

What are best practices in identifying interaction effects?

Other than literally testing each possible combination of variable(s) in a model (x1:x2 or x1*x2 ... xn-1 * xn). How do you ...
user avatar
26 votes
1 answer
16k views

Logistic Regression - Multicollinearity Concerns/Pitfalls

In Logistic Regression, is there a need to be as concerned about multicollinearity as you would be in straight up OLS regression? For example, with a logistic regression, where multicollinearity ...
user avatar
50 votes
2 answers
41k views

Regression: Transforming Variables

When transforming variables, do you have to use all of the same transformation? For example, can I pick and choose differently transformed variables, as in: Let, $x_1,x_2,x_3$ be age, length of ...
user avatar
19 votes
4 answers
14k views

How to add periodic component to linear regression model?

I have some cumulative frequency data. A line $y=ax+b$ looks like it fits the data extremely well, but there is cyclic/periodic wiggle in the line. I would like to estimate when the cumulative ...
user avatar
2 votes
1 answer
8k views

F-test for Lack-of-Fit in SPSS

Some googling revealed that doing the F-test for Lack-of-Fit in SPSS is not so trivial. It seems one has to “trick” SPSS to do that. See for example this. Can anybody suggest a better source of ...
user avatar
40 votes
6 answers
18k views

Least-angle regression vs. lasso

Least-angle regression and the lasso tend to produce very similar regularization paths (identical except when a coefficient crosses zero.) They both can be efficiently fit by virtually identical ...
user avatar
  • 5,381
5 votes
1 answer
208 views

Toy regression question with latent variables

I originally asked this on a machine learning site, but one of the responses made me think that maybe this site is more suitable. Suppose you have two weighted coins, and every day you flip each one ...
user avatar
  • 415
19 votes
2 answers
55k views

Interpreting the drop1 output in R

In R, the drop1command outputs something neat. These two commands should get you some output: example(step)#-> swiss ...
user avatar
  • 491
12 votes
1 answer
904 views

Updating the lasso fit with new observations

I am fitting an L1-regularized linear regression to a very large dataset (with n>>p.) The variables are known in advance, but the observations arrive in small chunks. I would like to maintain the ...
user avatar
  • 5,381
5 votes
3 answers
3k views

Java implementations of the lasso [closed]

Are there any open-source Java implementations of lasso or least angles regression? Pure Java code would be best, but clean implementations in other languages would also be of interest. I am already ...
user avatar
  • 5,381
3 votes
2 answers
3k views

Explaining variation in a dependent variable based on a factorial experiment

I have run a factorial type test in a processing plant and have run a forward and backward step regression in R. How can I use the regression results and the anova created from the regression to ...
user avatar
11 votes
1 answer
5k views

In R, does "glmnet" fit an intercept?

I am fitting a linear model in R using glmnet. The original (non-regularized) model was fitted using lm and did not have a ...
user avatar
  • 5,381
2 votes
1 answer
1k views

Regression on a triangular shaped region of points representing a symmetric relation

I plotted a set of about 200,000 points and got a triangular shaped region. The shape is roughly like the triangle made by the points $(1,0)$, $(0,1)$ and $(0,0)$. My points have the property that ...
user avatar
  • 1,479
68 votes
8 answers
190k views

Regression with multiple dependent variables?

Is it possible to have a (multiple) regression equation with two or more dependent variables? Sure, you could run two separate regression equations, one for each DV, but that doesn't seem like it ...
user avatar
  • 3,555
1 vote
2 answers
18k views

How to compute goodness of fit for a linear model in R

I have fit a linear model using the lm function in R... model <- lm(trans.baseline.CD4 ~ hiv$Julian.Date) ... and I would ...
user avatar
4 votes
2 answers
2k views

Validating a linear model with R, lm() [closed]

I've created a model (cue ominous music) in R based on previous months data using lm(). Now, I would like to see how well it predicts the current months data. For example, my model predicts sales ...
user avatar