Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
5 votes
2 answers
5k views

R-squared result in linear regression and "unexplained variance"

I did a linear regression in R and got the following result: ...
user avatar
  • 20.1k
2 votes
1 answer
2k views

Interpretation of log likelihood and covariate significance in Cox regression

Another newbie question here (probably piece of cake for you guys). When I run a Cox Regression and one of my covariates come out as significant: ...
user avatar
  • 459
2 votes
1 answer
738 views

How to handle age at measure-start in Cox regression?

I'm a statistics newbie (medical student) trying my luck with a Cox regression for a survival analysis on the outcome of a specific type of operation. And I'm trying to determine which variables to ...
user avatar
  • 459
111 votes
3 answers
114k views

Does an unbalanced sample matter when doing logistic regression?

Okay, so I think I have a decent enough sample, taking into account the 20:1 rule of thumb: a fairly large sample (N=374) for a total of 7 candidate predictor variables. My problem is the following: ...
user avatar
  • 1,213
4 votes
2 answers
6k views

Evaluating effect sizes of interactions in multiple regression

I have been running 3-level multilevel models with HLM, and my main interest is in some cross-level interaction effects that I am finding. My concern is that the effect sizes of these interactions ...
user avatar
20 votes
2 answers
82k views

How do I interpret Exp(B) in Cox regression?

I'm a medical student trying to understand statistics(!) - so please be gentle! ;) I'm writing an essay containing a fair amount of statistical analysis including survival analysis (Kaplan-Meier, Log-...
user avatar
  • 459
5 votes
2 answers
194 views

Effectively fitting this kind of model: $y = c_1 (x_3 - x_4) + c_2 (x_1 - x_9)$

Given observations of $\{y, x_1, x_2, \cdots, x_n\}$, we can always do a linear regression and get all the coefficients $\{c_i\}$ for the model $$y = c_0 + c_1 x_1 + \cdots + c_n x_n.$$ However, ...
user avatar
29 votes
7 answers
26k views

Estimating a distribution based on three percentiles

What methods can I use to infer a distribution if I know only three percentiles? For example, I know that in a certain data set, the fifth percentile is 8,135, the 50th percentile is 11,259, and the ...
user avatar
3 votes
3 answers
1k views

Regression with an unknown dependent variable - estimating "likelihood" to do something

This probably seems like a really strange question, but let me try to explain what I want to do; hopefully it will make sense. I have a data set with a couple dozen variables, such as age, level of ...
user avatar
5 votes
3 answers
2k views

Can you use heteroskedastic time series variables within a regression model?

We are working on a multivariate linear regression model. Our objective is to forecast the quarterly % growth in mortgage loans outstanding. The independent variables are: 1) Dow Jones level. 2) % ...
user avatar
  • 6,942
46 votes
5 answers
1k views

Understanding regressions - the role of the model

How can a regression model be any use if you don't know the function you are trying to get the parameters for? I saw a piece of research that said that mothers who breast fed their children were less ...
user avatar
10 votes
1 answer
6k views

Plotting a piecewise regression line

Is there a way of plotting the regression line of a piecewise model like this, other than using lines to plot each segment separately, or using ...
user avatar
13 votes
3 answers
6k views

Understanding SVM regression: objective function and "flatness"

SVMs for classification make intuitive sense to me: I understand how minimizing $||\theta||^2$ yields the maximum margin. However, I don't understand that objective in the context of regression. ...
user avatar
  • 3,001
16 votes
4 answers
7k views

Confidence intervals for regression parameters: Bayesian vs. classical

Given two arrays x and y, both of length n, I fit a model y = a + b*x and want to calculate a 95% confidence interval for the slope. This is (b - delta, b + delta) where b is found in the usual way ...
user avatar
  • 561
10 votes
2 answers
13k views

Knot selection for cubic regression splines [duplicate]

I was wondering if anybody had experience in how to set the knot points when using cubic regression splines. Some background: I have a response and predictor variable, and I want to determine the ...
user avatar
  • 7,699
6 votes
1 answer
2k views

Leave-one-out cross validation and boosted regression trees

Colleagues of mine recently presented a work where they calibrate boosted regression trees (BRT) models on small data sets ($n= 30$). They validated the models using leave-one-out cross validation (...
user avatar
7 votes
1 answer
4k views

How to tell if the slope of a line is 0 or there is just no relationship?

I am attempting to examine the change in slope between a predictor and response over two years. In year 1, it is definitely positive. (Linear regression, the 95% CI of the slope doesn't overlap 0). ...
user avatar
  • 885
3 votes
1 answer
179 views

When is it better to average observations at the same abscissa?

If you want to regress y on x, where multiple y's are observed at each x, is it ever better to instead take the mean at each x, and the use those means for the regression? Does it depend on the ...
user avatar
9 votes
4 answers
13k views

Partialling or regressing out a categorical variable?

Occasionally I see in literature that a categorical variable such as sex is “partialled” or “regressed” out in (fixed-effects or mixed-effects) regression analysis. I'm troubled with the following ...
user avatar
  • 2,386
11 votes
6 answers
11k views

Finding the change point in data from a piecewise linear function

Greetings, I'm performing research that will help determine the size of observed space and the time elapsed since the big bang. Hopefully you can help! I have data conforming to a piecewise linear ...
user avatar
4 votes
3 answers
982 views

Confidence interval of slope in linear regression

When computing a confidence interval of slope in linear regression, should you use the z- or t-statistic?
user avatar
3 votes
2 answers
3k views

Average effect of coefficients across multiple linear models?

I have several OLS models with robust s.e.'s that predict an outcome variable Y. For instance: Model 1: $Y=B_0 +B_1X_1$ Model 2: $Y=B_0 + B_1X_1 + B_2X_2$ Model 3: $Y=B_0 +B_1X_1 + B_2X_2 +...
user avatar
11 votes
3 answers
6k views

Comparing regression models on count data

I recently fit 4 multiple regression models for the same predictor/response data. Two of the models I fit with Poisson regression. ...
user avatar
5 votes
6 answers
23k views

Get the number of parameters of a linear model

Is there a way to get the number of parameters of a linear model like that? model <- lm(Y~X1+X2) I would like to get the number 3 somehow (intercept + X1 + X2)...
user avatar
  • 307
27 votes
4 answers
29k views

What if interaction wipes out my direct effects in regression?

In a regression, the interaction term wipes out both related direct effects. Do I drop the interaction or report the outcome? The interaction was not part of the original hypothesis.
user avatar
  • 273
5 votes
3 answers
9k views

No valid coefficients for NegBin regression

I am doing multiple regression with some data (5 predictors, 1 response). Since the response is discrete and non-negative, I thought I would try Poisson regression. However, the data are significantly ...
user avatar
10 votes
2 answers
4k views

How to compute the confidence intervals on regression coefficients in PLS?

The underlying model of PLS is that a given $n \times m$ matrix $X$ and $n$ vector $y$ are related by $$X = T P' + E,$$ $$y = T q' + f,$$ where $T$ is a latent $n \times k$ matrix, and $E, ...
user avatar
  • 10.4k
1 vote
3 answers
1k views

Which test to find out best concentration (the one having maximum effect)?

Hi my apologies first, I'm a biologist and not so good in statistics. In my study I'm studying the effect of concentration of feed on growth of a certain specimen. I have with me the different ...
user avatar
10 votes
4 answers
26k views

What are the assumptions for applying a Tobit regression model?

My (very basic) knowledge of the Tobit regression model isn't from a class, like I would prefer. Instead, I have picked up pieces of information here and there through several Internet searches. My ...
user avatar
  • 1,827
49 votes
3 answers
34k views

Why is there a difference between manually calculating a logistic regression 95% confidence interval, and using the confint() function in R?

Dear everyone - I've noticed something strange that I can't explain, can you? In summary: the manual approach to calculating a confidence interval in a logistic regression model, and the R function <...
user avatar
  • 5,548
29 votes
5 answers
92k views

What is the expected correlation between residual and the dependent variable?

In multiple linear regression, I can understand the correlations between residual and predictors are zero, but what is the expected correlation between residual and the criterion variable? Should it ...
user avatar
2 votes
1 answer
7k views

How to do meta-regression analysis with SPSS?

I have done a meta analysis and heterogeneity is too high. I am working with (even/Total) for experimental and control groups to calculate the Odds Ratio. I have done fixed-effect and random-effect ...
user avatar
6 votes
2 answers
293 views

Discerning between two different linear regression models in one sample

Suppose I observe a sample $(y_i,x_i)$, $i=1,...,n$. Suppose that I know the following: $y_i=\alpha_0+\alpha_1x_i+\varepsilon_i$, $i \in J\subset\{1,...,n\}$ $y_i=\beta_0+\beta_1x_i+\varepsilon_i$, ...
user avatar
  • 33.3k
2 votes
1 answer
218 views

What type of statistical analysis solves this problem?

I have database of 78706 resident incidents in aged care facilities (5 years of data). I want to to learn and implement a tool allowing analyzing these data using following attributes: Resident Date/...
user avatar
10 votes
3 answers
2k views

Resources for learning about spurious time series regression

"Spurious regression" (in the context of time series) and associated terms like unit root tests are something I've heard a lot about, but never understood. Why/when, intuitively, does it occur? (I ...
user avatar
  • 9,200
7 votes
4 answers
4k views

Determining trend significance in a time series

I have some time series data and want to test for the existence of and estimate the parameters of a linear trend in a dependent variable w.r.t. time, i.e. time is my independent variable. The time ...
user avatar
  • 7,445
274 votes
2 answers
214k views

Interpretation of R's lm() output

The help pages in R assume I know what those numbers mean, but I don't. I'm trying to really intuitively understand every number here. I will just post the output and comment on what I found out. ...
user avatar
2 votes
2 answers
2k views

How can I find the best linear combination of a set of matrices to approximate a target matrix?

I want to find $\theta$ such that $ \theta = argmin_{\theta} \left( \left|\left| Y - \sum_{i=1}^k \theta_i X_i \right|\right| \right) $ where $X_i$ and $Y$ are N x N matrices and $\theta$ is a ...
user avatar
4 votes
1 answer
218 views

In a linear regression whose components can also be broken down, is it better to do multi-layered regression, or flatten to final components?

Consider a series like CPI (inflation), which I know is composed of a series of component prices (e.g. meat prices, grain prices, non-food prices, etc.), which in turn are also composed of a series of ...
user avatar
  • 1,327
9 votes
2 answers
2k views

How to test if the slopes in the linear model are equal to a fixed value?

Suppose we have a simple linear regression model $Z = aX + bY$ and would like to test the null hypothesis $H_0: a=b=\frac{1}{2}$ against the general alternative. I think one can use the estimate of $...
user avatar
  • 1,251
6 votes
1 answer
639 views

Stochastic coordinate descent for $\ell_1$ regularization

I recently came across the following paper: "Stochastic Methods for $\ell_1$ Regularized Loss Minimization" by Shai Shalev-Shwartz and Ambuj Tewari, ICML 2009. In the paper, the authors propose a ...
user avatar
  • 5,381
11 votes
4 answers
6k views

Lasso fitting by coordinate descent: open-source implementations? [closed]

What open-source implementations -- in any language -- exist out there that can compute lasso regularisation paths for linear regression by coordinate descent? So far I am aware of: glmnet scikits....
user avatar
  • 5,381
30 votes
1 answer
22k views

Is interaction possible between two continuous variables?

All of my variables are continuous. There are no levels. Is it possible to even have interaction between the variables?
user avatar
14 votes
1 answer
4k views

LARS vs coordinate descent for the lasso

What are the pros and cons of using LARS [1] versus using coordinate descent for fitting L1-regularized linear regression? I am mainly interested in performance aspects (my problems tend to have ...
user avatar
  • 5,381
14 votes
2 answers
7k views

Coordinate descent for the lasso or elastic net

Are there any good papers or books dealing with the use of coordinate descent for L1 (lasso) and/or elastic net regularization for linear regression problems?
user avatar
  • 5,381
20 votes
4 answers
28k views

Can I simply remove one of two predictor variables that are highly linearly correlated?

Using Pearson's Correlation Coefficient, I have several variables that are highly correlated ($\rho = 0.978$ and $\rho = 0.989$ for 2 pairs of variables that are in my model). The reason some of the ...
user avatar
42 votes
6 answers
35k views

What are best practices in identifying interaction effects?

Other than literally testing each possible combination of variable(s) in a model (x1:x2 or x1*x2 ... xn-1 * xn). How do you ...
user avatar
26 votes
1 answer
16k views

Logistic Regression - Multicollinearity Concerns/Pitfalls

In Logistic Regression, is there a need to be as concerned about multicollinearity as you would be in straight up OLS regression? For example, with a logistic regression, where multicollinearity ...
user avatar
50 votes
2 answers
40k views

Regression: Transforming Variables

When transforming variables, do you have to use all of the same transformation? For example, can I pick and choose differently transformed variables, as in: Let, $x_1,x_2,x_3$ be age, length of ...
user avatar
19 votes
4 answers
14k views

How to add periodic component to linear regression model?

I have some cumulative frequency data. A line $y=ax+b$ looks like it fits the data extremely well, but there is cyclic/periodic wiggle in the line. I would like to estimate when the cumulative ...
user avatar