Questions tagged [regression]
Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.
28,341
questions
3
votes
3
answers
7k
views
What does RMS stand for?
A physics application I'm using reports for a first order fit of the three points below as $11.388612x - 301.878$.
x, y
35, 0
430, 4861
656, 7000
It ...
- 166
18
votes
1
answer
17k
views
Fitting models in R where coefficients are subject to linear restriction(s)?
How should I define a model formula in R, when one (or more) exact linear restrictions binding the coefficients is available. As an example, say that you know that b1 = 2*b0 in a simple linear ...
- 6,334
15
votes
5
answers
12k
views
How to model prices?
I asked this question on the matemathics stackexchange site and was recommended to ask here.
I'm working on a hobby project and would need some help with the following problem.
A bit of context
Let'...
- 323
6
votes
2
answers
27k
views
Can the multiple linear correlation coefficient be negative?
I am using IDL regression function to compute the multiple linear correlation coefficient...
...
Jason Box
5
votes
1
answer
2k
views
What is the prediction error while using deming regression (weighted total least squares)
Deming Regression is a regression technique taking into account uncertainty in both the explanatory and dependent variable.
Although I have found some interesting references on the calculation of ...
- 183
3
votes
1
answer
2k
views
Why are my constraints getting dropped?
I'm currently applying the Roy Zelner test of poolability as shown
in the excellent article of Andrea Vaona,
in fact I'm working with panel N=17 T=5, and my model looks like
this :
$$Y_{it}= a_0+...
- 125
5
votes
2
answers
5k
views
R-squared result in linear regression and "unexplained variance"
I did a linear regression in R and got the following result:
...
- 20.5k
2
votes
1
answer
2k
views
Interpretation of log likelihood and covariate significance in Cox regression
Another newbie question here (probably piece of cake for you guys).
When I run a Cox Regression and one of my covariates come out as significant:
...
- 459
2
votes
1
answer
757
views
How to handle age at measure-start in Cox regression?
I'm a statistics newbie (medical student) trying my luck with a Cox regression for a survival analysis on the outcome of a specific type of operation. And I'm trying to determine which variables to ...
- 459
120
votes
3
answers
125k
views
Does an unbalanced sample matter when doing logistic regression?
Okay, so I think I have a decent enough sample, taking into account the 20:1 rule of thumb: a fairly large sample (N=374) for a total of 7 candidate predictor variables.
My problem is the following: ...
- 1,303
4
votes
2
answers
7k
views
Evaluating effect sizes of interactions in multiple regression
I have been running 3-level multilevel models with HLM, and my main interest is in some cross-level interaction effects that I am finding. My concern is that the effect sizes of these interactions ...
user2686
20
votes
2
answers
84k
views
How do I interpret Exp(B) in Cox regression?
I'm a medical student trying to understand statistics(!) - so please be gentle! ;)
I'm writing an essay containing a fair amount of statistical analysis including survival analysis (Kaplan-Meier, Log-...
- 459
5
votes
2
answers
201
views
Effectively fitting this kind of model: $y = c_1 (x_3 - x_4) + c_2 (x_1 - x_9)$
Given observations of $\{y, x_1, x_2, \cdots, x_n\}$, we can always do a linear regression and get all the coefficients $\{c_i\}$ for the model
$$y = c_0 + c_1 x_1 + \cdots + c_n x_n.$$
However, ...
Steven
30
votes
7
answers
29k
views
Estimating a distribution based on three percentiles
What methods can I use to infer a distribution if I know only three percentiles?
For example, I know that in a certain data set, the fifth percentile is 8,135, the 50th percentile is 11,259, and the ...
- 749
3
votes
3
answers
2k
views
Regression with an unknown dependent variable - estimating "likelihood" to do something
This probably seems like a really strange question, but let me try to explain what I want to do; hopefully it will make sense.
I have a data set with a couple dozen variables, such as age, level of ...
- 31
5
votes
3
answers
2k
views
Can you use heteroskedastic time series variables within a regression model?
We are working on a multivariate linear regression model. Our objective is to forecast the quarterly % growth in mortgage loans outstanding.
The independent variables are:
1) Dow Jones level.
2) % ...
- 7,188
47
votes
5
answers
2k
views
Understanding regressions - the role of the model
How can a regression model be any use if you don't know the function you are trying to get the parameters for?
I saw a piece of research that said that mothers who breast fed their children were less ...
- 471
10
votes
1
answer
6k
views
Plotting a piecewise regression line
Is there a way of plotting the regression line of a piecewise model like this, other than using lines to plot each segment separately, or using ...
- 6,334
13
votes
3
answers
6k
views
Understanding SVM regression: objective function and "flatness"
SVMs for classification make intuitive sense to me: I understand how minimizing $||\theta||^2$ yields the maximum margin. However, I don't understand that objective in the context of regression. ...
- 3,121
16
votes
4
answers
7k
views
Confidence intervals for regression parameters: Bayesian vs. classical
Given two arrays x and y, both of length n, I fit a model y = a + b*x and want to calculate a 95% confidence interval for the slope. This is (b - delta, b + delta) where b is found in the usual way ...
- 561
10
votes
2
answers
14k
views
Knot selection for cubic regression splines [duplicate]
I was wondering if anybody had experience in how to set the knot points when using cubic regression splines.
Some background: I have a response and predictor variable, and I want to determine the ...
- 7,849
6
votes
1
answer
2k
views
Leave-one-out cross validation and boosted regression trees
Colleagues of mine recently presented a work where they calibrate boosted regression trees (BRT) models on small data sets ($n= 30$). They validated the models using leave-one-out cross validation (...
- 61
7
votes
1
answer
5k
views
How to tell if the slope of a line is 0 or there is just no relationship?
I am attempting to examine the change in slope between a predictor and response over two years. In year 1, it is definitely positive. (Linear regression, the 95% CI of the slope doesn't overlap 0). ...
- 1,015
3
votes
1
answer
192
views
When is it better to average observations at the same abscissa?
If you want to regress y on x, where multiple y's are observed at each x, is it ever better to instead take the mean at each x, and the use those means for the regression? Does it depend on the ...
grg s
9
votes
4
answers
13k
views
Partialling or regressing out a categorical variable?
Occasionally I see in literature that a categorical variable such as sex is “partialled” or “regressed” out in (fixed-effects or mixed-effects) regression analysis. I'm troubled with the following ...
- 2,456
11
votes
6
answers
12k
views
Finding the change point in data from a piecewise linear function
Greetings,
I'm performing research that will help determine the size of observed space and the time elapsed since the big bang. Hopefully you can help!
I have data conforming to a piecewise linear ...
- 2,402
4
votes
3
answers
995
views
Confidence interval of slope in linear regression
When computing a confidence interval of slope in linear regression, should you use the z- or t-statistic?
- 377
3
votes
2
answers
4k
views
Average effect of coefficients across multiple linear models?
I have several OLS models with robust s.e.'s that predict an outcome variable Y. For instance:
Model 1:
$Y=B_0 +B_1X_1$
Model 2:
$Y=B_0 + B_1X_1 + B_2X_2$
Model 3:
$Y=B_0 +B_1X_1 + B_2X_2 +...
Thomas
12
votes
3
answers
6k
views
Comparing regression models on count data
I recently fit 4 multiple regression models for the same predictor/response data. Two of the models I fit with Poisson regression.
...
- 1,199
5
votes
6
answers
25k
views
Get the number of parameters of a linear model
Is there a way to get the number of parameters of a linear model like that?
model <- lm(Y~X1+X2)
I would like to get the number 3 somehow (intercept + X1 + X2)...
- 307
28
votes
4
answers
30k
views
What if interaction wipes out my direct effects in regression?
In a regression, the interaction term wipes out both related direct effects. Do I drop the interaction or report the outcome? The interaction was not part of the original hypothesis.
- 283
6
votes
3
answers
10k
views
No valid coefficients for NegBin regression
I am doing multiple regression with some data (5 predictors, 1 response). Since the response is discrete and non-negative, I thought I would try Poisson regression. However, the data are significantly ...
- 1,199
10
votes
2
answers
5k
views
How to compute the confidence intervals on regression coefficients in PLS?
The underlying model of PLS is that a given $n \times m$ matrix $X$ and $n$ vector $y$ are related by
$$X = T P' + E,$$
$$y = T q' + f,$$
where $T$ is a latent $n \times k$ matrix, and $E, ...
- 14.1k
1
vote
3
answers
2k
views
Which test to find out best concentration (the one having maximum effect)?
Hi my apologies first, I'm a biologist and not so good in statistics. In my study I'm studying the effect of concentration of feed on growth of a certain specimen. I have with me the different ...
John adams
11
votes
4
answers
28k
views
What are the assumptions for applying a Tobit regression model?
My (very basic) knowledge of the Tobit regression model isn't from a class, like I would prefer. Instead, I have picked up pieces of information here and there through several Internet searches. My ...
- 2,067
52
votes
3
answers
38k
views
Why is there a difference between manually calculating a logistic regression 95% confidence interval, and using the confint() function in R?
Dear everyone - I've noticed something strange that I can't explain, can you? In summary: the manual approach to calculating a confidence interval in a logistic regression model, and the R function <...
- 5,908
31
votes
5
answers
99k
views
What is the expected correlation between residual and the dependent variable?
In multiple linear regression, I can understand the correlations between residual and predictors are zero, but what is the expected correlation between residual and the criterion variable? Should it ...
- 573
2
votes
1
answer
7k
views
How to do meta-regression analysis with SPSS?
I have done a meta analysis and heterogeneity is too high. I am working with (even/Total) for experimental and control groups to calculate the Odds Ratio. I have done fixed-effect and random-effect ...
Nikki
6
votes
2
answers
295
views
Discerning between two different linear regression models in one sample
Suppose I observe a sample $(y_i,x_i)$, $i=1,...,n$. Suppose that I know the following:
$y_i=\alpha_0+\alpha_1x_i+\varepsilon_i$, $i \in J\subset\{1,...,n\}$
$y_i=\beta_0+\beta_1x_i+\varepsilon_i$, ...
- 34.1k
2
votes
1
answer
218
views
What type of statistical analysis solves this problem?
I have database of 78706 resident incidents in aged care facilities (5 years of data).
I want to to learn and implement a tool allowing analyzing these data using following attributes:
Resident
Date/...
Igor
10
votes
3
answers
2k
views
Resources for learning about spurious time series regression
"Spurious regression" (in the context of time series) and associated terms like unit root tests are something I've heard a lot about, but never understood.
Why/when, intuitively, does it occur? (I ...
- 9,470
7
votes
4
answers
4k
views
Determining trend significance in a time series
I have some time series data and want to test for the existence of and estimate the parameters of a linear trend in a dependent variable w.r.t. time, i.e. time is my independent variable. The time ...
- 8,439
277
votes
2
answers
225k
views
Interpretation of R's lm() output
The help pages in R assume I know what those numbers mean, but I don't.
I'm trying to really intuitively understand every number here. I will just post the output and comment on what I found out. ...
- 4,241
2
votes
2
answers
2k
views
How can I find the best linear combination of a set of matrices to approximate a target matrix?
I want to find $\theta$ such that
$ \theta = argmin_{\theta} \left( \left|\left| Y - \sum_{i=1}^k \theta_i X_i \right|\right| \right) $
where $X_i$ and $Y$ are N x N matrices and $\theta$ is a ...
- 123
4
votes
1
answer
230
views
In a linear regression whose components can also be broken down, is it better to do multi-layered regression, or flatten to final components?
Consider a series like CPI (inflation), which I know is composed of a series of component prices (e.g. meat prices, grain prices, non-food prices, etc.), which in turn are also composed of a series of ...
- 1,397
10
votes
2
answers
2k
views
How to test if the slopes in the linear model are equal to a fixed value?
Suppose we have a simple linear regression model $Z = aX + bY$ and would like to test the null hypothesis $H_0: a=b=\frac{1}{2}$ against the general alternative.
I think one can use the estimate of $...
- 1,371
6
votes
1
answer
656
views
Stochastic coordinate descent for $\ell_1$ regularization
I recently came across the following paper: "Stochastic Methods for $\ell_1$ Regularized Loss Minimization" by Shai Shalev-Shwartz and Ambuj Tewari, ICML 2009.
In the paper, the authors propose a ...
- 5,461
11
votes
4
answers
6k
views
Lasso fitting by coordinate descent: open-source implementations? [closed]
What open-source implementations -- in any language -- exist out there that can compute lasso regularisation paths for linear regression by coordinate descent?
So far I am aware of:
glmnet
scikits....
- 5,461
31
votes
1
answer
23k
views
Is interaction possible between two continuous variables?
All of my variables are continuous. There are no levels. Is it possible to even have interaction between the variables?
- 779
14
votes
1
answer
4k
views
LARS vs coordinate descent for the lasso
What are the pros and cons of using LARS [1] versus using coordinate descent for fitting L1-regularized linear regression?
I am mainly interested in performance aspects (my problems tend to have ...
- 5,461