# Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

9,425 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
2k views

### How does a Relevance Vector Machine (RVM) work?

Relevance Vector Machines (RVMs) are really interesting models when contrasted with the highly geometrical (and popular) SVMs. In the light of a question like How does a Support Vector Machine (SVM) ...
3k views

### Gamma hurdle model for continuous response?

I am modelling invertebrate.biomass ~ habitat.type * calendar.day + habitat.type * calendar.day ^ 2, with a random intercept of ...
495 views

### Asymptotic property of tuning parameter in penalized regression

I'm currently working on asymptotic properties of penalized regression. I've read a myriad of papers by now, but there is an essential issue that I cannot get my head around. To keep things simple, I'...
639 views

### Adjustments to (Linear Regression) Forecast

Full disclosure: I am not a statistician, nor do I claim to be one. I am a lowly IT administrator. Please play gentle with me. :) I am responsible for collecting and forecasting disk storage use ...
11k views

### Time series regression with overlapping data

I am seeing a regression model which is regressing Year-on-Year stock index returns on lagged (12 months) Year-on-Year returns of the same stock index, credit spread (difference between monthly mean ...
258 views

### Is autocorrelation not worth addressing with small N?

Consider a simple regression context in which there is a small set of response values, $Y$, and corresponding dates, $X$. (For simplicity, we can assume the dates are equally spaced.) We would like ...
512 views

### Interpreting regression coefficients based on Andrew Gelman's re-scaling method

I have two predictors in a binary logistic regression model: One binary and one continuous. My primary goal is to compare the coefficients of the two predictors within the same model. I have come ...
146 views

### Logistic regression for classification: are there any analytical solutions for the out-of-sample accuracy?

I run a binary logistic regression, with a binary dependent variable and a continuous independent one. Now I want to evaluate the out-of-sample performance of the classification algorithm so obtained. ...
152 views

### Pope effect on pizza - Regression with presence absence and similarity data as dependent variables

I'm trying to figure out the right way to set up a regression when the dependent variables are presence absence data (of pizzas), and the similarity between the present pizzas. Bear with the story: ...
406 views

### Bootstrap Prediction Interval: which residuals to use and which method?

I ask this question referring to the post: Bootstrap prediction interval, where a step by step method for calculating the prediction interval for linear regression models is explained. In the ...
571 views

### Does there exist an analogous statement to BLUE (Gauss-Markov) for GLMs?

I recall from my graduate school days that the Gauss-Markov (GM) theorem states that the Best Linear Unbiased Estimator (BLUE) in a linear regression is $\vec{\beta}=(X^TX)^{-1}X^T\vec{y}$. An amazing ...
2k views

### does serial correlation have something to do with endogeneity?

I'm a beginner of econometrics, and I've construed that endogeneity is caused by omitted variable bias, measurement error, and reverse causality, and it makes OLS estimator be biased. And also I've ...
9k views

### Regress residuals in second regression

I am wondering if anyone can point me to a paper/lecture notes on the rationale behind first running an OLS on a set of variables, and then in a second regression using the residuals of that ...
2k views

2k views

### Fixed Effects vs Lagged DV vs. First Differences Regression

What are the differences between using unit fixed effects, unit fixed effects and time fixed effects, lagged DV, or first differences to analyze a time series with 4-5 time periods and 35-50 units per ...
1k views

### Calculating and plotting confidence interval for Theil-Sen estimator

I'm using Wilcox's R functions (specifically, regplot) to plot a Theil-Sen estimator with a single predictor. However, regplot ...
908 views

### Dantzig Selector, LASSO, LAD LASSO

I am wondering about this. When is it best to use Dantzig Selector (the infinity normed error measure plus the L1 regularizer) , the LASSO (the mean square error measure plus the L1 regularizer), and ...
5k views

### Poisson regression for binary data

I've been trying to read up on Poisson regression models, and it looks like it is possible to estimate such a model with a binary outcome. This has come up before on this site here (and somewhat here ...
98 views

### Identifying non-linearities in relationship between variables

Logistic regression is often used to identify the effect of $x$ on a binary variable $y$ after adjusting for potential confounders $x_1,...,x_n$. In the medical literature, I will sometimes encounter ...
309 views

### Robust Gamma Regression

I am modeling some spectroscopic data where the response of the instrument to the size of the input is strictly positive and non-linear. Gamma regression seems like a good choice to explain the data, ...
219 views

### Regression with dependent data with low dependence

Suppose you have data that is grouped in one way or another and therefore the assumption of independence is suspect. But you look at the intraclass correlation (or autocorrelation) and it is very ...
241 views

### Intuition: What is the difference between linear factor models and regular linear regression?

So, I have a very vexing theoretical question that I hope some experienced econometrician can help me with. Being in finance, I have recently been exposed to linear factor models, which are models ...
916 views

### Geometrical interpretation of L1 regression

I have found the following image (or a similar version) in a lot of books related to penalized linear models. I get the insight of this image. The ellipsoids are the solution of the linear regression ...
235 views

### Is there any geometric intuition on least absolute deviation regression?

There are a lot of geometric intuitions for regression with least square, e.g., projection, orthogonal, etc. (This and this answers are good examples.) Is there similar geometric intuition for least ...
691 views

### Is lasso always outperformed by adaptive lasso?

I have been reading some papers and I understood that adaptive lasso has the Oracle properties which lasso lacks. Does that mean adaptive lasso always better than lasso (let's focus on the simple ...
80 views

### Can I use bootstrap results at the observation level?

I have read quite a bit of bootstrapping, but the issue I want to address seem not to appear. Consider a simple regression model: $$y_{i} = \beta_{0} + \beta_{1}x_{i} + e_{i}$$ I am aware that ...
658 views

### Interpretation of smoothing spline

This question is about interpreting the results from non-linear regression models, especially when using regression splines. The numerical output is not very informative when interpreting the effects, ...
846 views

### Can I calculate Cohen's $d$ from multiple regression coefficient?

Question: Is it appropriate to calculate Cohen's $d$ (effect size) from the regression coefficient of an independent categorical variable? Background: My regression coefficient represents ...
1k views

### What techniques are there to measure goodness of fit of Deming (orthogonal) regression?

Questions: Even if there is no "widely accepted" technique, is there a useful-and-above-average technique for estimating goodness of fit in orthogonal regressions? What are the pros/cons of this ...
788 views

### Rule of thumb for excluded variable in Heckman selection model?

I'm working on a project that involves the use of a Heckman selection model (more specifically a Roy or move-stay model, which is essentially a two-sided Heckman) of the following form:  Y_{i1} = ...
390 views

706 views

### Studentized residuals and goodness-of-fit with robust linear regression

Could you please advise whether studentized residuals are meaningful when computed on a robust linear regression model using an M-estimator? I'd like to use it to detect outliers by doing something ...