Skip to main content

# Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

9,973 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
17 votes
0 answers
14k views

### Time series regression with overlapping data

I am seeing a regression model which is regressing Year-on-Year stock index returns on lagged (12 months) Year-on-Year returns of the same stock index, credit spread (difference between monthly mean ...
16 votes
0 answers
571 views

### Asymptotic property of tuning parameter in penalized regression

I'm currently working on asymptotic properties of penalized regression. I've read a myriad of papers by now, but there is an essential issue that I cannot get my head around. To keep things simple, I'...
• 12.9k
13 votes
0 answers
249 views

### Logistic regression for classification: are there any analytical solutions for the out-of-sample accuracy?

I run a binary logistic regression, with a binary dependent variable and a continuous independent one. Now I want to evaluate the out-of-sample performance of the classification algorithm so obtained. ...
• 2,311
13 votes
0 answers
721 views

### Interpreting regression coefficients based on Andrew Gelman's re-scaling method

I have two predictors in a binary logistic regression model: One binary and one continuous. My primary goal is to compare the coefficients of the two predictors within the same model. I have come ...
• 423
13 votes
0 answers
379 views

### Is autocorrelation not worth addressing with small N?

Consider a simple regression context in which there is a small set of response values, $Y$, and corresponding dates, $X$. (For simplicity, we can assume the dates are equally spaced.) We would like ...
11 votes
0 answers
1k views

### Bootstrap Prediction Interval: which residuals to use and which method?

I ask this question referring to the post: Bootstrap prediction interval, where a step by step method for calculating the prediction interval for linear regression models is explained. In the ...
• 311
11 votes
1 answer
5k views

### Generalized additive model: choosing between cubic and thin-plate splines

I am using the gam function (from the mgcv package) to model a continuous response (a soil nutrient) in relation to a continuous ...
• 241
10 votes
0 answers
179 views

### Pope effect on pizza - Regression with presence absence and similarity data as dependent variables

I'm trying to figure out the right way to set up a regression when the dependent variables are presence absence data (of pizzas), and the similarity between the present pizzas. Bear with the story: ...
• 241
10 votes
0 answers
573 views

• 1,532
9 votes
0 answers
214 views

### Efficient nonparametric estimation of confidence intervals and p-values for nonlinear regression

I'm estimating parameters for a complex, "implicit" nonlinear model $f(\mathbf{x}, \boldsymbol{\theta})$. It's "implicit" in the sense that I don't have an explicit formula for $f$: its value is the ...
• 18.3k
9 votes
0 answers
502 views

### How do sufficiency statistics help in the interpretation of regression results?

One of the results why canonical link functions are widely used in GLMs is the existence of sufficiency statistics for the regression parameters, which in turn allow for: ... minimal sufficient ...
• 4,492
9 votes
0 answers
1k views

### Why would I use ratio estimation instead of regression estimation to estimate means?

I am taking a graduate course on survey data analysis. I was recently introduced to ratio estimation and regression estimation. I understand that using ratio estimator may be easier if we are ...
• 306
9 votes
0 answers
719 views

### Errors-in-Variables model for logistic regression

Simple question: I am familiar (though don't have tons of experience) with errors-in-variables regression. From what I have seen, this mostly is used with continuous outcomes in a linear model. A) Is ...
• 2,783
9 votes
3 answers
532 views

• 411
8 votes
0 answers
3k views

### Fixed Effects vs Lagged DV vs. First Differences Regression

What are the differences between using unit fixed effects, unit fixed effects and time fixed effects, lagged DV, or first differences to analyze a time series with 4-5 time periods and 35-50 units per ...
• 412
8 votes
0 answers
290 views

### Generalization of cumulative probability models for ordinal Y

There are many models in existence for ordinal $Y$, for example the proportional odds ordinal logistic model, the continuation ratio model, and the cumulative probit model. The first and third of ...
• 96.4k
8 votes
0 answers
994 views

### Dantzig Selector, LASSO, LAD LASSO

I am wondering about this. When is it best to use Dantzig Selector (the infinity normed error measure plus the L1 regularizer) , the LASSO (the mean square error measure plus the L1 regularizer), and ...
• 391
8 votes
1 answer
185 views

### What do the terms "nearly-optimal rate", "near-minimax rate", "minimax optimal rate" and "minimax rate" mean in the context of posterior consistency?

Definition: A sequence $\epsilon_n$ is a posterior contraction rate at the parameter $θ_0$ if $$\Pi_n(θ: d(θ, θ_0) ≥ M_n \epsilon_n| X^{(n)}) → 0$$ in $P^{(n)}_{θ_0}$-probability, for every $M_n → ∞$. ...
• 308
8 votes
1 answer
789 views

### How to subset alternatives in nested multinomial logistic regression?

I am trying to predict whether or not captains in a particular groundfish fishery choose to fish on any given day and what variables may influence that decision. Originally I had planned on using ...
7 votes
1 answer
166 views

• 765
7 votes
0 answers
7k views

### What is difference between interrupted time series and regression discontinuity design

Say that one has data over time, t, on an outcome, y. There is an event that happens at t==0....
• 337
7 votes
0 answers
424 views

### Robust Gamma Regression

I am modeling some spectroscopic data where the response of the instrument to the size of the input is strictly positive and non-linear. Gamma regression seems like a good choice to explain the data, ...
• 223
7 votes
0 answers
1k views

### Geometrical interpretation of L1 regression

I have found the following image (or a similar version) in a lot of books related to penalized linear models. I get the insight of this image. The ellipsoids are the solution of the linear regression ...
7 votes
0 answers
256 views

### Is there any geometric intuition on least absolute deviation regression?

There are a lot of geometric intuitions for regression with least square, e.g., projection, orthogonal, etc. (This and this answers are good examples.) Is there similar geometric intuition for least ...
• 37.2k
7 votes
0 answers
800 views

### Is lasso always outperformed by adaptive lasso?

I have been reading some papers and I understood that adaptive lasso has the Oracle properties which lasso lacks. Does that mean adaptive lasso always better than lasso (let's focus on the simple ...
• 81
7 votes
2 answers
1k views

### Linear model with hidden variable

I have come across a somewhat unusual (I think) estimation problem. I have two "coupled" linear regression models, $$Y = a + b x + \epsilon, \quad Z = c + d x + \nu$$ where $Y,Z,\epsilon,\nu$ are ...
• 181
7 votes
0 answers
89 views

### Can I use bootstrap results at the observation level?

I have read quite a bit of bootstrapping, but the issue I want to address seem not to appear. Consider a simple regression model: $$y_{i} = \beta_{0} + \beta_{1}x_{i} + e_{i}$$ I am aware that ...
• 2,748
7 votes
1 answer
947 views

### Assumptions for PCR and PLS

I am writing up a report on fitting Principal Component Regression (PCR) and Partial Least Squares (PLS) to my data-set. A similar question: Model assumptions of partial least squares (PLS) ...
• 281
7 votes
1 answer
479 views

• 271
7 votes
0 answers
1k views

### Bias Variance tradeoff from a Bayesian perspective

I know the general question about bias variance has been asked before. I understand the frequentist approach and the concept of model selection and the impact of bias and variance on "accuracy" of a ...
7 votes
0 answers
157 views

### R packages that work with biased samples

I'm working with a biased sample of web users. I'm only able to track responses of users who have navigated my site in a certain way, and I'd like to run an analysis to determine how certain factors (...
• 645
7 votes
0 answers
2k views

### Standardizing count variables in panel data with overdispersion - R or Stata

I'm running a regression where the dependent (response) variable is a highly dispersed (slightly zero-inflated) count and the explanatory (independent or predictor) variables are continuous, counts as ...
• 535