Unanswered 'regression' Questions

25

votes

0answers

1k views

How does a Relevance Vector Machine (RVM) work?

Relevance Vector Machines (RVMs) are really interesting models when contrasted with the highly geometrical (and popular) SVMs. In the light of a question like How does a Support Vector Machine (SVM) ...

asked Sep 28 '16 at 13:55

Firebug

13.3k3 gold badges49 silver badges113 bronze badges

13

votes

0answers

3k views

Gamma hurdle model for continuous response

I am modelling invertebrate.biomass ~ habitat.type * calendar.day + habitat.type * calendar.day ^ 2, with a random intercept of transect.id (50 transects were repeated 5 times) My response is zero-...

r regression glmm zero-inflation

asked Aug 6 '15 at 8:05

Tom Finch

2312 silver badges4 bronze badges

13

votes

0answers

436 views

Asymptotic property of tuning parameter in penalized regression

I'm currently working on asymptotic properties of penalized regression. I've read a myriad of papers by now, but there is an essential issue that I cannot get my head around. To keep things simple, I'...

regression lasso parameterization regularization

asked Apr 25 '13 at 13:35

Nick Sabbe

11.7k2 gold badges33 silver badges43 bronze badges

12

votes

1answer

9k views

Cause of a high condition number in a python statsmodels regression?

I'm pretty new to regression analysis, and I'm using python's statsmodels to look at the relationship between GDP/health/social services spending and health outcomes (DALYs) across the OECD. Just to ...

regression python statsmodels condition-number

asked Oct 28 '16 at 18:44

pst0102

1211 silver badge5 bronze badges

12

votes

2answers

536 views

Adjustments to (Linear Regression) Forecast

Full disclosure: I am not a statistician, nor do I claim to be one. I am a lowly IT administrator. Please play gentle with me. :) I am responsible for collecting and forecasting disk storage use ...

regression forecasting adjustment

asked Feb 11 '16 at 20:27

sbrown

2211 silver badge2 bronze badges

11

votes

0answers

6k views

“Zero-inflated” predictors in regression?

I know that zero-inflated models (e.g. zero-inflated Poisson or negative binomial models) can be used for dependent variables. I also know that in general there are no assumptions for the independent ...

regression poisson-distribution zero-inflation

asked Jul 1 '13 at 3:10

KuJ

1,2513 gold badges15 silver badges23 bronze badges

10

votes

0answers

267 views

When/why not to use studentized residuals for regression diagnostics?

Consider a linear regression $$ y=X\beta+\varepsilon. $$ Residuals $e:=y-X\hat\beta$ are often used as substitutes for the unobserved model errors $\varepsilon$ for validating assumptions such as ...

regression residuals diagnostic

asked May 3 '19 at 12:59

Richard Hardy

45.3k9 gold badges75 silver badges193 bronze badges

10

votes

1answer

2k views

Hidden state models vs. stateless models for time series regression

This is a quite generic question: assume I want to build a model to predict the next observation based on the previous $N$ observations ($N$ can be a parameter to optimize experimentally). So we ...

regression time-series modeling prediction hidden-markov-model

asked Apr 27 '14 at 20:53

Mannaggia

1516 bronze badges

10

votes

1answer

388 views

Interpreting regression coefficients based on Andrew Gelman's re-scaling method

I have two predictors in a binary logistic regression model: One binary and one continuous. My primary goal is to compare the coefficients of the two predictors within the same model. I have come ...

regression logistic regression-coefficients standardization scales

asked May 7 '18 at 21:54

ksroogl

3531 silver badge11 bronze badges

9

votes

0answers

126 views

Pope effect on pizza - Regression with presence absence and similarity data as dependent variables

I'm trying to figure out the right way to set up a regression when the dependent variables are presence absence data (of pizzas), and the similarity between the present pizzas. Bear with the story: ...

regression bayesian spatial

asked Dec 17 '19 at 15:01

elsherbini

2312 silver badges5 bronze badges

9

votes

0answers

176 views

Bootstrap Prediction Interval: which residuals to use and which method?

I ask this question referring to the post: Bootstrap prediction interval, where a step by step method for calculating the prediction interval for linear regression models is explained. In the ...

regression bootstrap reliability prediction-interval

asked Aug 9 '19 at 12:04

user2683832

1711 silver badge4 bronze badges

9

votes

0answers

1k views

Are there unbiased, non-linear estimators with lower variance than the OLS estimator?

Consider an ordinary least squares model, $$y = \beta X + \epsilon \qquad \epsilon\sim N(0, \sigma)$$ The Gauss-Markov theorem tells us that the ordinary least-squares (OLS) estimator is the minimum-...

regression mathematical-statistics linear-model

asked Jul 4 '17 at 4:05

user126350

2711 silver badge4 bronze badges

8

votes

0answers

154 views

Understanding equation used by Hastie et al

I am trying to recreate FIGURE 3.6 from Elements of Statistical Learning. The only information about the figure is included in the caption. I am not clear on what the equation on the Y-axis means ...

regression machine-learning self-study stepwise-regression

asked Jun 30 '19 at 16:44

Seraf Fej

3541 silver badge13 bronze badges

8

votes

1answer

2k views

does serial correlation have something to do with endogeneity?

I'm a beginner of econometrics, and I've construed that endogeneity is caused by omitted variable bias, measurement error, and reverse causality, and it makes OLS estimator be biased. And also I've ...

regression autocorrelation endogeneity

asked May 10 '17 at 13:13

Kevin Kang

3792 silver badges10 bronze badges

8

votes

0answers

397 views

How do sufficiency statistics help in the interpretation of regression results?

One of the results why canonical link functions are widely used in GLMs is the existence of sufficiency statistics for the regression parameters, which in turn allow for: ... minimal sufficient ...

regression generalized-linear-model interpretation sufficient-statistics link-function

asked Mar 1 '17 at 1:27

Alex

3,6082 gold badges23 silver badges45 bronze badges

8

votes

0answers

198 views

Is autocorrelation not worth addressing with small N?

Consider a simple regression context in which there is a small set of response values, $Y$, and corresponding dates, $X$. (For simplicity, we can assume the dates are equally spaced.) We would like ...

regression time-series autocorrelation covariance-matrix small-sample

asked Jul 5 '16 at 17:13

gung - Reinstate Monica

126k76 gold badges333 silver badges622 bronze badges

7

votes

0answers

179 views

Efficient nonparametric estimation of confidence intervals and p-values for nonlinear regression

I'm estimating parameters for a complex, "implicit" nonlinear model $f(\mathbf{x}, \boldsymbol{\theta})$. It's "implicit" in the sense that I don't have an explicit formula for $f$: its value is the ...

regression confidence-interval p-value bootstrap nonlinear-regression

asked Apr 15 '17 at 9:00

DeltaIV

14.5k4 gold badges54 silver badges99 bronze badges

7

votes

0answers

580 views

Assumptions for PCR and PLS

I am writing up a report on fitting Principal Component Regression (PCR) and Partial Least Squares (PLS) to my data-set. A similar question: Model assumptions of partial least squares (PLS) ...

regression assumptions

asked Jun 23 '16 at 15:20

Kivis

2811 silver badge7 bronze badges

7

votes

1answer

265 views

Logistic Regression with (Normal) Distributions for Independent Variables

Consider the logistic regression where $Y_i \in {0,1}$ are dependent variable observations and $X_i \in \mathbb{R}$ are the independent variables. However we do not observe the $X_i$ themselves. ...

regression logistic normal-distribution regression-strategies

asked Mar 3 '16 at 11:03

rwolst

5724 silver badges11 bronze badges

7

votes

0answers

4k views

Poisson regression for binary data

I've been trying to read up on Poisson regression models, and it looks like it is possible to estimate such a model with a binary outcome. This has come up before on this site here (and somewhat here ...

regression poisson-distribution binary-data

asked Sep 26 '12 at 3:34

Dimitriy V. Masterov

27.9k5 gold badges56 silver badges121 bronze badges

7

votes

1answer

3k views

Using Fieller's theorem to calculate the confidence interval of a ratio (paired measurements)

If you have two means (with their own confidence intervals) and want to represent them as a ratio, how do calculate the confidence interval for the ratio? An answer that was given to me, mentions ...

regression confidence-interval ratio

asked Jun 24 '15 at 9:06

Marc

711 silver badge2 bronze badges

6

votes

0answers

37 views

Interactions in Multiple Linear Regression (Divide Vs Multiply)

My question is about the difference (in general) between the interaction terms $x_1x_2$ and $x_1/x_2$ in multiple linear regression. Suppose you are performing multiple linear regression and you have ...

regression machine-learning interaction

asked Mar 2 at 8:33

Mike Tauber

5891 gold badge4 silver badges11 bronze badges

6

votes

0answers

74 views

In reality, there is almost always measurement error in the independent variable(s), so why is this ignored in almost every linear regression model?

In the vast majority of cases, linear regression models are used in practice as opposed to the more complicated errors-in-variables models. For the sake of example, consider modelling height $Y$ vs ...

regression modeling measurement-error errors-in-variables

asked Sep 10 '20 at 10:59

Bertus101

6456 bronze badges

6

votes

0answers

212 views

Robust Gamma Regression

I am modeling some spectroscopic data where the response of the instrument to the size of the input is strictly positive and non-linear. Gamma regression seems like a good choice to explain the data, ...

regression bayesian generalized-linear-model heteroscedasticity pymc

asked Apr 10 '19 at 20:50

udushu

2131 silver badge6 bronze badges

6

votes

0answers

210 views

Regression with dependent data with low dependence

Suppose you have data that is grouped in one way or another and therefore the assumption of independence is suspect. But you look at the intraclass correlation (or autocorrelation) and it is very ...

regression multilevel-analysis non-independent

asked Mar 20 '19 at 20:18

Peter Flom

91.4k34 gold badges136 silver badges266 bronze badges

6

votes

0answers

5k views

Calculating R-squared using standard errors

I have the following estimated model: $\hat{y} = 0.2857 + 0.8019x_1 - 0.0741x_2$ (the $t$-statistics are $1.8959$, $8.4198$, and $-3.7017$, respectively). Furthermore, I know the sample size $N = 92$,...

regression r-squared

asked Mar 19 '19 at 10:38

Anna

1931 silver badge5 bronze badges

6

votes

0answers

697 views

Geometrical interpretation of L1 regression

I have found the following image (or a similar version) in a lot of books related to penalized linear models. I get the insight of this image. The ellipsoids are the solution of the linear regression ...

regression lasso ridge-regression quantile-regression

asked May 20 '18 at 17:32

Álvaro Méndez Civieta

1,0075 silver badges15 bronze badges

6

votes

0answers

1k views

Is the OLS estimator the UMVUE (assuming Normality)?

Suppose $$ \mathbf{y} = \mathbf{X} \mathbf{b} + \mathbf{e} \, , \\ \mathbf{e} \sim \mathcal{N}(0,\mathbf{I}_P) \, . $$ We know that $\mathbf{\hat{b}} = (\mathbf{X}^T \mathbf{X})^{-1} \mathbf{X}^T \...

regression normal-distribution least-squares umvue blue

asked May 15 '18 at 9:49

Patrick

7194 silver badges13 bronze badges

6

votes

0answers

201 views

Is there any geometric intuition on least absolute deviation regression?

There are a lot of geometric intuitions for regression with least square, e.g., projection, orthogonal, etc. (This and this answers are good examples.) Is there similar geometric intuition for least ...

regression loss-functions intuition geometry least-absolute-deviations

asked Jan 23 '18 at 19:11

Haitao Du

30.3k14 gold badges101 silver badges196 bronze badges

6

votes

0answers

575 views

Is lasso always outperformed by adaptive lasso?

I have been reading some papers and I understood that adaptive lasso has the Oracle properties which lasso lacks. Does that mean adaptive lasso always better than lasso (let's focus on the simple ...

regression lasso linear oracle

asked Oct 20 '17 at 19:59

MZ75

714 bronze badges

6

votes

0answers

326 views

Does there exist an analogous statement to BLUE (Gauss-Markov) for GLMs?

I recall from my graduate school days that the Gauss-Markov (GM) theorem states that the Best Linear Unbiased Estimator (BLUE) in a linear regression is $\vec{\beta}=(X^TX)^{-1}X^T\vec{y}$. An amazing ...

regression normal-distribution multiple-regression generalized-linear-model optimal

asked Sep 19 '17 at 2:20

Lucas Roberts

3,42912 silver badges40 bronze badges

6

votes

0answers

949 views

Signatures of underfitting and overfitting in logistic regression calibration curves

My confusion stems from reading the following paper http://www.bmj.com/content/351/bmj.h3868 It states in its abstract (and they later show an empirical study that conforms to the claim) - "...

regression logistic overfitting calibration

asked Aug 18 '17 at 16:56

Ayush Saxena

613 bronze badges

6

votes

0answers

2k views

When is oversampling poor practice?

For my particular domain and problem, I have data on the entire population. However, my "event" only occurs in 0.5% of the cases. I want my model to be able to pick up on significant characteristics ...

regression classification sampling

asked Aug 14 '17 at 18:34

blacksite

4021 gold badge8 silver badges22 bronze badges

6

votes

0answers

483 views

Cumulative hazard in the setting of Cox regression of repeated events

Cox regression is commonly extended to estimate repeated events processes (for a quick review see [ 1 ] and [ 2 ]). In Clark et al's first article in their excellent review series of survival ...

regression survival cox-model hazard recurrent-events

asked Jun 9 '17 at 10:43

user6571411

2101 silver badge7 bronze badges

6

votes

0answers

867 views

Does there exist zero-inflated linear regression?

I have a non-count data with huge number of zeros in the target variable. I need to fit a model being a mixture of Dirac delta function and normal distribution parametrized by mean $X\beta$ and ...

regression zero-inflation

asked Jun 1 '17 at 14:04

Tim♦

89.2k17 gold badges188 silver badges321 bronze badges

6

votes

0answers

1k views

How to test a linear relationship between log odds and predictors before performing logistic regression?

In case of a linear regression, it's easy to test a linear relationship between a continuous dependent variable and each independent variable. For example, I can plot a scatter plot between the ...

regression logistic logit linear odds

asked Jan 23 '17 at 14:52

Cha.Po

614 bronze badges

6

votes

0answers

74 views

Can I use bootstrap results at the observation level?

I have read quite a bit of bootstrapping, but the issue I want to address seem not to appear. Consider a simple regression model: $$ y_{i} = \beta_{0} + \beta_{1}x_{i} + e_{i}$$ I am aware that ...

regression bootstrap standard-error

asked Aug 21 '16 at 15:16

luchonacho

2,4222 gold badges16 silver badges37 bronze badges

6

votes

0answers

898 views

Why would I use ratio estimation instead of regression estimation to estimate means?

I am taking a graduate course on survey data analysis. I was recently introduced to ratio estimation and regression estimation. I understand that using ratio estimator may be easier if we are ...

regression estimation survey ratio

asked Nov 15 '15 at 8:29

Oğuzhan Öğreden

1566 bronze badges

6

votes

0answers

719 views

Can I calculate Cohen's $d$ from multiple regression coefficient?

Question: Is it appropriate to calculate Cohen's $d$ (effect size) from the regression coefficient of an independent categorical variable? Background: My regression coefficient represents ...

regression regression-coefficients cohens-d

asked Sep 10 '15 at 0:39

user89128

611 bronze badge

6

votes

0answers

2k views

Fixed Effects vs Lagged DV vs. First Differences Regression

What are the differences between using unit fixed effects, unit fixed effects and time fixed effects, lagged DV, or first differences to analyze a time series with 4-5 time periods and 35-50 units per ...

regression fixed-effects-model lags

asked Jul 31 '15 at 0:31

Michael

3503 silver badges13 bronze badges

6

votes

0answers

7k views

Regress residuals in second regression

I am wondering if anyone can point me to a paper/lecture notes on the rationale behind first running an OLS on a set of variables, and then in a second regression using the residuals of that ...

regression multivariate-analysis modeling residuals instrumental-variables

asked Jun 25 '15 at 15:30

Z_D

1231 silver badge6 bronze badges

6

votes

1answer

702 views

Calculating and plotting confidence interval for Theil-Sen estimator

I'm using Wilcox's R functions (specifically, regplot) to plot a Theil-Sen estimator with a single predictor. However, regplot ...

r regression confidence-interval data-visualization robust

asked Jun 20 '15 at 20:30

coding dummy

612 bronze badges

6

votes

0answers

680 views

Rule of thumb for excluded variable in Heckman selection model?

I'm working on a project that involves the use of a Heckman selection model (more specifically a Roy or move-stay model, which is essentially a two-sided Heckman) of the following form: $$ Y_{i1} = ...

regression econometrics instrumental-variables identifiability heckman

asked Feb 6 '15 at 18:27

NickCHK

2061 silver badge7 bronze badges

6

votes

1answer

334 views

minimizer weighted linear regression

In a regression problem, with $y=X\theta+\epsilon$ and $X$ is an $n$ by $p$ matrix the ‘weighted least squares estimate is the minimizer $\theta^{*}$ of $f(\theta)=\sum_{i=1}^{n}\omega_{i}(y_i-x_i^{'}\...

regression mathematical-statistics covariance weighted-regression extreme-value

asked Dec 1 '14 at 5:31

M.Krov

1611 bronze badge

6

votes

0answers

949 views

Bias Variance tradeoff from a Bayesian perspective

I know the general question about bias variance has been asked before. I understand the frequentist approach and the concept of model selection and the impact of bias and variance on "accuracy" of a ...

regression bayesian variance bias

asked Jun 5 '14 at 10:02

dazedandconfused

1213 bronze badges

6

votes

0answers

134 views

R packages that work with biased samples

I'm working with a biased sample of web users. I'm only able to track responses of users who have navigated my site in a certain way, and I'd like to run an analysis to determine how certain factors (...

r regression bias unbiased-estimator

asked Apr 18 '14 at 13:52

user1956609

5252 gold badges5 silver badges8 bronze badges

6

votes

0answers

1k views

Logistic regression and maximum entropy

I have read (e.g. here) that a (multinomial) logistic regressor corresponds to a maximum entropy classifier. My question is, how does one end up with the formula for logistic regression starting with ...

regression logistic classification entropy maximum-entropy

asked Nov 22 '13 at 17:09

Josh

2,9682 gold badges20 silver badges39 bronze badges

6

votes

0answers

247 views

Generalization of cumulative probability models for ordinal Y

There are many models in existence for ordinal $Y$, for example the proportional odds ordinal logistic model, the continuation ratio model, and the cumulative probit model. The first and third of ...

regression ordered-logit ordered-probit

asked May 9 '13 at 16:28

Frank Harrell

65.3k4 gold badges131 silver badges285 bronze badges

6

votes

0answers

101 views

Zero values and discontinuity in explanatory variable

One of my independent variables measures worker productivity through the variable $\frac{\log{sales}}{\text{# of workers}}$, and I'm creating one variable for skilled and another for unskilled workers....

regression

asked Apr 26 '13 at 3:14

user62117

1025 bronze badges

6

votes

0answers

631 views

Studentized residuals and goodness-of-fit with robust linear regression

Could you please advise whether studentized residuals are meaningful when computed on a robust linear regression model using an M-estimator? I'd like to use it to detect outliers by doing something ...

r regression robust

asked Mar 28 '13 at 10:19

a11msp

6636 silver badges19 bronze badges

Questions tagged [regression]

Related Tags