Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

18
votes
0answers
668 views

How does a Relevance Vector Machine (RVM) work?

Relevance Vector Machines (RVMs) are really interesting models when contrasted with the highly geometrical (and popular) SVMs. In the light of a question like How does a Support Vector Machine (SVM) ...
11
votes
0answers
1k views

Ratios in Regression, aka Questions on Kronmal

Recently, randomly browsing questions triggered a memory of on off-hand comment from one of my professors a few years back warning about the usage of ratios in regression models. So I started reading ...
10
votes
0answers
379 views

Asymptotic property of tuning parameter in penalized regression

I'm currently working on asymptotic properties of penalized regression. I've read a myriad of papers by now, but there is an essential issue that I cannot get my head around. To keep things simple, I'...
9
votes
0answers
2k views

Gamma hurdle model for continuous response

I am modelling invertebrate.biomass ~ habitat.type * calendar.day + habitat.type * calendar.day ^ 2, with a random intercept of transect.id (50 transects were repeated 5 times) My response is zero-...
7
votes
0answers
150 views

Efficient nonparametric estimation of confidence intervals and p-values for nonlinear regression

I'm estimating parameters for a complex, "implicit" nonlinear model $f(\mathbf{x}, \boldsymbol{\theta})$. It's "implicit" in the sense that I don't have an explicit formula for $f$: its value is the ...
7
votes
0answers
263 views

Should I use unpenalized logistic regression, lasso or ridge for explanatory modelling?

When using logistic regression for predictive modelling, the choice between 'standard' logistic regression vs ridge vs LASSO versions of logistic regression seems relatively straightforward - just ...
6
votes
0answers
39 views

Robust Gamma Regression

I am modeling some spectroscopic data where the response of the instrument to the size of the input is strictly positive and non-linear. Gamma regression seems like a good choice to explain the data, ...
6
votes
0answers
185 views

Regression with dependent data with low dependence

Suppose you have data that is grouped in one way or another and therefore the assumption of independence is suspect. But you look at the intraclass correlation (or autocorrelation) and it is very ...
6
votes
0answers
142 views

Is there any geometric intuition on least absolute deviation regression?

There are a lot of geometric intuitions for regression with least square, e.g., projection, orthogonal, etc. (This and this answers are good examples.) Is there similar geometric intuition for least ...
6
votes
0answers
66 views

Can I use bootstrap results at the observation level?

I have read quite a bit of bootstrapping, but the issue I want to address seem not to appear. Consider a simple regression model: $$ y_{i} = \beta_{0} + \beta_{1}x_{i} + e_{i}$$ I am aware that ...
6
votes
0answers
335 views

Deep Learning for Ordinal Classification

Can we use any of the various types of Deep Neural Networks for Ordinal Classification? If yes, then how? If no, then what is the limiting problem? I know that CovNets with a softmax output can be ...
6
votes
0answers
552 views

Rule of thumb for excluded variable in Heckman selection model?

I'm working on a project that involves the use of a Heckman selection model (more specifically a Roy or move-stay model, which is essentially a two-sided Heckman) of the following form: $$ Y_{i1} = ...
6
votes
0answers
5k views

“Zero-inflated” predictors in regression?

I know that zero-inflated models (e.g. zero-inflated Poisson or negative binomial models) can be used for dependent variables. I also know that in general there are no assumptions for the independent ...
6
votes
0answers
208 views

Generalization of cumulative probability models for ordinal Y

There are many models in existence for ordinal $Y$, for example the proportional odds ordinal logistic model, the continuation ratio model, and the cumulative probit model. The first and third of ...
6
votes
0answers
97 views

Zero values and discontinuity in explanatory variable

One of my independent variables measures worker productivity through the variable $\frac{\log{sales}}{\text{# of workers}}$, and I'm creating one variable for skilled and another for unskilled workers....
6
votes
0answers
826 views

Dantzig Selector, LASSO, LAD LASSO

I am wondering about this. When is it best to use Dantzig Selector (the infinity normed error measure plus the L1 regularizer) , the LASSO (the mean square error measure plus the L1 regularizer), and ...
6
votes
0answers
3k views

Poisson regression for binary data

I've been trying to read up on Poisson regression models, and it looks like it is possible to estimate such a model with a binary outcome. This has come up before on this site here (and somewhat here ...
6
votes
0answers
521 views

Average Structural Function Calculation

EDIT: I have solved this problem myself. The problem with the simulation below is that the omitted variable should not be included in the 'true model'. I have written a blog post with a more detailed ...
6
votes
0answers
257 views

Making new variable instead of correcting for temporal autocorrelation in a GLMM. Is it a valid alternative?

I am doing some forest disturbance research, in which the aim is to predict the probabilities of wind damage occurrence in forest stands of different site (altitude, slope steepness) and stand ...
5
votes
0answers
37 views

Compare if two regression model behaves similarly

this is my first asking question here... forgive me if my question is not clear enough. I have two datasets; one is ground truth dataset and another is perturbed version of the same dataset. I want ...
5
votes
0answers
38 views

“Return values” of univariate logistic regression

I read an interesting article on an approach to calibrate probabilistic classifiers (Kull et al. 2017, "Beyond sigmoids: How to obtain well-calibrated probabilities from binary classifiers with beta ...
5
votes
0answers
82 views

Why does “mixtools” return the model with highest AIC as the “winner” if lower AIC is better?

Mixtools package is used to fit mixtures of normal/regressions. The package documentation is given here The regmixmodel.sel fits the mixture model for varying ...
5
votes
0answers
183 views

Is linear regression equivalent to the support vector regression with a linear kernel?

I notice the objective function of the linear regression and the support vector regression (SVR) with a linear kernel could be the same, except for the SVR has two error constraints for each data ...
5
votes
0answers
299 views

Predictive modeling using GAM (mgcv)

I have seven years of continuous insect population data, along with temperature and humidity parameters. I’d like to use this data to predict future populations in a given year using a generalized ...
5
votes
0answers
325 views

Bayesian linear regression - posterior distribution

This is about bayesian linear regression. In this link http://fourier.eng.hmc.edu/e161/lectures/gaussianprocess/node2.html there's a derivation for = The part that I don't understand is how it is ...
5
votes
0answers
601 views

Signatures of underfitting and overfitting in logistic regression calibration curves

My confusion stems from reading the following paper http://www.bmj.com/content/351/bmj.h3868 It states in its abstract (and they later show an empirical study that conforms to the claim) - "...
5
votes
0answers
760 views

When is oversampling poor practice?

For my particular domain and problem, I have data on the entire population. However, my "event" only occurs in 0.5% of the cases. I want my model to be able to pick up on significant characteristics ...
5
votes
0answers
544 views

Are there unbiased, non-linear estimators with lower variance than the OLS estimator?

Consider an ordinary least squares model, $$y = \beta X + \epsilon \qquad \epsilon\sim N(0, \sigma)$$ The Gauss-Markov theorem tells us that the ordinary least-squares (OLS) estimator is the minimum-...
5
votes
0answers
479 views

Does there exist zero-inflated linear regression?

I have a non-count data with huge number of zeros in the target variable. I need to fit a model being a mixture of Dirac delta function and normal distribution parametrized by mean $X\beta$ and ...
5
votes
0answers
202 views

Orthogonality in ANOVA and Regression Analysis

I read the following (Wikipedia) regarding contrast coding of categorical variables: Unlike when used in ANOVA, where it is at the researcher’s discretion whether they choose coefficient values ...
5
votes
0answers
162 views

Wishart Conditional

I am looking for the conditional probability of a Wishart distribution, i.e., if I have a Wishart distributed variable $ S \sim W(\Sigma,n), $ where $$ S = \begin{bmatrix} S_1 \quad S_{12} \\ S_{21} ...
5
votes
0answers
522 views

How to test a linear relationship between log odds and predictors before performing logistic regression?

In case of a linear regression, it's easy to test a linear relationship between a continuous dependent variable and each independent variable. For example, I can plot a scatter plot between the ...
5
votes
0answers
867 views

Forecasting seasonal time series in R with weekly data

I have a data set that contains weekly orders for a product. Since the data is a time series, I have the following line: ...
5
votes
0answers
3k views

Cause of a high condition number in a python statsmodels regression?

I'm pretty new to regression analysis, and I'm using python's statsmodels to look at the relationship between GDP/health/social services spending and health outcomes (DALYs) across the OECD. Just to ...
5
votes
0answers
7k views

Comparing models using the deviance and log-likelihood ratio tests

This is an update to a previous question that I have posted. I am looking for clarification on comparing glm models using deviance and log-likelihood ratio tests (I have updated my question to make it ...
5
votes
0answers
93 views

Is autocorrelation not worth addressing with small N?

Consider a simple regression context in which there is a small set of response values, $Y$, and corresponding dates, $X$. (For simplicity, we can assume the dates are equally spaced.) We would like ...
5
votes
0answers
248 views

Is there an appropriate order to apply bagging and filter feature selection?

I'm training a (regression) learner on a $p \gg n$ problem, including bagging and filter feature selection (information gain). I'm in doubt though regarding the order of the procedures: Apply the ...
5
votes
0answers
440 views

Why the focus on variance reduction for $R^2$?

It seems to odd to me that we measure the explanatory power of a regression model in "percent of variance explained", or $R^2 = {\rm cor}(\hat{y},y)^2 = r^2$ even though we all know that variance is ...
5
votes
0answers
805 views

How to deal with failing the proportional odds assumption in ordinal logistic regression

I am attempting to do ordinal logistic regression but I keep failing to pass the proportional odds assumption. Almost all of my features are shown to have high significance, but the only model that I ...
5
votes
0answers
731 views

Why would I use ratio estimation instead of regression estimation to estimate means?

I am taking a graduate course on survey data analysis. I was recently introduced to ratio estimation and regression estimation. I understand that using ratio estimator may be easier if we are ...
5
votes
0answers
232 views

Interpretation of smoothing spline

This question is about interpreting the results from non-linear regression models, especially when using regression splines. The numerical output is not very informative when interpreting the effects, ...
5
votes
0answers
481 views

Can I calculate Cohen's $d$ from multiple regression coefficient?

Question: Is it appropriate to calculate Cohen's $d$ (effect size) from the regression coefficient of an independent categorical variable? Background: My regression coefficient represents ...
5
votes
0answers
725 views

What techniques are there to measure goodness of fit of Deming (orthogonal) regression?

Questions: Even if there is no "widely accepted" technique, is there a useful-and-above-average technique for estimating goodness of fit in orthogonal regressions? What are the pros/cons of this ...
5
votes
0answers
445 views

Regression on large samples: can aggregation of the dependent variable by covariate pattern increase speed of estimation?

Is there a way to build a regression model for continuous output using aggregate data instead of individual data points when all input variables are categorical? I have a moderately large dataset (...
5
votes
0answers
4k views

Regress residuals in second regression

I am wondering if anyone can point me to a paper/lecture notes on the rationale behind first running an OLS on a set of variables, and then in a second regression using the residuals of that ...
5
votes
0answers
144 views

How to evaluate uncertainty estimates in regression?

Some regression algorithms (e.g. Gaussian process regression) can produce uncertainties along with point predictions at test time. These should also be evaluated. How about calculating the Pearson ...
5
votes
0answers
392 views

Ideal statistical or machine learning technique to model highly cross-correlated data

I'm trying to build a model that can predict streamflow for an alpine (snowmelt-fed) watershed using snow albedo (roughly, the energy reflectance of the snow) data. Albedo controls the melt of the ...
5
votes
0answers
244 views

Restriction matrix for a VAR

In New Introduction to Multiple Time Series Analysis by Luetkepohl (2005), section 5.2.1, it says that one can specify linear restraints for a VAR, $Y = \beta X + U$, in the form $$ \operatorname{vec}{...
5
votes
0answers
94 views

R packages that work with biased samples

I'm working with a biased sample of web users. I'm only able to track responses of users who have navigated my site in a certain way, and I'd like to run an analysis to determine how certain factors (...
5
votes
0answers
1k views

Standardizing count variables in panel data with overdispersion - R or Stata

I'm running a regression where the dependent (response) variable is a highly dispersed (slightly zero-inflated) count and the explanatory (independent or predictor) variables are continuous, counts as ...