# Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

6,503 questions
581 views

### How does a Relevance Vector Machine (RVM) work?

Relevance Vector Machines (RVMs) are really interesting models when contrasted with the highly geometrical (and popular) SVMs. In the light of a question like How does a Support Vector Machine (SVM) ...
1k views

### Ratios in Regression, aka Questions on Kronmal

Recently, randomly browsing questions triggered a memory of on off-hand comment from one of my professors a few years back warning about the usage of ratios in regression models. So I started reading ...
360 views

### Asymptotic property of tuning parameter in penalized regression

I'm currently working on asymptotic properties of penalized regression. I've read a myriad of papers by now, but there is an essential issue that I cannot get my head around. To keep things simple, I'...
136 views

### Efficient nonparametric estimation of confidence intervals and p-values for nonlinear regression

I'm estimating parameters for a complex, "implicit" nonlinear model $f(\mathbf{x}, \boldsymbol{\theta})$. It's "implicit" in the sense that I don't have an explicit formula for $f$: its value is the ...
229 views

### Should I use unpenalized logistic regression, lasso or ridge for explanatory modelling?

When using logistic regression for predictive modelling, the choice between 'standard' logistic regression vs ridge vs LASSO versions of logistic regression seems relatively straightforward - just ...
2k views

### Gamma hurdle model for continuous response

I am modelling invertebrate.biomass ~ habitat.type * calendar.day + habitat.type * calendar.day ^ 2, with a random intercept of transect.id (50 transects were repeated 5 times) My response is zero-...
123 views

4k views

### “Zero-inflated” predictors in regression?

I know that zero-inflated models (e.g. zero-inflated Poisson or negative binomial models) can be used for dependent variables. I also know that in general there are no assumptions for the independent ...
205 views

### Generalization of cumulative probability models for ordinal Y

There are many models in existence for ordinal $Y$, for example the proportional odds ordinal logistic model, the continuation ratio model, and the cumulative probit model. The first and third of ...
96 views

### Zero values and discontinuity in explanatory variable

One of my independent variables measures worker productivity through the variable $\frac{\log{sales}}{\text{# of workers}}$, and I'm creating one variable for skilled and another for unskilled workers....
782 views

### Dantzig Selector, LASSO, LAD LASSO

I am wondering about this. When is it best to use Dantzig Selector (the infinity normed error measure plus the L1 regularizer) , the LASSO (the mean square error measure plus the L1 regularizer), and ...
3k views

### Poisson regression for binary data

I've been trying to read up on Poisson regression models, and it looks like it is possible to estimate such a model with a binary outcome. This has come up before on this site here (and somewhat here ...
496 views

### Average Structural Function Calculation

EDIT: I have solved this problem myself. The problem with the simulation below is that the omitted variable should not be included in the 'true model'. I have written a blog post with a more detailed ...
249 views

### Making new variable instead of correcting for temporal autocorrelation in a GLMM. Is it a valid alternative?

I am doing some forest disturbance research, in which the aim is to predict the probabilities of wind damage occurrence in forest stands of different site (altitude, slope steepness) and stand ...
44 views

### Why does “mixtools” return the model with highest AIC as the “winner” if lower AIC is better?

Mixtools package is used to fit mixtures of normal/regressions. The package documentation is given here The regmixmodel.sel fits the mixture model for varying ...
141 views

### Is linear regression equivalent to the support vector regression with a linear kernel?

I notice the objective function of the linear regression and the support vector regression (SVR) with a linear kernel could be the same, except for the SVR has two error constraints for each data ...
223 views

### Predictive modeling using GAM (mgcv)

I have seven years of continuous insect population data, along with temperature and humidity parameters. I’d like to use this data to predict future populations in a given year using a generalized ...
498 views

### Signatures of underfitting and overfitting in logistic regression calibration curves

My confusion stems from reading the following paper http://www.bmj.com/content/351/bmj.h3868 It states in its abstract (and they later show an empirical study that conforms to the claim) - "...
497 views

### When is oversampling poor practice?

For my particular domain and problem, I have data on the entire population. However, my "event" only occurs in 0.5% of the cases. I want my model to be able to pick up on significant characteristics ...
472 views

### Are there unbiased, non-linear estimators with lower variance than the OLS estimator?

Consider an ordinary least squares model, $$y = \beta X + \epsilon \qquad \epsilon\sim N(0, \sigma)$$ The Gauss-Markov theorem tells us that the ordinary least-squares (OLS) estimator is the minimum-...
167 views

### Orthogonality in ANOVA and Regression Analysis

I read the following (Wikipedia) regarding contrast coding of categorical variables: Unlike when used in ANOVA, where it is at the researcher’s discretion whether they choose coefficient values ...
151 views

837 views

### Model diagnostics for a glmmPQL in R mixed-effects model

Several texts (both online and published books) have been reviewed prior to asking this. What diagnostics are accepted as best practise for a generalised linear mixed-effects model fitted in R using ...
93 views

### R packages that work with biased samples

I'm working with a biased sample of web users. I'm only able to track responses of users who have navigated my site in a certain way, and I'd like to run an analysis to determine how certain factors (...
1k views

### Standardizing count variables in panel data with overdispersion - R or Stata

I'm running a regression where the dependent (response) variable is a highly dispersed (slightly zero-inflated) count and the explanatory (independent or predictor) variables are continuous, counts as ...
462 views

### Is it ever appropriate to put dependent variables in a model when they calculate the independent variable directly?

Is it ever appropriate to fit a multivariable regression model with dependent variables that directly calculate the independent variable? For example, I know that fitting a model to predict BMI by ...
879 views

### Logistic regression and maximum entropy

I have read (e.g. here) that a (multinomial) logistic regressor corresponds to a maximum entropy classifier. My question is, how does one end up with the formula for logistic regression starting with ...
475 views

### Studentized residuals and goodness-of-fit with robust linear regression

Could you please advise whether studentized residuals are meaningful when computed on a robust linear regression model using an M-estimator? I'd like to use it to detect outliers by doing something ...
861 views

### Rule of thumb - number of predictors - Poisson regression rates

I am interested in estimating a Poisson regression for mortality rates, with number of deaths as the dependent variable and log(population size) as the offset. I have 50 observations (states). I am ...
2k views

### Time series forecasting using autoregressive and linear terms in R

I have real daily market data which I'm looking at to create a model for forecasting. The model that I created (below) used autoregressive terms within a linear regression. I was sharing this with a ...
1k views

### Log-Ratio \ Compositional analysis

I am not a trained statistician but I am trying to improve on my own MBA thesis which was essentially regression analysis of the factors affecting opening cinema box-office in the UK. I am now ...
392 views

### Regression line envelope from Census ACS data

Context: I’m working with the Census Bureau’s American Community Survey (ACS) data which are samples (not complete enumerations) aggregated at different spatial scales. Each ACS estimate is provided ...
Given a collection of $(x,y)$ data, one might do a non/semiparametric regression of $y$ on $x$ to understand how to predict the $E(Y|X)$. Similarly, if one has enough data, it might be useful take a ...