# Tagged Questions

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

479 views

### Negative binomial regression question - is it a poor model?

I am reading a very interesting article by Sellers and Shmueli on regression models for count data. Near the beginning (p. 944) they cite McCullaugh and Nelder (1989) saying that negative binomial ...
203 views

### Two ways of using bootstrap to estimate the confidence interval of coefficients in regression

I am applying a linear model to my data: $$y_{i}=\beta_{0}+\beta_{1}x_{i}+\epsilon_{i}, \quad\epsilon_{i} \sim N(0,\sigma^{2}).$$ I would like to estimate the confidence interval (CI) of the ...
280 views

### Advice on identifying curve shape using quantreg

I'm using the quantreg package to make a regression model using the 99th percentile of my values in a data set. Based on advice from a previous stackoverflow question I asked, I used the following ...
87 views

### Testing certain contrasts: Is this provably a hard problem, or not?

I posted this to mathoverflow and no one's answering: Scheffé's method for identifying statistically significant contrasts is widely known. A contrast among the means $\mu_i$, $i=1,\ldots,r$ of $r$ ...
244 views

### How to test for mediation when working with binary data?

I want to assess if a single variable is mediating the effects of a set of IVs on a single DV. All variables in the model (IV, DV and the mediator variable) are dichotomous (0, 1) and observed. What ...
544 views

### Random forests for multivariate regression

I have a multi-output regression problem with $d_x$ input features and $d_y$ outputs. The outputs have a complex, non-linear correlation structure. I'd like to use random forests to do the ...
188 views

### What does an infinite AIC mean and what can be done about it?

I have a question about performing stepwise regression. I realize that there are issues with using stepwise methods, but I have about 30 or so predictors and have constructed an ...
309 views

### Mean comparisons following multiple imputation

I need to do some simple mean comparisons between groups (basic ANOVA F-tests) on data with missing values. I use the mice package in R for multiple imputation, but I can only pool results for the ...
979 views

### How do you select variables in a regression model?

The traditional approach to variable selection is to find variables that contribute the most to predicting a new response. Recently I learned of an alternative to this. In modeling variables that ...
50 views

### Using a set of binary logistic regressions with multiple choice categorical response variable

I have categorical survey data on people's attitudes towards a certain policy area from 13 countries. The response variable is categorical, and includes 4 distinct answers that cannot be ordered. I ...
80 views

### Using percentiles as predictors - good idea?

I am thinking about a problem which is to predict log(spend) of a customer using linear regression. I am considering what features to use as input and wondering if it would be OK to use the ...
141 views

### How to interpret the significant interaction of two non-significant main predictors?

I have two non-significant main binary variables in my binary logistic regression model, but their interaction is significant. The variables are centered and no multicollinearity is the case (all VIFs ...
80 views

### Randomly distributed residuals or not?

Using Minitab I have checked for regression assumptions. Here I can see that the residuals (errors) of x4 are distributed normally. (y is response). However, are they distributed randomly? Looking at ...
111 views

### Regression model with heteroskedasticity in both variables

I've been learning (lurking) from this site for a while and I finally have a question I haven't seen answered yet. I'm doing a flight test and trying to fit the resulting data to linear line. From a ...
166 views

### Average Structural Function Calculation

EDIT: I have solved this problem myself. The problem with the simulation below is that the omitted variable should not be included in the 'true model'. I have written a blog post with a more detailed ...
199 views

### Regularization $L_1$ norm and $L_2$ norm empirical study

There are many methods to perform regularization -- $L_0$, $L_1$, and $L_2$ norm based regularization for example. According to Friedman Hastie & Tibsharani, the best regularizer depends on the ...
177 views

### Regression line envelope from Census ACS data

Context: I’m working with the Census Bureau’s American Community Survey (ACS) data which are samples (not complete enumerations) aggregated at different spatial scales. Each ACS estimate is provided ...
61 views

### Normalize binary features for logistic regression

I am applying logistic regression and I am using a mixture of continuous and binary features. My question would be how do you go about binary features when trying to normalize. I would very much ...
95 views

### Generalised least squares: from regression coefficients to correlation coefficients?

For least squares with one predictor: $y = \beta x + \epsilon$ If $x$ and $y$ are standardised prior to fitting (i.e. $\sim N(0,1)$), then: $\beta$ is the same as the Pearson correlation ...
47 views

### Zero values and discontinuity in explanatory variable

One of my independent variables measures worker productivity through the variable $\frac{\log{sales}}{\text{# of workers}}$, and I'm creating one variable for skilled and another for unskilled ...
536 views

### Unable to fit negative binomial regression in R (attempting to replicate published results)

Attempting to replicate the results from the recently published article, Aghion, Philippe, John Van Reenen, and Luigi Zingales. 2013. "Innovation and Institutional Ownership." American Economic ...
170 views

### Predicting count data with random forest

Can a Random Forest be trained to appropriately predict count data? How would this proceed? I have quite a extensive range of values so classification doesn't really make sense. If I would use ...
228 views

### Is standardization needed before fitting logistic regression?

My question is do we need to standardize the data set to make sure all variables have the same scale, between [0,1], before fitting logistic regression. The formula is ...
159 views

### Logistic regression model for analysis of many IVs with a relatively small sample size

I'm trying to determine the influence (direction and relative strength) of certain attributes of incoming students to an academic program on their successful completion of the program. My sample size ...
111 views

### Relationship between LASSO T and LARS number of steps k

We can see on the figure (cf Least Angle Regression p30, Efron, Hastie, Johnstone, Tibshirani - link: Least Angle Regression) that there is a direct relationship between: LASSO T absolute norm of ...
66 views

### Regressing on many outcomes in a spatial model

I'm measuring many microbe populations in a 'river model' ecosystem by sampling at two points. I want to test the hypothesis that key species upstream are the best predictors for any given population ...
141 views

### Variable Selection One by One vs Simultaneously

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
107 views

### Making new variable instead of correcting for temporal autocorrelation in a GLMM. Is it a valid alternative?

I am doing some forest disturbance research, in which the aim is to predict the probabilities of wind damage occurrence in forest stands of different site (altitude, slope steepness) and stand ...
254 views

### Multivariate ordered logit or probit

I have two ordinal dependent variables, each having three response levels. You can use an ordered logit or probit model for such data if you have one dependent variable. I've seen some papers about ...
368 views

### Fixed-effects or mixed-effects models to population based data

There are various definitions of fixed and random effects in linear mixed-effects models in the statistics literature, and even in the fixed-effects models from econometrics. The data I have now ...
58 views

### Simultaneous Non-parametric regression and Non-parametric density estimation

Given a collection of $(x,y)$ data, one might do a non/semiparametric regression of $y$ on $x$ to understand how to predict the $E(Y|X)$. Similarly, if one has enough data, it might be useful take a ...
560 views

### Methods to best test lead/lag relationships

I was wondering if you can share your experiences on what you feel is the best method to test lead / lag relationships between I(1) time series variables (i.e stock prices) and advantages and ...
93 views

### Determing noise type and level of noise

Context I have a set of data that was collected from several inertial measurement units (orientation and acceleration data). I want to determine to what extent an inference method degrades when the ...
25 views

### Method/workflow for analyzing data with changing structure?

I am analyzing data relating to networks that evolve over times (more precisely, a snap shot of the network at every discrete time step). Each node of the network denotes a person who perform some ...
33 views

### Truncated Explanatory Variable

I am running a model where one of the explanatory variables is truncated. In particular, the variable measures the duration of unemployment (retrospectively) in months and it is truncated at 2 years, ...
41 views

### Negative Binomial Regression and Heteroskedasticity test

I am running a negative binomial regression in Stata and would like to know if I need to include the vce(robust) option in the model. I know the negative binomial ...
19 views

### Test incorrect functional form when residuals have non-normal distribution

J. B. Ramsey (in "Tests for specification errors in classical linear least-squares regression analysis." Journal of the Royal Statistical Society. 1969) says that the RESET test assumes that the ...
58 views

### How to determine whether web traffic has gone up or down over a 12 month period?

I have traffic data for a website that simply shows total visitor each month over 12 months. I'm trying to establish if there's a general trend upwards or downwards. I seem to remember from high ...
59 views

### Cross validation for lasso logistic regression

I am writing a routine for logistic regression with lasso in matlab. So the problem is to minimize the negative log-likelihood function with the penalty term ...
54 views

### Determine linear relationship between input and output with multiple input variables

Say that your input variables are in $\mathbb{R}^2$ with a univariate output variable. Say that you want to determine whether the output variable is a linear combination of the input variables. If ...
46 views

### What is the analysis techniques for comparing two sets of ratings

Say, I have collected 50 comments. For each comment, I am planning to get 10 ratings from non-expert raters for the following criteria: "interesting vs. not interesting". Based on the plurality of ...
103 views

### On nonlinear regression, fits, and transformations

I am trying to fit a nonlinear regression model in R using nls(). I have a form of the equation I want to fit to: y = (a \times x_{1}^c +b \times x_{2}^d) ...
62 views

### Predicting for month in R

I'm trying to understand some concepts related to predictive modeling. So let's say that I have the following data sample and am trying to regress sales on ...
283 views

### Are categorical variables standardized differently in penalized regression?

In penalized/regularized regression (lasso, ridge, etc.) the predictors are typically standardized to be centered at 0 and often to have variance 1. Are categorical predictors treated differently. If ...
94 views

### Regression model of large, correlated, heavy-tailed data

I have a large panel-like data set: about 15,000 individuals, on average 350 time points, two dozen variables (plus some more variables we left out for context-specific reasons). What I want to ...
28 views

### Variable for logistic regression is categorical and continuous so creates “missingness” in R

I am doing a logistic regression analysis using the glm command in R. It is to identify causes of valve narrowing beyond a certain threshold; 0=no narrowing, 1=narrowed. One of my variables is the ...
83 views

### Help with repeated measures regression (in R)

I have some small datasets (example data given below) where I am interested in describing relationships between $Y_1$ and $Y_2$ across a gradient $X$ at two time points (date = "early" & "late"). ...
57 views

### What does a huge gap between P values of the same predictor computed in bootstrapped and non-bootstrapped versions of the same regression mean?

My binary logistic regression gives P values significant at 0.05 level for some of the modeled independent variables. However, bootstrapping the same regression model gives P values significant at ...