# Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

18,439 questions
10 views

### Regression shows high multicollinearity even after a PCA

I am comparing men and women using a measure of personality which has 24 variables. I did a oblimin rotated PCA for women and men separately so that they are truly representative of the population and ...
5k views

### In a GLM, is the log likelihood of the saturated model always zero?

As part of the output of a generalised linear model, the null and residual deviance are used to evaluate the model. I often see the formulas for these quantities expressed in terms of the log ...
2 views

### What kind of discrete limited dependent variable regression do I need? Or other?

I have a sample of 78 scientific articles plus 385 software and data resources used by these papers. Each paper has between 1-26 such resources. Sometimes, the creators of software/data will request ...
63 views

### Should this regression be tested for stationarity?

This is a fairly basic question I am struggling with. I am using the oil prices I have currently to forecast the number of cars bought 8 months from now using the regression below. Would this ...
23 views

### ISLR - Interpretation of Confidence Interval in Linear Regression

I'm reading about Linear Regression in Introduction to Statistical Learning (Chapter 3) I see the confidence interval defined as A 95% confidence interval is defined as range of values such that ...
61 views

### removing collinarity of resulting dummy variable columns

I am currently reviewing some work which uses 'dummy variable encoding' as described here: ...
263 views

### feature embedding for categorical features

I'm training a model and among the features, I have the language of the users. Right now I have done one-hot encoding on the language feature. But I think it would make more sense to have the language ...
32 views

### MultiTaskLasso vs. Lasso with dummies

I am trying to do a Lasso regression, where one of the features is a categorical string e.g. suppose we have Price,Year,Make for a car. One option would be to use one-hot encoding for Make, and do ...
115 views

### Decision Tree vs Regression for Multiple Categorical Inputs

I have a problem with multiple categorical inputs. These categories do not intuitively map to integers, while preserving their adjacent relationship. Does it make more sense to us a Decision Tree ...
42 views

### Linear Regression and High Dimensional Categorical Data

I've read that mean encoding is useful for classification tasks with high dimensional categorical data. My question: What kinds of encodings are effective for high dimensional categorical data in ...
1 view

### Feature-construction for regression on samples consisting of varying numbers of sub-samples

I recently came across the following regression problem: The goal is to predict some value (e.g. the duration) of an event. One such event consists of multiple sub-events. The amount of sub-events ...
319 views

### Using pairwise differences as variable in regression

I have a dataset consisting of 72 data points, with 1 dependent variable $y$ and 37 predictors $X_j$. The typical way to perform a linear regression would be to model $y$ as $$y = XB + \epsilon$$ ...
18 views

### Can I regress a country-level variable with a firm-level variable?

Can I regress a country-level variable with a firm-level variable, for example, the country's GDP and the profitability of the firm? If so, then in this case all companies from the same country would ...
29 views

### Regressing on residuals of linear mixed model

So for a linear mixed model like this: $y=X\beta+Zu+e$ I can estimate the BLUP for $u$: $\hat{u}=GZ^TV^{-1}(y-X\hat{\beta})$ Where $G$ is the co-variance matrix of $u$ and $V=ZGZ^T+R$, where $R$ ...
50 views

### Machine learning models for regression on small data sets

What are the "best" models to be used for simple regression of 1 numerical variable using only a small data set of e.g. 250 samples and up to 10 features? I understand that the data set is super ...
51 views

### Clear explaination of dummy variable trap [duplicate]

I have a confusion in multiple regression about dummy variable trap, so far I had seen tutorials explaining about dummy variable trap and multicollinearity but I'm unable to understand it fully.
15 views

### Translate data into parameter coefficients - Bayesian regression

I Have a data set of accident rates from a population in which I'm attempting to identify which factors have the most effect on how many injuries occur from each accident. Since I am trying to ...
22 views

### Why Normality is considered as an important assumption for dependent and independent variables? [on hold]

While going through one kernel on Kaggle regarding Regression in that it was mentioned that the data should look like a normal distribution. But I am not getting why? I know this question might be ...
12 views

### Method for fusing multiple repeating parameters into a single none-repeating parameter space

I have a system with an input (x) and multiple outputs (A, B, C, etc.) that are all measured on the same scale (y). What I want to do is take a measurement of y to obtain my x value, the challenge is ...
48 views

### Only the intercept is significant in regression model (with dummy variable?)

This is a similar question to Intercept significant but not the variables in GLM, but in more detail: my model's dependent variable is change in population density of a state, and the independent ...
13 views

### Can I use standardized residuals for outlier detection when I have population data?

I have a data set containing the entirety of the population I'm interested in, not just a sample. However, there is an abundance of outliers in this data, which we have determined is due to a lot of ...
18 views

### Label encoding vs Dummy variable/one hot encoding - correctness?

i understand that when label encoding is used, the numeric number can be interpreted to have an order and a model could assume a linear relationship. However shouldn't this be a problem when there are ...
282 views

### Using estimated parameters as dependent variables

I'm doing a project on gold mining firms. I've taken the daily prices of the stocks of the 10 biggest gold mining companies in the last 15 years (I'm using the returns in order to make the data ...
43 views

### Why do non-linear models like randomForest() and h2o.gbm() have the $R^2$ as one of evaluation metrics

The models I used for predicting cost are randomForest(),h20.gbm(),glmnet() and ...
14 views

### Use z-scores as measure for regression analysis/ANOVA

I want to calculate the difference between a certain value associated with a decision alternative and the value associated with the objectively correct alternative as a measure of decision accuracy in ...
4k views

### Should we standardize the data while doing Gaussian process regression?

I am performing Gaussian process regression (GPR) and optimizing over hyper-parameters. I am using minFunc to perform all optimizations. My question is should we (...
2k views

### Difference between correlation and covariance: is covariance only useful if the relation is linear?

I'm trying to understand better the difference between covariance and correlation, besides the fact that the correlation coefficient is a dimensional and has values between $-1$ and $1$. One ...
6 views

### Model for Predicting Sales Drop in Drop Ship Model E-commerce [on hold]

I am trying to build a rule based model to predict the sales drop at product level on a particular week...I have data at week level for about 8 weeks and the independent variable that I am having are ...
12 views

### Finding the exponents of a multiple power law: is linear regression valid?

I want to fit a multiple power law equation of the form $y = {x_1}^{\alpha_1} {x_2}^{\alpha_2}$ where I have many examples of $y, x_1, x_2$. (Note there is no intercept.) Is it possible for me to ...
461 views

### Auto.arima is not fitting the data well

I have two variables speed and vibration and you can see that speed causes vibration. I am trying to fit this using auto.arima. But when i plot the fitted model, it gives bad result ...
16k views

### Why isn't Logistic Regression called Logistic Classification?

Since Logistic Regression is a statistical classification model dealing with categorical dependent variables, why isn't it called Logistic Classification? Shouldn't the "Regression" name be reserved ...
53 views

### Machine Learning Lottery [on hold]

I have lottery data and I want to use Scikit-Learn to create several different models starting with Regression(One vs All). I know the lottery is random, and impossible to predict, but I want to use ...
31 views

### Converting orthogonal coefficients to monomial form using specific data [on hold]

Using mtcars dataset and its first 2 variables, mpg and disp, can someone show me how to convert orthogonal coefficients made with poly 3 to monomial so I can evaluate using “basic” equation. I am ...
46 views

### Interpretation of logit model without reference category

I have developed a logit model with binary dependent variable in R using the glm function. Overall, the independent variables include seven nominal, 23 ordinal, and two numeric variables. The nominal ...
16 views

### Difference-in-differences analysis with multiple time periods

I am looking at the effects of a law on listed companies. I have a control group and a treatment group, but instead of having a pre treatment period and a treatment period I have 3 time periods. I ...
9 views

### 2x2 crossover design linear mixed model [on hold]

I have a 2x2 crossover design study data. I am looking to find the effect of a particular diet on the study participants. My factors are Diet- (Chicken and Pork), Intervention ( Pre and Post), Sex (...
16 views

### What is difference between interrupted time series and regression discontinuity design

Say that one has data over time, t, on an outcome, y. There is an event that happens at t==0....
15 views

### Pattern in residual plot

I am still learning about regression and I am currently trying to perform some basic analysis on my experimental data but I do not know how to deal with or interpret this obvious linear line in my ...
50 views

### Writing up findings from a summary given for a linear regression model

I am new to linear regression and came across the following paper when looking for examples to practice: https://www.maths.cam.ac.uk/sites/www.maths.cam.ac.uk/files/pre2014/postgrad/mathiii/pastpapers/...
3k views

### Cross validation and ordinal logistic regression

I am trying to understand cross-validation for ordinal logistic regression. The aim of the game is to validate the model used in an analysis... I first construct a toy data set: ...
11 views

### Differences between probabilistic regression + threshold and classification?

Working on probabilistic models, we often end up thresholding the result to decide if we should take some action or not. This method allow simple and explicit decisions, while being adaptative to our ...
7 views

### Ordinal regression on probabilities

Say I have a data set X, and a binary output y. I use a probabilistic model to determine the probability of y being one. Then for practical reasons I put my data in ordered buckets, with simple or ...
7 views

### How to interpret the ratio between a coefficient on a dummy and the coefficient of a log income variable?

I was reading this paper by Kahneman & Deaton: Kahneman, D., & Deaton, A. (2010). High income improves evaluation of life but not emotional well-being. Proceedings of the national academy of ...
16 views
+100

### Proportion log-regression with constant population and events

I have a population n, which is subject to event e. I classify my population into buckets $n_1,...,n_m$ with associated number of event $e_1,...e_m$. The goal of the classification is to distinguish ...
6 views

### feeding distribution over time into regression model - many cooks spoil the broth

This is a simplified/contrived example. Let us say I want to model the overall time of a meal including cooking time and prep time depending on features of the dish and the types of cooks. Let us say ...
6 views

### Moran's eigenvector spatial regression vs autocorrelation term

Comparing two methods for spatial regression: Moran's Eigenvector-based spatial regression Regression with a term for spatial autocorrelation e.g. where $Y = \beta X + \gamma MY + \epsilon$ for ...
22 views

### How to fit the following model in R? [duplicate]

'Rwml' is the monthly log return So the first column is clear, I got nearly the same values, at least the same magnitude. But: If I regress on the variance, my input values are way too low to get a ...
70k views

### Difference between forecast and prediction?

I was wondering what difference and relation are between forecast and prediction? Especially in time series and regression? For example, am I correct that: In time series, forecasting seems to mean ...