Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

Recommend a research-level book on econometrics, particularly for the theory of linear regression for random regressors X

Can anyone suggest a research monograph (with proofs or at least detailed references) for econometrics? Specifically, I'm looking to learn about the theory of linear regression and all the usual ...
1 vote
0 answers
12 views

What happens to the Correlation coefficient for a multilevel random intercept regression?

Suppose we have a two level random intercept regression model: that is, for i = 1,...n, $$X_{i} \sim N(\mu_{xi}, \sigma_{xi}) $$ and $$Y_{i} \sim N(\mu_{yi}, \sigma_{yi}) $$ where $$\mu_{yi} = a_{i} +...
  • 21
3 votes
3 answers
187 views

Linear probability model with crossentropy (log) loss

For better or for worse, some people shoehorn binary $y$ variables into an ordinary least squares linear regression. $$ \mathbb E[Y\vert X]=\hat y=X\beta $$ If we encode the $y_i$ as either $0$ or $1$,...
  • 35.7k
3 votes
1 answer
63 views

How do Measure "Robustness" in Statistics?

I am an MBA Student taking courses in Statistics. Our prof was comparing two different methods of estimating the parameters for a regression model: General Method of Moments (GMM) and Maximum ...
3 votes
2 answers
133 views

Is there a Relationship Between Variance and Chi-Square?

I am an MBA Student that is taking courses in Statistics. Up until now, we had only encountered "Chi-Square" in the context of Contingency Tables. That is, how to find if the difference ...
0 votes
1 answer
27 views

What does an xtreg equation look like?

I ran some FE regressions in Stata using xtreg. One “normal” fixed effects model and a second generalized difference-in-differences model. Now, I am wondering how ...
  • 95
0 votes
0 answers
6 views

linear regression removing interce [duplicate]

I have 4 continuous x variables and it is a linear regression problem. I built the first model and recorded performance on the test data - Mean absolute % error. I also noticed that some x variables ...
1 vote
1 answer
19 views

Derivation of factors associated with increase in HIV DNA

I would just like to verify a statement made in the following paper: Peripheral blood HIV-1 DNA dynamics in antiretroviral-treated HIV/HCV co-infected patients receiving directly-acting antivirals The ...
  • 123
0 votes
0 answers
7 views

MLR OLS computing slope and intercept separately

In multiple linear regression, when we want to derive the slopes and intercept separately, I have seen the following formulas: $\hat \beta = (X^T_c X_c)^{-1} X^T_c y_c$ $\beta_0 = \overline y - \beta^...
  • 111
-1 votes
0 answers
9 views

Regression and Distribution of Inverses

We know that $\beta = (X^TX)^{-1}X^Ty,$ but when can we "distribute" the -1 in to get $(X^TX)^{-1}=X^{-1}X^{T^{-1}}$?
2 votes
1 answer
30 views

Same variable on both sides of a regression model

I am sketching a regression model for examining the effects of multiple variables on the difference between a perceived value $B_i$ and predicted value $\hat{B}_i$ at any given timepoint $i$. The $\...
  • 23
0 votes
0 answers
10 views

Absolute effect of one dummy in a linear regression with multiple dummies

I am regressing meat consumption (a continuous variable) on several socio-demographic characteristics (all categorical variables) converted into dummies : sex (1/2), age category (1/2/3), level of ...
  • 1
0 votes
0 answers
22 views

Linear regression output interpretation [duplicate]

I ran a linear model using 2 variables which I "log+1" transformed prior to analysis. I included a fixed effect of month of sample collection (factor with 3 levels - September, October and ...
1 vote
0 answers
11 views

Linear regression with interaction variables need all interacted variables as independent variables? [duplicate]

Say we have a dependent variable $Y$ and two independent variables $X_1$ and $X_2$. If we are doing the linear regression with interacted variables, do we need to include both $X_1$ and $X_2$ as ...
2 votes
1 answer
27 views

Centering vs standardizing in ridge regression

I have read that to apply ridge regression, we first need to standardize the predictive variables. That is because the variables should be in a homogenous scale so that lambda has an effect of the ...
  • 111
0 votes
1 answer
45 views

Can I combine dichotomous and continuous outcomes into a single regression model?

I am doing analysis on an educational product that aims to predict what impacts whether or not a student gets a question correct or incorrect. The DV includes item scores from four different question ...
  • 11
1 vote
0 answers
84 views

Can we isolate the effect of two different control variables in a semi-partial correlation?

I know that we are able to use a partial correlation when we want to correlate X and Y but Z affects both of them and that we may use a semi-partial correlation when we want to correlate X and Y and ...
1 vote
0 answers
63 views

Difference between fixed effects model and Random effects with varying intercept

According to my understanding fixed effects gives different intercepts for different clusters and so does random effects model with fixed slope and varying intercept. This is the sample data used. I ...
0 votes
0 answers
43 views

How to find the regression model log likelihood of data $(x, z, b) $ where $b$ indicates whether $y > z? $

I have a dataset where I don't have the exact output labels $y$ but what I have is if $y$ is larger or smaller than another value $z.$ There is another binary parameter $b $ that decides if y is ...
  • 1
0 votes
0 answers
10 views

How is an Autoregression Model actually fitted?

Suppose I have a time series dataset X and want to fit an autoregression model with lag 1. How does the fitting process work under the hood? Is the data set broken up to two datasets $X_0 = x_{0},x_{2}...
  • 101
0 votes
0 answers
13 views

Techniques that can avoid log transformation bias in a linear model

Suppose one prediction task is for long tail positive values, log transformation can be applied to get a distribution shape that's close to normal distribution. When we get the predicted value, we ...
1 vote
1 answer
24 views

What exactly is a "true" population model in linear regression?

What do we mean by a true population model when talking about linear regression? Say I want to study the effects of years of schooling $S$ on wages. I posit the following two models: $log(wage)=β_0+...
  • 55
0 votes
1 answer
17 views

What is the "True" population model in linear regression?

What do we mean by a true population model when talking about linear regression? Say I want to study the effects of years of schooling $S$ on wages. I posit the following two models: $ log(wage) = \...
  • 55
7 votes
3 answers
476 views

Interpret neural network like the linear regression equation such as how much will Y change if we change X1 and keep the other variables fixed

In linear regression, assume we get the following equation : Y = 0.8X1+1.9X2+2.4X3+4X4. We can interpret the linear equation: Keep the other predictors fix, one ...
2 votes
2 answers
66 views

Would there be any alternatives to a Logistic Regression or way to modify the Regression for what I'm looking for?

To provide more information I am looking for an alternative to logistic Regression or a way to modify it. This is because of two reasons: My data is widely dispersed across the X axis for both my 1s ...
  • 41
1 vote
1 answer
26 views

Is regression a good model for predicting disk usage growth? [closed]

I am struggling a bit with the logic. I need to estimate growth based on historical data. My data is basically composed of dates + a snapshot of disk usage. date disk_usage growth 2001.01.01 520 ...
0 votes
0 answers
18 views

Regression coefficient of y~x_1, y~x_2 and y~x_1+x_2

If we regress $y$ on $x_1$ and get $y = b_1 x_1$, regress $y$ on $x_2$ and get $y = b_2x_2$, regress $y$ on $x_1$ and $x_2$ and get $y = b_1'x_1 + b_2' x_2$, what's the relationship between $b_1b_2$ ...
1 vote
1 answer
23 views

Which analysis to choose for panel data with three timepoints? (Not interested in change but influence of other variable)

I have data from a 3-wave panel study which I don't know how to analyze. I am interested in how an independent variable (IV) affects a dependent variable (DV). Therefore, I am not interested in the ...
  • 13
0 votes
1 answer
24 views

Error in Fixed Effects Model

I want to do a fixed effects model on a dummy dataset which has 3 columns depicting severity of patients and their survival rate in different hospitals. The SURVIVALRATE is the dependent variable and ...
0 votes
0 answers
10 views

How should I interpret the results of the OLS regression I did for 2 cointegrated variables?

So I've been doing cointegration between two variables that are both I(1). I run the OLS regression between the variables to possibly check the stationary of the residuals. However when I checked the ...
  • 1
5 votes
4 answers
363 views

Dependent Variable takes on the values 0, 1, 2, 3 - What is the right (logistic) regression model to use?

I am looking for help to analyze the data from my online experiment. For my master thesis I conducted an online experiment where participants had to conduct a shopping task where they were provided ...
  • 61
2 votes
1 answer
46 views

cv.glmnet vs glmnet

I'm using glmnet to fit a ridge regression model on some data and evaluate the model's test MSE. The lambda value I select is derived from cross-validation. I'm ...
  • 43
11 votes
1 answer
975 views

Aren't ALL Parameters Eventually "Nuisance Parameters"?

I am an MBA student taking some courses in statistics. We attended a seminar on GLM Models for Count Data in which the presenter was introducing us to the concept of "Nuisance" Parameters. I ...
0 votes
0 answers
12 views

Are pairwise combinations of an additive quantity valid ‘samples’?

I have a subject that is measured for a continuous quantity V at multiple subsequent points in time (t1, t2, t3 etc). At each measurement time, the quantity V is essentially ‘reset’ back to 0 and V ...
1 vote
1 answer
31 views

The results from 2 programs are conflicting on convergence issues in my multivariate logistic regression, how do I deal with this?

Currently I am analyzing a dataset using logistic regression, I ran it in R using the glm function to run a multivariate logistic regression with 12 predictors. Some of these are quite collinear as ...
0 votes
0 answers
13 views

Deriving average partial effect with interaction terms

I am attempting to derive the average partial effect of the following equation for two random variables $x_1$ and $x_2$ on a third random variable $y$. $$ y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \...
0 votes
0 answers
17 views

How does a 'basic' panel regression model with fixed effects looks like?

I have modeled a basic panel data regression model under fixed effects in Stata. Specifications and results are clear. But, now I want to write down the model in lets say general formation. I spent ...
  • 95
1 vote
1 answer
22 views

Backend process of lmer function [closed]

I am trying to learn mixed models using lmer function and I am able to use it. But I am not able to understand what is going on in the backend like how it is taking care of fixed effects and random ...
1 vote
0 answers
18 views

Is maximal spectral entropy of residuals a poor loss function because phase information is lost?

Suppose I define a custom loss function SpectralEntropy as follows: ...
  • 4,962
4 votes
1 answer
61 views

Should we scale the data if our response(Y) is numeric, a large number, and 99% of other variables are dummy variables?

Our response variable is something like Sales which should be very large(Mean at the million level), and one predictor is also numeric but with mean at ten thousand ...
1 vote
1 answer
23 views

What is the hat matrix and why is it inappropriate for GLMM standardized residuals?

When I run this code to plot standardized residuals for a standard logistic regression: ...
0 votes
0 answers
18 views

Two different answers when calculating MSEP

I would appreciate help with understanding why I get two different answers when I calculate MSEP (mean square error prediction) in two different ways using R, both for simple linear regression and ...
  • 23
2 votes
2 answers
76 views

How does ridge regression reduce the variance of the estimates of $\beta$

In the scikit-learn library, Ridge class, there is a note that reads: "Regularization improves the conditioning of the problem and reduces the variance of the estimates." Given the ...
  • 111
1 vote
2 answers
249 views

Application of Maximum Likelihood estimation (MLE) to the step of Feasible Generalized Least Square (FGLS)

I have the following regression $$y = X\beta +u$$ where $y$ and $u$ are $(n\times 1)$ and $X$ is a fixed $(n \times k)$ matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of ...
  • 976
1 vote
0 answers
25 views

Is standarization necessary for ridge regression?

Is variable normalization necessary in Ridge regression (for both X and y)? If so, what happens (mathematically) if we don't do it?
  • 111
0 votes
0 answers
24 views

Showing the unbiased estimator of variance for GLS estimator

I have the following regression $$y = X\beta +u$$ where $y$ and $u$ are $(n\times 1)$ and $X$ is a fixed $(n \times k)$ matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of ...
  • 976
2 votes
0 answers
18 views

Intercept and slope of ridge regression model

When we compute a Ridge regression model, do we need to compute the intercept separately from the slopes? As you know, the estimated $\beta$ for the ridge regression model is given by: $\hat \beta = (...
  • 111
0 votes
0 answers
17 views

Overcome selection bias

I'm new to stata and I have a question regarding selection bias. For example, I am testing the impact of risk management on firm total risk. So if firms use risk management, they should have a lower ...
14 votes
2 answers
1k views

Why are "Linear" Models so Important?

I am an MBA student that is taking courses in statistics. Yesterday, I attended a statistics seminar in which some graduate students presented their research on some psychology experiments (e.g. ...
2 votes
0 answers
31 views

OLS solution to linear regression via SVD decomposition

I'm solving a linear regression problem. In a textbook that I follow, the author says that directly computing the OLS vector: $\beta = (X^TX)^{-1}X^T y$ can lead to problems when $(X^TX)$ is singular ...
  • 111