Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
3 votes
2 answers
30 views

What is the 'right' slope formula of a regression? deltas or Pearson?

this may be a silly question, but still: I've been told that the slope formula equals the rise/run ratio, like this: $$ m = \frac{rise}{run} = \frac{y_2 - y_1}{x_2 - x_1} $$ in which rise equals ...
0 votes
0 answers
2 views

Testing equality of mean responses

In the context of linear regression, we know that $E(y|x_0)$ for a given new data $x_0$ is estimated by $\hat{y_0}=x_0^t\hat{\beta}$. If given another set of new observations $x_1$, is there a way to ...
1 vote
1 answer
200 views

Counterexample where E(u|x)=0 in a regression model cannot hold in the population?

Edit: Background information: I have two variables of interest, $y$ and $x$ that are linearly related via the following: $y = a + bx + u$, where "$a$" and "$b$" are fixed ...
0 votes
0 answers
14 views

do you need to specify the direction of b for a moderated regression

I conducted a moderated regression, but didn't specify the direction of moderation by M. that is I didnt specify whether the relationship between X and Y would be greater at lower or higher levels of ...
1 vote
2 answers
202 views
+50

Application of Maximum Likelihood estimation (MLE) to the step of Feasible Generalized Least Square (FGLS)

I have the following regression $$y = X\beta +u$$ where $y$ and $u$ are $(n\times 1)$ and $X$ is a fixed $(n \times k)$ matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of ...
1 vote
0 answers
41 views
+50

Can we isolate the effect of two different control variables in a semi-partial correlation?

I know that we are able to use a partial correlation when we want to correlate X and Y but Z affects both of them and that we may use a semi-partial correlation when we want to correlate X and Y and ...
0 votes
0 answers
7 views

Approaches for running separate monthly regressions on a time series

I have a time series with daily granularity. The time series under consideration depends on an independent variable x (say). In order to account for seasonality effects - I run a separate regression ...
0 votes
1 answer
19 views

Logistic GLM vs GLMM diagnostic issues

Problem I am running through the diagnostics of two logistic regressions and two equivalent GLMMs with their only differences being crossed random effects (intercepts only). The output for the ...
1 vote
0 answers
12 views

how well would a robust mixed model fit these data? R (rlmer)

I want to investigate Y ~ X1 * X2 + (1|ID on this dataset (there's a plot of these data in that post too, it's the same dataframe) Y is a continuos outcome ...
0 votes
0 answers
8 views

Change versus level regression

I tried running a panel regression of y on contemporaneous x variables, and again using the change in y from year t-1 to year t. I am getting the opposite sign on my variable of interest (VOI) if I ...
0 votes
0 answers
33 views

The GMM estimator is reliable under the given conditions

Consider the linear model $$y_t= x_t'\beta +\epsilon_t$$ for $t=1,...,T$. where $x_t= ( x_{1t} \ \ x_{2t} \ \ ... \ \ x_{kt}) $ and $\beta$ is $(k\times 1)$ vector of unknown coefficients. Given $z_t= ...
0 votes
0 answers
8 views

How best to quantify how much one part (i.e., a group of items in a scale) contributes to the whole pool?

I am trying to determine how much specific types of stressors contribute to the overall pool of stress experienced, e.g., X type of stress contributes X% to the overall pool of stress, or be able to ...
0 votes
0 answers
18 views

How to prove that L2 loss converges faster than L1 loss

I see many sources claim this result, however, I was not able to find a proof for it. I think this should be given in some paper or book. Can someone point me to some resources, or even better, show ...
2 votes
1 answer
278 views

What is the purpose of using cbind in r for multivariate regression? [closed]

I am trying to see if A1_IS (internalised stigma) predicts A1_CSI (couple satisfaction at time1) and A2_CSI(couple satisfaction at time 2). So i think this is multivariate regression right? My R code: ...
7 votes
3 answers
439 views

Interpret neural network like the linear regression equation such as how much will Y change if we change X1 and keep the other variables fixed

In linear regression, assume we get the following equation : Y = 0.8X1+1.9X2+2.4X3+4X4. We can interpret the linear equation: Keep the other predictors fix, one ...
1 vote
0 answers
16 views

Connection Between Bayesian Prior and Variable Selection in Lasso [duplicate]

I am interested in learning more about the Bayesian interpretation of the Lasso model. The Lasso model assumes a Laplace distribution of coefficients and the optimal coefficients maximize the ...
0 votes
0 answers
11 views

Multivariate regression with dependent variables on an interval

I want to do a regression what the probability is of event being True (=P(E)). This dependends on three variables, let's say A, B and C. SO: P(E) = X1 * A + X2 * B + X3 * C My data consists of the ...
0 votes
1 answer
21 views

Probit/Logit or Linear regression Model?

I have data about the occurence of a data breach at certain companies for the periode 2005-2018. Now I have a question about the model I should use. I have two options: Probit/Logit: I set the ...
0 votes
0 answers
14 views

How to improve the predictions of a model when we have too few predictor variables?

I tried to use a linear model to explain a variable "age" with two variables "x1" and "x2". I can clearly see a decreasing slope inside my scatterplot for age vs x1, or ...
3 votes
1 answer
207 views

Generalized least squares error estimation

First of all, I have to admit that I am not statistician so some of my nomenclature could not be very rigorous and maybe a bit confusing; pleas ask me to clarify if necessary. The Problem Let's say ...
0 votes
0 answers
8 views

Regression with paired repeated measures design [closed]

I have the following data: ...
2 votes
1 answer
24 views

GLM: how to treat multiple variables that all measure a confounding aspect in a slightly different way?

For a response variable $y$ and predictor $x_0$, I have data for a number of additional variables $x_n$, $n = 1, ..., 7$. I would like to control for a confounder in my GLM, let's call it "size&...
1 vote
0 answers
14 views

Violating assumptions of linear mixed model

I am currently analyzing a dataset using a linear mixed model. In the study - using a within-subjects design - participants had to rate the intensity of a stimulus (this is my dependent variable, DV) ...
1 vote
1 answer
21 views

What exactly is a "true" population model in linear regression?

What do we mean by a true population model when talking about linear regression? Say I want to study the effects of years of schooling $S$ on wages. I posit the following two models: $log(wage)=β_0+...
0 votes
0 answers
34 views

Regression on multiple variables as a whole and separately

I have been asked to predict the effect of Covid-19 on two variables as a whole and individually. The two variables are prices for commodities of two types. The data includes a daily record of prices ...
0 votes
1 answer
200 views

Conditional logistic regression for calculation odds ratios

I want to calculate the crude and adjusted odds ratios for exposure to occupational risk factors such as aluminum and fossil fuels in my case control study. My cases are 180 demented patients and I ...
1 vote
1 answer
27 views

determining the effect of the change in an independent variable using regression

I'm creating a regression model that predicts a customer's spending based on their income, while adjusting for age, gender , and region. The model looks as follows: ...
3 votes
1 answer
58 views

How do Measure "Robustness" in Statistics?

I am an MBA Student taking courses in Statistics. Our prof was comparing two different methods of estimating the parameters for a regression model: General Method of Moments (GMM) and Maximum ...
0 votes
1 answer
196 views

Can we fit a regression model when the dependent variable is poorly correlated with the independent variables?

I have a requirement, I need to predict Y from 2 X_variables, I plotted two scattered plots ie Y vs X1, Y vs X2, As you can see the below pics, the plots are sparced. There is poor correlation between ...
2 votes
0 answers
30 views

separate models vs joint model

My goal is to estimate the association between children BMI and distance to the nearest fast food restaurants. The hypothesis is that children BMI increases with increasing proximity of fast food ...
2 votes
1 answer
227 views

Whether it is okay to dichotomise a 4-point Likert-scale outcome item

I'm using an already collected data-set within my research, meaning I was very limited with the outcomes that I could use. One of my two main outcome variables was an item using a 4-point Likert-scale ...
1 vote
0 answers
10 views

What happens to the Correlation coefficient for a multilevel random intercept regression?

Suppose we have a two level random intercept regression model: that is, for i = 1,...n, $$X_{i} \sim N(\mu_{xi}, \sigma_{xi}) $$ and $$Y_{i} \sim N(\mu_{yi}, \sigma_{yi}) $$ where $$\mu_{yi} = a_{i} +...
2 votes
1 answer
1k views

Averaging data then fitting vs fitting then averaging in non-linear regression?

I have a very similar problem like in this question. The difference is that I am dealing with non-linear regression. Moreover, the answer to that question suggests that there should be no difference ...
3 votes
3 answers
175 views

Linear probability model with crossentropy (log) loss

For better or for worse, some people shoehorn binary $y$ variables into an ordinary least squares linear regression. $$ \mathbb E[Y\vert X]=\hat y=X\beta $$ If we encode the $y_i$ as either $0$ or $1$,...
1 vote
0 answers
14 views

How to learn steep functions using neural network?

I am trying to use a neural network to learn the below function. In total, I have 25 features and 19 outputs. The above image shows the distribution of two features with respect to one of the outputs....
3 votes
1 answer
195 views

References Request (Least-Squares Estimates for non i.i.d. Processes)

I am interested in suggestions concerning possible applications/problems within applied statistics with respect to estimates of least-squares for non-stationary designs. In particular, I would like to ...
1 vote
1 answer
19 views

Standard error logic

Obviously the closer the standard error to zero the better. However, what if the values the in which the standard errors are from are extremely large and the standard errors are much higher than zero ...
1 vote
1 answer
29 views

Robust standard errors with splines

I realize that large changes in model results between using robust and non-robust standard errors can suggest a misspecified model. My case refers to using a Cox regression and I have experimented ...
0 votes
1 answer
194 views

Time series to forecast a probability

I'm looking to forecast a time series, that is, forecast the next period give historical time series data. At the end of the day, what I want is to forecast a probability. The metric that I'm ...
0 votes
0 answers
10 views

Recommend a research-level book on econometrics, particularly for the theory of linear regression for random regressors X

Can anyone suggest a research monograph (with proofs or at least detailed references) for econometrics? Specifically, I'm looking to learn about the theory of linear regression and all the usual ...
3 votes
2 answers
330 views

Why in the Ridge regression, the coefficients cannot be 0?

In the second answer (https://stats.stackexchange.com/a/368426/287815) to the question (Why will ridge regression not shrink some coefficients to zero like lasso?), the OP found out that, $β = 𝑥𝑦/(𝑥...
1 vote
1 answer
15 views

Derivation of factors associated with increase in HIV DNA

I would just like to verify a statement made in the following paper: Peripheral blood HIV-1 DNA dynamics in antiretroviral-treated HIV/HCV co-infected patients receiving directly-acting antivirals The ...
0 votes
0 answers
35 views

Are "Moments" More Robust Then "MLE"?

I am an MBA Student taking courses in Statistics. We are learning about different ways to estimate the parameters (i.e. coefficients) of a Regression Model. Our professor indicated that there are two ...
1 vote
1 answer
391 views

Assessing overfitting via learning curves

I have intended to find overfitting or underfitting cases. I have used MLP classifier and Logistic regression of scikit-learn. How do I know which is a good fit? Or Which one underfitting or ...
1 vote
0 answers
30 views

Is there a Relationship Between Variance and Chi-Square?

I am an MBA Student that is taking courses in Statistics. Up until now, we had only encountered "Chi-Square" in the context of Contingency Tables. That is, how to find if the difference ...
169 votes
10 answers
193k views

When is it ok to remove the intercept in a linear regression model?

I am running linear regression models and wondering what the conditions are for removing the intercept term. In comparing results from two different regressions where one has the intercept and the ...
0 votes
0 answers
16 views

How does an xtreg equation looks like?

I ran some FE regressions in Stata using xtreg. One “normal” FE Model and a second generalized DiD Model. Now, I am wondering how the code would look translated into mathematical equation. Please ...
0 votes
0 answers
6 views

linear regression removing interce [duplicate]

I have 4 continuous x variables and it is a linear regression problem. I built the first model and recorded performance on the test data - Mean absolute % error. I also noticed that some x variables ...
0 votes
0 answers
6 views

MLR OLS computing slope and intercept separately

In multiple linear regression, when we want to derive the slopes and intercept separately, I have seen the following formulas: $\hat \beta = (X^T_c X_c)^{-1} X^T_c y_c$ $\beta_0 = \overline y - \beta^...
-1 votes
0 answers
9 views

Regression and Distribution of Inverses

We know that $\beta = (X^TX)^{-1}X^Ty,$ but when can we "distribute" the -1 in to get $(X^TX)^{-1}=X^{-1}X^{T^{-1}}$?

1
2 3 4 5
545