Questions tagged [multiple-regression]
Regression that includes two or more non-constant independent variables.
5,101
questions
2
votes
1
answer
35
views
Causal Diagram and multiple regression
I have 4 nodes:
A causes B and C, and C by itself causes D. However, C is not measurable, and my interest is to test the association between B and D. What would be the right causal diagram and ...
- 143
0
votes
0
answers
13
views
What is the correct method to compare linear mixed models and select the best one?
Suppose we have four variables, say, x, y, z and w, and we are fitting a linear mixed effects model with x as the target and y ...
1
vote
1
answer
30
views
Does multicollinearity among control variables matter?
I am conducting a regression analysis between X and Y, where X is the main independent variable. However, I want to control for several variables that are related to Y. For example if my dependent ...
- 11
2
votes
1
answer
15
views
Simple Linear Regression Question: How does correlation between X and Y affect MSE?
Suppose we know that the correlation between $X_1$ and $Y$ is $\rho_{X_1Y}$ and the correlation between $X_2$ and $Y$ is $\rho_{X_2Y}$. Furthermore, suppose $0 < \rho_{X_2Y} < \rho_{X_1Y}$. Now, ...
- 105
1
vote
1
answer
16
views
Calculation of leave-one-out cross-validation statistic in linear regression when design matrix is square
Suppose we are performing a least-squares multiple linear regression of the form
$$
\mathbf{Y}=\mathbf{X}\boldsymbol{\beta}+\boldsymbol{\varepsilon}\,,
$$
where $\mathbf{Y}=(y_1,y_2,...,y_N)$ are the ...
- 13
1
vote
3
answers
245
views
Understanding multiple linear regression residuals
I'm a bit confused whether residual vectors in OLS are orthogonal to every vector in X.
The problem I have is: Which of the following are true statements for our multiple linear regression with ...
- 11
4
votes
1
answer
58
views
bootstrapping a linear mixed model with R's lmeresampler or lme4 or a robust regression?
considering that I have a very small sample and that my residuals are non-normally distributed, I've decided to perform a lmer() with bootstrapping. This is my very ...
- 471
0
votes
0
answers
31
views
Is this relationship between variables linear?
I have four research questions and all of them has the same main "independent variable", as such the relationship between x and y1, y2, y3, y4. This is not a multivariate analysis, I am just ...
- 51
0
votes
0
answers
12
views
Detection of Multivariate Outliers (in a multiple linear regression problem) [duplicate]
In a multiple regression problem, suppose we have responses $Y_1, Y_2, \cdots , Y_n$ corresponding to data $\mathbf{X}_1, \mathbf{X}_2, \cdots, \mathbf{X}_n$ where each $\mathbf{X}_i$ is a $d$-...
- 561
0
votes
2
answers
53
views
Why $\text{Explained Variance} = Cov(y, \hat{y})$?
Context:
We all know:
$$
\text{Total Variance}=\text{Unexplained Variance} + \text{Explained Variance}
$$
where,
$$
\text{Explained Variance} = \text{SSReg} = \frac{1}{n-1} \sum_{i=1}^n(\hat y_i-\bar ...
- 153
0
votes
0
answers
17
views
Lagged regression with more than one predictor time-series
I have n + 1 different discrete time series. One of them is {Yt}, which I call ‘response time series’, and the other n are {Xt,i} (i = 1,…,n) which I call predictor series. I define n time lag ...
- 1
0
votes
0
answers
19
views
How to perform multilevel regression on Y?
We know multiple linear regression has the equation
$$
Y_1 = \beta_0 + \beta_1x_{11} + ... + \beta_k x_{1k} + \epsilon_1 \\
Y_2 = \beta_0 + \beta_1x_{21} + ... + \beta_k x_{2k} + \epsilon_2 \\
\dots \...
- 161
0
votes
0
answers
12
views
Build three models in one study
I build three models. First model has two mediators, one is not significant, then I removed it from the second model. The second one has one moderator and one mediator. The third one has another ...
- 1
0
votes
0
answers
10
views
Calculate uncertainty of surface normal based on points uncertainty
How to calculate angular variation of fitted plane from points that has positional uncertainty with normal probability distribution?
For example, I have 4 points
P1 = 102.0000 84.0000 139.5443
P2 = ...
0
votes
0
answers
7
views
Knockoff filters simple explanation and importance!
Knockoff filters are new in the field of variable selection. Can someone provide (or refer to) a slightly simple understanding of the topic? Also, what is the fuzz about this new method compared to ...
- 31
0
votes
0
answers
35
views
Interpretation of significant predictors in logistic regression
I am approaching an exercise about data analysis, but I have some doubts about interpreting my results.
I have $p$ predictors, some categorical, others continuous (let's call them $X_1$, $X_2$, $\dots$...
- 1
0
votes
0
answers
11
views
Reasoning about modelling uncertainty w.r.t input
I am trying to build up my reasoning about uncertainty modelling and ways of modelling it.
What I am trying to essentially get at is how changes in input variables results in different posteriors(...
1
vote
0
answers
32
views
How do the ridge or lasso coefficient changes when we add more variables
Suppose we run ridge or lasso regression over a bunch of features. And now suppose we add one more feature into the regressions. What will happen to the coefficients of the "old" features?
I ...
- 213
0
votes
0
answers
4
views
Aggregation in (cross-sectional) factor model
Suppose I have a large factor model for security returns, i.e. I have a vector $\mathbf{Y}(t) \in \mathbb{R}^{P}$, with factor loadings $\mathbf{\beta} \in \mathbb{R}^{P \times K}$ over a set of $K$ ...
- 1
1
vote
1
answer
39
views
Finding critical value for F statistic
I am performing a hypothesis test and therefore intersted in finding the F critical value for my F statistic. Below is the anova output when I compare two models.
...
- 13
0
votes
0
answers
15
views
Testing equality of mean responses
In the context of linear regression, we know that $E(y|x_0)$ for a given new data $x_0$ is estimated by $\hat{y_0}=x_0^t\hat{\beta}$. If given another set of new observations $x_1$, is there a way to ...
- 1
2
votes
0
answers
35
views
separate models vs joint model
My goal is to estimate the association between children BMI and distance to the nearest fast food restaurants. The hypothesis is that children BMI increases with increasing proximity of fast food ...
2
votes
1
answer
39
views
Intuition behind the formula for multiple linear regression coefficients from Econometric Analysis Greene
I was looking into the maths why coefficients change with the addition of new variables and so looked up the formula for multiple linear regression coefficients. This is what I found from section 3.2....
- 81
1
vote
0
answers
16
views
Is non-stationarity an issue in this setting?
I am trying to model the development of European spot prices of gas. My purpose is to explain what has caused past movements in the gas price, rather than forecasting future gas prices.
Naturally, ...
- 11
2
votes
1
answer
30
views
Same variable on both sides of a regression model
I am sketching a regression model for examining the effects of multiple variables on the difference between a perceived value $B_i$ and predicted value $\hat{B}_i$ at any given timepoint $i$. The $\...
- 23
1
vote
0
answers
11
views
Linear regression with interaction variables need all interacted variables as independent variables? [duplicate]
Say we have a dependent variable $Y$ and two independent variables $X_1$ and $X_2$.
If we are doing the linear regression with interacted variables, do we need to include both $X_1$ and $X_2$ as ...
- 11
0
votes
0
answers
18
views
Regression coefficient of y~x_1, y~x_2 and y~x_1+x_2
If we regress $y$ on $x_1$ and get $y = b_1 x_1$, regress $y$ on $x_2$ and get $y = b_2x_2$, regress $y$ on $x_1$ and $x_2$ and get $y = b_1'x_1 + b_2' x_2$, what's the relationship between $b_1b_2$ ...
- 1
0
votes
0
answers
35
views
How do we report a anova() of multiple model comparisons in R?
My question is pretty similar to this one, but I don't seem to get an F-statistic.
Hi, I know that we would usually report an ANOVA result as ...
- 471
1
vote
2
answers
249
views
Application of Maximum Likelihood estimation (MLE) to the step of Feasible Generalized Least Square (FGLS)
I have the following regression
$$y = X\beta +u$$
where $y$ and $u$ are $(n\times 1)$ and $X$ is a fixed $(n \times k)$ matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of ...
- 976
0
votes
0
answers
7
views
What regression/analysis should i do for the following case?
I am analyzing a survey where I check for the importance that certain variables have in the referral decision making of individuals in the workplace. In the survey I have one normal scenario and two ...
- 11
0
votes
0
answers
15
views
Analyzing frequency of events in R
I have data on fire incidences:
structure(list(Season = c("Winter", "Winter", "Winter", "Winter",
"Winter", "Winter", "Winter", &...
- 1
1
vote
0
answers
23
views
Subtracting predictions from linear regression model & stochastic error [closed]
Understanding that a linear regression model includes stochastic error, can I eliminate error from my model's predictions by subtracting one prediction from the other?
Let's say we have a model of the ...
3
votes
1
answer
234
views
Alternative interpretation of multiple regression coefficients?
If we have a linear regression of the form
$$
Y = \beta_0 + \beta_1X_1 + \beta_2X_2
$$
is it valid to interpret the coefficient $\beta_1$ as the associated change in $Y$ when $X_1$ increases by a unit ...
- 2,315
2
votes
1
answer
48
views
Is a general linear model a "system" of linear regression models?
I was trying to understand the difference between a multiple linear regression model, a general linear model, and a generalized linear model. I have seen very similar questions have already been ...
1
vote
0
answers
12
views
Where do I obtain R squared for the power analysis of multiple regression
I have a regression dataset and want to check if the amount of samples is sufficient. For this I conduct a power analysis.
For the power analysis of a multiple regression model, I need to obtain the ...
- 11
0
votes
0
answers
61
views
Interpret Coefficient of Determination in matrix form
In matrix form, a linear regression can be represented in the following form:
$$
Y \sim \mathbf{X} \beta + \epsilon; \\
\epsilon \sim N(0, \sigma^2 \mathbf{I})
$$
The definition of $R^2$ is the ...
- 143
2
votes
1
answer
63
views
Should I do pairwise NA deletion for the predictors in regression and delete NAs from outcome?
I have a very straight-forward question. THIS is my dataset
I have NAs in both the outcome (y = "scores") and in the continuos predictor (X1_c):
...
- 471
0
votes
0
answers
11
views
How to test if the impact of binary independent variables is significant on the numerical dependent variable?
Here's the dataset with the most important columns (use raw paste data):
https://pastebin.com/pxYnuWWg
I have a dataset of 20 marathon runners and their corresponding finishing times. After a year, ...
3
votes
2
answers
65
views
Which groups are reference groups in a regression model with interaction?
What are the reference groups in a regression model where there are interaction categories? Using the iris dataset in R, I've created a category with three levels ...
- 357
1
vote
1
answer
18
views
Interpretation of interaction coefficients and p-values with categorical predictors
Let's say that we have gene expression measurements from bacteria after heatshock. The expression changes are collected as log2 changes, and are approximately normal and centered around 0. Ribosomal ...
- 11
2
votes
2
answers
122
views
Lmer violating residuals' normality assumption: What should I do? When "enough data" is enough?
I'm trying to plot the following lmer:
mod1 <- lmer(SCORE ~ X1_c * X2 + (1|PARTICIPANT), data = data)
With THIS dataset. (this is a Git link)
However, I can't ...
- 471
2
votes
1
answer
56
views
simple slope analysis for curvilinear mixed model
I am a student currently working on a simple slope for mixed model and came upon a question.
As I am interested in a two-way interaction that involves a quadratic term of X variable and a moderator, I ...
- 21
0
votes
0
answers
16
views
How to interpret mixed level logistic regression with contrast coding?
I'm currently trying to interpret several mixed-level logistic regressions with contrast coding and it is my first time using this method.
My main research of interest is the intercept, which is ...
- 33
3
votes
1
answer
20
views
compare model fit logistic regression negative two times log likelihood
I'm trying to decide between two logistic regression models. I think I've used the negative two times log likelihood criterion before. My two models are not nested - can I still use that approach? ...
- 53
-1
votes
0
answers
26
views
How to calculate a financial beta and theta (time decay) in R linear regression?
I have the following code trying to calculate a beta (if priceA goes up 1, then priceY goes up X) and theta (each month that passes priceY goes down Z).
...
- 99
1
vote
1
answer
27
views
hierarchical regression models interpretation (with interaction term)
I am running multiple regression to test my hypothesis, which includes interaction terms. I have some control variables and three key predictors A, B and C. I used hierarchical regression models which ...
- 23
0
votes
0
answers
12
views
Regression coefficients with different signs (one positive, one negative) in a mediation analysis
I ran a mediation analysis following this article (which goes over the Baron & Kenny method).
Step 1. lm(Y ~ X, data); My X (independent) variable has a significant relationship with my Y (...
- 41
1
vote
1
answer
22
views
Interactions in multiple regression model for more than 2 predictors
Currently I am analyzing a multiple regression model to test the effect of 2 continuous predictors (P1 and P2) and 1 discrete predictor (P3, two levels) on a continuous response variable (RV). The ...
- 21
2
votes
1
answer
48
views
How to establish relationship between regressions on subsets of data?
From classical OLS, the regression of $y\in\mathbb{R}^n$ on $X\in\mathbb{R}^{n\times k}$ yields $\beta = (X^TX)^{-1} X^Ty$. Suppose we were to partition $X$ into two blocks as: $X = \begin{pmatrix} ...
- 557
2
votes
1
answer
53
views
Frisch-Waugh Theorem: Why the estimates are slightly different between multiple regression and partitioned regression?
I run two kinds of regressions to compare generic version of multiple regression and partitioned regression.
My model is
$$\textrm{ wage}=\beta_0+\beta_1 \textrm{educ}+\beta_2 \textrm{exper}+\beta_3 \...
- 137