# Questions tagged [multiple-regression]

Regression that includes two or more non-constant independent variables.

5,101 questions
Filter by
Sorted by
Tagged with
35 views

### Causal Diagram and multiple regression

I have 4 nodes: A causes B and C, and C by itself causes D. However, C is not measurable, and my interest is to test the association between B and D. What would be the right causal diagram and ...
• 143
13 views

### What is the correct method to compare linear mixed models and select the best one?

Suppose we have four variables, say, x, y, z and w, and we are fitting a linear mixed effects model with x as the target and y ...
1 vote
30 views

### Does multicollinearity among control variables matter?

I am conducting a regression analysis between X and Y, where X is the main independent variable. However, I want to control for several variables that are related to Y. For example if my dependent ...
• 11
15 views

### Simple Linear Regression Question: How does correlation between X and Y affect MSE?

Suppose we know that the correlation between $X_1$ and $Y$ is $\rho_{X_1Y}$ and the correlation between $X_2$ and $Y$ is $\rho_{X_2Y}$. Furthermore, suppose $0 < \rho_{X_2Y} < \rho_{X_1Y}$. Now, ...
• 105
1 vote
16 views

### Calculation of leave-one-out cross-validation statistic in linear regression when design matrix is square

Suppose we are performing a least-squares multiple linear regression of the form $$\mathbf{Y}=\mathbf{X}\boldsymbol{\beta}+\boldsymbol{\varepsilon}\,,$$ where $\mathbf{Y}=(y_1,y_2,...,y_N)$ are the ...
1 vote
245 views

### Understanding multiple linear regression residuals

I'm a bit confused whether residual vectors in OLS are orthogonal to every vector in X. The problem I have is: Which of the following are true statements for our multiple linear regression with ...
58 views

### bootstrapping a linear mixed model with R's lmeresampler or lme4 or a robust regression?

considering that I have a very small sample and that my residuals are non-normally distributed, I've decided to perform a lmer() with bootstrapping. This is my very ...
• 471
31 views

### Is this relationship between variables linear?

I have four research questions and all of them has the same main "independent variable", as such the relationship between x and y1, y2, y3, y4. This is not a multivariate analysis, I am just ...
• 51
12 views

### Detection of Multivariate Outliers (in a multiple linear regression problem) [duplicate]

In a multiple regression problem, suppose we have responses $Y_1, Y_2, \cdots , Y_n$ corresponding to data $\mathbf{X}_1, \mathbf{X}_2, \cdots, \mathbf{X}_n$ where each $\mathbf{X}_i$ is a $d$-...
• 561
53 views

• 161
12 views

### Build three models in one study

I build three models. First model has two mediators, one is not significant, then I removed it from the second model. The second one has one moderator and one mediator. The third one has another ...
• 1
10 views

### Calculate uncertainty of surface normal based on points uncertainty

How to calculate angular variation of fitted plane from points that has positional uncertainty with normal probability distribution? For example, I have 4 points P1 = 102.0000 84.0000 139.5443 P2 = ...
7 views

### Knockoff filters simple explanation and importance!

Knockoff filters are new in the field of variable selection. Can someone provide (or refer to) a slightly simple understanding of the topic? Also, what is the fuzz about this new method compared to ...
• 31
35 views

### Interpretation of significant predictors in logistic regression

I am approaching an exercise about data analysis, but I have some doubts about interpreting my results. I have $p$ predictors, some categorical, others continuous (let's call them $X_1$, $X_2$, $\dots$...
11 views

### Reasoning about modelling uncertainty w.r.t input

I am trying to build up my reasoning about uncertainty modelling and ways of modelling it. What I am trying to essentially get at is how changes in input variables results in different posteriors(...
1 vote
32 views

### How do the ridge or lasso coefficient changes when we add more variables

Suppose we run ridge or lasso regression over a bunch of features. And now suppose we add one more feature into the regressions. What will happen to the coefficients of the "old" features? I ...
• 213
4 views

### Aggregation in (cross-sectional) factor model

Suppose I have a large factor model for security returns, i.e. I have a vector $\mathbf{Y}(t) \in \mathbb{R}^{P}$, with factor loadings $\mathbf{\beta} \in \mathbb{R}^{P \times K}$ over a set of $K$ ...
• 1
1 vote
39 views

### Finding critical value for F statistic

I am performing a hypothesis test and therefore intersted in finding the F critical value for my F statistic. Below is the anova output when I compare two models. ...
• 13
15 views

### Testing equality of mean responses

In the context of linear regression, we know that $E(y|x_0)$ for a given new data $x_0$ is estimated by $\hat{y_0}=x_0^t\hat{\beta}$. If given another set of new observations $x_1$, is there a way to ...
35 views

### separate models vs joint model

My goal is to estimate the association between children BMI and distance to the nearest fast food restaurants. The hypothesis is that children BMI increases with increasing proximity of fast food ...
39 views

### Intuition behind the formula for multiple linear regression coefficients from Econometric Analysis Greene

I was looking into the maths why coefficients change with the addition of new variables and so looked up the formula for multiple linear regression coefficients. This is what I found from section 3.2....
• 81
1 vote
16 views

### Is non-stationarity an issue in this setting?

I am trying to model the development of European spot prices of gas. My purpose is to explain what has caused past movements in the gas price, rather than forecasting future gas prices. Naturally, ...
• 11
30 views

I am sketching a regression model for examining the effects of multiple variables on the difference between a perceived value $B_i$ and predicted value $\hat{B}_i$ at any given timepoint $i$. The $\... • 23 1 vote 0 answers 11 views ### Linear regression with interaction variables need all interacted variables as independent variables? [duplicate] Say we have a dependent variable$Y$and two independent variables$X_1$and$X_2$. If we are doing the linear regression with interacted variables, do we need to include both$X_1$and$X_2$as ... • 11 0 votes 0 answers 18 views ### Regression coefficient of y~x_1, y~x_2 and y~x_1+x_2 If we regress$y$on$x_1$and get$y = b_1 x_1$, regress$y$on$x_2$and get$y = b_2x_2$, regress$y$on$x_1$and$x_2$and get$y = b_1'x_1 + b_2' x_2$, what's the relationship between$b_1b_2$... 0 votes 0 answers 35 views ### How do we report a anova() of multiple model comparisons in R? My question is pretty similar to this one, but I don't seem to get an F-statistic. Hi, I know that we would usually report an ANOVA result as ... • 471 1 vote 2 answers 249 views ### Application of Maximum Likelihood estimation (MLE) to the step of Feasible Generalized Least Square (FGLS) I have the following regression $$y = X\beta +u$$ where$y$and$u$are$(n\times 1)$and$X$is a fixed$(n \times k)$matrix with full column rank and$\beta$is an unknown$(k\times 1)$vector of ... • 976 0 votes 0 answers 7 views ### What regression/analysis should i do for the following case? I am analyzing a survey where I check for the importance that certain variables have in the referral decision making of individuals in the workplace. In the survey I have one normal scenario and two ... • 11 0 votes 0 answers 15 views ### Analyzing frequency of events in R I have data on fire incidences: structure(list(Season = c("Winter", "Winter", "Winter", "Winter", "Winter", "Winter", "Winter", &... • 1 1 vote 0 answers 23 views ### Subtracting predictions from linear regression model & stochastic error [closed] Understanding that a linear regression model includes stochastic error, can I eliminate error from my model's predictions by subtracting one prediction from the other? Let's say we have a model of the ... 3 votes 1 answer 234 views ### Alternative interpretation of multiple regression coefficients? If we have a linear regression of the form $$Y = \beta_0 + \beta_1X_1 + \beta_2X_2$$ is it valid to interpret the coefficient$\beta_1$as the associated change in$Y$when$X_1$increases by a unit ... • 2,315 2 votes 1 answer 48 views ### Is a general linear model a "system" of linear regression models? I was trying to understand the difference between a multiple linear regression model, a general linear model, and a generalized linear model. I have seen very similar questions have already been ... 1 vote 0 answers 12 views ### Where do I obtain R squared for the power analysis of multiple regression I have a regression dataset and want to check if the amount of samples is sufficient. For this I conduct a power analysis. For the power analysis of a multiple regression model, I need to obtain the ... 0 votes 0 answers 61 views ### Interpret Coefficient of Determination in matrix form In matrix form, a linear regression can be represented in the following form: $$Y \sim \mathbf{X} \beta + \epsilon; \\ \epsilon \sim N(0, \sigma^2 \mathbf{I})$$ The definition of$R^2$is the ... • 143 2 votes 1 answer 63 views ### Should I do pairwise NA deletion for the predictors in regression and delete NAs from outcome? I have a very straight-forward question. THIS is my dataset I have NAs in both the outcome (y = "scores") and in the continuos predictor (X1_c): ... • 471 0 votes 0 answers 11 views ### How to test if the impact of binary independent variables is significant on the numerical dependent variable? Here's the dataset with the most important columns (use raw paste data): https://pastebin.com/pxYnuWWg I have a dataset of 20 marathon runners and their corresponding finishing times. After a year, ... 3 votes 2 answers 65 views ### Which groups are reference groups in a regression model with interaction? What are the reference groups in a regression model where there are interaction categories? Using the iris dataset in R, I've created a category with three levels ... 1 vote 1 answer 18 views ### Interpretation of interaction coefficients and p-values with categorical predictors Let's say that we have gene expression measurements from bacteria after heatshock. The expression changes are collected as log2 changes, and are approximately normal and centered around 0. Ribosomal ... • 11 2 votes 2 answers 122 views ### Lmer violating residuals' normality assumption: What should I do? When "enough data" is enough? I'm trying to plot the following lmer: mod1 <- lmer(SCORE ~ X1_c * X2 + (1|PARTICIPANT), data = data) With THIS dataset. (this is a Git link) However, I can't ... • 471 2 votes 1 answer 56 views ### simple slope analysis for curvilinear mixed model I am a student currently working on a simple slope for mixed model and came upon a question. As I am interested in a two-way interaction that involves a quadratic term of X variable and a moderator, I ... 0 votes 0 answers 16 views ### How to interpret mixed level logistic regression with contrast coding? I'm currently trying to interpret several mixed-level logistic regressions with contrast coding and it is my first time using this method. My main research of interest is the intercept, which is ... 3 votes 1 answer 20 views ### compare model fit logistic regression negative two times log likelihood I'm trying to decide between two logistic regression models. I think I've used the negative two times log likelihood criterion before. My two models are not nested - can I still use that approach? ... • 53 -1 votes 0 answers 26 views ### How to calculate a financial beta and theta (time decay) in R linear regression? I have the following code trying to calculate a beta (if priceA goes up 1, then priceY goes up X) and theta (each month that passes priceY goes down Z). ... 1 vote 1 answer 27 views ### hierarchical regression models interpretation (with interaction term) I am running multiple regression to test my hypothesis, which includes interaction terms. I have some control variables and three key predictors A, B and C. I used hierarchical regression models which ... • 23 0 votes 0 answers 12 views ### Regression coefficients with different signs (one positive, one negative) in a mediation analysis I ran a mediation analysis following this article (which goes over the Baron & Kenny method). Step 1. lm(Y ~ X, data); My X (independent) variable has a significant relationship with my Y (... 1 vote 1 answer 22 views ### Interactions in multiple regression model for more than 2 predictors Currently I am analyzing a multiple regression model to test the effect of 2 continuous predictors (P1 and P2) and 1 discrete predictor (P3, two levels) on a continuous response variable (RV). The ... • 21 2 votes 1 answer 48 views ### How to establish relationship between regressions on subsets of data? From classical OLS, the regression of$y\in\mathbb{R}^n$on$X\in\mathbb{R}^{n\times k}$yields$\beta = (X^TX)^{-1} X^Ty$. Suppose we were to partition$X$into two blocks as:$X = \begin{pmatrix} ...
• 557
I run two kinds of regressions to compare generic version of multiple regression and partitioned regression. My model is \textrm{ wage}=\beta_0+\beta_1 \textrm{educ}+\beta_2 \textrm{exper}+\beta_3 \...