Questions tagged [multiple-regression]

Regression that includes two or more non-constant independent variables.

Filter by
Sorted by
Tagged with
2 votes
1 answer
35 views

Causal Diagram and multiple regression

I have 4 nodes: A causes B and C, and C by itself causes D. However, C is not measurable, and my interest is to test the association between B and D. What would be the right causal diagram and ...
0 votes
0 answers
13 views

What is the correct method to compare linear mixed models and select the best one?

Suppose we have four variables, say, x, y, z and w, and we are fitting a linear mixed effects model with x as the target and y ...
1 vote
1 answer
30 views

Does multicollinearity among control variables matter?

I am conducting a regression analysis between X and Y, where X is the main independent variable. However, I want to control for several variables that are related to Y. For example if my dependent ...
  • 11
2 votes
1 answer
15 views

Simple Linear Regression Question: How does correlation between X and Y affect MSE?

Suppose we know that the correlation between $X_1$ and $Y$ is $\rho_{X_1Y}$ and the correlation between $X_2$ and $Y$ is $\rho_{X_2Y}$. Furthermore, suppose $0 < \rho_{X_2Y} < \rho_{X_1Y}$. Now, ...
  • 105
1 vote
1 answer
16 views

Calculation of leave-one-out cross-validation statistic in linear regression when design matrix is square

Suppose we are performing a least-squares multiple linear regression of the form $$ \mathbf{Y}=\mathbf{X}\boldsymbol{\beta}+\boldsymbol{\varepsilon}\,, $$ where $\mathbf{Y}=(y_1,y_2,...,y_N)$ are the ...
1 vote
3 answers
245 views

Understanding multiple linear regression residuals

I'm a bit confused whether residual vectors in OLS are orthogonal to every vector in X. The problem I have is: Which of the following are true statements for our multiple linear regression with ...
4 votes
1 answer
58 views

bootstrapping a linear mixed model with R's lmeresampler or lme4 or a robust regression?

considering that I have a very small sample and that my residuals are non-normally distributed, I've decided to perform a lmer() with bootstrapping. This is my very ...
0 votes
0 answers
31 views

Is this relationship between variables linear?

I have four research questions and all of them has the same main "independent variable", as such the relationship between x and y1, y2, y3, y4. This is not a multivariate analysis, I am just ...
  • 51
0 votes
0 answers
12 views

Detection of Multivariate Outliers (in a multiple linear regression problem) [duplicate]

In a multiple regression problem, suppose we have responses $Y_1, Y_2, \cdots , Y_n$ corresponding to data $\mathbf{X}_1, \mathbf{X}_2, \cdots, \mathbf{X}_n$ where each $\mathbf{X}_i$ is a $d$-...
0 votes
2 answers
53 views

Why $\text{Explained Variance} = Cov(y, \hat{y})$?

Context: We all know: $$ \text{Total Variance}=\text{Unexplained Variance} + \text{Explained Variance} $$ where, $$ \text{Explained Variance} = \text{SSReg} = \frac{1}{n-1} \sum_{i=1}^n(\hat y_i-\bar ...
  • 153
0 votes
0 answers
17 views

Lagged regression with more than one predictor time-series

I have n + 1 different discrete time series. One of them is {Yt}, which I call ‘response time series’, and the other n are {Xt,i} (i = 1,…,n) which I call predictor series. I define n time lag ...
  • 1
0 votes
0 answers
19 views

How to perform multilevel regression on Y?

We know multiple linear regression has the equation $$ Y_1 = \beta_0 + \beta_1x_{11} + ... + \beta_k x_{1k} + \epsilon_1 \\ Y_2 = \beta_0 + \beta_1x_{21} + ... + \beta_k x_{2k} + \epsilon_2 \\ \dots \...
0 votes
0 answers
12 views

Build three models in one study

I build three models. First model has two mediators, one is not significant, then I removed it from the second model. The second one has one moderator and one mediator. The third one has another ...
  • 1
0 votes
0 answers
10 views

Calculate uncertainty of surface normal based on points uncertainty

How to calculate angular variation of fitted plane from points that has positional uncertainty with normal probability distribution? For example, I have 4 points P1 = 102.0000 84.0000 139.5443 P2 = ...
0 votes
0 answers
7 views

Knockoff filters simple explanation and importance!

Knockoff filters are new in the field of variable selection. Can someone provide (or refer to) a slightly simple understanding of the topic? Also, what is the fuzz about this new method compared to ...
0 votes
0 answers
35 views

Interpretation of significant predictors in logistic regression

I am approaching an exercise about data analysis, but I have some doubts about interpreting my results. I have $p$ predictors, some categorical, others continuous (let's call them $X_1$, $X_2$, $\dots$...
0 votes
0 answers
11 views

Reasoning about modelling uncertainty w.r.t input

I am trying to build up my reasoning about uncertainty modelling and ways of modelling it. What I am trying to essentially get at is how changes in input variables results in different posteriors(...
1 vote
0 answers
32 views

How do the ridge or lasso coefficient changes when we add more variables

Suppose we run ridge or lasso regression over a bunch of features. And now suppose we add one more feature into the regressions. What will happen to the coefficients of the "old" features? I ...
  • 213
0 votes
0 answers
4 views

Aggregation in (cross-sectional) factor model

Suppose I have a large factor model for security returns, i.e. I have a vector $\mathbf{Y}(t) \in \mathbb{R}^{P}$, with factor loadings $\mathbf{\beta} \in \mathbb{R}^{P \times K}$ over a set of $K$ ...
  • 1
1 vote
1 answer
39 views

Finding critical value for F statistic

I am performing a hypothesis test and therefore intersted in finding the F critical value for my F statistic. Below is the anova output when I compare two models. ...
  • 13
0 votes
0 answers
15 views

Testing equality of mean responses

In the context of linear regression, we know that $E(y|x_0)$ for a given new data $x_0$ is estimated by $\hat{y_0}=x_0^t\hat{\beta}$. If given another set of new observations $x_1$, is there a way to ...
2 votes
0 answers
35 views

separate models vs joint model

My goal is to estimate the association between children BMI and distance to the nearest fast food restaurants. The hypothesis is that children BMI increases with increasing proximity of fast food ...
2 votes
1 answer
39 views

Intuition behind the formula for multiple linear regression coefficients from Econometric Analysis Greene

I was looking into the maths why coefficients change with the addition of new variables and so looked up the formula for multiple linear regression coefficients. This is what I found from section 3.2....
  • 81
1 vote
0 answers
16 views

Is non-stationarity an issue in this setting?

I am trying to model the development of European spot prices of gas. My purpose is to explain what has caused past movements in the gas price, rather than forecasting future gas prices. Naturally, ...
  • 11
2 votes
1 answer
30 views

Same variable on both sides of a regression model

I am sketching a regression model for examining the effects of multiple variables on the difference between a perceived value $B_i$ and predicted value $\hat{B}_i$ at any given timepoint $i$. The $\...
  • 23
1 vote
0 answers
11 views

Linear regression with interaction variables need all interacted variables as independent variables? [duplicate]

Say we have a dependent variable $Y$ and two independent variables $X_1$ and $X_2$. If we are doing the linear regression with interacted variables, do we need to include both $X_1$ and $X_2$ as ...
0 votes
0 answers
18 views

Regression coefficient of y~x_1, y~x_2 and y~x_1+x_2

If we regress $y$ on $x_1$ and get $y = b_1 x_1$, regress $y$ on $x_2$ and get $y = b_2x_2$, regress $y$ on $x_1$ and $x_2$ and get $y = b_1'x_1 + b_2' x_2$, what's the relationship between $b_1b_2$ ...
0 votes
0 answers
35 views

How do we report a anova() of multiple model comparisons in R?

My question is pretty similar to this one, but I don't seem to get an F-statistic. Hi, I know that we would usually report an ANOVA result as ...
1 vote
2 answers
249 views

Application of Maximum Likelihood estimation (MLE) to the step of Feasible Generalized Least Square (FGLS)

I have the following regression $$y = X\beta +u$$ where $y$ and $u$ are $(n\times 1)$ and $X$ is a fixed $(n \times k)$ matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of ...
  • 976
0 votes
0 answers
7 views

What regression/analysis should i do for the following case?

I am analyzing a survey where I check for the importance that certain variables have in the referral decision making of individuals in the workplace. In the survey I have one normal scenario and two ...
  • 11
0 votes
0 answers
15 views

Analyzing frequency of events in R

I have data on fire incidences: structure(list(Season = c("Winter", "Winter", "Winter", "Winter", "Winter", "Winter", "Winter", &...
  • 1
1 vote
0 answers
23 views

Subtracting predictions from linear regression model & stochastic error [closed]

Understanding that a linear regression model includes stochastic error, can I eliminate error from my model's predictions by subtracting one prediction from the other? Let's say we have a model of the ...
3 votes
1 answer
234 views

Alternative interpretation of multiple regression coefficients?

If we have a linear regression of the form $$ Y = \beta_0 + \beta_1X_1 + \beta_2X_2 $$ is it valid to interpret the coefficient $\beta_1$ as the associated change in $Y$ when $X_1$ increases by a unit ...
  • 2,315
2 votes
1 answer
48 views

Is a general linear model a "system" of linear regression models?

I was trying to understand the difference between a multiple linear regression model, a general linear model, and a generalized linear model. I have seen very similar questions have already been ...
1 vote
0 answers
12 views

Where do I obtain R squared for the power analysis of multiple regression

I have a regression dataset and want to check if the amount of samples is sufficient. For this I conduct a power analysis. For the power analysis of a multiple regression model, I need to obtain the ...
0 votes
0 answers
61 views

Interpret Coefficient of Determination in matrix form

In matrix form, a linear regression can be represented in the following form: $$ Y \sim \mathbf{X} \beta + \epsilon; \\ \epsilon \sim N(0, \sigma^2 \mathbf{I}) $$ The definition of $R^2$ is the ...
2 votes
1 answer
63 views

Should I do pairwise NA deletion for the predictors in regression and delete NAs from outcome?

I have a very straight-forward question. THIS is my dataset I have NAs in both the outcome (y = "scores") and in the continuos predictor (X1_c): ...
0 votes
0 answers
11 views

How to test if the impact of binary independent variables is significant on the numerical dependent variable?

Here's the dataset with the most important columns (use raw paste data): https://pastebin.com/pxYnuWWg I have a dataset of 20 marathon runners and their corresponding finishing times. After a year, ...
3 votes
2 answers
65 views

Which groups are reference groups in a regression model with interaction?

What are the reference groups in a regression model where there are interaction categories? Using the iris dataset in R, I've created a category with three levels ...
1 vote
1 answer
18 views

Interpretation of interaction coefficients and p-values with categorical predictors

Let's say that we have gene expression measurements from bacteria after heatshock. The expression changes are collected as log2 changes, and are approximately normal and centered around 0. Ribosomal ...
  • 11
2 votes
2 answers
122 views

Lmer violating residuals' normality assumption: What should I do? When "enough data" is enough?

I'm trying to plot the following lmer: mod1 <- lmer(SCORE ~ X1_c * X2 + (1|PARTICIPANT), data = data) With THIS dataset. (this is a Git link) However, I can't ...
2 votes
1 answer
56 views

simple slope analysis for curvilinear mixed model

I am a student currently working on a simple slope for mixed model and came upon a question. As I am interested in a two-way interaction that involves a quadratic term of X variable and a moderator, I ...
0 votes
0 answers
16 views

How to interpret mixed level logistic regression with contrast coding?

I'm currently trying to interpret several mixed-level logistic regressions with contrast coding and it is my first time using this method. My main research of interest is the intercept, which is ...
3 votes
1 answer
20 views

compare model fit logistic regression negative two times log likelihood

I'm trying to decide between two logistic regression models. I think I've used the negative two times log likelihood criterion before. My two models are not nested - can I still use that approach? ...
  • 53
-1 votes
0 answers
26 views

How to calculate a financial beta and theta (time decay) in R linear regression?

I have the following code trying to calculate a beta (if priceA goes up 1, then priceY goes up X) and theta (each month that passes priceY goes down Z). ...
1 vote
1 answer
27 views

hierarchical regression models interpretation (with interaction term)

I am running multiple regression to test my hypothesis, which includes interaction terms. I have some control variables and three key predictors A, B and C. I used hierarchical regression models which ...
  • 23
0 votes
0 answers
12 views

Regression coefficients with different signs (one positive, one negative) in a mediation analysis

I ran a mediation analysis following this article (which goes over the Baron & Kenny method). Step 1. lm(Y ~ X, data); My X (independent) variable has a significant relationship with my Y (...
1 vote
1 answer
22 views

Interactions in multiple regression model for more than 2 predictors

Currently I am analyzing a multiple regression model to test the effect of 2 continuous predictors (P1 and P2) and 1 discrete predictor (P3, two levels) on a continuous response variable (RV). The ...
  • 21
2 votes
1 answer
48 views

How to establish relationship between regressions on subsets of data?

From classical OLS, the regression of $y\in\mathbb{R}^n$ on $X\in\mathbb{R}^{n\times k}$ yields $\beta = (X^TX)^{-1} X^Ty$. Suppose we were to partition $X$ into two blocks as: $X = \begin{pmatrix} ...
  • 557
2 votes
1 answer
53 views

Frisch-Waugh Theorem: Why the estimates are slightly different between multiple regression and partitioned regression?

I run two kinds of regressions to compare generic version of multiple regression and partitioned regression. My model is $$\textrm{ wage}=\beta_0+\beta_1 \textrm{educ}+\beta_2 \textrm{exper}+\beta_3 \...
  • 137

1
2 3 4 5
103