Questions tagged [linear-model]
Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.
2,586
questions
0
votes
0
answers
5
views
Correction for multiple comparison, after pairwise comparison on 4 models with related dependant variables, when to do it?
I am statistically evaluating my experiment by using linear mixed effects models. The experiment is comprised of several different intervention protocols done within subject at a random sequence. ...
- 133
1
vote
1
answer
27
views
How should I handle exposure to different topics between participants in a panel dataset using Stata?
I'm looking to analyze data using Stata in which participants were randomly assigned to read 2 out of 4 possible topics. After reading a topic, participants were exposed to five levels of treatment (...
- 11
1
vote
0
answers
15
views
Checking linearity assumption in regression with quadratic terms
I have a few questions about checking the assumptions for linear regression: What is the best way to check linearity? Many recommendations I saw said to check scatterplots, but since linearity refers ...
- 63
2
votes
2
answers
37
views
Does ceiling effect of outcome variable violate linearity assumption of linear regression
If there is a ceiling effect in the outcome variable, e.g. in my case the outcome variable is limited to a certain value and 25% of data points have that highest possible value, does this mean that ...
- 63
0
votes
0
answers
13
views
How do I compare multiple groups of count data (that are either millions or zeros, with big variances)?
I have three independent groups: treatment, drug 1, and drug 2. For each group, I have bacterial counts in CFU. I'm not very good at statistics, so please bear with me. I know I can't use ANOVA ...
- 1
1
vote
1
answer
23
views
Mixed Design Anova with or without aggregation?
I have carried out a reaction time experiment and would now like to evaluate it. It is a repeated measures experiment with two conditions (let's call them condition A and condition B). There are also ...
- 37
0
votes
0
answers
11
views
Restoring partial heredity with Category:Numeric interactions keeps parameter count constant when adding terms
I want to understand the following minimal example.
...
- 297
0
votes
0
answers
30
views
Waldtest:Error in modelCompare(objects[[i - 1]], objects[[i]], vfun = vcov0) : nesting of models cannot be determined
I hope that someone can help me with the following problem with Waldtest and robustlmm package. I want to compare two robust regression models. I want to test ...
- 1
0
votes
0
answers
32
views
Interpreting β in regression as unit change
In a linear regression where variables have been standardized, a change of one SD in X is associated with a change of β*1 SD in Y, and I have seen people interpreting this in units since we know what ...
- 63
1
vote
0
answers
16
views
Prediction intervals of functions of inputs
I have a linear regression model I have estimated which is of the form Y = a + b*X + e.
I know how to construct the prediction interval for the outcomes of Y given some value X (say X1), but I was ...
0
votes
0
answers
18
views
Is regression on aggregated continuous independent variable adequate?
I´m trying to analyze some cohorts that reported aggregate data (mean, SD, and n) of a physiological parameter for the outcome of interest, which is a nominal variable but fairly lineal with the ...
- 83
2
votes
1
answer
40
views
Power Analysis for Interaction in Linear Mixed Model
I am trying to run a power analysis for an interaction in a linear mixed model to figure out the necessary sample size.
The model has the following structure:
Y ~ C * X + (1|Subject)
Y and X are ...
- 111
1
vote
1
answer
66
views
Given a predictor $x$. under what circumstance would you have high $R^2$ but low $\beta$
Assume I have a time series $y$ and a predictor $x$. Let's say they are both centered at zero.
$$R^2 = 1 - \frac{ \sum (y_i - x_i)^2 }{\sum y_i^2}$$
Now I run a new regression $y \sim \beta x$, in an ...
- 230
2
votes
1
answer
61
views
Linear model with interaction - pairwise comparison
I have the following model in R:
lme(log_weight ~ log_weight0 + Group*Day, random = ~ 1 | ID, data = mydata)
The interaction term is significant. I would like to ...
- 97
0
votes
0
answers
8
views
Do I need to set contrasts in my model matrix when using the car package for type 3 ANOVA tables?
I have been running several general and generalized linear models (not linear regressions) using glmmTMB and lme4. After I ...
- 43
0
votes
0
answers
12
views
Newton-Raphson on Cox PH
I am currently working on my research, namely comparing the conventional optimization method, namely Newton-Raphson, in estimating Coxph parameters with the SGD optimization method. What if I want to ...
4
votes
2
answers
155
views
How to determine relative contribution of explanatory variables in a linear regression
I estimate a linear regression model, for instance,
$$Y = \alpha + \beta_1 X_1 + \beta_2 X_2 + u$$
and I want to determine how much the variables contribute to $Y$ on average. Put in other words, I ...
- 41
0
votes
0
answers
40
views
In a bivariate linear regression why is $Y = \alpha X + \beta + U$ where $\alpha$ and $\beta$ are real constants & $U$ is an r.v. an assumption?
Suppose that I want to conduct a bivariate regression between random variables $Y$ and $X$. In the textbooks that I'm reading from, primarily Introductory Econometrics and Estimation and Inference in ...
1
vote
2
answers
84
views
ADALINE simple implementation with 2 features bug
I am reading Machine Learning with PyTorch and Ski-kit learn book by Sebastian Raschka
While plotting the decision boundary (a line in this case, since the number of features considered = 2) I can't ...
- 21
0
votes
0
answers
7
views
Is the OECD BCI Dataset fit for use with Linear Regression?
I am wondering if the OECD Business Confidence Index can be utilised by a linear regression model for time-series data.
I have had a look at the ‘basis of prep, for the data and I am rather confused (...
- 1
1
vote
1
answer
47
views
on a linear regression analysis, the determination coefficient is 0.99, but the residuals are not distributed normally. How do I interpret this?
So to preface I'd like to say that this is for homework and I am not very good at statistics. Please explain things to me like I am 5 years old.Also english is not my first language.
So the homework ...
- 11
0
votes
0
answers
37
views
Interpreting coefficients in Log-linear model vs. Poisson regression model
I am trying to understand the difference in interpreting coefficients between log-linear regression and Poisson regression models. To clarify, when I use the term "log-linear regression", I ...
0
votes
0
answers
14
views
Uncertainties when fitting an image
I know how to fit a straight line to a set of 2d points with uncertainties on both coordinates, in order to obtain estimators, goodness-of-fit, and uncertainties - see for instance Press & ...
- 378
3
votes
2
answers
63
views
Perform a weighted linear regression on $x_i, y_i$ by doing a standard linear regression on $X_i, Y_i$?
Let's say we want to do a weighted linear regression between two series $(x_i)$ and $(y_i)$, with weights $(w_i)$, and get the coefficients from the line $y = mx + p$, and the $r^2$ coefficient.
Is ...
- 612
0
votes
0
answers
18
views
Matching on actual earnings versus matching on the kind of unexpectedness in earnings
For simplicity, lets assume this is a question about linear modeling, although I am actually looking at some non-linear models and am willing to consider other models if they would be more ...
- 3,187
0
votes
1
answer
46
views
Proving the equivalence of two distinct approaches to multiple regression for binary classification
I'm stuck with this peculiar problem that uses multiple linear regression in order to solve a binary classification problem (note: it's not considering the logistic version or any other GLM approach).
...
- 3
2
votes
1
answer
92
views
Online updating of $t$-value for simple linear regression
Suppose I am regressing a dependent variable $y$ onto a single independent variable $x$ using a simple ordinary least squares regression model $y = \beta_1 x + \beta_0$. Suppose I start with $n$ data ...
- 123
1
vote
1
answer
44
views
Understanding the coefficients of highly correlated features in generalized linear models
I am trying to fit a generalized linear model, for simplicity assume that is a linear regression. I have a bunch of features and I fitted a linear model to it, the feature ...
- 13
3
votes
2
answers
71
views
Normality condition met with outliers in residuals
I am not sure if the normality condition for my multiple regression is met. Below are the graphs of my residuals. In the second graph (residuals vs fit) the red points are the outliers in the fitted ...
- 155
4
votes
1
answer
52
views
Why do OLS libraries fit models using the MP Pseudoinverse of the design matrix?
For the linear model $y = X\beta$ for design matrix $X$, it's well known that the optimal solution is $\hat{\beta} = (X'X)^{-1}X'y$.
Some statistical libraries (such as Python's statsmodels) estimate ...
- 612
2
votes
1
answer
54
views
Derivative of Linear Model with respect to Residual
I am looking at two sections on the wikipedia page for total least squares, specifically:
#Allowing_observation_errors_in_all_variables and #Example
I have two questions, the first is how does one ...
- 425
0
votes
0
answers
34
views
Coefficient of determination in a linear regression model with a covaring predictor
Given a model:
\begin{align}Y_{i}=Z_{i}*\beta * X_{i} + Z_{i}\tag{Eq. 1}&\end{align}
I am interested in a closed formula for the proportion of variance explained by the predictor variable $X$, ...
- 215
0
votes
1
answer
83
views
Z-score and standard error in linear regression
I am reading Elements of statistical learning, and in the chapter on linear regression, I cannot understand the following:
We have estimated the regression parameters $\beta_1, ..., \beta_p$ from $N$ ...
- 13
1
vote
0
answers
43
views
Non-informative prior in Bayesian Linear Regression
It's known that in Bayesian Linear Regression with $\text{Inv-}\Gamma(a_0, b_0)$ prior on variance parameter $\sigma^2$, the posterior distribution after $n$ observations $(X, Y)$ is $f(\beta, \sigma^...
- 11
1
vote
1
answer
49
views
Need help interpreting interaction effects
I have two categorical independent variables (Category: Before/After and Treatment [A,B,C]) and a binary response.
My regression output in R is as follows:
...
- 13
3
votes
1
answer
67
views
Using F-test for restricted model vs unrestricted, wrong answer
studying Econometrics I come across this question for which I cannot find the right answer:
Assuming a percentage of household's expenses on food is linearly dependent on the total expenditure and ...
- 31
2
votes
1
answer
101
views
Power analysis for ANCOVA
I'm interested in conducting power analysis for experiment design and inference using ANCOVA. I see questions A,B,C vary in terms of quality, applicably and answers; whereas I'm interested in an ...
- 3,386
0
votes
0
answers
31
views
Why are beta coefficients in the two linear regressions same? [duplicate]
I tried two linear regressions with the same dependent variable $y$. Let us assume that the dependent and independent variables are centered around 0 to avoid the need of intercept.
The first with 2 ...
3
votes
0
answers
44
views
Dry skull vs live skull measurement adjustments
I am working on a dataset that has dry skull measurements from a museum's collection (twice as many samples) or a live specimen (less common). As you can imagine, the live specimen are on average a ...
1
vote
0
answers
23
views
Same p-value of overall model and binary predictor level 2 versus Intercept (lm function in R)
I have a response variable PC1 (it is PCA scores for a bunch of observations). I have a response variable category with two ...
- 143
0
votes
0
answers
55
views
Which statistical model will be best for this data?
I'm trying to identify the relationship between the dependent variable and the independent variables. I've utilized linear regression, but I'm not sure if it's suitable given the distribution of my ...
1
vote
1
answer
117
views
Independent variable becomes insignificant after adding control variable. Mediation is significant but doesnt make sense
I am doing a linear regression analysis and have the problem as stated above.
When only the independent variable (IV) and the dependent variable (DV) are included in my model, I get this:
...
- 21
0
votes
0
answers
19
views
Endogeneity or omitted variable bias in a causal model
I am estimating a regression of the form:
My variable of interest is "X1", and based on the information here I can confidently say that the goal of my analysis is a causal inference.
Now to ...
- 1
7
votes
3
answers
994
views
Is multicollinearity a "warning sign" for causal inference?
Suppose we are inferring whether $A$ causes $B$, while holding $N = [N_0, N_1, \ldots, N_n]$ constant and we find $N_i$ correlates well but not perfectly with $A$. There are four reasons to exclude $...
- 173
3
votes
2
answers
91
views
Didactic example of mean-variance dependency in linear models
I'd like to illustrate the importance of accounting for the dependency between mean and variance in inference with linear models. Is my example below a good one? Do you agree with my comments on it?
...
- 4,794
0
votes
1
answer
30
views
How to show that a stable discrete stochastic process converges to a stationary process?
So I have a discrete stochastic process defined by
$x_{k+1}=Ax_k+Bw_k$
where $w_k$ is zero mean Gaussian white noise with covariance $R_w$, and where $A$ has its eigenvalues in the unit disk.
I can ...
0
votes
0
answers
22
views
Sample Variance of the regression coefficient - why does it reduce for more dispersed data?
Thinking on this and I can't see an intuitive reason for this. Given
$$
Var(\hat{\beta}) = \frac{\sigma^2}{S_{xx}}
$$
where
$$
S_{xx} = \sum_{i=1}^{n} (x_i - \bar{x})^2
$$
Intuitively, if we have data ...
0
votes
0
answers
18
views
Coefficient matrix in terms of covariance [duplicate]
I'm currently reading a paper (White et al 2001) on the regression calibration method for addressing measurement error in studies, but am getting stuck on the set up in section 3.1
We have that $A$ ...
0
votes
2
answers
45
views
What is the meaning of the coefficient of an interaction term for this particular regression? [duplicate]
If run the OLS regression (with an interaction term):
$$
y = c + \beta_1 x_{1} + \beta_2 x_{2} + \beta_3 x_{1} x_{2}
$$
What would the meaning of the $\beta_1$ and $\beta_2$ mean? Should the meanings ...
- 103
1
vote
0
answers
38
views
Mediation with PROCESS - Why is my b-path insignificant but my lm significant?
I´m currently working on my bachelor's thesis and ran a mediation with PROCESS (Model 4). My IV is self-compassion, my DV is general mental health and my mediator is loneliness. I am a little bit ...
