Skip to main content

Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

Filter by
Sorted by
Tagged with
1 vote
1 answer
37 views
+50

Power analysis for ANCOVA

I'm interested in conducting power analysis for experiment design and inference using ANCOVA. I see questions A,B,C vary in terms of quality, applicably and answers; whereas I'm interested in an ...
jbuddy_13's user avatar
  • 3,204
0 votes
0 answers
29 views

Why are beta coefficients in the two linear regressions same? [duplicate]

I tried two linear regressions with the same dependent variable $y$. Let us assume that the dependent and independent variables are centered around 0 to avoid the need of intercept. The first with 2 ...
Ashish Gupta's user avatar
3 votes
0 answers
39 views

Dry skull vs live skull measurement adjustments

I am working on a dataset that has dry skull measurements from a museum's collection (twice as many samples) or a live specimen (less common). As you can imagine, the live specimen are on average a ...
KellyForrester's user avatar
1 vote
0 answers
23 views

Same p-value of overall model and binary predictor level 2 versus Intercept (lm function in R)

I have a response variable PC1 (it is PCA scores for a bunch of observations). I have a response variable category with two ...
Shakir's user avatar
  • 143
0 votes
0 answers
54 views

Which statistical model will be best for this data?

I'm trying to identify the relationship between the dependent variable and the independent variables. I've utilized linear regression, but I'm not sure if it's suitable given the distribution of my ...
Chemokine1's user avatar
1 vote
1 answer
57 views

Independent variable becomes insignificant after adding control variable. Mediation is significant but doesnt make sense

I am doing a linear regression analysis and have the problem as stated above. When only the independent variable (IV) and the dependent variable (DV) are included in my model, I get this: ...
user9011032's user avatar
0 votes
0 answers
17 views

Endogeneity or omitted variable bias in a causal model

I am estimating a regression of the form: My variable of interest is "X1", and based on the information here I can confidently say that the goal of my analysis is a causal inference. Now to ...
mpinzonc's user avatar
6 votes
3 answers
870 views

Is multicollinearity a "warning sign" for causal inference?

Suppose we are inferring whether $A$ causes $B$, while holding $N = [N_0, N_1, \ldots, N_n]$ constant and we find $N_i$ correlates well but not perfectly with $A$. There are four reasons to exclude $...
charmoniumQ's user avatar
0 votes
0 answers
15 views

Didactic example of mean-variance dependency in linear models

I'd like to illustrate the importance of accounting for the dependency between mean and variance in inference with linear models. Is my example below a good one? Do you agree with my comments on it? ...
dariober's user avatar
  • 4,539
0 votes
1 answer
26 views

How to show that a stable discrete stochastic process converges to a stationary process?

So I have a discrete stochastic process defined by $x_{k+1}=Ax_k+Bw_k$ where $w_k$ is zero mean Gaussian white noise with covariance $R_w$, and where $A$ has its eigenvalues in the unit disk. I can ...
Minecraft dirt block's user avatar
0 votes
0 answers
21 views

Sample Variance of the regression coefficient - why does it reduce for more dispersed data?

Thinking on this and I can't see an intuitive reason for this. Given $$ Var(\hat{\beta}) = \frac{\sigma^2}{S_{xx}} $$ where $$ S_{xx} = \sum_{i=1}^{n} (x_i - \bar{x})^2 $$ Intuitively, if we have data ...
InvestingScientist's user avatar
0 votes
0 answers
17 views

Coefficient matrix in terms of covariance [duplicate]

I'm currently reading a paper (White et al 2001) on the regression calibration method for addressing measurement error in studies, but am getting stuck on the set up in section 3.1 We have that $A$ ...
Jessica F's user avatar
0 votes
2 answers
37 views

What is the meaning of the coefficient of an interaction term for this particular regression? [duplicate]

If run the OLS regression (with an interaction term): $$ y = c + \beta_1 x_{1} + \beta_2 x_{2} + \beta_3 x_{1} x_{2} $$ What would the meaning of the $\beta_1$ and $\beta_2$ mean? Should the meanings ...
KaiSqDist's user avatar
  • 103
1 vote
0 answers
35 views

Mediation with PROCESS - Why is my b-path insignificant but my lm significant?

I´m currently working on my bachelor's thesis and ran a mediation with PROCESS (Model 4). My IV is self-compassion, my DV is general mental health and my mediator is loneliness. I am a little bit ...
statistic noob 666's user avatar
0 votes
0 answers
13 views

Is it advisable to use PLSR components in another linear equation?

I currently possess biological data from two groups (healthy vs disease) in the form of protein concentrations and I was interested in determining what the relationship was between these protein ...
Syuma's user avatar
  • 115
0 votes
0 answers
19 views

GLM with canonical link: Why linear regression over the natural parameter?

So I've been wondering why it is "natural" to extend linear models where we assume $Y\sim N(\mu,\Sigma)$ and try to fit $E[Y|X]=X\beta$ to generalized linear models where we assume $Y_i\sim ...
R.V.N.'s user avatar
  • 123
1 vote
1 answer
39 views

Unsure about assumptions of linear model with time series variables, spurious regression and periodic patterns

Background I'm learning about time series in context of linear regression. The goal of this question is to understand how seasonality of either X or Y can affect the model. Linear model assumptions $...
Brzoskwinia's user avatar
3 votes
1 answer
118 views

How small coefficient of variation should be accepted for dependent variable and how does it affect linear regression?

I was wondering what happens to a linear regression model if the coefficient of variation of $Y$ variable is small. Also, what is considered as "too small CV"? What is the minimal accepted ...
Brzoskwinia's user avatar
0 votes
0 answers
40 views

Is an interaction term essentially interpreted as just another independent variable?

I'm trying to understand if the linear regression model interprets an interaction term as just another independent variable? This is the formula for linear regression with 2 independent variables ...
Mandem's user avatar
  • 135
3 votes
1 answer
38 views

Mixed effects clarification - Ecology

I have data from an experiment which measured a variety of response variables (e.g. soil carbon) across a number of sites each with one of 4 different treatments. These sites are all grouped within a ...
Eco_Analysis's user avatar
2 votes
1 answer
38 views

Simple Regression Coefficient Formula for Categorical Variable?

For an indepdent numerical variable X the B1 coefficient is COV(X,Y)/Var(X). Since Categorical Variables don't have things like Means(from which things like COV and VAR are derived) how would it work ...
Mandem's user avatar
  • 135
0 votes
0 answers
12 views

Estimating treatment effect with/without intercept [duplicate]

I am trying to estimate the treatment effect based on the above two model: $$Y(Z)=\beta_0+\tau Z+\varepsilon.$$ $$Y(Z)=\tau Z+\varepsilon.$$ Based on result from my data, I found the intercept is not ...
Fangzhi Luo's user avatar
0 votes
0 answers
10 views

Decomposing model volatility with respect to factor contributions

Consider a linear model $\textbf{y} = \textbf{x}\pmb{\beta} + \pmb{\varepsilon}$ with $\textbf{y}$ a $T \times 1$ vector of random variables, $\pmb{\beta}$ a $K \times 1$ vector and $\textbf{x}$ a $T \...
user9875321__'s user avatar
1 vote
3 answers
87 views

How does $y = ax + b + \epsilon$ induce a probability distribution on $R^2$?

This is with regards to the example given in the wikipedia article on statistical models. In the example it is claimed that "Each possible value of $\theta = (b_0, b_1, σ^2)$ determines a ...
John Doe's user avatar
4 votes
3 answers
507 views

In linear regression, do the errors overall have a normal distribution, or do the errors at each value of x have a normal distribution?

In linear regression with fixed effects (i.e. with constant $x_i$, not random $X_i$), the model states that $\epsilon_i \sim N(0, \sigma^2), \ \ i = 1, 2, ..., n$ Does this say that the overall set ...
Iterator516's user avatar
1 vote
2 answers
63 views

In linear regression, does the formula for error contain the marginal expectation or conditional expectation?

In linear regression, let $\epsilon_i$ be the $i$th error term. Is the formula for $\epsilon_i$ $\epsilon_i = Y_i - E(Y_i)$ or $\epsilon_i = Y_i - E(Y_i | X_i = x_i)$? I have seen both definitions....
Iterator516's user avatar
1 vote
1 answer
77 views

Should I contain time as a random variable in repeated measurement?

I have a dataset which contains happiness scores of several subjects, measured three times within a day (in morning, afternoon and evening) and repeated across several days. But during how many days ...
Eve's user avatar
  • 51
1 vote
0 answers
19 views

When error propagation is necessary in modelling?

This is a somewhat philosophical question. When executing classical statistical modeling, such as regression, LM, GLM, mixed modeling, etc., there is often no mention of propagating the error of the ...
JMenezes's user avatar
  • 539
1 vote
1 answer
53 views

How to express Hotelling T² test as a Likelihood ratio between two multivariate linear models?

Is it possible, given that Hotelling's T² (or Hotelling-Lawley Trace for that matter) is just a generalization of Student's T, to reformulate the same testing procedure (to test if two vectors differ) ...
ratatosk's user avatar
0 votes
0 answers
12 views

When is it appropriate to adjust an independent variable by regression before including it in a regression on the dependent variable of interest?

I want to run a linear mixed regression in which, basically, I see whether the expression of a particular gene is associated with a trait exhibited by the animal. I'll use growth rate as an example. ...
Made's user avatar
  • 1
1 vote
1 answer
48 views

Comparing Deming/Orthogonal Regression to Null Hypothesis

I have some data of with the relationship Y=commonFactor+error1 and X=Alpha+Beta*commonFactor+error2 I want to test the hypothesis that Beta is non-zero, or that there is a significant relationship ...
A Friendly Fish's user avatar
1 vote
0 answers
12 views

Computing Type 3 Sum of Squares of AN(C)OVA

I have a two-way ANCOVA model of the form $$y_{ijk} = \mu + \alpha_i +\beta_j+\gamma_{ij}+\delta c_{ijk} + \epsilon_{ijk}.$$ Rewriting this model in matrix notation gives $$\boldsymbol y = \boldsymbol ...
Quertiopler's user avatar
0 votes
0 answers
20 views

anova_lm function for a single multiple linear regression model [duplicate]

When using anova_lm from statsmodels with a single multiple linear regression model, how are statistics for each predictor calculated ? E.g. ...
Dalibor's user avatar
2 votes
1 answer
33 views

Is it possible to test if the response difference over two time points is significantly different between groups when there are no repeated subjects?

I currently have a dataset where I've collected data between two groups (disease versus control). The experimental design is to evaluate how the response variable changes over time. Thus, we have 4 ...
Syuma's user avatar
  • 115
0 votes
0 answers
23 views

Deriving MLEs for $\sigma_{\epsilon}^2$, $\beta$, and $Q$ in a Linear Mixed Model

I'm currently working on a problem involving a linear mixed model of the form: $Y_i = X_i \beta + Z_i b_i + \epsilon_i,$ where $\epsilon_i \sim N (0, \sigma_{\epsilon}^2 I_{Ji})$. The model can also ...
David's user avatar
  • 1
5 votes
1 answer
221 views

What are average comparisons in the `marginaleffects` package?

I am confused about what the avg_comparison function does in the marginaleffects package. ...
Carol Eisen's user avatar
0 votes
0 answers
13 views

Comparing the ranking of effect sizes?

I have a lot of effect sizes, estimated from the same linear model, but where the tested explanatory variable is different. These models are run in two different groups, but it is the same model and ...
lo2's user avatar
  • 75
2 votes
1 answer
53 views

Sampling Variance of OLS Estimators of Regression Coefficients

I am confused about whether the value of the sampling variance of the OLS estimator of a regression coefficient (e.g. slope) differs from sample to sample. Assume we have the following simple linear ...
Jingyang Zhang's user avatar
1 vote
1 answer
39 views

Confidence box for coefficients of linear regression?

I am learning linear regression and I am trying to create a visualisation. Say I want to estimate a power model $y=ax^b$ using linear regression. I take the logarithm to get $$\ln(y)=\ln(a)+b\ln(x)$$ ...
Sorfosh's user avatar
  • 111
2 votes
1 answer
42 views

Is It valid to use a Linear Mixed effect Model to quantify a group 'summary' value and plot it when the factors are all categorical?

I currently have a dataset with two factors: Gene and Timepoint. Both of these factors are categorical in nature where Gene has 2 levels defined as control vs disease, and timepoint has 4 levels: ...
Syuma's user avatar
  • 115
4 votes
1 answer
44 views

Partial Correlation and Partial (Linear) Regression

Consider the linear regression model $$\boldsymbol y = \alpha + \beta \boldsymbol x + \gamma \boldsymbol z + \boldsymbol u,$$ and denote the OLSE of $\alpha$, $\beta$ and $\gamma$ by $\hat\alpha$, $\...
Syd Amerikaner's user avatar
0 votes
1 answer
27 views

How can I show that none of the other variables in the model were potential mediators

I got a revision for my research paper recently and the following is the reviewer's comment on my paper: the authors should show that for any estimate from the linear regression that is reported in ...
zhiheng yi's user avatar
0 votes
1 answer
43 views

Multiple linear regression slopes inconsistent with graph - why?

I am doing a multiple linear regression, with 3 categorical predictor variables (Flow, Drug, Pesticide) each with two levels (0 vs. 1). The response variable is the abundance of invertebrates. I have ...
blue_earth's user avatar
2 votes
1 answer
46 views

Regression using unordered combinations, sign of predictor depends on order

First time poster here. Is there anything wrong with the following regression approach? EDIT: After following the advice of the accepted answer to include the multiple-membership effect, I wanted to ...
Evan Stegner's user avatar
0 votes
0 answers
29 views

Conditionally conjugate prior in heteroskedastic model

I am researching a linear model where the noise is a function of the slope parameter as follows $$y_i = \beta_0 + \beta_1x_i + \beta_1\epsilon_i$$ $$\epsilon_i \sim N(0, \sigma^2 g)$$ where $g$ is ...
spencergw's user avatar
  • 141
2 votes
2 answers
326 views

Why are error properties in linear regression assumptions if they are true by construction?

The following two results on the residuals ($\epsilon$) in the case of linear regression get stated as assumptions of the linear regressions $E(\epsilon) = 0$ $cov(X, \epsilon) = 0$ Here is MIT 18....
figs_and_nuts's user avatar
1 vote
0 answers
63 views

Is this regression problem solvable? [closed]

I have a random vector $\pmb{x}=(X_1,...,X_p)^T\in \mathbb{R}^p$, a symmetric matrix $$\Theta = \left(\begin{matrix}0 & \theta_{12} & \theta_{13} & \cdots & \theta_{1p}\\ \theta_{12} &...
Hepdrey's user avatar
  • 79
1 vote
1 answer
54 views

What's the justification for comparing two separate models built on subsets of data versus using one model that uses the whole dataset?

I've noticed that there are some data analysis being done in some scientific field where the authors would split out an entire dataset into subsets based on a particular property. One classic example ...
Syuma's user avatar
  • 115
0 votes
1 answer
26 views

ANOVA comparison different subsets of same data frame

I am trying to compare two model which are based either on the male or female gender in my data. There is the same number of people in every gender group. Why is the anova function not giving a p-...
Han's user avatar
  • 1
0 votes
0 answers
15 views

How to interpret the coefficient of a residualized variable in a linear model?

I was fitting a linear model and there was strong multicollinearity present in the data. So, I decided to residualize one regressor variable to reduce the multicollinearity and fitted the model again. ...
Peter's user avatar
  • 1

1
2 3 4 5
52