Questions tagged [linear-model]
Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.
2,547
questions
1
vote
1
answer
25
views
How to express Hotelling T² test as a Likelihood ratio between two multivariate linear models?
Is it possible, given that Hotelling's T² (or Hotelling-Lawley Trace for that matter) is just a generalization of Student's T, to reformulate the same testing procedure (to test if two vectors differ) ...
0
votes
0
answers
42
views
Translate lm results into math equation [closed]
I don't understand how to use my lm() results as a math formula to retrieve my model's formula.
...
0
votes
0
answers
7
views
When is it appropriate to adjust an independent variable by regression before including it in a regression on the dependent variable of interest?
I want to run a linear mixed regression in which, basically, I see whether the expression of a particular gene is associated with a trait exhibited by the animal. I'll use growth rate as an example. ...
0
votes
0
answers
26
views
How to force lm() and glm() functions not to refactor weights for linear regression? [closed]
I just want to run lm() and glm() for linear regression without refactoring weights, i.e., to utilize the weights just as they ...
1
vote
1
answer
34
views
Comparing Deming/Orthogonal Regression to Null Hypothesis
I have some data of with the relationship
Y=commonFactor+error1 and X=Alpha+Beta*commonFactor+error2
I want to test the hypothesis that Beta is non-zero, or that there is a significant relationship ...
1
vote
0
answers
10
views
Computing Type 3 Sum of Squares of AN(C)OVA
I have a two-way ANCOVA model of the form
$$y_{ijk} = \mu + \alpha_i +\beta_j+\gamma_{ij}+\delta c_{ijk} + \epsilon_{ijk}.$$
Rewriting this model in matrix notation gives $$\boldsymbol y = \boldsymbol ...
0
votes
0
answers
20
views
anova_lm function for a single multiple linear regression model [duplicate]
When using anova_lm from statsmodels with a single multiple linear regression model, how are statistics for each predictor calculated ?
E.g.
...
2
votes
1
answer
26
views
Is it possible to test if the response difference over two time points is significantly different between groups when there are no repeated subjects?
I currently have a dataset where I've collected data between two groups (disease versus control). The experimental design is to evaluate how the response variable changes over time. Thus, we have 4 ...
0
votes
0
answers
23
views
Deriving MLEs for $\sigma_{\epsilon}^2$, $\beta$, and $Q$ in a Linear Mixed Model
I'm currently working on a problem involving a linear mixed model of the form:
$Y_i = X_i \beta + Z_i b_i + \epsilon_i,$
where $\epsilon_i \sim N (0, \sigma_{\epsilon}^2 I_{Ji})$. The model can also ...
5
votes
1
answer
143
views
What are average comparisons in the `marginaleffects` package?
I am confused about what the avg_comparison function does in the marginaleffects package.
...
0
votes
0
answers
13
views
Comparing the ranking of effect sizes?
I have a lot of effect sizes, estimated from the same linear model, but where the tested explanatory variable is different. These models are run in two different groups, but it is the same model and ...
2
votes
1
answer
48
views
Sampling Variance of OLS Estimators of Regression Coefficients
I am confused about whether the value of the sampling variance of the OLS estimator of a regression coefficient (e.g. slope) differs from sample to sample.
Assume we have the following simple linear ...
0
votes
1
answer
34
views
Confidence box for coefficients of linear regression?
I am learning linear regression and I am trying to create a visualisation. Say I want to estimate a power model $y=ax^b$ using linear regression. I take the logarithm to get
$$\ln(y)=\ln(a)+b\ln(x)$$
...
1
vote
1
answer
37
views
Is It valid to use a Linear Mixed effect Model to quantify a group 'summary' value and plot it when the factors are all categorical?
I currently have a dataset with two factors: Gene and Timepoint. Both of these factors are categorical in nature where Gene has 2 levels defined as control vs disease, and timepoint has 4 levels: ...
4
votes
1
answer
39
views
Partial Correlation and Partial (Linear) Regression
Consider the linear regression model $$\boldsymbol y = \alpha + \beta \boldsymbol x + \gamma \boldsymbol z + \boldsymbol u,$$ and denote the OLSE of $\alpha$, $\beta$ and $\gamma$ by $\hat\alpha$, $\...
0
votes
1
answer
27
views
How can I show that none of the other variables in the model were potential mediators
I got a revision for my research paper recently and the following is the reviewer's comment on my paper: the authors should show that for any estimate from the linear regression that is reported in ...
0
votes
1
answer
40
views
Multiple linear regression slopes inconsistent with graph - why?
I am doing a multiple linear regression, with 3 categorical predictor variables (Flow, Drug, Pesticide) each with two levels (0 vs. 1). The response variable is the abundance of invertebrates. I have ...
2
votes
1
answer
37
views
Regression using unordered combinations, sign of predictor depends on order
First time poster here. Is there anything wrong with the following regression approach?
Background:
I am dealing with fuel consumption $F$ of a vehicle under a variety of conditions. I have developed ...
0
votes
0
answers
27
views
Conditionally conjugate prior in heteroskedastic model
I am researching a linear model where the noise is a function of the slope parameter as follows
$$y_i = \beta_0 + \beta_1x_i + \beta_1\epsilon_i$$
$$\epsilon_i \sim N(0, \sigma^2 g)$$
where $g$ is ...
2
votes
2
answers
313
views
Why are error properties in linear regression assumptions if they are true by construction?
The following two results on the residuals ($\epsilon$) in the case of linear regression get stated as assumptions of the linear regressions
$E(\epsilon) = 0$
$cov(X, \epsilon) = 0$
Here is MIT 18....
1
vote
0
answers
57
views
Is this regression problem solvable? [closed]
I have a random vector $\pmb{x}=(X_1,...,X_p)^T\in \mathbb{R}^p$, a symmetric matrix
$$\Theta = \left(\begin{matrix}0 & \theta_{12} & \theta_{13} & \cdots & \theta_{1p}\\
\theta_{12} &...
0
votes
1
answer
42
views
What's the justification for comparing two separate models built on subsets of data versus using one model that uses the whole dataset?
I've noticed that there are some data analysis being done in some scientific field where the authors would split out an entire dataset into subsets based on a particular property. One classic example ...
0
votes
1
answer
23
views
ANOVA comparison different subsets of same data frame
I am trying to compare two model which are based either on the male or female gender in my data. There is the same number of people in every gender group. Why is the anova function not giving a p-...
0
votes
0
answers
11
views
How to interpret the coefficient of a residualized variable in a linear model?
I was fitting a linear model and there was strong multicollinearity present in the data. So, I decided to residualize one regressor variable to reduce the multicollinearity and fitted the model again. ...
1
vote
1
answer
23
views
Can I use an autoregressive (AR1) model to determine if longitudinal data can be treated as individual single time points?
I am currently in a position where I have two datasets, one consisting of longitudinal data collected at 4 different time points, and another consisting of only single time points (e.g. data collected ...
0
votes
0
answers
25
views
Dealing with Ratios in Linear Regression
I am studying the relationship between X and Y using linear models. Both are composed of a left and right scalar value, and I'm interested in the relationship between both the totals and ratios for ...
0
votes
1
answer
45
views
Can I violate assumptions of normality for categorical linear regression models?
I'm using packages included in this R/rStudio tutorial to set up some linear regression models comparing a continuous dependent variable (eccentricity) to three categorical variables (year, bird ...
0
votes
0
answers
53
views
Can I specify the signs on coefficients of variables in Regression models?
I have a Regression model where the coefficients of each variable need specific signs.
For Example, say I have a standard OLS model: Y = β1X1 + β2X2 + … + β7X7. I want coefficients β2 and β5 to be ...
0
votes
0
answers
14
views
How to fix signs of variable coefficients apriori in a linear regression model? [duplicate]
I want to restrict the signs of variable coefficients in a linear regression model. For example, if I am using the mtcars dataset from R to build a linear regression model I will get these ...
1
vote
1
answer
33
views
linear Combination of Normal and T-Distributions
Consider the following probability distribution function (PDF):
\begin{equation}
p(x) = a\mathcal{N}(x; \mu, \sigma^2) + b \mathcal{T}(x; \mu, \tau^2, v) \; \; st. \; \;a + b = 1
\end{equation}
$p(x)$ ...
5
votes
1
answer
158
views
Confusion regarding the criteria for defining a ML model as a linear model
I am confused about the criteria which determines whether a model is linear or not. As far as I understand, the following statements are equivalent :
A model is linear
Output class label is a linear ...
0
votes
0
answers
17
views
least squares in regression with covariate-dependent model [duplicate]
Classical least squares results in regression in statistics state that if $(Y, X)$ follow a model where $$\mathbb{E}[Y\mid X=x] = \alpha + \beta x,$$ we can estimate $\beta$ from a random sample ...
2
votes
1
answer
120
views
Weighted least squares for a linear model
Background
I have a 2-dimensional dataset $\{y_i, x_i\}_{i=1}^N$ in the coordinates $y,x$. I'm trying to fit the dataset with the trivial model
$$\tag{*}y=mx$$
where $m$ is a (scalar) parameter that ...
0
votes
0
answers
24
views
What's the interpretation of the intercept of a linear mixed effect model with only one group
I'm trying to understand the meaning of the intercept term in a linear mixed model. The caveat here is that the dataset the LME will be built upon will only consist of subjects from a single class (...
3
votes
1
answer
30
views
Converting Adjusted R²
I just examined the $R^2_\text{adj}$ Formula on Wikipedia and found two ways to calculate the adjusted $R^2$.
Firstly as
$$R^2_\text{adj}=1-\frac{\frac{SS_\text{res}}{(n-p-1)}}{\frac{SS_\text{tot}}{(n-...
0
votes
0
answers
28
views
Compare the intercept and y-intercept of the new model with the old model
If we have two simple linear models (old and new), is there a good way to evaluate whether the newly developed model is sufficiently different from the previous model?
For example, let's assume there ...
0
votes
1
answer
49
views
Expectation of residuals in linear regression
Consider a linear regression model
$$ Y=X\beta + \epsilon, $$
where $Y\in R^n$, $X = (x_1,...,x_n)^T\in R^{n\times p}$ are i.i.d. $p$-dimensional observations, $\beta\in R^p$, and $\epsilon = (\...
1
vote
1
answer
84
views
How to show that OLS and MLE are the same for $\beta$
I have several questions regarding this proof:
Shouldn't the $\propto$ be used instead of the $=$ when we leave out the $\frac{1}{\sqrt{2\pi\sigma²}}$ ?
Is a maximization problem simply inverted to a ...
2
votes
0
answers
38
views
Interpreting eigenvalues of non-normalized covariance matrix of physical system
Cross-posted from physics stackexchange
Summary: Eigenvalues of a "non-normalized" covariance matrix of time-series measurements from a linear system have units of Action (energy * time). ...
5
votes
2
answers
183
views
How to deal with correlated variables
I would like to know how to deal with correlated variables, with this kind of correlation matrix:
Is there a way to combine the correlated variables such as all the AV.. variables or FF.. variables?
...
1
vote
1
answer
36
views
How to estimate the coefficients in OLS (all steps) [duplicate]
I'm a B.S. Math graduate who likes to (attempt) to teach myself statistics on my own time because I can't afford a masters degree.
It really bothers me when I can't understand at a fundamental level ...
1
vote
1
answer
89
views
Is Linear Regression a good algorithm or even applicable with the distribution shown in the scatter plot I have shared in this question?
I am trying to use Linear Regression on a dataset using scikit-learn with python. And my understanding is that Linear Regression requires "some linearity" to exist between independent and ...
0
votes
1
answer
74
views
In a linear model, why do we have $-2X^T \vec{y} + 2X^T X \vec{\beta}=0$? [duplicate]
When we derive the estimates of $\vec{\beta}$ such that they minimize the sum of squared error ($SSE$) we begin with $\sum_{i=1}^{n} (y_i - (\beta_0 + \beta_1x_1 + ... + \beta_kx_k))^2$. This is ...
1
vote
1
answer
57
views
Exact Meaning of Controlled Variables in terms of the OLS estimators
Consider a two-regressors linear regression model:
$$Y=\beta_0+\beta_1X+\beta_2D+u$$
where $X$ is continuous and $D$ is binary.
In this case, we say that we have controlled $D$.
Here, I want to know ...
11
votes
7
answers
2k
views
Why do we use Linear Models when tree based models often work better than linear models?
In Supervised Machine Learning, and specifically on Kaggle, it is usually seen that tree models often outperform linear models. And even in the tree-based models, it is usually XGBoost that ...
0
votes
0
answers
19
views
Is there a method to estimate the distribution of error term in linear model?
Consider the linear model where $A$ is not known
$$
y = Ax + \epsilon
$$
where we want to estimate the distribution $\epsilon$ from a set of samples. To prevent over-fitting, we want to impose some ...
1
vote
0
answers
59
views
Linear model for maximizing rank correlation between observed and predicted response
linear regression is modelled as
$$Y = X\beta + \epsilon$$
for response variable $Y$ (vector), design matrix $X$, and iid Gaussian noise $\epsilon$ (vector).
instead of minimizing the mean squared ...
0
votes
0
answers
18
views
Difference in tests for linear models and significance levels adjustment
Context
Imagine we measure two outcomes $\mathbf{Y}$ and $\mathbf{Z}$ that are real vectors. We also have the following covariates $G,T, X_1,\cdots,X_k$ for a given integer $k>1$. $G$ represents a ...
1
vote
1
answer
63
views
How can we think about linear regression geometrically?
The simple linear regression model is given by $y_i = \beta_0 + \beta_1x_1 + e$
It is my understanding that it can be rewritten in matrix vector form as $\vec{y} = X\vec{\beta} + \vec{e}$ where $X$ is ...
1
vote
0
answers
59
views
How to mathematically prove the "transitive property of nested predictors"?
QUESTION
I am studying the structure of experiment data sets, and I want to propose a rule that I call the "transitive property of nested predictors".
The general idea is that…
if there are ...