Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

Filter by
Sorted by
Tagged with
1 vote
1 answer
25 views

How to express Hotelling T² test as a Likelihood ratio between two multivariate linear models?

Is it possible, given that Hotelling's T² (or Hotelling-Lawley Trace for that matter) is just a generalization of Student's T, to reformulate the same testing procedure (to test if two vectors differ) ...
ratatosk's user avatar
0 votes
0 answers
42 views

Translate lm results into math equation [closed]

I don't understand how to use my lm() results as a math formula to retrieve my model's formula. ...
Aurélien Lengrand's user avatar
0 votes
0 answers
7 views

When is it appropriate to adjust an independent variable by regression before including it in a regression on the dependent variable of interest?

I want to run a linear mixed regression in which, basically, I see whether the expression of a particular gene is associated with a trait exhibited by the animal. I'll use growth rate as an example. ...
Made's user avatar
  • 1
0 votes
0 answers
26 views

How to force lm() and glm() functions not to refactor weights for linear regression? [closed]

I just want to run lm() and glm() for linear regression without refactoring weights, i.e., to utilize the weights just as they ...
Viktor's user avatar
  • 1,007
1 vote
1 answer
34 views

Comparing Deming/Orthogonal Regression to Null Hypothesis

I have some data of with the relationship Y=commonFactor+error1 and X=Alpha+Beta*commonFactor+error2 I want to test the hypothesis that Beta is non-zero, or that there is a significant relationship ...
A Friendly Fish's user avatar
1 vote
0 answers
10 views

Computing Type 3 Sum of Squares of AN(C)OVA

I have a two-way ANCOVA model of the form $$y_{ijk} = \mu + \alpha_i +\beta_j+\gamma_{ij}+\delta c_{ijk} + \epsilon_{ijk}.$$ Rewriting this model in matrix notation gives $$\boldsymbol y = \boldsymbol ...
Quertiopler's user avatar
0 votes
0 answers
20 views

anova_lm function for a single multiple linear regression model [duplicate]

When using anova_lm from statsmodels with a single multiple linear regression model, how are statistics for each predictor calculated ? E.g. ...
Dalibor's user avatar
2 votes
1 answer
26 views

Is it possible to test if the response difference over two time points is significantly different between groups when there are no repeated subjects?

I currently have a dataset where I've collected data between two groups (disease versus control). The experimental design is to evaluate how the response variable changes over time. Thus, we have 4 ...
Syuma's user avatar
  • 73
0 votes
0 answers
23 views

Deriving MLEs for $\sigma_{\epsilon}^2$, $\beta$, and $Q$ in a Linear Mixed Model

I'm currently working on a problem involving a linear mixed model of the form: $Y_i = X_i \beta + Z_i b_i + \epsilon_i,$ where $\epsilon_i \sim N (0, \sigma_{\epsilon}^2 I_{Ji})$. The model can also ...
David's user avatar
  • 1
5 votes
1 answer
143 views

What are average comparisons in the `marginaleffects` package?

I am confused about what the avg_comparison function does in the marginaleffects package. ...
Carol Eisen's user avatar
0 votes
0 answers
13 views

Comparing the ranking of effect sizes?

I have a lot of effect sizes, estimated from the same linear model, but where the tested explanatory variable is different. These models are run in two different groups, but it is the same model and ...
lo2's user avatar
  • 75
2 votes
1 answer
48 views

Sampling Variance of OLS Estimators of Regression Coefficients

I am confused about whether the value of the sampling variance of the OLS estimator of a regression coefficient (e.g. slope) differs from sample to sample. Assume we have the following simple linear ...
Jingyang Zhang's user avatar
0 votes
1 answer
34 views

Confidence box for coefficients of linear regression?

I am learning linear regression and I am trying to create a visualisation. Say I want to estimate a power model $y=ax^b$ using linear regression. I take the logarithm to get $$\ln(y)=\ln(a)+b\ln(x)$$ ...
Sorfosh's user avatar
  • 101
1 vote
1 answer
37 views

Is It valid to use a Linear Mixed effect Model to quantify a group 'summary' value and plot it when the factors are all categorical?

I currently have a dataset with two factors: Gene and Timepoint. Both of these factors are categorical in nature where Gene has 2 levels defined as control vs disease, and timepoint has 4 levels: ...
Syuma's user avatar
  • 73
4 votes
1 answer
39 views

Partial Correlation and Partial (Linear) Regression

Consider the linear regression model $$\boldsymbol y = \alpha + \beta \boldsymbol x + \gamma \boldsymbol z + \boldsymbol u,$$ and denote the OLSE of $\alpha$, $\beta$ and $\gamma$ by $\hat\alpha$, $\...
Syd Amerikaner's user avatar
0 votes
1 answer
27 views

How can I show that none of the other variables in the model were potential mediators

I got a revision for my research paper recently and the following is the reviewer's comment on my paper: the authors should show that for any estimate from the linear regression that is reported in ...
zhiheng yi's user avatar
0 votes
1 answer
40 views

Multiple linear regression slopes inconsistent with graph - why?

I am doing a multiple linear regression, with 3 categorical predictor variables (Flow, Drug, Pesticide) each with two levels (0 vs. 1). The response variable is the abundance of invertebrates. I have ...
blue_earth's user avatar
2 votes
1 answer
37 views

Regression using unordered combinations, sign of predictor depends on order

First time poster here. Is there anything wrong with the following regression approach? Background: I am dealing with fuel consumption $F$ of a vehicle under a variety of conditions. I have developed ...
Evan Stegner's user avatar
0 votes
0 answers
27 views

Conditionally conjugate prior in heteroskedastic model

I am researching a linear model where the noise is a function of the slope parameter as follows $$y_i = \beta_0 + \beta_1x_i + \beta_1\epsilon_i$$ $$\epsilon_i \sim N(0, \sigma^2 g)$$ where $g$ is ...
spencergw's user avatar
2 votes
2 answers
313 views

Why are error properties in linear regression assumptions if they are true by construction?

The following two results on the residuals ($\epsilon$) in the case of linear regression get stated as assumptions of the linear regressions $E(\epsilon) = 0$ $cov(X, \epsilon) = 0$ Here is MIT 18....
figs_and_nuts's user avatar
1 vote
0 answers
57 views

Is this regression problem solvable? [closed]

I have a random vector $\pmb{x}=(X_1,...,X_p)^T\in \mathbb{R}^p$, a symmetric matrix $$\Theta = \left(\begin{matrix}0 & \theta_{12} & \theta_{13} & \cdots & \theta_{1p}\\ \theta_{12} &...
Hepdrey's user avatar
  • 63
0 votes
1 answer
42 views

What's the justification for comparing two separate models built on subsets of data versus using one model that uses the whole dataset?

I've noticed that there are some data analysis being done in some scientific field where the authors would split out an entire dataset into subsets based on a particular property. One classic example ...
Syuma's user avatar
  • 73
0 votes
1 answer
23 views

ANOVA comparison different subsets of same data frame

I am trying to compare two model which are based either on the male or female gender in my data. There is the same number of people in every gender group. Why is the anova function not giving a p-...
Han's user avatar
  • 1
0 votes
0 answers
11 views

How to interpret the coefficient of a residualized variable in a linear model?

I was fitting a linear model and there was strong multicollinearity present in the data. So, I decided to residualize one regressor variable to reduce the multicollinearity and fitted the model again. ...
Peter's user avatar
  • 1
1 vote
1 answer
23 views

Can I use an autoregressive (AR1) model to determine if longitudinal data can be treated as individual single time points?

I am currently in a position where I have two datasets, one consisting of longitudinal data collected at 4 different time points, and another consisting of only single time points (e.g. data collected ...
Syuma's user avatar
  • 73
0 votes
0 answers
25 views

Dealing with Ratios in Linear Regression

I am studying the relationship between X and Y using linear models. Both are composed of a left and right scalar value, and I'm interested in the relationship between both the totals and ratios for ...
Celongar's user avatar
0 votes
1 answer
45 views

Can I violate assumptions of normality for categorical linear regression models?

I'm using packages included in this R/rStudio tutorial to set up some linear regression models comparing a continuous dependent variable (eccentricity) to three categorical variables (year, bird ...
ElizaBeso000's user avatar
0 votes
0 answers
53 views

Can I specify the signs on coefficients of variables in Regression models?

I have a Regression model where the coefficients of each variable need specific signs. For Example, say I have a standard OLS model: Y = β1X1 + β2X2 + … + β7X7. I want coefficients β2 and β5 to be ...
user402101's user avatar
0 votes
0 answers
14 views

How to fix signs of variable coefficients apriori in a linear regression model? [duplicate]

I want to restrict the signs of variable coefficients in a linear regression model. For example, if I am using the mtcars dataset from R to build a linear regression model I will get these ...
T_S's user avatar
  • 21
1 vote
1 answer
33 views

linear Combination of Normal and T-Distributions

Consider the following probability distribution function (PDF): \begin{equation} p(x) = a\mathcal{N}(x; \mu, \sigma^2) + b \mathcal{T}(x; \mu, \tau^2, v) \; \; st. \; \;a + b = 1 \end{equation} $p(x)$ ...
Snowy Baboon's user avatar
5 votes
1 answer
158 views

Confusion regarding the criteria for defining a ML model as a linear model

I am confused about the criteria which determines whether a model is linear or not. As far as I understand, the following statements are equivalent : A model is linear Output class label is a linear ...
TarS's user avatar
  • 51
0 votes
0 answers
17 views

least squares in regression with covariate-dependent model [duplicate]

Classical least squares results in regression in statistics state that if $(Y, X)$ follow a model where $$\mathbb{E}[Y\mid X=x] = \alpha + \beta x,$$ we can estimate $\beta$ from a random sample ...
Albert Paradek's user avatar
2 votes
1 answer
120 views

Weighted least squares for a linear model

Background I have a 2-dimensional dataset $\{y_i, x_i\}_{i=1}^N$ in the coordinates $y,x$. I'm trying to fit the dataset with the trivial model $$\tag{*}y=mx$$ where $m$ is a (scalar) parameter that ...
matteogost's user avatar
0 votes
0 answers
24 views

What's the interpretation of the intercept of a linear mixed effect model with only one group

I'm trying to understand the meaning of the intercept term in a linear mixed model. The caveat here is that the dataset the LME will be built upon will only consist of subjects from a single class (...
Syuma's user avatar
  • 73
3 votes
1 answer
30 views

Converting Adjusted R²

I just examined the $R^2_\text{adj}$ Formula on Wikipedia and found two ways to calculate the adjusted $R^2$. Firstly as $$R^2_\text{adj}=1-\frac{\frac{SS_\text{res}}{(n-p-1)}}{\frac{SS_\text{tot}}{(n-...
Linus's user avatar
  • 53
0 votes
0 answers
28 views

Compare the intercept and y-intercept of the new model with the old model

If we have two simple linear models (old and new), is there a good way to evaluate whether the newly developed model is sufficiently different from the previous model? For example, let's assume there ...
S. Jeon's user avatar
  • 35
0 votes
1 answer
49 views

Expectation of residuals in linear regression

Consider a linear regression model $$ Y=X\beta + \epsilon, $$ where $Y\in R^n$, $X = (x_1,...,x_n)^T\in R^{n\times p}$ are i.i.d. $p$-dimensional observations, $\beta\in R^p$, and $\epsilon = (\...
Hepdrey's user avatar
  • 63
1 vote
1 answer
84 views

How to show that OLS and MLE are the same for $\beta$

I have several questions regarding this proof: Shouldn't the $\propto$ be used instead of the $=$ when we leave out the $\frac{1}{\sqrt{2\pi\sigma²}}$ ? Is a maximization problem simply inverted to a ...
BlankerHans's user avatar
2 votes
0 answers
38 views

Interpreting eigenvalues of non-normalized covariance matrix of physical system

Cross-posted from physics stackexchange Summary: Eigenvalues of a "non-normalized" covariance matrix of time-series measurements from a linear system have units of Action (energy * time). ...
user3716267's user avatar
5 votes
2 answers
183 views

How to deal with correlated variables

I would like to know how to deal with correlated variables, with this kind of correlation matrix: Is there a way to combine the correlated variables such as all the AV.. variables or FF.. variables? ...
Siueg's user avatar
  • 51
1 vote
1 answer
36 views

How to estimate the coefficients in OLS (all steps) [duplicate]

I'm a B.S. Math graduate who likes to (attempt) to teach myself statistics on my own time because I can't afford a masters degree. It really bothers me when I can't understand at a fundamental level ...
AdmiralMunson's user avatar
1 vote
1 answer
89 views

Is Linear Regression a good algorithm or even applicable with the distribution shown in the scatter plot I have shared in this question?

I am trying to use Linear Regression on a dataset using scikit-learn with python. And my understanding is that Linear Regression requires "some linearity" to exist between independent and ...
letdatado's user avatar
  • 325
0 votes
1 answer
74 views

In a linear model, why do we have $-2X^T \vec{y} + 2X^T X \vec{\beta}=0$? [duplicate]

When we derive the estimates of $\vec{\beta}$ such that they minimize the sum of squared error ($SSE$) we begin with $\sum_{i=1}^{n} (y_i - (\beta_0 + \beta_1x_1 + ... + \beta_kx_k))^2$. This is ...
AdmiralMunson's user avatar
1 vote
1 answer
57 views

Exact Meaning of Controlled Variables in terms of the OLS estimators

Consider a two-regressors linear regression model: $$Y=\beta_0+\beta_1X+\beta_2D+u$$ where $X$ is continuous and $D$ is binary. In this case, we say that we have controlled $D$. Here, I want to know ...
MinChul Park's user avatar
11 votes
7 answers
2k views

Why do we use Linear Models when tree based models often work better than linear models?

In Supervised Machine Learning, and specifically on Kaggle, it is usually seen that tree models often outperform linear models. And even in the tree-based models, it is usually XGBoost that ...
letdatado's user avatar
  • 325
0 votes
0 answers
19 views

Is there a method to estimate the distribution of error term in linear model?

Consider the linear model where $A$ is not known $$ y = Ax + \epsilon $$ where we want to estimate the distribution $\epsilon$ from a set of samples. To prevent over-fitting, we want to impose some ...
Ma Joad's user avatar
  • 163
1 vote
0 answers
59 views

Linear model for maximizing rank correlation between observed and predicted response

linear regression is modelled as $$Y = X\beta + \epsilon$$ for response variable $Y$ (vector), design matrix $X$, and iid Gaussian noise $\epsilon$ (vector). instead of minimizing the mean squared ...
fool's user avatar
  • 2,440
0 votes
0 answers
18 views

Difference in tests for linear models and significance levels adjustment

Context Imagine we measure two outcomes $\mathbf{Y}$ and $\mathbf{Z}$ that are real vectors. We also have the following covariates $G,T, X_1,\cdots,X_k$ for a given integer $k>1$. $G$ represents a ...
lulufofo's user avatar
  • 482
1 vote
1 answer
63 views

How can we think about linear regression geometrically?

The simple linear regression model is given by $y_i = \beta_0 + \beta_1x_1 + e$ It is my understanding that it can be rewritten in matrix vector form as $\vec{y} = X\vec{\beta} + \vec{e}$ where $X$ is ...
AdmiralMunson's user avatar
1 vote
0 answers
59 views

How to mathematically prove the "transitive property of nested predictors"?

QUESTION I am studying the structure of experiment data sets, and I want to propose a rule that I call the "transitive property of nested predictors". The general idea is that… if there are ...
Chris Science's user avatar

1
2 3 4 5
51