Questions tagged [linear]

For statistical topics which involve the assumption of linearity, for example, linear regression or linear mixed models, or for the discussion of linear algebra as applied to statistics.

Filter by
Sorted by
Tagged with
1 vote
1 answer
24 views

What is the definition of a non-linear estimator? I heard that ratio of estimators is non-linear

Why don't we consider nonlinear estimators for the parameters of linear regression models? says that LASSO is a non-linear estimator. I think LASSO has a solution via matrix multiplication. I don'...
0 votes
0 answers
16 views

Regression with paired repeated measures design

I have the following data: ...
0 votes
1 answer
30 views

What's the difference between statsmodels' RLM and robustbase's glmrob?

The Python package statsmodels comes with robust models of linear regression (RLM, https://www.statsmodels.org/stable/rlm.html). ...
  • 1
5 votes
2 answers
129 views

Intuition for $RSS_2 - RSS_1$ having chi-square distribution in F-test for linear models

In https://en.wikipedia.org/wiki/F-test#Regression_problems, an application of the F-statistic to comparing linear models is given: Consider two models, 1 and 2, where model 1 is 'nested' within ...
  • 153
1 vote
1 answer
105 views

Elastic Net Collinearity

When performing linear regression it is often assumed that the predictors are independent with Gaussian noise: \begin{equation} Y = X\beta + \epsilon \quad \epsilon \sim \mathcal{N}(0, \sigma) \end{...
  • 121
2 votes
1 answer
28 views

Interpreting plot of interaction effects

In this vignette, the plot_model function is described and some examples are given on how to plot two way interactions along with confidence bands. I am wondering how to bets interpret such plots. For ...
  • 1,146
1 vote
1 answer
23 views

RMSE and R2 with different training splits

I am running 2 linear regression models using the same data with different data splits. n=205 70/30 split RMSE: 2341 R2: 0.85 50/50 split RMSE: 2474 R2: 0.88 Seems counterintuitive that the R2 ...
2 votes
2 answers
464 views

Why is a linear regression not linear when you plot it?

I can't find a proper explanation for my question on Cross Validated. The closest explanation was this one from Medium, but still, I don't see the difference visually among the four cases in that ...
  • 343
2 votes
1 answer
44 views

How do you optimize multiple objective functions simultaneously?

For example, suppose we want to maximize the 3 expressions on the right, subject to some constraints. To give some context, this is a problem about generating prototypes in unsupervised learning. In ...
13 votes
3 answers
1k views

Why do we need regularization for linear least squares given that a line is the simplest model possible?

In linear least squares we are trying to fit a line to data. A line is the simplest model that can be fit to the data. How is it possible for a linear model to over-fit the data? In short why do we ...
  • 231
0 votes
0 answers
23 views

Linear mixed effect model for time series data

I am analyzing 9 years of data on methane uptake (consumption) in forest soil. The measurements were done two times each month. I want to check if there is any difference between the years by ...
  • 1
1 vote
0 answers
9 views

Linear Regresssion: Multiple variables or build multiple models?

Hypothetical example: I want to regress a vehicle's fuel efficiency to the following variables: Vehicle type (car, truck, van, SUV) Year built Weight Manufacturer Gas vs Diesel etc.. When (if ever) ...
0 votes
0 answers
43 views

The violation of the normality in simple linear regression model

I ran simple linear regression models, however my model could not meet all the assumptions (e.g., the normality of the residuals, the homogeneity of the variance). I know that both are quite important ...
  • 101
0 votes
0 answers
14 views

Simple linear regression with stochastic regressors formulation through conditional expectation

Just recently I discovered there could be deterministic and stochastic regressors. Could somebody please correct me if my following reasoning is off? The conditional expectation $\mathbb{E}[Y|X]$ ...
  • 1
0 votes
0 answers
14 views

Polygenic Categorical Variable (Allele Presence/Absence) within Linear Regression

I want to understand some differences to represent the allele presence/absence with lm() function in R. As a dependent variable, I have the ...
2 votes
0 answers
29 views

Non-normal residual distribution using mixed models

I am pretty much a beginner in statistics/R and I would need your advice. I have tried to look for the answer for my problem (believe me, I prefer to search rather than post) but it only increased my ...
  • 21
0 votes
0 answers
32 views

Is it possible to calculate prediction interval without info on the predictor x?

I can see from here that prediction interval for a new response Y (setting is simple linear regression) is However I've read here that Apparently, no calculation related to x is needed according to ...
  • 393
0 votes
0 answers
24 views

Does inverse of linear regression also work for prediction of input?

for y=Xβ, we often find β and use it for predicting output (y). Then if I calculate (pseudo)inverse of β, can I use it for predicting inputs (x) with known y? Will the answer of this question change ...
1 vote
1 answer
18 views

Estimating interactions from non-interacting features

Suppose I have a sample $\mathcal{D}=\{(\mathbf{x}^{i}, y^{i})\}_{i=1\dots M}$ of binary variables $\mathbf{X}$ ($N$ of them) and a continuous variable $Y$ that I want to predict based on a linear ...
  • 272
0 votes
0 answers
46 views

Converting effect size from linear and logistic regression

I want to calculate a same effect size from a linear regression and a logistic regression to be able to compare them. More specifically, I would like to calculate standardized mean-difference (SMD). I ...
  • 1
0 votes
2 answers
21 views

Multiple Linear Regression - Can independent variables with a very weak relationship to the DV be used in a model?

To conduct a MLR it is required that each variable has a linear relationship to the DV. However, in my current study, the variables have a weak relationship with the DV. This makes it quite hard to ...
1 vote
1 answer
21 views

High correlation between predictor and outcome

I am fitting a linear model which is trying to predict a certain quantitative variable (volume after treatment). I am trying to make inference in which other variables influences this volume. One of ...
3 votes
3 answers
103 views

Is one-hot encoding required for a binary categorical variable?

We are performing multiple linear regression. Dataset: let's call the response Y and the predictors X1, ...
  • 151
4 votes
2 answers
287 views

Multivariate Probability Distribution with Linear Conditional Expectation

I want to know what probability distribution has the linearity property of the conditional expectation. To be specific, suppose that we have three random variables named $v_1,\;v_2,\;v_3$. Then, if $[...
2 votes
1 answer
48 views

Linear mixed model with unstructured repeated measures

I have a dataset with some growth measurements pre and post vaccination and 3 different groups (3 different types of vaccine) so there are both a within (pre/post) and between factor (group). Problem ...
0 votes
0 answers
59 views

Linear Hypothesis Tests

I have questions about linearHypothesis() function in the car package. I would like to see the post hoc comparisons of the interaction A:B, e.g. is A significant when B is "1"? I read the ...
  • 35
0 votes
1 answer
26 views

How to find a mapping to a higher dimension that separates the data, given a data set

We have the following dataset: $$ \begin{bmatrix} x_1 & x_2 & y\\ +1 & 0 & +1\\ -1 & 0 & +1\\ 0 & +2 & +1\\ 0 & +1 & -1 \end{bmatrix} $$ I was asked to find ...
2 votes
1 answer
34 views

Linear Separator in Higher Dimension - Theory Example Explanation

We have the following dataset: $$ \begin{bmatrix} x_1 & x_2 & y\\ +1 & +1 & +1\\ -1 & +1 & +1\\ 0 & -1 & +1\\ 0 & 0 & -1 \end{bmatrix} $$ I was asked to ...
1 vote
1 answer
24 views

Linear regression decimal residuals below and above don't match exactly

I'm completely new to statistics and I've been trying to learn it by watching some videos: https://youtu.be/ZkjP5RJLQF4?t=731 In that video, it says that if you sum the top residuals and bottom ...
  • 113
2 votes
1 answer
45 views

How does a small range of the dependent variable affects linear regression?

In my regression model, the dependent variable has just a range of 96-100 with a mean of 99.73. My questions are: 1.) (How) Does this affect the quality of the linear regression model? 2.) What would ...
  • 95
1 vote
1 answer
34 views

Linear regression with Poisson distributed error term?

I am working on gut microbiome data (counts) and I came across a paper where they are trying to predict bacteria counts in time using a linear regression model with Poisson distributed error term. I ...
  • 11
0 votes
0 answers
21 views

What is the formula for the p value of a tendline?

I can't find any no matter how hard I google. Every search result just tells me either how to calculate it on excel or explains what the p value is.
0 votes
0 answers
41 views

Linear Regression Model assumptions

I am trying to understand wether I am allowed to do a linear regression or if I am not respecting some assumptions. It is my understanding that it is possible to confirm all assumptions by looking at ...
  • 109
7 votes
3 answers
1k views

If X=Y+Z, Is it ever useful to regress X on Y?

If we have X and Y that are mathematically dependent: X = Y + Z, is it 'forbidden' to use Y as a predictor to X in linear regression? I'm trying to find a concise explanation for why it is, or isn't. ...
  • 85
0 votes
0 answers
28 views

Regression for linear model with dummy variable

I have the following linear model, for which I want to build a regression. $$ r_{i,t} = \alpha_i + \beta_i r_{m,t} + \gamma_i r_{m,t}d_t + \varepsilon_{i,t} $$ $r_{i,t}$ and $r_{i,t}$ are the returns ...
  • 1
1 vote
0 answers
26 views

How to test if both the mean and variance of the treatment effect is less than the control?

Statistics background I'm relatively new to experimentation and causal inference. I took up to graduate level stats in school but forget a ton after entering industry. Situation I am running an ...
  • 11
1 vote
2 answers
63 views

Simple linear regression exercise

I'm trying to do an exercise but I can't figure out how to proceed. OLS is run for this model, with 100 observations. $$ y_i=b_0+b_1x_i+\epsilon_i $$ The results are $$ \hat\beta =\begin{pmatrix} 9 \\...
0 votes
0 answers
31 views

What is the 1 standard deviation error in papers

I have recently come across some papers in my field. They had a large (X,Y) data set, it was binned into 4 bins. Least square method was used to find slope (m) and y intercept (b). In the table that ...
  • 21
1 vote
1 answer
35 views

Quantify the proportion of data not explained by explanatory variables in linear regression

Say I'm in a classic linear regression situation: $Y = \beta_0X_0 + \beta_1X_1 + \dots + \beta_NX_N + \epsilon$ And that I know that my explanatory variables $X$ only explain some part of my ...
1 vote
1 answer
34 views

Statistical test to check if data come from two different lines or just one

I have a very basic problem, can you give me some insights on how to solve it, or just some keywords to search online? I have two sets of points in $\mathbb{R}^2$, let say $S_1 = \{(x_i, y_i)\}_{i=1}^{...
1 vote
1 answer
61 views

Would the Mantel-Haenszel test for linear trend be appropriate for this data/hypothesis?

I am testing whether or not there is an association between the parents education level and the pupils grades in the oral exam in social studies in the last year of danish primary school from the ...
's user avatar
0 votes
1 answer
88 views

What does it mean if my confidence interval includes zero with a significant p value in linear regression analysis?

I performed linear regression analysis to assess the associations between continuous variables. I found a significant p-value but my confidence interval includes zero. What does it mean? Here are the ...
  • 61
3 votes
0 answers
50 views

Gauss-Markov with $p>n$

Let $p$ be the number of parameters in a linear regression model, let $n$ be the number of observations, and let $p>n$. $$\mathbb E[Y\vert X] = \beta_0 +\beta_1X_1 +...+\beta_pX_p$$ Does the Gauss-...
  • 35.3k
0 votes
1 answer
52 views

Multiple linear regression: Do all independent variables need to have good adjusted R-squared independently?

I'm very sorry if this should be obvious, I'm just feeling a little lost with this assignment.. I have four independent variables X1,X2,X3,X4 plus a constant, modelled against Y. I know X4 to be ...
  • 1
1 vote
0 answers
31 views

How to compare the goodness of fit between linear and logit? Why linear deviance is less than logit?

How can I evaluate which model - between linear and logit - determine the best fit to the data? The models use the same input variables and I thought that comparing the deviances was the proper choice ...
0 votes
0 answers
16 views

Biased, linear MMSE estimator from biased measurement data?

I am trying to find out if what I am looking at is a known problem. I am considering the case of weighted least squares, and I am trying to find the optimal weights of biased measurements. I have ...
  • 101
1 vote
0 answers
20 views

Question about confidence interval and standard errors on graphs

So I made a graph in R, it’s a linear regression of the same sample at different concentrations. It has a confidence interval built into it of 95%. Then I had to determine the concentration of an ...
  • 11
0 votes
0 answers
22 views

which model should be used for calculate the residuals of DV in twin data

I want to use the residuals of the dependent variables as new variables for analyses. these variables are continuous mammographic density measures from twin data. so which model should I use for ...
  • 143
1 vote
1 answer
214 views

Linear Regression with Lasso Regularization by using scikitlearn and scipy.optimize

i am trying to apply lasso linear regression with both scikitlearn and scipy.optimize min method. However, i cannot reach same result. Code that i created with scipy.optimize can't shrink redundant ...
1 vote
1 answer
17 views

Comparing Activities with regression

I am trying to find out what is the effect of activities(like jumping, weight lifting etc.) on behavior (such as attitude towards participating in a marathon). (sample size of 60 observations for each ...
  • 11

1
2 3 4 5
27