# Questions tagged [linear]

For statistical topics which involve the assumption of linearity, for example, linear regression or linear mixed models, or for the discussion of linear algebra as applied to statistics.

1,315 questions
Filter by
Sorted by
Tagged with
1 vote
24 views

### What is the definition of a non-linear estimator? I heard that ratio of estimators is non-linear

Why don't we consider nonlinear estimators for the parameters of linear regression models? says that LASSO is a non-linear estimator. I think LASSO has a solution via matrix multiplication. I don'...
16 views

### Regression with paired repeated measures design

I have the following data: ...
30 views

### What's the difference between statsmodels' RLM and robustbase's glmrob?

The Python package statsmodels comes with robust models of linear regression (RLM, https://www.statsmodels.org/stable/rlm.html). ...
129 views

### Intuition for $RSS_2 - RSS_1$ having chi-square distribution in F-test for linear models

In https://en.wikipedia.org/wiki/F-test#Regression_problems, an application of the F-statistic to comparing linear models is given: Consider two models, 1 and 2, where model 1 is 'nested' within ...
• 153
1 vote
105 views

### Elastic Net Collinearity

When performing linear regression it is often assumed that the predictors are independent with Gaussian noise: \begin{equation} Y = X\beta + \epsilon \quad \epsilon \sim \mathcal{N}(0, \sigma) \end{...
• 121
28 views

### Interpreting plot of interaction effects

In this vignette, the plot_model function is described and some examples are given on how to plot two way interactions along with confidence bands. I am wondering how to bets interpret such plots. For ...
• 1,146
1 vote
23 views

### RMSE and R2 with different training splits

I am running 2 linear regression models using the same data with different data splits. n=205 70/30 split RMSE: 2341 R2: 0.85 50/50 split RMSE: 2474 R2: 0.88 Seems counterintuitive that the R2 ...
464 views

### Why is a linear regression not linear when you plot it?

I can't find a proper explanation for my question on Cross Validated. The closest explanation was this one from Medium, but still, I don't see the difference visually among the four cases in that ...
• 343
44 views

### How do you optimize multiple objective functions simultaneously?

For example, suppose we want to maximize the 3 expressions on the right, subject to some constraints. To give some context, this is a problem about generating prototypes in unsupervised learning. In ...
1k views

### Why do we need regularization for linear least squares given that a line is the simplest model possible?

In linear least squares we are trying to fit a line to data. A line is the simplest model that can be fit to the data. How is it possible for a linear model to over-fit the data? In short why do we ...
• 231
23 views

### Linear mixed effect model for time series data

I am analyzing 9 years of data on methane uptake (consumption) in forest soil. The measurements were done two times each month. I want to check if there is any difference between the years by ...
1 vote
9 views

### Linear Regresssion: Multiple variables or build multiple models?

Hypothetical example: I want to regress a vehicle's fuel efficiency to the following variables: Vehicle type (car, truck, van, SUV) Year built Weight Manufacturer Gas vs Diesel etc.. When (if ever) ...
• 111
43 views

### The violation of the normality in simple linear regression model

I ran simple linear regression models, however my model could not meet all the assumptions (e.g., the normality of the residuals, the homogeneity of the variance). I know that both are quite important ...
• 101
14 views

### Simple linear regression with stochastic regressors formulation through conditional expectation

Just recently I discovered there could be deterministic and stochastic regressors. Could somebody please correct me if my following reasoning is off? The conditional expectation $\mathbb{E}[Y|X]$ ...
• 1
14 views

### Polygenic Categorical Variable (Allele Presence/Absence) within Linear Regression

I want to understand some differences to represent the allele presence/absence with lm() function in R. As a dependent variable, I have the ...
29 views

### Non-normal residual distribution using mixed models

I am pretty much a beginner in statistics/R and I would need your advice. I have tried to look for the answer for my problem (believe me, I prefer to search rather than post) but it only increased my ...
• 21
32 views

### Is it possible to calculate prediction interval without info on the predictor x?

I can see from here that prediction interval for a new response Y (setting is simple linear regression) is However I've read here that Apparently, no calculation related to x is needed according to ...
• 393
24 views

### Does inverse of linear regression also work for prediction of input?

for y=Xβ, we often find β and use it for predicting output (y). Then if I calculate (pseudo)inverse of β, can I use it for predicting inputs (x) with known y? Will the answer of this question change ...
1 vote
18 views

### Estimating interactions from non-interacting features

Suppose I have a sample $\mathcal{D}=\{(\mathbf{x}^{i}, y^{i})\}_{i=1\dots M}$ of binary variables $\mathbf{X}$ ($N$ of them) and a continuous variable $Y$ that I want to predict based on a linear ...
• 272
46 views

### Converting effect size from linear and logistic regression

I want to calculate a same effect size from a linear regression and a logistic regression to be able to compare them. More specifically, I would like to calculate standardized mean-difference (SMD). I ...
21 views

### Multiple Linear Regression - Can independent variables with a very weak relationship to the DV be used in a model?

To conduct a MLR it is required that each variable has a linear relationship to the DV. However, in my current study, the variables have a weak relationship with the DV. This makes it quite hard to ...
1 vote
21 views

### High correlation between predictor and outcome

I am fitting a linear model which is trying to predict a certain quantitative variable (volume after treatment). I am trying to make inference in which other variables influences this volume. One of ...
103 views

### Is one-hot encoding required for a binary categorical variable?

We are performing multiple linear regression. Dataset: let's call the response Y and the predictors X1, ...
• 151
287 views

• 35.3k
52 views

### Multiple linear regression: Do all independent variables need to have good adjusted R-squared independently?

I'm very sorry if this should be obvious, I'm just feeling a little lost with this assignment.. I have four independent variables X1,X2,X3,X4 plus a constant, modelled against Y. I know X4 to be ...
1 vote
31 views

### How to compare the goodness of fit between linear and logit? Why linear deviance is less than logit?

How can I evaluate which model - between linear and logit - determine the best fit to the data? The models use the same input variables and I thought that comparing the deviances was the proper choice ...
• 11
16 views

### Biased, linear MMSE estimator from biased measurement data?

I am trying to find out if what I am looking at is a known problem. I am considering the case of weighted least squares, and I am trying to find the optimal weights of biased measurements. I have ...
• 101
1 vote
20 views

### Question about confidence interval and standard errors on graphs

So I made a graph in R, it’s a linear regression of the same sample at different concentrations. It has a confidence interval built into it of 95%. Then I had to determine the concentration of an ...
• 11
22 views

### which model should be used for calculate the residuals of DV in twin data

I want to use the residuals of the dependent variables as new variables for analyses. these variables are continuous mammographic density measures from twin data. so which model should I use for ...
• 143
1 vote
214 views

### Linear Regression with Lasso Regularization by using scikitlearn and scipy.optimize

i am trying to apply lasso linear regression with both scikitlearn and scipy.optimize min method. However, i cannot reach same result. Code that i created with scipy.optimize can't shrink redundant ...
• 13
1 vote