# Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

2,452 questions
Filter by
Sorted by
Tagged with
26 views

### How to accomodate for missingness in column dependent on value of other column

Please run this code in order to create a reproducible example: ...
42 views

### Calculating the distribution of the sum of the squares of the predictors in linear regression

I'm calculating the distribution of the sum of the squares of the components of the MLE $hat{\beta}$ in linear regression with normal errors. We are assuming that $\beta = 0$. The distribution of the ...
1 vote
63 views

### Multiple predictors that measure the same concept in regression

I have used three questionnaires in a study that all measure musical training. Each of these three questionnaires (MT1/MT2/MT3) consist of various Likert scales that are averaged to calculate the ...
37 views

### How to reconcile these two matrix equations for obtaining the coefficients for a linear least squares fit?

In ordinary least squares linear regression, given a set of data points $(x_1,y_1),(x_2,y_2),...(x_N,y_N)$, that we want to fit to the function $y=\beta_0 + \beta_1 x$, we would usually write the ...
34 views

### Too good results with linear regression on a non-linear dataset due to training on seen data?

I plotted some time series data that looks non-linear as can be seen below. ![Text] [Looks pretty non-linear, but I decided to implement a linear regression model for learnings sake. ...
1 vote
22 views

### Statistical methods for ranking units

The problem: Suppose I have an ordered sample of $n$ observations (e.g. a playlist of songs) that are ranked according to some latent (unobserved) feature, for each of which $p$ covariates are ...
23 views

1 vote
18 views

1 vote
164 views

### The uncertainity about the weight matrix in linear model

Textbook literature often denotes the estimated weight matrix of a linear regression model $y = Wx + \epsilon, \epsilon \sim \mathcal{N}(0,\sigma^2)$ by $\hat W$ due to the inherent variability in the ...
12 views

### Understand inconsistent results between direct effect of mediation analysis and linear regression

I have a question concerning different results between linear regression and the direct effect of mediation analysis. Indeed, linear regression between predictor X and outcome Y showed a non-...
29 views

### Is there an analytic solution for linear classifier?

Is it possible to calculate the weight vector anlyticaly for linear classifier? Just like we can do it for linear regression where $w_* = (X^T*X)^{-1} * X^T * y$ is the vector of weights.
1 vote
21 views

### Presenting separate linear regression models in one regression table? [closed]

I've created this regression table, but I'm unsure if this is the correct way to present my results. The dependent variable is the proportion of each frame, and the independent variable is time (from ...
18 views

### Building a hybrid model? From a Random Forest and a OLS linear regressions

Cureently, I am conducting a regression study of household expenditure (target variable) from a set of determiants (income, household size, ...) in Malaysia using OLS and Random Forest. It is a long ...
1 vote
36 views

### Fixed or random effect in linear mixed effect

Let say I've a dataframe as : image_id image_group group x y nn I've 20 images defined by image_id ; each image belongs to a group G1 or G2 defined in the ...
1 vote
10 views

### How to apply spatial correction to field data based on many randomly dispersed check varieties with varying levels of replication?

I have inherited a dataset with a truly chaotic trial design. 32/155 varieties are duplicated at least once, most having 2-4 reps, but one is repeated x5, another x8, and another x19. I was told that ...
42 views

### RStudio: Help on Linear Mixed Models [closed]

I am a beginner in linear mixed models on RStudio and would like some advice on what I would like to do with my data. I work in the field of cognitive neuroscience and my research focuses on ...
1 vote
10 views

### RM-ANOVA vs Mixed Model

I am trying to find the best analysis approach for my data. Simply put, I am looking at a treatment for the impacts of drug exposure before birth. My groups are: 1 - no drug, no treatment 2 - drug, no ...
10 views

### Analysing three-directional relationships

Suppose there are three variables A, B and C, that are thought to have a three-directional relationship. This means A is related to B and C, B also influences A and C, and C affects A and B. I ...
39 views

### Is the average of residuals zero in a parabolic regression?

This post answers for when you fit your data with a line, but my teacher's notes at university claim that the mean of residuals is zero for all linear (in its parameters) models, such as the parabolic ...
1 vote
38 views

### Propagation of uncertainty in a derivative of a function [duplicate]

I've performed an ordinary least squares on a data set with one variable. For simplicity, let's say I've fitted a polynomial function $$f(x)=a+bx+cx^2+dx^3.$$ I obtain the best fit and the standard ...
56 views

### Why is my polynomial regression with gradient descent not overfitting?

I wanted to implement linear regression with gradient descent from scratch and demonstrate how you can overfit when using too many polynomials. Unfortunately my model does not really overfit the data. ...
95 views

### Subgroup analysis: post hoc test interpretation

I am modeling treatment effect in a hypothetical case where only a subset of the sample has disease-related impairment on the outcome of interest. I only expect treatment effects in this subset, but ...
1 vote
31 views

### No significant interaction in lmer - do you refit the model without interaction?

I'm running/ reporting lmer analysis for the first time and have a hypothetical question related to a few of my models. I'm using R lmer package - my models are generally of the formula: ...
28 views

### Covariance of linear models with Second Order Assumptions [duplicate]

I have been studying linear models recently and I'm confused why $cov(Y) = cov(\epsilon)$ holds for $Y = X\beta + \epsilon$. This was just kinda assumed in my course notes I was looking at this ...
1 vote
43 views

### Multidimensional linear regression (not multiple linear regression)

Let $p$ be a positive integer and suppose that each observation in my data set is a length-$p$ multivariate normal vector, and I have $n$ (an integer) observations of the length-$p$ multivariate ...
827 views

### Linear regression vs. average of slopes

Suppose that we want to know how the price of a house changes per meter square of the area of the house. Further suppose that I have a dataset as the following: ...
1 vote
28 views

### Holding covariates constant to plot MLR model on 2d scatterplot in R

I am currently working with a MLR model comprising 1 numeric/continuous predictor variable (x1), several nominal categorical variables (x2 ... xi), and an interaction term between the continuous ...
18 views

### How to choose appropriate parameter vector for linear regression?

If we have a dataset $D := f(x_i, y_i)^n_{i=1}$ where $x_i = [x_{i_1}, x_{i_2}, ... , x_{i_p}]^T$ is a p-dimensional predictor and $y_i \in R$ is the response to $x_i$. Now, shall we select as our ...
1 vote
15 views

### Does min-norm least squares solve regular least squares in some basis?

For a data matrix $X$ of dimension $n \times p$ where $p > n$ and corresponding label vector $y$ of dimension $n$, the standard least squares fit, $\hat{\beta} = (X^TX)^{-1}X^Ty$, is ...
131 views

### Linear regression with Cauchy distribution for errors

I have ran the below linear regression model and using the performance package in R I however checked whether the distribution of the residuals is normal. The performance package suggests I should be ...
9 views

### can I put a time(months) variable as interaction term in linear fixed effects model to account for time-varying changes?

This is what my data looks like. ...
60 views

### Large sample limit of linear and ridge regression

I wanted to check if these reasonings are correct. The formula for a multilinear regression, input $X_{s,i}$, where $s$ is the sample and $i$ the features, and output $Y$, is given by: \beta=(X^+X)^{...
13 views

### Multivariate linear regression project into the future

I have following data columns: output, year, data1, data2...
8 views

### Difference between LDA and LR? [duplicate]

I want a brief difference about Linear Discriminant Analysis and Linear Regression. Isnt it the same process? I heard the difference is that when it comes to multidimensional target/categories LDA is ...
22 views

### Regression model comparison query - AIC suggesting a non-significant model is better than an alternative significant model

Wondering if anyone can help. I’m trying to compare two regression models with one predictor to see which best describes the data. Model one is a linear model (y = ax + b) with R2 = .036, F = 3.047, p ...
1 vote
22 views

### Predicted dependent variable with some conditions in R

I have a data including province, district, number of household in need (nohhinneed) (dependent variable), a score about socioeconomic status (ses), total number of households (sumhh), etc. When I ...
90 views

### Practical power analysis on linear model

Power analysis for linear model test asks for an analytical expression of the power of a linear model. The answer claims that Note that this depends on the correlation between $X$ and $T$, $r_{TX}$. ...
19 views

### Nested Fixed Effects

I have been reading up on nesting in Linear Mixed Effects modelling, and typically nesting is for random effects. However, if I want to estimate the effects of language and type of word for each ...
20 views

### Constraint Linear regression in finance

Below you can find the problem that I am trying to understand. My main problem is to understand where do the reparametrization that they propose come from. Alternatively I have try to connect it with ...
65 views

### How to correctly set up my mixed-effect model?

I have data on days in which the greening of trees happen across America in 2015. This includes meteorological and topography data etc. I want to predict the day of greening happens through a linear ...
33 views

### Forecasting day-ahead electricity prices using a linear time series regression

I have a dataset of hourly day-ahead electricity prices and hourly forecasted day-ahead demand (from governmental agency) in the Norwegian price area NO2. I am trying to use the forecasted day-ahead ...
1 vote
25 views

### Combine variance from two linear models

I used here linear models for simplifying my final approach. Let's assume that a first linear model is estimated to obtain a response that will be then used as covariate in a successive linear model. ...
1 vote
26 views

### If we have a binary variable in our linear regression, the VIF for its coefficient estimate uses the $R^2$ of a linear probability model. What gives?

The variance inflation factor (VIF) in an ordinary least squares linear regression coefficient is calculated using the $R^2$ of a linear model that uses the other features to predict the feature to ...
33 views

### Only observing sign of the output of a linear model under Gaussian assumption

Suppose the linear model is $y = \beta x + \epsilon$, where $X \sim \mathcal{N}(0, 1), \epsilon \sim \mathcal{N}(0, s^2)$. If we only observe the sign of the output $y_i$, and the number of ...