Questions tagged [linear-model]
Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.
2,452
questions
0
votes
1
answer
26
views
How to accomodate for missingness in column dependent on value of other column
Please run this code in order to create a reproducible example:
...
0
votes
2
answers
42
views
Calculating the distribution of the sum of the squares of the predictors in linear regression
I'm calculating the distribution of the sum of the squares of the components of the MLE $hat{\beta}$ in linear regression with normal errors. We are assuming that $\beta = 0$. The distribution of the ...
- 196
1
vote
1
answer
63
views
Multiple predictors that measure the same concept in regression
I have used three questionnaires in a study that all measure musical training. Each of these three questionnaires (MT1/MT2/MT3) consist of various Likert scales that are averaged to calculate the ...
- 61
0
votes
1
answer
37
views
How to reconcile these two matrix equations for obtaining the coefficients for a linear least squares fit?
In ordinary least squares linear regression, given a set of data points $(x_1,y_1),(x_2,y_2),...(x_N,y_N)$, that we want to fit to the function $y=\beta_0 + \beta_1 x$, we would usually write the ...
- 169
0
votes
1
answer
34
views
Too good results with linear regression on a non-linear dataset due to training on seen data?
I plotted some time series data that looks non-linear as can be seen below.
![Text] [Looks pretty non-linear, but I decided to implement a linear regression model for learnings sake.
...
- 1
1
vote
0
answers
22
views
Statistical methods for ranking units
The problem: Suppose I have an ordered sample of $n$ observations (e.g. a playlist of songs) that are ranked according to some latent (unobserved) feature, for each of which $p$ covariates are ...
- 56
0
votes
0
answers
23
views
Nonlinear transformation in simple linear regression and almost inverse function
Let $X$ be an independent variable and $Y$ the dependent variable. Suppose we have the relationship $Y = f(X) + \epsilon$ for some unknown function $f(x)$ and some noise $\epsilon \sim N(0,1)$. If $f(...
0
votes
0
answers
16
views
Interpreting r.regression.multi
This is the data output of the tool r.regression.multi on QGIS. I am new to this and cannot find good sources on how to draw conclusions for this. Therefore could someone explain; how adequately do ...
- 1
1
vote
0
answers
25
views
Do most GLM predictors' effects depend on other covariates' values because of non-linear link function, error distribution, or both?
For many generalized linear models (GLMs), the effect of changing a predictor $x_m \in X$ on the predicted mean outcome $E[\hat{y}|X]$ depends on the values of other predictors in the model $x_{n \neq ...
- 671
1
vote
1
answer
18
views
Drawing samples from a joint distribution defined by limits?
Assume that I want to efficiently draw samples from a (for simplicity bivariate) joint distribution $p(x,y)$, with $x \in \mathbb{R}$ and $y \in \mathbb{R}$. I don't have a closed-form expression for $...
- 533
0
votes
1
answer
17
views
How is this linear model producing non-linear output?
I trained a 1 unit 1 layer (which I assume is limited to being a linear model) on temperature data, which follows a sinusoidal pattern over time. I expected this limited model to just produce a line ...
- 3
1
vote
4
answers
158
views
Why the OLS underestimates the variances of the coefficients
CONSEQUENCES OF HETEROSCEDASTICITY
$\textbf{1}$. The presence of heteroscedasticity does not make the OLS estimates of coefficients biased, but it causes the variances of OLS estimates to increase.
$\...
- 310
1
vote
1
answer
164
views
The uncertainity about the weight matrix in linear model
Textbook literature often denotes the estimated weight matrix of a linear regression model $y = Wx + \epsilon, \epsilon \sim \mathcal{N}(0,\sigma^2)$ by $\hat W$ due to the inherent variability in the ...
- 276
0
votes
0
answers
12
views
Understand inconsistent results between direct effect of mediation analysis and linear regression
I have a question concerning different results between linear regression and the direct effect of mediation analysis.
Indeed, linear regression between predictor X and outcome Y showed a non-...
0
votes
1
answer
29
views
Is there an analytic solution for linear classifier?
Is it possible to calculate the weight vector anlyticaly for linear classifier? Just like we can do it for linear regression where $w_* = (X^T*X)^{-1} * X^T * y$ is the vector of weights.
1
vote
0
answers
21
views
Presenting separate linear regression models in one regression table? [closed]
I've created this regression table, but I'm unsure if this is the correct way to present my results. The dependent variable is the proportion of each frame, and the independent variable is time (from ...
- 11
0
votes
0
answers
18
views
Building a hybrid model? From a Random Forest and a OLS linear regressions
Cureently, I am conducting a regression study of household expenditure (target variable) from a set of determiants (income, household size, ...) in Malaysia using OLS and Random Forest. It is a long ...
- 11
1
vote
1
answer
36
views
Fixed or random effect in linear mixed effect
Let say I've a dataframe as :
image_id image_group group x y nn
I've 20 images defined by image_id ; each image belongs to a group G1 or G2 defined in the ...
- 556
1
vote
0
answers
10
views
How to apply spatial correction to field data based on many randomly dispersed check varieties with varying levels of replication?
I have inherited a dataset with a truly chaotic trial design. 32/155 varieties are duplicated at least once, most having 2-4 reps, but one is repeated x5, another x8, and another x19. I was told that ...
- 11
2
votes
0
answers
42
views
RStudio: Help on Linear Mixed Models [closed]
I am a beginner in linear mixed models on RStudio and would like some advice on what I would like to do with my data. I work in the field of cognitive neuroscience and my research focuses on ...
- 21
1
vote
0
answers
10
views
RM-ANOVA vs Mixed Model
I am trying to find the best analysis approach for my data.
Simply put, I am looking at a treatment for the impacts of drug exposure before birth.
My groups are:
1 - no drug, no treatment
2 - drug, no ...
- 11
0
votes
0
answers
10
views
Analysing three-directional relationships
Suppose there are three variables A, B and C, that are thought to have a three-directional relationship. This means A is related to B and C, B also influences A and C, and C affects A and B.
I ...
- 133
0
votes
1
answer
39
views
Is the average of residuals zero in a parabolic regression?
This post answers for when you fit your data with a line, but my teacher's notes at university claim that the mean of residuals is zero for all linear (in its parameters) models, such as the parabolic ...
1
vote
0
answers
38
views
Propagation of uncertainty in a derivative of a function [duplicate]
I've performed an ordinary least squares on a data set with one variable. For simplicity, let's say I've fitted a polynomial function
$$f(x)=a+bx+cx^2+dx^3.$$
I obtain the best fit and the standard ...
- 11
0
votes
1
answer
56
views
Why is my polynomial regression with gradient descent not overfitting?
I wanted to implement linear regression with gradient descent from scratch and demonstrate how you can overfit when using too many polynomials. Unfortunately my model does not really overfit the data. ...
- 97
0
votes
1
answer
95
views
Subgroup analysis: post hoc test interpretation
I am modeling treatment effect in a hypothetical case where only a subset of the sample has disease-related impairment on the outcome of interest. I only expect treatment effects in this subset, but ...
- 145
1
vote
1
answer
31
views
No significant interaction in lmer - do you refit the model without interaction?
I'm running/ reporting lmer analysis for the first time and have a hypothetical question related to a few of my models. I'm using R lmer package - my models are generally of the formula:
...
- 115
0
votes
0
answers
28
views
Covariance of linear models with Second Order Assumptions [duplicate]
I have been studying linear models recently and I'm confused why $cov(Y) = cov(\epsilon)$ holds for $Y = X\beta + \epsilon$. This was just kinda assumed in my course notes
I was looking at this ...
1
vote
1
answer
43
views
Multidimensional linear regression (not multiple linear regression)
Let $p$ be a positive integer and suppose that each observation in my data set is a length-$p$ multivariate normal vector, and I have $n$ (an integer) observations of the length-$p$ multivariate ...
- 791
9
votes
2
answers
827
views
Linear regression vs. average of slopes
Suppose that we want to know how the price of a house changes per meter square of the area of the house. Further suppose that I have a dataset as the following:
...
- 1,174
1
vote
1
answer
28
views
Holding covariates constant to plot MLR model on 2d scatterplot in R
I am currently working with a MLR model comprising 1 numeric/continuous predictor variable (x1), several nominal categorical variables (x2 ... xi), and an interaction term between the continuous ...
- 31
3
votes
0
answers
18
views
How to choose appropriate parameter vector for linear regression?
If we have a dataset $D := f(x_i, y_i)^n_{i=1}$ where $x_i = [x_{i_1}, x_{i_2}, ... , x_{i_p}]^T$ is a p-dimensional
predictor and $y_i \in R$ is the response to $x_i$.
Now, shall we select as our ...
- 31
1
vote
0
answers
15
views
Does min-norm least squares solve regular least squares in some basis?
For a data matrix $X$ of dimension $n \times p$ where $p > n$ and corresponding label vector $y$ of dimension $n$, the standard least squares fit, $\hat{\beta} = (X^TX)^{-1}X^Ty$, is ...
- 506
2
votes
2
answers
131
views
Linear regression with Cauchy distribution for errors
I have ran the below linear regression model and using the performance package in R I however checked whether the distribution of the residuals is normal. The performance package suggests I should be ...
- 13.4k
0
votes
0
answers
9
views
can I put a time(months) variable as interaction term in linear fixed effects model to account for time-varying changes?
This is what my data looks like.
...
- 1
3
votes
1
answer
60
views
Large sample limit of linear and ridge regression
I wanted to check if these reasonings are correct.
The formula for a multilinear regression, input $X_{s,i}$, where $s$ is the sample and $i$ the features, and output $Y$, is given by:
$$\beta=(X^+X)^{...
- 820
0
votes
0
answers
13
views
Multivariate linear regression project into the future
I have following data columns: output, year, data1, data2...
- 101
0
votes
0
answers
8
views
Difference between LDA and LR? [duplicate]
I want a brief difference about Linear Discriminant Analysis and Linear Regression. Isnt it the same process? I heard the difference is that when it comes to multidimensional target/categories LDA is ...
- 1
2
votes
1
answer
22
views
Regression model comparison query - AIC suggesting a non-significant model is better than an alternative significant model
Wondering if anyone can help. I’m trying to compare two regression models with one predictor to see which best describes the data.
Model one is a linear model (y = ax + b) with R2 = .036, F = 3.047, p ...
1
vote
0
answers
22
views
Predicted dependent variable with some conditions in R
I have a data including province, district, number of household in need (nohhinneed) (dependent variable), a score about socioeconomic status (ses), total number of households (sumhh), etc.
When I ...
- 11
4
votes
1
answer
90
views
Practical power analysis on linear model
Power analysis for linear model test asks for an analytical expression of the power of a linear model. The answer claims that
Note that this depends on the correlation between $X$ and $T$, $r_{TX}$.
...
- 170
0
votes
0
answers
19
views
Nested Fixed Effects
I have been reading up on nesting in Linear Mixed Effects modelling, and typically nesting is for random effects. However, if I want to estimate the effects of language and type of word for each ...
0
votes
0
answers
20
views
Constraint Linear regression in finance
Below you can find the problem that I am trying to understand. My main problem is to understand where do the reparametrization that they propose come from. Alternatively I have try to connect it with ...
- 227
0
votes
1
answer
65
views
How to correctly set up my mixed-effect model?
I have data on days in which the greening of trees happen across America in 2015. This includes meteorological and topography data etc. I want to predict the day of greening happens through a linear ...
- 448
0
votes
1
answer
33
views
Forecasting day-ahead electricity prices using a linear time series regression
I have a dataset of hourly day-ahead electricity prices and hourly forecasted day-ahead demand (from governmental agency) in the Norwegian price area NO2. I am trying to use the forecasted day-ahead ...
- 11
1
vote
0
answers
25
views
Combine variance from two linear models
I used here linear models for simplifying my final approach. Let's assume that a first linear model is estimated to obtain a response that will be then used as covariate in a successive linear model. ...
- 11
1
vote
0
answers
26
views
If we have a binary variable in our linear regression, the VIF for its coefficient estimate uses the $R^2$ of a linear probability model. What gives?
The variance inflation factor (VIF) in an ordinary least squares linear regression coefficient is calculated using the $R^2$ of a linear model that uses the other features to predict the feature to ...
- 50.8k
2
votes
1
answer
33
views
Only observing sign of the output of a linear model under Gaussian assumption
Suppose the linear model is $y = \beta x + \epsilon$, where $X \sim \mathcal{N}(0, 1), \epsilon \sim \mathcal{N}(0, s^2)$. If we only observe the sign of the output $y_i$, and the number of ...
- 231
7
votes
2
answers
477
views
What type of regression to use when outcome is integers from 0 to 5
Say I have a questionnaire with 5 questions about anxiety. For each question, their response is rated a 1 or 0. Their total anxiety score is the sum, so an integer between 0 and 5.
Now I would like to ...
- 91
0
votes
1
answer
51
views
Investigate relationship between independent variables with multiple timepoints; repeated measures linear model?
I have a dataset of 2 variables collected repeatedly at 5 different timepoints on a group of individuals, structured like this:
...
- 123