Questions tagged [linear]
For statistical topics which involve the assumption of linearity, for example, linear regression or linear mixed models, or for the discussion of linear algebra as applied to statistics.
1,430
questions
0
votes
0
answers
7
views
Dealing with Dominant Past Sales Predictor in Linear Regression for Store Sales Inference
I am using linear regression to do inference and know how much each predictor affects sales.
In have data for several stores with features and sales during a certain time frame.
There is not much ...
- 101
0
votes
0
answers
34
views
Which linear model should I use?
I have data collected from an experiment in which 24 subjects did the same task 4 times. So in total I have 96 data points. My research hypothesis is that there are always linear relationships between ...
0
votes
0
answers
10
views
Can you use linear regression to predict individua or group player performance from team performance in a Public Good Game?
I am doing an analysis of the experimental data I have collected. In particular, I should do a panel regression.
I have collected data from my experiment on public goods. My variable of interest is ...
0
votes
0
answers
23
views
Two contradicting derivations of the Covariance Matrix for Linear Regression
I am looking to compute the variance covariance matrix for the standard linear regression coefficients $\hat{\beta}$ when:
$$Y = X \beta + \epsilon $$
and $\epsilon \sim N(0,\sigma^2)$. I have derived ...
- 610
4
votes
3
answers
90
views
Obtaining Residual Sum of Squares from a large OLS problem using a naive sequential approach - why doesn't it work?
Suppose we have an OLS problem with a large number of predictors: $Y = X_1 + X_2 + \cdots + X_p$.
I want to obtain its RSS. I don't need to know the regression coefficients or individual residuals, ...
- 2,780
0
votes
1
answer
49
views
Why do we differentiate the RSS with respect to minimizers?
I am having a hard time understanding simple linear regression. I got to a point, in this website to which I can see the closest answer to my question :
To minimize our error function, S, we must find ...
- 1
0
votes
0
answers
24
views
If the error term in a regression is squared can it still be a linear regression?
So basically I’ve been taught that in a linear regression model the parameters alpha and beta cannot be squared when defining the equation of our model, does this also apply for the error term (...
- 115
0
votes
0
answers
14
views
Understanding the kernel trick [duplicate]
I need to understand the kernel trick in order to understand methods like KRR and GPR for machine learning and I think I am getting too confused over some very basic questions. I have read in various ...
- 101
0
votes
0
answers
4
views
What is the conditions to use a weights argument to a linear model, when the dependent variable is a proportion?
My data consists of the independent variable (x) which is slope gradient (°) and the dependent variable (y) is collar GPS point density/km². For each slope gradient, the independent variable was ...
0
votes
2
answers
40
views
Should I transform my positively skewed predictor in hierarchical regression?
I'm doing a hierarchical regression trying to understand how intelligence (first predictor) and personality traits (second predictor) influence general knowledge (dependent variable). The problem is ...
- 1
0
votes
0
answers
39
views
Can linear regressions be used, given these diagnostic plots? [duplicate]
I'm using several linear regressions on a big dataset (about 1000 datapoints) with one numerical dependent variable and several independent variables (both dummy and numerical): ...
- 15
0
votes
1
answer
23
views
Questions about adding polynomial features to a dataset for linear regression
Apologies if this belongs to Data Science instead of here (I can move the question) but this seems related to the math aspect more than ML. In our course we just ...
- 111
0
votes
1
answer
25
views
Linear model with categorical variables (and constant variable) in R
I'm trying to estimate days spent in hospital (length of stay, continuous variable) based on a clinical severity score (...
- 103
0
votes
0
answers
26
views
All OLS assumptions not satisfied
I sorted some portfolios using the constituents of the S&P500 and then estimate the linear multi factor models. Now, based on my first criterion I constructed 3 portfolios where all of them had ...
2
votes
2
answers
125
views
Why is $\beta^{T}X_{c}^{T}X_{c}\beta$ divided by $(k-1)\sigma^{2}$ in multiple regression?
In single linear regression mmodel, to find expectation of regression, we use the following formula:
$$E[MSR] = \sigma^{2}+\hat \beta^{2}(X-\bar X)^{2}.$$
$MSR = \frac{\sum_{i=0}^n (\hat y-\bar y)^{2}}...
- 21
0
votes
0
answers
12
views
About Estimating Parameters from Unpaired Datasets
I possess three datasets: $x$, $y$ and $z$. It's hypothesized that a relationship exists between these variables, represented by the equation $z=a*x+b*y$.
My goal is to estimate the values of $a$ and $...
- 1
0
votes
0
answers
28
views
Model comparsion for robust linear mixed models (robustlmm)
I'm currently working on a project where I've fitted 4 robust linear mixed models. However, I've hit a bit of a roadblock when it comes to model selection. I've been using the AIC (Akaike Information ...
0
votes
0
answers
44
views
Linear Regression with Multiple Input and Output Variables, Matrix Invertibility Condition?
I'm trying to work out a linear regression model where both inputs and outputs are multidimensional.
Suppose we have $n$ input variables, $m$ output variables and $k$ observations. Each observation ...
- 59
0
votes
0
answers
28
views
Bootstrapping a linear mixed model with R's lmeresampler
I have a data of participants in 3 Groups, with 6,6,10 participants respectively. Each participant are measured 6 times in the combination of 2 conditions A and B, A of 2 levels and B of 10 levels. ...
- 15
0
votes
1
answer
61
views
Determine 'w' and 'b' in hard margin SVM
I have been asked the following question related to SVM (Hard Margin) in the exam, and I failed to answer it. Can anyone help me find the solution?
Consider the dataset M:
\begin{align*}
& \left(\...
0
votes
0
answers
8
views
Sensor To Sensor Variability - Generalizing One Sensor Calibration To Many
I have 7 different sensors of the same type that try to qualitatively estimate soil water content (SWC) based off of the capacitance of the soil/medium they're touching. However, I have read a few ...
0
votes
0
answers
8
views
Linear regression model: adding a boolean dependent on y [duplicate]
Ciao,
I have to perform an estimation of price fluctations. There are some outliers in the price, namely above 350€ and above 950€. My first feeling is to add two booleans representing prices between ...
- 1
4
votes
1
answer
83
views
How to calculate standard error of a regression coefficient given R output
Given the following regression equation
$mpg_i=\beta_0 + \beta_1 weight_i+ \beta_2 ln(hp_i)+ \beta_3 diesel_i + \beta_4 ln(torque_i) + \beta_5 ln(torque_i) + \beta_6 year_i+\epsilon_i$
with the ...
- 81
6
votes
2
answers
331
views
Mediation analysis with a log-transformed mediator
The very basic framework for mediation analysis (as I understand it) is below (DV = dependent variable, IV = independent variable):
Step 1: DV ~ IV
Step 2: Mediator ~ IV
Step 3: DV ~ IV + Mediator – ...
- 63
0
votes
0
answers
17
views
Do I want the linear regressions fixed, random or marginal effects as the "adjusted values"?
my apologies ahead of time if it's not as clear as I would like it to be. I'm using a linear mixed effect (nlme package) to determine the association between a modularity score (on a range of -1 to 1)...
7
votes
1
answer
281
views
Categorical variable in simple linear regression
I'm new to statistics. I have data that measures the effects of two different drugs (A and B) on the size of rabbits. The data consists of triplets $(\textrm{dose}, \mathrm{size}, \mathrm{type})$. I'm ...
- 193
0
votes
0
answers
32
views
Doing OLS, we can add new features by G-S orthogonalising. What is the fastest way to compare MSE improvement between two potential new regressors?
[Q] Working through ESL, looking at QR decomp for OLS. Lets say we want to add a few new features: we can iteratively add them by orthogonalising. If we only want to include a subset, how could you ...
- 1
0
votes
0
answers
79
views
Why is linear regression taught differently from what I have learned? [duplicate]
I took a machine learning course using the book "Learning from Data: A Short Course" by Hsuan-Tien Lin, Malik Magdon-Ismail, and Yaser Abu-Mostafa (LFD)
You are given a set of examples $\{...
0
votes
0
answers
22
views
Regression distribution of Residuals - Chi square test
I'm trying to solve a conundrum but can't figure out where I'm making the mistake.
Let me define the Linear model as
$$
\begin{align}
y &= X \beta + \epsilon \\
\end{align}
$$
Where $...
- 153
2
votes
0
answers
43
views
Comparing many event studies
I am studying the impact of an event on around 100 timeseries $i$ with weekly observations $t$.
$y_{i,t} = \beta_{1}\text{post-event} + \beta_{2} \text{week number} + \beta_{3} \text{(post-event X ...
- 121
2
votes
1
answer
204
views
Linear mixed model beta coefficient larger than one
My question is related to the linear mixed model. I have data on the built land area for three sampling sites from which we collected samples for 11 consecutive days. We collected the community of ...
- 21
0
votes
1
answer
44
views
Sampling distribution of ordinary least squares confusion
I was reviewing the derivation for the variance of ordinary least squares estimators and experienced some confusion.
$$
\Large
Var(\hat{\beta}) = \frac{\sigma^2}{\Sigma^n_{i=1}(X_i-\bar{X})^2}
$$
From ...
- 3
0
votes
0
answers
13
views
Linear regression with mean and standard deviation instead of values of each repetition possible?
I have a question about a linear regression I would like to perform.
In my experiment I have two treatments and a control which have 4 repetitions of 20 genotypes each. I have performed linear ...
- 1
1
vote
1
answer
51
views
Can All Regression Supervised Machine Learning Models Be Viewed as Linear Models Over Transformed Features?
I've been studying various supervised machine learning algorithms for regression tasks, and I've come across an interesting perspective suggesting all machine learning models could be represented as ...
0
votes
1
answer
43
views
The OLS prediction is different after standardizing the data
I have a dataset where $X\in N\times1$ is the carats and $y\in N\times 1$ is the diamond price in USD.
I modelled the data using $$
y\sim\mathcal N(Xw,\sigma)
$$I removed the bias from the model so ...
- 45
0
votes
0
answers
75
views
Exploding coefficients in a Linear Regression Model with single input
This is my first post here, so tips are appreciated for future posts!
For homework in my CS course I wanted to build a very simple Linear Regression Model (although this was not explicitly required) ...
1
vote
1
answer
39
views
real world example of two variables that cannot explain a dependent variable univariately but perfectly explain the dependent variable jointly
An interesting statistical curiosity I like is the following:
Suppose you have 3 non-random variables X, Y and Z.
Suppose that univariately X & Y can explain 2% and 5% of the variation in Z ...
- 121
2
votes
0
answers
41
views
Relationship between conditional expectation and regression
I would be grateful if you could help me clear up some confusion regarding conditional expectation and regression.
I have seen two formulations of the linear regression framework:
$$Y=a+bX+\varepsilon\...
1
vote
1
answer
24
views
Analysis of causal relationship of within-subject choice patterns among different groups using R - emmeans / hlm / Mixed Effects Logistic Regression
I am conducting a research and investigate the relationship between persona type (independent variable; a vs. b vs. c vs. d) and luxury perfume choice (dependent variable; niche vs prestige)(H1), ...
0
votes
2
answers
50
views
Calculating the distribution of the sum of the squares of the predictors in linear regression
I'm calculating the distribution of the sum of the squares of the components of the MLE $hat{\beta}$ in linear regression with normal errors. We are assuming that $\beta = 0$. The distribution of the ...
- 206
1
vote
1
answer
47
views
Is there a name for data that, when plotted, seem to have two (or more) distinct trends?
I have the following plot, which appears to have two pretty distinct trends.
The plots shown here and here depict similar situations. Is there a name for data that have two or more distinct trends, ...
- 225
0
votes
0
answers
15
views
Interaction effects in modeling with multicollinearity
I have a large model in r like:
sales = ai+ x1 + x2 + x3 + x4 +(x1x4) + (x2x4) + (x3*x4), where x4 is a dummy for a certain interventoin. As a result, i wanted to analyse the effect of x1,x2,x3 during ...
- 1
0
votes
0
answers
61
views
Multiple linear regression: F-test significant, but most predictors are not [duplicate]
In a multiple linear regression, what is the reason for the most predictors to be non-significant ($p > 0.05$), but ANOVA shows significances? What could be some of the reasons? Or a calculation ...
- 1
1
vote
1
answer
23
views
Drawing samples from a joint distribution defined by limits?
Assume that I want to efficiently draw samples from a (for simplicity bivariate) joint distribution $p(x,y)$, with $x \in \mathbb{R}$ and $y \in \mathbb{R}$. I don't have a closed-form expression for $...
- 545
0
votes
0
answers
19
views
Effects of multiple non-independent treatments
I need to run an analysis of efficiency of automated recommendations for call-centre agents on how to handle customer support tickets. It works like that:
A ticket get 0, 1 or more recommendations ...
- 1
1
vote
0
answers
23
views
Interpretation and validity of linear regression when dependent variables has negative, positive and lot of 0 values [closed]
I am new on this site, so excuse me, if I made any mistakes writing this post.
I want the describe the results of a linear regression of a variable "cost_difference" which has some positive ...
- 11
1
vote
0
answers
38
views
Finding inequality relationships in stochastic data
Consider random variables $X_i$ such that the following holds:
$$ a_1 X_1 + \dots + a_n X_n -k \geq 0 + \epsilon $$
where the $a_i$ are constants and $\epsilon$ is random noise.
How is it possible to ...
- 642
0
votes
0
answers
27
views
The variance of the predicted variable in a linear regression problem
Given the model $y = f(x) + \epsilon, f(x) = Wx$, I want to find an estimate of $Var(Y)$. Note here I don't account for the randomness in the input $x$, but rather I consider it a deterministic value.
...
- 236
0
votes
0
answers
17
views
How to transform one distribution into another using a linear transform of Mean and linear transform of Std?
I'm trying to account for a time correction between two streams of data, let's say Stream A and Stream B. Stream A and Stream B each have different message arrival/latency characteristics. Each ...
- 1
2
votes
0
answers
48
views
RStudio: Help on Linear Mixed Models [closed]
I am a beginner in linear mixed models on RStudio and would like some advice on what I would like to do with my data. I work in the field of cognitive neuroscience and my research focuses on ...
- 21