Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

0
votes
0answers
6 views

Proof of robustness of ``Linear Discriminant Analysis"

Assume one has been given $N$ data points in $\mathbb{R}^{d_1}$ each of which comes with a label from some set $\{1,\ldots,q\}$. Now I guess the claim is that doing (Linear Discriminant Analysis) LDA ...
1
vote
0answers
9 views

uncertainty in dependent variable at single covariate

I have a dataset consisting of responses of a dependent variable measured at the same independent variables over multiple trials. It looks something like trial $i$: $(x^{(i)}_1, y^{(i)}_1) = (1.0,\...
1
vote
0answers
20 views

Linear discriminant analysis model plot in R [on hold]

I'm reading the Introduction to statistical learning with R currently, but I blocked through a Lab about Discriminant analysis. So the thing is that we trying to fit a linear discriminant analysis ...
0
votes
1answer
70 views

Linear regression very significant βs with multiple variables, not significant alone

Could anyone provide intuition on why for y ~ β1x1 + β2x2 + β3x3, β1 β2 and β3 can be significant in a multiple variable model (p range 7x10-3 to 8x10-4), but the βs are not significant in separate ...
0
votes
2answers
19 views

Statistical measure for linear regression with two distinct clusters of points

In the following plot, I have a linear regression of 30 points, representing 10 treatments with three replicates each. As you can see, the r-squared value is quite strong (0.83) and the p-value is ...
0
votes
0answers
35 views

Linear regression - assume residual mean is zero

One of the assumptions of linear regression is that the residual mean is zero. As far as I can tell though, the residual mean is always zero i.e. it is not an assumption, it is a fact. The formula ...
0
votes
0answers
19 views

Fitting quadratic model for non-normal data

I am regressing between the body size and eye size values of different bat species. They are not linearly related (picture attached) and I want to fit a quadratic model to it but the values are non-...
1
vote
0answers
9 views

R: anova(lm): What is the Sum Sq when we have two inputs

What is the formula to calculate the Sum sq column for the inputs? Answers to some other questions state, that it should be $RSS = \sum (\hat Y_i -\bar Y)^2$, yet this is false, it only gives the ...
2
votes
0answers
69 views
+50

Difference Between Two Tikhonov Regularization Schemes

For the solution of $Ax = b$, where $A$ is a square matrix, what is the difference between these two regularized solutions: $x = (A + \alpha I)^{-1}b$ -- coressponding to eq.3 below $x = (A^TA + \...
0
votes
0answers
18 views

How to interpolate LogOR results from logit GAM

I am having trouble interpreting the results of crude and adjusted logistic GAM regression. My outcome variable is binary (0 or 1, event no/yes respectively). My predictor variable for crude model is <...
0
votes
0answers
17 views

Singular Normal matrix (GP for ML book)

In reading section 2.2, page 14 of this book, it appears the term sigular Gaussian which means that the covariance matrix is singular. But I wonder why this distribution is singular, I introduce the ...
0
votes
1answer
24 views

What does it mean when conditional mean is linear in predictors?

I am trying to understand this equation: If conditional mean is linear in $x$, so that $E[y|x] = x'\beta$... I don't understand why conditional mean ($E[y|x]$) should be equal to $x'\beta$ if its ...
0
votes
0answers
29 views

How to solve systems of linear equations with random variables? How to identify model parameters?

I want to learn know how to solve systems of linear equations with randomness. Example of a deterministic version of the sort of problem I want to solve: ...
1
vote
0answers
19 views

R2 - contribution of explanatory variables x1, x2 and x3 to the variance of the explained variable y

I wish to look at the contribution of explanatory variables x1, x2 and x3 to the variance of the explained variable y. To summarize the contribution of the explanatory variables alone to the ...
0
votes
0answers
10 views

Running a linear model on zero-inflated data — which type of error?

When I have a response variable made up of count data with lots of zeroes, I often run a negative binomial GLM to test hypotheses. An example: if I counted butterfly eggs on 100 plants, most plants ...
1
vote
0answers
18 views

Mixed integer programming R: Least absolute deviation with cost associated with each regressor

I have been presented with a problem, regarding the minimization of the absolute error, the problem know as LAD(Least absolute deviation) but, being each regressor the result of expenssive test with ...
1
vote
2answers
46 views

Method of moments for linear regression?

I have been reading about the method of moments, and now I understand how to obtain the method of moments estimator for a random sample $x_1,...,x_n$ from a distribution $f(x;\theta)$, in the ...
1
vote
1answer
67 views

Difference between linear regression and neural network

I am obviously confused with terms, and different concepts behind it. Each websites gives different intuitions. With all intuitions my brain is full of confusion now. Please help me to address what is ...
0
votes
0answers
11 views

Scale-Location Graph question

This graph is confusing me. I am assuming there is no equal variance because the residuals are spread out so much; however I am not sure. Please verify this for me.
0
votes
0answers
5 views

Low t Values Multicollinearity

I'm studying a model which has large datasets of empirical data and analysis available. In the particular dataset that I consideted (with a few a priori expectations) I tried out a simple linear ...
0
votes
0answers
75 views

Inference from regression in presence of multicolinearity

I would like to estimate the effect of one independent variable on the predicted variable in the purely observational study. On the other hand I know that there exists another independent variable, ...
0
votes
0answers
13 views

Estimability of 2way ANOVA

Consider the model, yijk = µ + Ai + Bij + eijk ; i, j, k= 1,...,5 I tried testing for estimability by trying to reducing the coefficient matrix. But the matrix is really big that it's getting ...
0
votes
0answers
19 views

Proving that $S_{Y_x}^2 = \frac{S_{xy}^2}{n-2}(1-r_{xy}^2)$

Exercise : Show that for the simple linear model, it is : $$S_{Y_x}^2 = \frac{\sum(y_i-\hat{y}_i)^2}{n-2} = \frac{1}{n-2}\bigg(S_{yy}-\frac{S_{xy}^2}{S_{xx}}\bigg) = \frac{S_{xy}^2}{n-2}(1-r_{xy}^...
0
votes
0answers
40 views

Proving that $V(\hat{y}_{x_0}) = \sigma^2\bigg[\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}\bigg]$ [duplicate]

Exercise : Prove that the variance of $\hat{y}_{x_0} = \hat{b_0} + \hat{b_1}x_0$ is : $$\text{Var}(\hat{y}_{x_0}) = \frac{\sigma^2\sum x_i^2}{n\sum(x_i-\bar{x})^2}+\frac{\sigma^2x_0^2}{\sum(x_i-\...
1
vote
1answer
9 views

What is the difference between a design matrix and a “factor loading” matrix?

I realise this is probability a naive question, but what exactly is the difference between your standard design matrix $X$ in $$y = X\beta + \epsilon$$ and a "factor loading" matrix $\Lambda$ $$y = ...
0
votes
0answers
43 views

Showing that $\sum_{i=1}^n y_i = \sum_{i=1}^n \hat{y_i}$

Exercise : Prove that for the Generalized Linear Model with a constant intercept $b_0$, the sum of the observed values equals the sum of the fitted values : $$\sum_{i=1}^n y_i = \sum_{i=1}^n \hat{...
1
vote
1answer
40 views

Linear Model - accuracy of predictions with confidence interval

I am fitting a linear model to predict a variable which is a type of performance of animal behaviour. Let's call it performance. When the model makes a prediction ...
0
votes
0answers
19 views

Can I average my standard deviation across the levels of the main effects when computing the difference standard deviation for an interaction?

I am currently trying to perform a meta-analysis of an interaction effect between 2 factors (2x2 ANOVA). I am using the raw mean difference as my effect size, because the interaction effect size is ...
4
votes
2answers
316 views

How useful are linear hypotheses?

In a linear model $Y=X\beta + \varepsilon$, one can easily test linear hypotheses of the form $H_0: C\beta = \gamma, $ where $C$ is a matrix and $\gamma$ is a vector with dimension equal to the number ...
1
vote
0answers
10 views

How to compute sigma in linear discriminant analysis

How to compute sigma in linear discriminant analysis? The formula that I found in Elements of Statistical Learning (https://web.stanford.edu/~hastie/Papers/ESLII.pdf) on page 109 is somewhat ...
1
vote
1answer
42 views

Linear regression model, SCE,SCT,SCM and model's error

Could you please check if what I've done is correct? and how could I improve some of them? Thank you in advance. Suppose I have the following data (the original data its like 20 data with decimal ...
1
vote
1answer
28 views

Prove that $\sum \hat y_i(y_i-\hat y_i)=0$ for linear regression model

Prove that $\sum \hat y_i(y_i-\hat y_i)=0$ for linear regression model. Attempt We have that $\sum \hat y_i(y_i-\hat y_i)=\sum x_i\hat\beta(x_i\beta-x_i\hat\beta)=(X\hat\beta)'(X\beta)-(X\hat\beta)'...
0
votes
0answers
31 views

MLR $H_0 : \beta_0 \geq 1$ versus $H_a : \beta_0 < 1$

Compute the Pvalue of the test $H_0: \beta_0 \geq 1$ versus $H_a: \beta_0 < 1$. $$Y_i = \beta_0 + \beta_1 X_{i1} + \beta_2 X_{i2} + \beta_3 X_{i1}^2 + \beta_4 X_{i1} X_{i2}+\sigma Z_i, Z_i \sim N(...
4
votes
1answer
30 views

Construct a $95\%$ confidence interval for $5\beta_4$

Construct a $95\%$ confidence interval for $5\beta_4$. If this question were about $\beta_4$ without the $5$, I would absolutely know what to do. But I have to idea how the $5$ comes into play. I can'...
1
vote
1answer
19 views

Coefficients in Anova

When considering anova as a linear model where the variables of the model are categorical, I've heard that the coefficient given to a variable is the mean of the response in the group of that variable....
0
votes
0answers
17 views

How to recover the original coefficients in principal components regression?

Let $y$ be a response variable of size $n\times1$, and $X$ be a covariates matrix of dimension $n\times p$, being $p>n$. Since $p>n$, I cannot directly solve the linear model $\tilde{y}=X\tilde{\...
1
vote
1answer
37 views

Fitting response surface using rsm package in R - Lack of fit test is missing

I am trying to fit an rsm model to a data set with three factors, to try to find optimum parameters of a simple empirical environmental model for disease risk prediction, or influence of a change in ...
0
votes
0answers
33 views

regression change point model

I am novice at best with linear regression models and I am interested to learn how to do a change point model in Python if possible. The data that I analyze is a years worth of electrical energy (kWh) ...
0
votes
0answers
24 views

Constant Terms in Linear Projection

In my time series textbook, it says, "Let $Y_i$ and $Y_j$ be two dependent variables in a time series process, e.g. $Y_{t+1}=\phi Y_{t}+\epsilon_{t+1}$, where $\phi$ is a constant coefficient. If a ...
1
vote
1answer
27 views

Is Elastic Net my best choice for finding sparse linear models in correlated features?

I have a linear regression problem, 1000 data points, but with 36 correlated features, those features are very highly correlated. And I know the ground truth must be linear. I know Lasso would give ...
1
vote
0answers
14 views

Use the features selected with RFE SVM linear for prediction of SVM rbf

I was wondering if the features selected with RFE with SVM linear kernel are still "good" features when we use a non linear model, like SVM rbf kernel. This comes in practice when you want to use SVM ...
0
votes
0answers
15 views

Regression for Interval and Continuous Predictors - Continuous Target Variable

I have in my dataset a continuous target variable sales and two predictors- one continuous and one interval. The continuous variable is ...
0
votes
1answer
25 views

How to incorporate AR(1) term in a multiple linear regression model

I was trying to model fish catch (CPUE) using a combination of some categorical and numrical predictors. I have the data for 10 years. The data has been collected only in the period from June to ...
0
votes
0answers
23 views

Time dummies in Linear Panel Data Models and Unobserved Effects Models

Wooldridge (2002, p. 129) says that with (independently) pooled cross sections over time (where different random samples are collected at different points in time - no individuals are observed more ...
0
votes
1answer
39 views

Mixed desgin ANOVA or LMM?

I am hitting a wall and would appreciate some help. Here is my data (raw data, before computing the means): ...
1
vote
0answers
18 views

How to determine the standard error of the prediction from an MLR?

I have a multiple linear regression model that I have built in SAS that's fairly simple: it predicts someone's blood pressure based on four variables: age, race, gender, and level of depression. ...
1
vote
1answer
29 views

How to properly test simple effects: factor recode or data subsetting

I have seen people test their simple effects in two different ways and would like to know which is the correct one and why. The two methods are briefly explained below: recoding method: Involves ...
1
vote
0answers
40 views

Deciding between get_dummies and LabelEncoder for categorical variables in a Linear Regression Model

I'm using the dataset http://www.stat.ufl.edu/~winner/data/airq402.dat whose description is here - http://www.stat.ufl.edu/~winner/data/airq402.txt. I'm planning to build a linear regression model ...
0
votes
0answers
14 views

Variance of linear regression estimator in stratified sampling (survey data)

What is the formula of the linearized variance of the multivariate linear regression estimator in a single-stage stratified sampling design? From the Stata manual, I know the formula for the estimator ...
4
votes
1answer
65 views

Why is Linear Discriminant Analysis a linear classifier?

Question As the title suggests, I am wondering why we consider LDA a linear classifier. More specifically, I would like to know why LDA is considered a linear classifier in the following case: ...