Questions tagged [linear-model]
Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.
2,554
questions
1
vote
0
answers
237
views
How can I perform Group Lasso?
Suppose I have an $n \times p$ matrix of data, $X$, and a $p \times p$ matrix of coefficients $\beta$. I'm interested in the folllowing:
$$ \min_{B} \Bigg\{ \lVert XB - X \lVert_2^2 + \lambda \sum_{...
3
votes
0
answers
326
views
Transform non-normal circular data
I have a set of time-series data showing event times for two different groups. Let's call them Site 1 and Site 2. My analysis shows one group is circular normal around a 24 hr clock (von Mises ...
16
votes
1
answer
25k
views
How to determine Degrees of Freedom in Linear (Mixed Effect) Regression
In a statistics class we had a final homework to work with the data set lexdec of languageR (full script at the end). The task ...
0
votes
1
answer
23
views
Heteroscedastic LR improves parameter estimates
Let's say I have 1d data generated using linear function, and heteroscedastic noise on top of it, whose distribution I happen to know. I can estimate parameters using least squares linear regression ...
7
votes
3
answers
3k
views
Standardizing quadratic variables in linear model [closed]
I have a fundamental question regarding standardization:
Say, I have predictor vector aand b (...
0
votes
1
answer
597
views
How to determine if a regression is feasible?
I am trying to build a regression model following the instructions from the book Introduction to linear regression analysis by D. Montgomery and others.
I am about to perform "all possible ...
3
votes
1
answer
58
views
Linear model terms within a linear model
I am still new to R and having issues searching the right terminology for my problem. I have the following equation:
...
2
votes
0
answers
39
views
When is it possible to estimate the non-linearity error when approximating data with a linear model?
The most common form of linear regression estimates the best values of $\vec{\beta}$ and $\sigma^2$ assuming that data is sampled from a model $y = \vec{\beta} \cdot \vec{x} + \vec{\epsilon}$ where $\...
6
votes
1
answer
5k
views
Trying to emulate linear regression using Keras [closed]
I am trying to build a very simple NN to approximate a linear function (literally).
I took a table data:
f(x) = 5 * x
Shapes:
Now I am building a very simple NN using Keras:
...
2
votes
1
answer
64
views
log log model where dependent variable is 0
I am fitting log log models (e.g. video link or non video link) like these (very much simplified):
log(y) = intercept + parameter * log(x);
I have a few rows ...
0
votes
0
answers
64
views
Linear model and mean value of covariates
Suppose a linear model $$Y_i=\beta_0+x_{i1}\beta_1+\ldots+x_{ip}\beta_p+e_i,\quad e_i\sim N(0,\sigma^2),\quad i=1,\ldots,n,$$ and its hat matrix $P=X(X^TX)^{-1}X^T$, where $Y=X\beta+e$.
Let $\bar{x}...
1
vote
1
answer
28
views
How to run powerful analyses with non-normal data
I coded the amount of speech that participants used during several tasks and I want to use this information to predict performance on other tasks. I also want to test whether speech on one task is ...
0
votes
0
answers
788
views
Log-scale for regression, graph, or both?
The below graphs denotes the relationship between prices (relative to the United States, so that US=1) and economic out per capita (also relative to the US, so that US=1).
The author graphs the ...
0
votes
1
answer
45
views
Which distribution has this Statistic model
For my exam I was given this statistic model:
$Y_i$=u+(1+a$x_i$)∗$ϵ_i$
where
$u$ is an unknown parameter with all real values,
$a$ is an unknown parameter in with values between ]-1;1[,
$e_i$ is ...
0
votes
0
answers
38
views
Cannot obtain a good regression model with features having significant correlation with the target variable
I have generated 42 features from the existing dataset for a prediction task. All these features are significantly correlated with the target variable (ranging from .25 to .05). For the dimension ...
0
votes
1
answer
482
views
Spearman or Kendall's tau-b correlation instead of simple Regression
Let's assume we make a regression, but the data does not fulfill the assumptions of normality and linearity.
Is it possible to use Spearman or Kendall's tau-b correlation instead of a regression? ...
4
votes
2
answers
13k
views
What does the additive assumption mean?
The additive assumption means the effect of changes in a predictor on a response is independent of the effect(s) of changes in other predictor(s).
However, with regression, say, with one continuous ...
2
votes
1
answer
163
views
Reason behind removing skewness?
While building predictive models we often see skewness in the target variable. Then we generally take transformations to make it more normal. We generally do it for linear models and not for tree ...
2
votes
2
answers
137
views
MLR doubt regarding invariance of predicted values
in the model $ Y= X\beta +\epsilon $
when they say the predicted value of y is invariant to full rank linear tranformation on xi's . what does it mean . ?
does it mean that any linear ...
1
vote
0
answers
75
views
Solution changes when explanatory variables and response variables are rotated in the same way for Linear Regression (Lasso)
Let's say we have a high dimension linear regression problem:
$$
y = X\beta + \epsilon
$$
where the dimension of $X$ is $n \times p$, and $y$ is just a $n \times 1$ vector.
We can solve for $\beta$ ...
2
votes
1
answer
267
views
Linear regression of B-splines with terms inside an integral?
I have encountered a problem that the literature suggests linear regression is able to solve, but I am at a loss.
I have a function $F$ that I want to estimate. This function obeys $N$ equations of ...
9
votes
3
answers
5k
views
High $R^2$ squared and high $p$-value for simple linear regression
Let's assume that we have simple linear regression:
$\hat{y} = bx + \text{intercept}$.
Is it possible to have a high p-value and high $R^2$ (or low p-value and low $R^2$)? I've been looking for ...
4
votes
1
answer
371
views
Fitting least squares when number of predictors are larger than instances
A statement from the book Introduction to Statistical learning with applications in R, didn't quite make sense to me. It says, "In cases when number of predictors are greater than the instances we ...
23
votes
4
answers
45k
views
Linear regression what does the F statistic, R squared and residual standard error tell us?
I'm really confused about the difference in meaning regarding the context of linear regression of the following terms:
F statistic
R squared
Residual standard error
I found this webstie which gave ...
0
votes
0
answers
434
views
Normal approximation for Negative Binomial regression
In negative binomial regression, the distribution is specified in terms of its mean, $\frac{pr}{1-p}$, which is then related to explanatory variables as in linear regression or other Generalized ...
1
vote
1
answer
565
views
Is a linear discriminant function actually "linear"?
Chris Bishop introduces the multi-class linear discriminant functions (a.k.a. linear machine) in PRML as follows (p. 183):
In proving the convexity of decision regions, it's assumed that $y_k(\cdot)$ ...
2
votes
1
answer
202
views
What is current consensus on analysing unbalanced datasets - type I/II/or III SS
We have a data-set with two variables - gender and education qualification. The data-set is severely unbalanced. The are only ~10 observations from 1 level of the education factor, ~20 from another ...
1
vote
1
answer
3k
views
Effect of combining predictor variables in a regression model
Let's say I first run a linear regression model Sales = f(TV Spend, Digital Spend).
Now I add TV Spend and Digital Spend and run the second model. My second model is Sales = f(TV Spend+Digital Spend)...
1
vote
1
answer
416
views
Skewed and fat-tailed error process in Kalman filter
I'm trying to filter a time series, of which I occasionally can observe the state variable variable but not always. I also have a noisy measure of this state variable all the time. By picking the time ...
14
votes
1
answer
3k
views
Why do lm and biglm in R give different p-values for the same data?
Here is a small example:
MyDf<-data.frame(x=c(1,2,3,4), y=c(1.2, .7, -.5, -3))
Now with thebase::lm:
...
0
votes
0
answers
50
views
Am I violating assumptions of mixed effects model?
I am using a mixed effects model to analyse data, but am unsure whether I am committing any violations due to the nature of the data I have.
My data comes from a game whereby people have to identify ...
13
votes
1
answer
4k
views
What is the meaning of double bars and 2 at the bottom in ordinary least squares?
I saw this notation for ordinary least squares here.
$$ \min_w \left\| Xw - y \right\|^2_2$$
I have never seen the double bars and the 2 at the bottom. What do these symbols mean? Do they have ...
8
votes
2
answers
8k
views
Linear regression with log transformed data - large error [duplicate]
I have a set of data which is has a very large positive skew, and has been transformed using a logarithm. I wish to predict one variable from another using the lm ...
0
votes
1
answer
168
views
Estimate parameters using OLS in Linear regression (demonstration)
I managed to demonstrate till the first part. The problem I'm facing is transforming it to look like the second part.
2
votes
1
answer
1k
views
How to choose between parametric and non-parametric regressions? [closed]
I am new to nonparametric regressions.
What tests should one perform to choose a non-parametric regression model over a parametric one(Or vice versa)?
Let's assume in our analysis we have a continuous ...
9
votes
2
answers
2k
views
How is it possible to get a high $R^²$ & still have 'poor predictions'? [duplicate]
The insulin-sensitivity check index (QUICKI) has an excellent linear correlation with the glucose clamp index of insulin sensitivity (SI_Clamp) that is better than that of many other surrogate indexes....
1
vote
0
answers
127
views
Interpeting residuals in a binary model (logistic regression, glm) [duplicate]
I know how to interpret residuals in a linear regression model. I am now working in a model that predicts a binary target combining numerical and categorical variables.
My residual plots is as ...
4
votes
2
answers
2k
views
Distribution of Y influenced by predictor X in simple linear regression?
One of the assumptions in simple linear regression is that the error term is supposed to be normally distributed. Now, I found on the internet the following quote:
"You’ll notice there is nothing ...
1
vote
1
answer
924
views
Percent deviation for linear regression
I have 2 sets of experimental data to which I applied a linear fit using Matlab. I can use the slope value to compare between both of them.
My question is: can I use the following percent deviation ...
4
votes
1
answer
240
views
Joint forecast with Linear Model and ARMA
This might be a rather trivial question, but I cannot seem to figure out how to do this in R. I have estimated a linear model for my time series, and modeled the residuals of the model using an ARMA ...
4
votes
1
answer
1k
views
How to calculate the logLik of a linear model acquired through matrix operations
If you were to run a linear model with the lm function, it is easy to obtain the log liklihood of the fit with the following:
...
0
votes
0
answers
52
views
Regression on non-normal DV + Heteroskedasticity
I have a DV scored from 1 to 3 from a questionnaire: respondents were given an integer score according to their responses. I need to compare the mean scores between two groups using a regression (...
0
votes
0
answers
46
views
Showing that an increase in uncertainty is significant
I have a linear model $y = ax+b$ and I estimate the coefficients $a$ and $b$ in the ordinary way.
I have found out that all of my values of $x$ were systematically overestimated, and also that they ...
1
vote
1
answer
563
views
Interactions in linear probability models
Suppose, I estimate a simple linear probability model:
$P(Y=1)=\beta_0+ \beta_1 X_1 + \beta_2 X_2 + \beta_3 X_1 \times X_2 + u$,
where $Y$, $X_1$, and $X_2$ are dummy variables. All standard OLS ...
1
vote
1
answer
43
views
Why $var(y_a - \hat{y}_a)=\sigma^2(1+ x_a' (X' X)^{-1} x_a)$?
Let $y_a$ be an observed value and $\hat{y}_a$ be predicted value.
Then I've read that
$$var(y_a- \hat{y}_a)=\sigma^(1+ x_a'(X'X)^{-1}x_a)$$
but what's the proof for this?
Additional info:
$X_1=x_{...
0
votes
2
answers
210
views
How to tell if it is worth to introduce polynomial terms?
I usually use Akaike's Information Criteria to compare linear models.
This time, I see that with a polynomial term the model's AIC is lower (better) than without it. In fact, the AIC keeps ...
1
vote
1
answer
268
views
Least squares system identification gives wrong coefficients
I am working on system identification using least squares method. I implemented the algorithm as recommended by the original paper.
This link describes what I implemented.
Example of desired model ...
6
votes
2
answers
2k
views
Understanding oscillating behaviour when using Q-learning on cart-pole problem
I am fairly new to RL and have been using OpenAI Gym to try implementing a few algorithms I've been learning about.
I've just been trying to get Q-learning working on the cart-pole environment using ...
2
votes
1
answer
218
views
White test with large model and few samples impossible?
I have a linear model with 12 regressors and sample size 42, which I want to test for heteroscedasticity. Hence, I applied a white test in the following way
regress $y=X\beta + e$
regress residuals $\...
5
votes
2
answers
23k
views
How do I avoid computationally singular matrices in R?
I'm fitting a logistic regression model (with R's caret package) to data here. I aim to predict whether Hillary or Trump will ...