Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

0
votes
0answers
15 views

How to check if i have strong linear relationship between dependent variable and independent variables in ols?

I want compare the out of sample prediction from an ols model and a regression tree. I read that ols outperforms regression tree if the relationship between the dependent variable and independent ...
0
votes
0answers
20 views

Parametrs of the Uniform Likelihood

Consider linear scalar model $y_k = x_k + w_k$ where $w_k \sim U(a,b)$. What we can say about the probability distribution $p(y_k|x_k)$? We can show that $p(y_k|x_k)$ have uniform density, but I ...
0
votes
0answers
10 views

Known Correlation between predictors

I have two predictors and an outcome, let's call them x, y and z. I know that x and y are correlated with correlation r. I am trying to construct a linear model: z = ax + by + c I have an ...
0
votes
0answers
9 views

How to plot Ranked Linear Regression values in original data plot?

Left Image Shows original EUR/USD values and the corresponding Regression line based on the original values. Right Image Shows the ranked values and ist corresponding ranked Regression line. Values ...
0
votes
1answer
19 views

Interpret effect of covariates in linear model with log-transformed response variable

I am having difficulty interpreting the effects of the covariates of a linear model with log-transformed response for two specific time points. This is the model: $log(Y_t) = \beta_0 + \beta_1 * X_{...
0
votes
0answers
8 views

Can you calculate Bayes Factors for effects in a non-significant regression model

I ran a linear regression model and want to calculate Bayes Factors (BF) for any non-significant effects that are generated by the model. However, the regression model itself is not significant (p = 0....
4
votes
0answers
63 views

What is the median of $y_{i}$ given $x_{i}$ for the function $y_i=\max\{0, x_{i}^{\prime}\beta + u_{i}\}$

$y_{i}$ is a kx1 matrix, $x_{i}$ is a kxk matrix, $\beta$ is a 1xk matrix of coefficients and $u_{i}$ is a kx1 matrix of error terms. $y_i=\max\{0, x_{i}^{\prime}\beta + u_{i}\}$ and $med(u_{i}|x_{i}...
0
votes
0answers
16 views

Looking for ways to transform time-series data recorded from object movement into equation describing the movement direction of the object

Looking for some time-series data transformation advice! I want to know what's the best way to transform data of 9-tuples time series data of IMU (Inertia Measurement Unit) sensor, recorded from a ...
0
votes
0answers
12 views

statistics associated with bootstrapped confidence intervals

Originally I posted this on StackedOverflow but quickly realized it wasn't an issue of coding but rather that of the statistics behind the estimating the 95% confidence intervals of the bootstrapped ...
-1
votes
0answers
18 views

Estimating bias in linear regression and linear mixed model in R simulation

I want to run simulations to estimate bias in linear model and linear mixed model. The bias is E(beta)-beta where beta is the association between my X and Y. I generated my X variable from a normal ...
3
votes
1answer
72 views

Interpretation of standardized (z-score rescaled) linear model coefficients

I have analyzed some data on vegetation change as a function of change in soil parameters. I compared a dataset from 2001 with a dataset from 2018 (fully balanced). To investigate the change in ...
4
votes
2answers
174 views

GLM: Modelling proportional data - account for variation in total sample size

When I am sampling the proportion of a sub-group of animals to the total number of animals within a sample, I can feel quite confident (after taking into account environmental factors) that I have a ...
0
votes
0answers
18 views

Multiple linear regression with dependent as a dummy predictor

I have a model $Y = \alpha + \beta_1X_1 + \beta_2X_2$. $Y$ has a bimodal normal(ish) distribution, so I'm looking to see if the relationship between the predictors and the response is different for ...
3
votes
1answer
61 views

Why normality assumption on linear model implies equivalence between least square estimation and maximum likelihood estimation?

Consider the following excerpt from the Alan Agresti's book on generalized linear models: "Having formed a model matrix $\textbf{X}$ and observed $\textbf{y}$, how do we obtain parameter estimates $\...
1
vote
0answers
34 views

Linear model for paired data [closed]

We have 100 subjects of varying and known age and sex with two strongly related dependent variables (X, Z) measured with two methods (A and B). Method B is known and expected to show more reduced ...
0
votes
1answer
20 views

Why do we use natural logarithm in categorical outcome variables

I'm reading a chapter on categorical outcome variables (chi-square & log linear analysis) and the author, in an effort of fitting a linear model, said that because the outcome variable is a ...
1
vote
0answers
41 views

Cross-Validation on a multiple linear regression model, negative values?

I'm trying to demonstrate that, using a linear model with too many predictors, that the correlation can be artificially inflated, and that k-fold cross validation can expose overfitting. To do this, ...
0
votes
1answer
19 views

nonlinear transformation vs. nonlinear regression

When the variance of a regression model is not a constant (heteroscedasticity problem), why would we have to make nonlinear transformations to linearise the model INSTEAD of fitting a nonlinear ...
0
votes
0answers
26 views

Why shifted predictor value would not change OLS estimator except intercept term?

This question comes from MånsT's answer of question The least squares estimators of $β_1$,$β_2$,… are not affected by shifting. The reason is that these are the slopes of the fitting surface - ...
0
votes
1answer
44 views

How to deal with different scales of significiance with a lmer model?

I did a linear mixed model and found only one significantly correlated variable. If I do a simple linear model other variables seemed to be correlate. How is it possible ? Is it because of different ...
1
vote
1answer
33 views

Literature / Books on Linear Models, Generalized Linear Models and Linear Mixed Models

As the title suggests, I'm looking for book recommendations on Linear Models, Generalized Linear Models and Linear Mixed Models. The book should give a good overview on the intuition behind ...
0
votes
1answer
24 views

Interpretation linear mixed model with interaction

I'm doing linear mixed model with an interaction between the time and my exposure. The fixed effects look like this : Y ~ T + T*GRP + X with T the time, GRP the exposure and X the adjustements ...
0
votes
1answer
18 views

No need for bias term if data is standardised? Linear classification models

For linear classification models, e.g. perceptron, bias term allows to move separating hyperplane away from origin. If data is scattered around the zero does that mean that we don't need bias term?
0
votes
1answer
32 views

Linear Forecasting with a small dataset

I am trying to get some forecast (5 years more) from a small dataset that is as follows: ...
0
votes
0answers
26 views

How to interpret plot residuals vs fitted values?

I run a ols regression and want now check the linearity assumption. I found out that i have to plot the residuals vs the fitted values and if there is no non linear pattern the linearity assumption ...
5
votes
1answer
63 views

Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
1
vote
1answer
31 views

Linear Regression confidence interval bounds

When performing a linear regression we first get a slope and intercept that is the best fit. How do we compute the confidence interval for predicted values? Here's an example: ...
0
votes
0answers
20 views

How to calculate the standard error of a slope in linear regression model? [duplicate]

I got a formula: $$ s(b_1) = \sqrt{\frac{1}{n-2}·\frac{\sum{(y_i-\hat{y}_i)^2}}{\sum{(x_i-\bar{x})^2}}} $$ , is this correct? and how can I calculate the standard error of the intercept? I am ...
0
votes
0answers
13 views

Statistical power and number of independent variables

I would like to understand properly the idea that increasing the number of independent variables in a linear regression decreases the statistical power of the estimated parameters. I have to include ...
0
votes
1answer
24 views

Intuition for Instrumental Variable estimator in Linear Regression Model

Suppose we have the linear regression model given by $y=X\beta+\epsilon$, but we have a violation of assumptions where $X$, the regressor matrix, and $\epsilon$ are correlated. Also, suppose there is ...
0
votes
0answers
25 views

Dealing with variable length time-series data for linear models

I am working with variable length time-series signals. I want to use a sliding window to extract features, things like mean, standard deviation, kurtosis, skewness. The length varies pretty ...
0
votes
0answers
10 views

Does an Interaction term **always** require “lesser order” term in the model? [duplicate]

Assuming a simple Linear model: $$\hat y = b_0 + b_1.x_1 + b_2.x_2$$ Where all coefficients are relevant to the model (p-value < $\alpha$). Then adding the interaction term: $$\hat y = b_0 + b_1....
0
votes
0answers
21 views

Handling Missing Values in the context of Time Series Data

I'm doing a study on one dataset that contains 70 financial ratios for all U.S. companies across eight different categories (Valuation, Liquidity, Profitability, and etc) from 1970 to 2018 monthly. ...
1
vote
0answers
17 views

Removing intercept from linear regression model [closed]

anyone knows how can i remove the intercept of my model while performing cross-validation? I have already tried using "-1" or "+0" in my formula but nothing. As you can see on my script my R2 is too ...
0
votes
0answers
32 views

Winsorizing data in small sample

I have a relatively small sample of panel data (quarterly data for 68 firms over 7 years). My dependent variable is positively skewed. In order to limit the influence of observations with large values,...
6
votes
4answers
171 views

Can i include the product of two random variables? Or do I risk collinearity?

I have a model in which I want to predict Y. My regressors X, are x1 and x2. For some reason I believe that it would also be useful to include into the model: ...
0
votes
0answers
20 views

Could this linear regression really has a probability of only 1 over a billion?

I am computing confidence intervals over linear regression, and I find the results to be rather counter-intuitive. I stack the two samples I want to regress together into a matrix. I compute the $2\...
1
vote
0answers
106 views

Show that $\text{Var}[\hat{\beta}_{1}] = \frac{\sigma^{2}}{\sum_{i=1}^{n}(x_{i1}-\overline{x}_{i})^{2}(1-r^{2})}$

For $i = 1,2,\ldots, n$, consider \begin{align*} Y_{i} = \beta_{0} + \beta_{1}(x_{i1} - \overline{x}_{1}) + \beta_{2}(x_{i2} - \overline{x}_{2}) + \epsilon_{i} \end{align*} where $\overline{x}_{j} = \...
0
votes
0answers
69 views

Given $Y_{i} = \beta_{0} + \beta_{1}x_{i} + \epsilon_{i}$, prove $\beta_{0}$ and $\beta_{1}$ are uncorrelated iff $\overline{x} = 0$

Let $Y_{i} = \beta_{0} + \beta_{1}x_{i} + \epsilon_{i}$ $(i = 1,2,\ldots,n)$, where $\textbf{E}[\epsilon] = 0$ and $\textbf{Var}[\epsilon] = \sigma^{2}\textbf{I}_{n}$. Find the least square estimates ...
0
votes
2answers
65 views

Feature selection using PCA for linear regression

I am using PCA to the training data set to do feature selection before applying linear regression to build a classifier model. In this scenario, would it be useful to use ridge regression to ensure ...
2
votes
1answer
72 views

Linear regression problem: how do I prove that $\sum_{i=1}^{n}(Y_{i} - \hat{Y}_{i}) = 0$? [closed]

If $\textbf{X}\in\textbf{R}^{n\times p}$ has full rank and $\textbf{Y}\in\textbf{R}^{n\times 1}$, prove that \begin{align*} \sum_{i=1}^{n}(Y_{i} - \hat{Y}_{i}) = 0 \end{align*} where $\hat{\textbf{Y}}...
1
vote
1answer
36 views

Write the null hypothesis for nested model test in algebraic form

The reduced model is: lm(y~Age+Sex, data = df); The full model is: lm(y~Age+Sex+Age*Sex, data = df). (...
1
vote
1answer
46 views

Whether there is significant difference between the slopes of different gender groups

The last question I have posted here: Whether there is significant difference between two gender groups There are 16 people in the dataset(using subjectID to ...
1
vote
0answers
38 views

Signal-to-noise ratio under heteroscedasticity

I want to be able to compare simulated datasets with and without heteroscedasticity, in the context of a linear regression: $y = X \beta + \epsilon$ Since I want to make sure that when I introduce ...
2
votes
0answers
25 views

Using transformation including a permutation matrix to fit a linear mixed model [closed]

reference: http://www.stat.wisc.edu/~bates/UseR2008/WorkshopD.pdf from page 97. I want to fit a linear mixed model. $Y|B=b\sim N(X\beta+Zb,\sigma^2 I)$ and $B\sim N(0,\Sigma(\theta))$. Here I choose ...
1
vote
1answer
37 views

How to account for incrementation in a log-linear model

I want to perform a mixed regression analysis with random intercept and uncorrelated random slope after multiple imputation. The dependent variable is continuous, namely a duration as number of days ...
1
vote
1answer
35 views

Can a linear and logit model have the same shape?

While I was working on an exercise based this book, I discovered something interesting. When I fit a logit and simple linear probability model on the data (see code below), the predictions are almost ...
1
vote
0answers
208 views

What statistics can I use?

I have done a research looking at different frequencies of abrasions (ablation, etc.) over time (in hrs) and my data mainly consists of zeros. As I am weak in statistics, I am unsure which statistics, ...
3
votes
2answers
186 views

Why does this expression simplify as such?

I'm reading through my professor's lecture notes on the multiple linear regression model and at one point he writes the following: $$E[(b-\beta)e']=E[(X'X)^{-1}\epsilon\epsilon'M_{[X]}]. $$ In the ...
0
votes
2answers
91 views

How to find all models that meet the pre-specified restrictions

Let's say I have a large number of predictors (e.g. 2000) and I'm facing the problem of choosing the linear regression model under following assumptions: There are few predictors that have to be ...