The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

Filter by
Sorted by
Tagged with
1
vote
0answers
237 views

Error in lda.default(x, grouping, …) : variable 74 appears to be constant within groups

I did a Linear Discriminant Analysis using R and the output given was Error in lda.default(x, grouping, ...) : variable 74 appears to be constant within groups. My codes are library(MASS) attach(...
4
votes
1answer
75 views

Should I use a linear mixed model or a generalized mixed model?

I have a test dataset with repeated measures, different individuals sampled at different time points, here measured in days. I want to know if I should use a GLMM or a LMM to see how well, if at all, ...
0
votes
1answer
17 views

Placing constraints on linear model coefficients

I have this bit of data : ...
0
votes
0answers
26 views

Need of p-value adjustment when analyzing all possible pairs of factor levels in linear model

I have a nominal (not ordinal) variable with $K$ levels (these are actualy some groups of patients) and I run linear model in which: this nominal variable is one of IV's (others are age, sex and so ...
1
vote
0answers
20 views

Combining regression models from separate data sets

What is the best way to combine regression betas from separate data sets? For example, a data set is split in two based on some fundamental characteristic, and the same two factor regression is run ...
0
votes
1answer
13 views

Feature selection on full training set, does information leak if using Filter Based Feature Selection or Linear discriminate analysis?

In order to test a potential classification set, usually some data is kept as a holdout set, and not used for inner-cross-validation or model training. However, what happens if too many features ...
1
vote
2answers
69 views

Bootstrapping with repeated measurements

I am trying to estimate a linear relation between body temperature and body mass, and I have a sample of measurements from subjects, with most subjects having one measurement, but several subjects ...
2
votes
1answer
24 views

Is it common to maximize correlation between response variable and a combination of explanatory variables as a first model?

I am doing a linear regression project and in the exploration part I did something I was asked to explain in more detail... There are only 4 variables: 1 response variable and 3 explanatory variables. ...
0
votes
1answer
52 views

In R (nlme/lme4), how do I compare two linear models with different sample sizes and different groups? [duplicate]

First off, I have a dataset with sparse longitudinal data. There are 30 individuals with 1 sample, 30 individuals with 2 samples, and 5 individuals with 3 samples. Various categorical variables are ...
1
vote
0answers
13 views

Monthly data vs aggregating weekly data to monthly in linear regression

Let's suppose I have weekly data available for making a forecast using a linear regression model. Let's also assume that the weekly forecasts can be aggregated to monthly ones. Would it make sense to ...
2
votes
2answers
54 views

Understanding simplification of constants in derivation of variance of regression coefficient

In looking over TooTone's answer in Derive Variance of regression coefficient in simple linear regression, there's a step in line 3 below where $(\beta_0 + \beta_1x_i + u_i )$ is simplified to $u_i$ ...
1
vote
0answers
11 views

An Interesting Model with Unknown Orthogonal Design Matrix

Consider a linear mixed model, $$\mathbf{y}_{ij}=\mathbf{\Gamma}\mathbf{\mu}+\mathbf{z}_i+\mathbf{e}_{ij}, ~~ ~~i=1,\ldots,m,~~j=1,\ldots,n_i, $$ where $\mathbf{y}_{ij}$ are $k\times 1$ observation ...
0
votes
2answers
84 views

Understanding cov.reduce argument in emmeans function

I would appreciate any help regarding emmeans package. I am fitting dummy-variable regression model (ANCOVA) with follow-up post hoc test in ...
0
votes
0answers
12 views

Cross-validation error of ridge regression

Problem In order to find the optimal parameter $\lambda$, each individual observation is taken out from design matrix $\mathbf{X}$ and solves $$ \text{minimize}_{\beta} \frac{1}{2}\Vert \mathbf{y}_{-...
2
votes
2answers
68 views

How to check if i have strong linear relationship between dependent variable and independent variables in linear regression (OLS)?

I want compare the out of sample prediction from an linear regression model (OLS) and a regression tree. I read that OLS outperforms regression tree if the relationship between the dependent variable ...
0
votes
0answers
23 views

Parametrs of the Uniform Likelihood

Consider linear scalar model $y_k = x_k + w_k$ where $w_k \sim U(a,b)$. What we can say about the probability distribution $p(y_k|x_k)$? We can show that $p(y_k|x_k)$ have uniform density, but I ...
0
votes
1answer
19 views

Known Correlation between predictors

I have two predictors and an outcome, let's call them x, y and z. I know that x and y are correlated with correlation r. I am trying to construct a linear model: z = ax + by + c I have an ...
0
votes
0answers
10 views

How to plot Ranked Linear Regression values in original data plot?

Left Image Shows original EUR/USD values and the corresponding Regression line based on the original values. Right Image Shows the ranked values and ist corresponding ranked Regression line. Values ...
0
votes
1answer
22 views

Interpret effect of covariates in linear model with log-transformed response variable

I am having difficulty interpreting the effects of the covariates of a linear model with log-transformed response for two specific time points. This is the model: $log(Y_t) = \beta_0 + \beta_1 * X_{...
0
votes
0answers
8 views

Can you calculate Bayes Factors for effects in a non-significant regression model

I ran a linear regression model and want to calculate Bayes Factors (BF) for any non-significant effects that are generated by the model. However, the regression model itself is not significant (p = 0....
4
votes
0answers
67 views

What is the median of $y_{i}$ given $x_{i}$ for the function $y_i=\max\{0, x_{i}^{\prime}\beta + u_{i}\}$

$y_{i}$ is a kx1 matrix, $x_{i}$ is a kxk matrix, $\beta$ is a 1xk matrix of coefficients and $u_{i}$ is a kx1 matrix of error terms. $y_i=\max\{0, x_{i}^{\prime}\beta + u_{i}\}$ and $med(u_{i}|x_{i}...
0
votes
0answers
25 views

Looking for ways to transform time-series data recorded from object movement into equation describing the movement direction of the object

Looking for some time-series data transformation advice! I want to know what's the best way to transform data of 9-tuples time series data of IMU (Inertia Measurement Unit) sensor, recorded from a ...
0
votes
0answers
13 views

statistics associated with bootstrapped confidence intervals

Originally I posted this on StackedOverflow but quickly realized it wasn't an issue of coding but rather that of the statistics behind the estimating the 95% confidence intervals of the bootstrapped ...
3
votes
1answer
108 views

Interpretation of standardized (z-score rescaled) linear model coefficients

I have analyzed some data on vegetation change as a function of change in soil parameters. I compared a dataset from 2001 with a dataset from 2018 (fully balanced). To investigate the change in ...
4
votes
2answers
228 views

GLM: Modelling proportional data - account for variation in total sample size

When I am sampling the proportion of a sub-group of animals to the total number of animals within a sample, I can feel quite confident (after taking into account environmental factors) that I have a ...
0
votes
0answers
18 views

Multiple linear regression with dependent as a dummy predictor

I have a model $Y = \alpha + \beta_1X_1 + \beta_2X_2$. $Y$ has a bimodal normal(ish) distribution, so I'm looking to see if the relationship between the predictors and the response is different for ...
3
votes
1answer
71 views

Why normality assumption on linear model implies equivalence between least square estimation and maximum likelihood estimation?

Consider the following excerpt from the Alan Agresti's book on generalized linear models: "Having formed a model matrix $\textbf{X}$ and observed $\textbf{y}$, how do we obtain parameter estimates $\...
1
vote
0answers
41 views

Linear model for paired data [closed]

We have 100 subjects of varying and known age and sex with two strongly related dependent variables (X, Z) measured with two methods (A and B). Method B is known and expected to show more reduced ...
0
votes
1answer
20 views

Why do we use natural logarithm in categorical outcome variables

I'm reading a chapter on categorical outcome variables (chi-square & log linear analysis) and the author, in an effort of fitting a linear model, said that because the outcome variable is a ...
1
vote
0answers
55 views

Cross-Validation on a multiple linear regression model, negative values?

I'm trying to demonstrate that, using a linear model with too many predictors, that the correlation can be artificially inflated, and that k-fold cross validation can expose overfitting. To do this, ...
0
votes
1answer
28 views

nonlinear transformation vs. nonlinear regression

When the variance of a regression model is not a constant (heteroscedasticity problem), why would we have to make nonlinear transformations to linearise the model INSTEAD of fitting a nonlinear ...
0
votes
0answers
33 views

Why shifted predictor value would not change OLS estimator except intercept term?

This question comes from MånsT's answer of question The least squares estimators of $β_1$,$β_2$,… are not affected by shifting. The reason is that these are the slopes of the fitting surface - ...
0
votes
1answer
53 views

How to deal with different scales of significiance with a lmer model?

I did a linear mixed model and found only one significantly correlated variable. If I do a simple linear model other variables seemed to be correlate. How is it possible ? Is it because of different ...
1
vote
1answer
46 views

Literature / Books on Linear Models, Generalized Linear Models and Linear Mixed Models

As the title suggests, I'm looking for book recommendations on Linear Models, Generalized Linear Models and Linear Mixed Models. The book should give a good overview on the intuition behind ...
0
votes
1answer
61 views

Interpretation linear mixed model with interaction

I'm doing linear mixed model with an interaction between the time and my exposure. The fixed effects look like this : Y ~ T + T*GRP + X with T the time, GRP the exposure and X the adjustements ...
1
vote
1answer
32 views

No need for bias term if data is standardised? Linear classification models

For linear classification models, e.g. perceptron, bias term allows to move separating hyperplane away from origin. If data is scattered around the zero does that mean that we don't need bias term?
0
votes
1answer
42 views

Linear Forecasting with a small dataset

I am trying to get some forecast (5 years more) from a small dataset that is as follows: ...
0
votes
0answers
30 views

How to interpret plot residuals vs fitted values?

I run a ols regression and want now check the linearity assumption. I found out that i have to plot the residuals vs the fitted values and if there is no non linear pattern the linearity assumption ...
5
votes
1answer
70 views

Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
1
vote
1answer
36 views

Linear Regression confidence interval bounds

When performing a linear regression we first get a slope and intercept that is the best fit. How do we compute the confidence interval for predicted values? Here's an example: ...
0
votes
0answers
29 views

How to calculate the standard error of a slope in linear regression model? [duplicate]

I got a formula: $$ s(b_1) = \sqrt{\frac{1}{n-2}·\frac{\sum{(y_i-\hat{y}_i)^2}}{\sum{(x_i-\bar{x})^2}}} $$ , is this correct? and how can I calculate the standard error of the intercept? I am ...
0
votes
0answers
22 views

Statistical power and number of independent variables

I would like to understand properly the idea that increasing the number of independent variables in a linear regression decreases the statistical power of the estimated parameters. I have to include ...
0
votes
1answer
44 views

Intuition for Instrumental Variable estimator in Linear Regression Model

Suppose we have the linear regression model given by $y=X\beta+\epsilon$, but we have a violation of assumptions where $X$, the regressor matrix, and $\epsilon$ are correlated. Also, suppose there is ...
0
votes
0answers
55 views

Dealing with variable length time-series data for linear models

I am working with variable length time-series signals. I want to use a sliding window to extract features, things like mean, standard deviation, kurtosis, skewness. The length varies pretty ...
0
votes
0answers
10 views

Does an Interaction term **always** require “lesser order” term in the model? [duplicate]

Assuming a simple Linear model: $$\hat y = b_0 + b_1.x_1 + b_2.x_2$$ Where all coefficients are relevant to the model (p-value < $\alpha$). Then adding the interaction term: $$\hat y = b_0 + b_1....
0
votes
0answers
32 views

Handling Missing Values in the context of Time Series Data

I'm doing a study on one dataset that contains 70 financial ratios for all U.S. companies across eight different categories (Valuation, Liquidity, Profitability, and etc) from 1970 to 2018 monthly. ...
1
vote
0answers
28 views

Removing intercept from linear regression model [closed]

anyone knows how can i remove the intercept of my model while performing cross-validation? I have already tried using "-1" or "+0" in my formula but nothing. As you can see on my script my R2 is too ...
0
votes
0answers
94 views

Winsorizing data in small sample

I have a relatively small sample of panel data (quarterly data for 68 firms over 7 years). My dependent variable is positively skewed. In order to limit the influence of observations with large values,...
6
votes
4answers
185 views

Can i include the product of two random variables? Or do I risk collinearity?

I have a model in which I want to predict Y. My regressors X, are x1 and x2. For some reason I believe that it would also be useful to include into the model: ...
0
votes
0answers
21 views

Could this linear regression really has a probability of only 1 over a billion?

I am computing confidence intervals over linear regression, and I find the results to be rather counter-intuitive. I stack the two samples I want to regress together into a matrix. I compute the $2\...