Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

learn more… | top users | synonyms

0
votes
1answer
33 views

fmincg vs fminunc

I have the following script in octave: ...
0
votes
0answers
5 views

One-way Fixed model

Duppose we have 12 treatments, and to start the experiment we choose these treatments, randomly. However, after selecting one treatment, all the samples are gathered and we shift to the next treatment ...
0
votes
0answers
30 views

Getting estimate and CI for dummy variable in linear model

I have a linear model based on some variables (age, gaming and tasks) on response time. It looks like this: ...
1
vote
0answers
13 views

What is the difference between Stochastic Regressor and Non-Stochastic Regressor in Linear Regression?

Suppose the regression specification is $$y_i=\beta_0+\beta_1x_i+\epsilon_i,$$ No matter $x_i$ is stochastic or not, we will need the assumption that $\epsilon_i$ is distributed the same for all $i$. ...
3
votes
0answers
51 views

Why Type III ANOVA is used for this analysis of coefficients

Note that I'm giving (what I believe) is minimal information to solve this problem. If more details than what are provided here are needed, please let me know, and I can provide them. I have a model ...
0
votes
0answers
27 views

Obtain lasso regression coeficient based LS when $X'X = I$

I need to obtain coefficients of lasso regression based in coefficients of Least Square regression method when $X'X = I $. any help will be appreciated.
0
votes
0answers
42 views

Can L1 linear regression perform worse than vanilla linear regression on fewer features?

I have a data set with 2 features and I'm trying to predict one real-valued variable. I use linear regression and I measure the error using 10-fold CV and absolute mean error as a metric. I noticed ...
5
votes
1answer
57 views

Bayesian model comparison in high school

I teach physics to high-school students, and I would like my students to conduct a rudimentary Bayesian model comparison for data from their experiments. I figured out a way for them to do so (see ...
0
votes
0answers
20 views

Derive a t-test for $\beta_1$ [on hold]

I'm having a bit of an issue trying to figure out how to derive a $t$-test for this question. I know I probably need to construct a likelihood function but I'm a bit confused about how to do that. ...
0
votes
0answers
20 views

How to deal with outliers and feature selection simultaneously?

I've been given some data and need to pick what I consider to be the best features from it and use them to build models that fit the data. My issue is that all the tests I've seen for outliers assume ...
1
vote
4answers
61 views

Does a GLM count as a linear least squares model?

I'm doing some work for a summer school project and I've been asked to model some data using a 'linear least squares' model. I've done all that and analysed the results and the summary statistics look ...
0
votes
0answers
9 views

How do I derive the Discriminant Function in Linear Discriminant Analysis

From An Introduction to Statistical Learning with Applications in R on page 143, the authors talk about obtaining the discriminant function in the case of LDA for >1 predictors. Assuming that we are ...
0
votes
2answers
56 views

Linear Model: Why is my R² positive while my abline shows negative trend?

This easy model is plotted with: ...
5
votes
1answer
114 views

Analytically linking coefficients from alternative linear models (OLS)

The general problem: I have two alternative models I could use for my estimation Model A: $y = \alpha^A+ X \beta^A_0 + Z\beta^A_1 + \varepsilon^A$ Model B: $y = \alpha^B + X \beta^B_0 + ...
1
vote
0answers
38 views

Lasso Regression - model predictions are not correct. low r-squared

I am attempting to use Lasso to choose the best variables from a set of 20. I have managed to construct a model using LassoCV, however when using the test data to compare the predicted returns to the ...
2
votes
1answer
123 views

Codification of Matrix $X$ in $Y=XB+\epsilon$

The variables for the data below is age, group (treatment 1,2,3), Y response variable. ...
0
votes
1answer
46 views

Lasso Regression - Finding multiple candidate models

I have 20 predictors and I am attempting to find several candidate models to then test. I am using the LassoCV library, my following code provides me with the alpha and co-efficients of a model. ...
4
votes
1answer
49 views

Self-studying the normal linear model: what can I do with it?

I am currently self-studying the normal linear model ($X_i \sim N(A \beta, \sigma^2I)$), and have learned about estimating $\beta$ and $\sigma^2$, testing certain types of hypothesis (although not too ...
0
votes
0answers
8 views

Correlation between binary predictor and numerical response using Linear Regression

I am trying to find correlations between several binary predictors and a continuous response variable. I am not sure what test it is supposed to work for this case, I got confused with all the things ...
0
votes
0answers
30 views

Is this way to check linearity correct?

Let’s say I want to explain one variable with six regressors. I don’t know if the relationship between my dependent variable and each regressor is linear, if any. I have six regressors, so I can’t ...
0
votes
0answers
35 views

How to interpret and explain negative coefficients when they do not make sense

I have seen appearance of negative coefficients where they do not make sense (the data is related to costs where negative coefficient should not appear). If regression models are fitted to individual ...
1
vote
0answers
35 views

Linear Regression Analysis with heavy tailed noise

I have 5 data sets, each can be fitted (this is given) with a linear model $y=a+x b+\epsilon$ but of different parameters $a,b$, where $\epsilon$ is a heavy-tailed noise of mean zero and $x$ ...
0
votes
0answers
12 views

to estimate bj regression category by category

I'm studying linear regression model with censored data. ~ Buckley James Estimator. For example, I have a data set which contains groups (two categories; 1,2). I estimated bj regression category by ...
0
votes
0answers
18 views

Linear model for an amplitude

I have some nerve cells registered that "fire" spikes during a time period of 60 minutes. These spikes are also encoded in "signal" amplitudes having their highs and lows between 50 and 600 points. ...
1
vote
0answers
35 views

Translate R lm poly output to Excel equation?

I have the results of a lm model and its a so-so fit but now I have to dump it into an Excel file to do predictions. Most of it I get but the POLY factors I don't understand and where there is data it ...
0
votes
0answers
16 views

PCA covariates to fit the linear model

I have pairwise relationship (relation) for items in ColA and ColB in table mydf which is also affected by various covariates ...
1
vote
3answers
173 views

Interpretation of moderately correlated predictors in linear model

I have understood why it's a bad idea having highly correlated predictors, what is puzzling me is a meaningful interpretation of moderately correlated predictors(correlation < 0.3). Let's suppose ...
3
votes
1answer
73 views

Is the least square estimator unique?

Given $X\in\mathbb R^{n\times p}$ and $y\in \mathbb R^n$, the least square coefficients are: $\hat{\beta} = \text{argmin} \| X\beta - y\|^2_2$. Is $\hat{\beta}$ unique in the case ...
1
vote
0answers
25 views

In linear regression, how to prove the equivalence of F-test and t-test? [duplicate]

In the setting of linear regression, when we want to test the hypothesis $H_0 : \beta_j = 0$, we can use either t-test or F-test, with test statistics $t = \frac{\hat{\beta_j}}{SE(\hat{\beta_j})}$ and ...
0
votes
0answers
25 views

Intuition regarding regression through the origin

An exercise asked to obtain properties of the lineal model $$E[y_i]=\beta x_i\qquad i=1,\cdots,n$$ where $Var[y_i]=\sigma^2$. In one of its sections, we had to calculate and estimator for $\beta$ ...
1
vote
0answers
23 views

when fitting non-linear data set to linear model

What is the residual standard deviation? Can I see whether the model I used is accurate or not by looking at this measure? In fact, I try to understand whether my data set is fitting to linear ...
1
vote
1answer
33 views

OLS when both independent and dependent variable are multiplied by another variable

I was given the following problem involving OLS: Suppose we have $(y_i,x_i,z_i)_{i=1}^n$ iid sequence, such that $x_i$ is a vector with K entries and $y_i$ and $z_i$ are scalars. Suppose $z_i$ is ...
8
votes
0answers
113 views

Restricted maximum likelihood with less than full column rank of $X$

This question deals with restricted maximum likelihood (REML) estimation in a particular version of the linear model, namely: $$ Y = X(\alpha)\beta + \epsilon, \epsilon\sim N_n(0, \Sigma(\alpha)), $$ ...
1
vote
0answers
18 views

Approximating GLM with a linear model

Let's say I have a model of $y$ with a single continuous independent variable $x$ that satisfies the GLM assumptions, but for computational reasons I prefer to run OLS instead. I replace $y$ with ...
0
votes
0answers
8 views

What makes for a “repeated measure” when using LMM?

I am conducting a LMM in SPSS and seem to gotten turned around. I have 3 time points and 2 treatment groups. But where does one specify the time points? Options seem to include: Specifying "time" ...
2
votes
0answers
24 views

How does Newey West affect inference?

I understand that the Newey-West estimator is an approach to estimate the covariance of the linear regression estimator (OLS) when there are heteroskedasticity and autocorrelation. Should it modify ...
0
votes
0answers
17 views

Are linear regression errors independent? Mean independent? Uncorrelated?

All I know is that we assume zero conditional mean (and hence zero mean) and conditional homoscedasticity (and hence homoscedasticity). When trying to prove that $E[(\hat{\beta_1} - \beta_1)\bar{u}] ...
0
votes
1answer
35 views

Overfitting a regression model

What are the effects of over-specificating a linear regression model? Could it be said that overfitting is less important when the goal is to only estimate the relationship between dependent and ...
4
votes
3answers
200 views

For linear classifiers, do larger coefficients imply more important features?

I'm a software engineer working on machine learning. From my understanding, linear regression (such as OLS) and linear classification (such as logistic regression and SVM) make a prediction based on ...
0
votes
0answers
18 views

Correcting the output of a linear regression using additional variable

I have the output of a linear regression model, $\hat{y}_i$, which was estimated as usual: $\hat{y}_i = \beta_0 + \beta_1 x_i + ... + \beta_k x_k$ However I do not have access to the data that was ...
1
vote
0answers
24 views

How can I help a student understand that PCA and EFA are examples of the GLM?

I know that principal component analysis (PCA) and [exploratory] factor analysis (EFA) are meant to be examples of the General Linear Model, but I've never been able to find a good way of explaining ...
5
votes
1answer
131 views

Difference between linear model and linear regression

I am interested in the difference between a linear regression and a linear model. In my understanding, linear regression is part of a larger family of linear models but both terms are often used as ...
0
votes
0answers
16 views

Linear regression quality of fit : the percentage of data points inside a confidential interval

When performing the linear regression, we can calculate a confidence interval for the regression. We can see the quality of fit graphically. However if we want to give one single number to summarize ...
0
votes
0answers
33 views

Why can't I recreate the same population of x, using linear-least-squares, for a known linear system, Ax=b?

I have a linear system of equations in the form Ax = b. \begin{equation} \begin{bmatrix} a_{11} \pm \sigma_{a_{11}} & a_{12} \pm \sigma_{a_{12}} & a_{13} \pm \sigma_{a_{13}} \\ a_{21} \pm ...
1
vote
0answers
28 views

Bias in the regression coefficients of a generalized linear model under MLE

Question: Are the regression coefficients of a generalized linear model biased when estimated through maximum likelihood? Imagine, we have a generalized linear model where $E[Y] = g^{-1}(\mu)$ for ...
4
votes
1answer
61 views

Equivalence of the OLS and GLS estimates

I am looking at a handful of problems where I am trying to fit a linear model using generalized least squares (GLS) where the covariance matrix of the error term is relatively "nice". I was wondering ...
2
votes
1answer
31 views

Why is it theoretically satisfactory to predict a random variable using constants and parameters?

Suppose we model the random variable $Y$ as follows: $$\mathbb{E}[Y]=\beta_0+\beta_1x_1.$$ Now many statistics textbooks treat $\beta_i$ as parameters, which is simply constants (correct me if I am ...
0
votes
0answers
5 views

Mediation with a dichtomous predictor

Suppose I have a model with a dichotomous predictor $X$ a continous mediator $M$ and a continous dependent variable $Y$. Let's say I have a game that people (randomly assigend) play either with or ...
0
votes
0answers
9 views

help me understand the proof in the paper “restricted ridge estimation”

I'm reading the paper "restricted ridge estimation" by Grob(2003). I can not understand the proof of theorem 1 in this paper. I don't know how this estimator $\hat{\beta}_{r}(k) = ...
0
votes
0answers
7 views

Control of assumptions in general linear model

In which ways can I "control" the assumptions I make in a linear model where I have $n$ independent, random variables with same variance and where the mean vector lies in some subspace (a strict ...