Refers to any model where the a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.
3
votes
1answer
245 views
AR1 and Law Of Iterated Expectations : No serial correlation
In the AR(1) model $y_{t}=\beta_{0}+\beta_{1}y_{t-1}+u_{t}$, assuming $E(u_{t-1}|y_{t-1},y_{t-2}...)=0$, how does the law of iterated expectations ensure that the errors must be uncorrelated: ...
1
vote
1answer
52 views
F Test: $F_{4, 81}(5\%) \approx F_{4,60}(5\%) - (81 - 60)(F_{4, 60}(5\%) - F_{4,120}(5\%))/60$. Where does this come from?
I had to compute the F-test in my ANOVA question and use interpolation if necassary.
The first time, I had to work out the values of $F_{4,90}(5\%)$ which I said was approximatley $F_{4, 60}(5 \%)$ ...
0
votes
0answers
70 views
Computing the LRT and Wald test from a subset of informations
The answer might be really easy, but I have got some doubts about it.
These are all the informations I have.
...
7
votes
1answer
215 views
Confusion related to linear dynamic systems
I was reading this book Pattern Recognition and Machine Learning by Bishop. I had a confusion related to a derivation of the linear dynamical system. In LDS we assume the latent variables to be ...
0
votes
1answer
70 views
Reducing the dimensionality of a problem
My particular application needs me to build a linear model with a strong correlation structure amongst the independent variables. The dimensions of the problem are high, for instance 1million X 200. ...
4
votes
1answer
106 views
Why arrange variables by causality in bivariate regression?
Suppose we have variables $(X,Y)$ and we have theory tell us that $X$ $\overset{\text{cause}}{\implies} Y$. Perhaps they're time-series variables and it would be common to see something like this:
...
1
vote
0answers
106 views
What are the conditions where we can regress non-stationary variables?
Obviously there are certain spots where it's okay to include a non-stationary predictor variable in a linear regression model. For example, a dummy variable interacted with a stationary variable must ...
16
votes
3answers
501 views
Fast linear regression robust to outliers
I am dealing with linear data with outliers, some of which are at more the 5 standard deviations away from the estimated regression line. I'm looking for a linear regression technique that reduces the ...
2
votes
1answer
118 views
What is an appropriate formula for residuals calculation in a model describing a single-factor repeated measures design?
For the repeated measures design D.C.Montgomery in his "Design and Analysis of Experiments" book provides the mathematical / statistical (linear) model: $y_{ij} = \mu + \tau_i+ \beta_j + ...
0
votes
2answers
520 views
How to derive the least square estimator for multiple linear regression?
In the simple linear regression case $y=\beta_0+\beta_1x$, you can derive the least square estimator $\hat\beta_1=\frac{\sum(x_i-\bar x)y}{\sum(x_i-\bar x)^2}$ such that you don't have to know ...
2
votes
0answers
60 views
Linear regression for samples of varying sizes
I have a data set of code patches and the bugs they produce. I'm using ordinary least squares to find a line which predicts bugs based on some attributes about the patch, such as the department which ...
0
votes
0answers
37 views
Cell Means Model Property
Could anybody demonstrate or direct me to a readily available proof of the following:
For the cell means model:
$$ y_{ij} = \mu_{i} + \epsilon_{ij},\ \text{ for }\ i = 1, \ldots, r\ \text{ and }\ j ...
0
votes
1answer
60 views
Learning the parameter of linear model
I was reading the slides from the following
http://www.slideshare.net/kunegis/searching-microblogs-coping-with-sparsity-and-document-quality
In slide 7, the author proposed just a linear model and ...
2
votes
1answer
128 views
Should raw data or residuals be used to check homogeneity of variance?
Unexpectedly for me (!) I've recently learnt that:
"We have assumed that the error terms, $\epsilon_{ij}$, of the variates in each sample will be independent, that the variances of the error terms ...
3
votes
2answers
98 views
Unable to figure out right transformation
I have obtained some data about how complexity of Java open source projects varies with time. I want to fit a curve to the data, however I am unable to figure out the right kind of transformation. I ...
1
vote
1answer
98 views
Modeling Outliers of Normal Distribtuion
I am using a linear model to predict under-nutrition in children under 5. The common metric discussed is stunting (a binary outcome) which is defined as being more than two standard deviations away ...
4
votes
1answer
217 views
Categorial features in linear machine learning algorithms
How can I deal with categorial features inside linear ML algorithms? The obvious solution is to represent each value as a binary feature. For example, if the categorical feature color has three ...
1
vote
0answers
112 views
Variance decomposition in linear regression model
Consider the linear model $y = \mathbf{X}\mathbf{\beta} + \epsilon$.
The residual variance-covariance matrix is given by $\text{Var}(\epsilon)$.
Greene's textbook* states that:
$$Var(\epsilon) = ...
1
vote
1answer
41 views
Rank of an expression in Linear Model (Reference Searle)
Linear model in matrix form is
$
\mathbf{y}=\mathbf{X}\beta+\epsilon\textrm{ where }\epsilon\sim\mathbb{N}\left(0,\sigma^{2}\mathbf{I}\right).
$
If
$
...
0
votes
1answer
36 views
What is it that's expected from the operation
I've got a regression model and I've got 12 observations. So, I could and I did create the matrix. I also calculated the OLS estimator. But my professor's question is to find X and Y. I'm new to stats ...
4
votes
1answer
226 views
Generalized Least Squares: Estimation of Variance-Covariance matrix
Linear model in matrix form is
$
\mathbf{y}=\mathbf{X}\beta+\epsilon\textrm{ where }\epsilon\sim\mathbb{N}\left(0,\sigma^{2}\mathbf{V}\right).
$
Then $\beta$ can be estimated through generalized ...
0
votes
0answers
61 views
Should I normalize for internal standard by taking residuals or by including the standard in the model?
I have some mass-spectroscopy data from several dozen samples, with the abundance of 60 compounds reported for each sample. Four internal standards were run with each sample, and their abundances are ...
1
vote
0answers
45 views
Derivative of $H$ with respect to $W$ when performing generalized linear squares
I am trying to solve a generalized linear squares model with the following form:
$\hat{Y}= X(X'\Omega^{-1}WX)^{-1}X'\Omega^{-1}WY $
$ H= X(X'\Omega^{-1}WX)^{-1}X'\Omega^{-1}W $
$ \Omega$ is the ...
9
votes
2answers
696 views
Choosing between LM and GLM for a log-transformed response variable
I'm trying to understand the philosophy behind using a Generalized Linear Model (GLM) vs a Linear Model (LM).
I've created an example data set below where:
$log(y) = x + \epsilon $.
The example ...
0
votes
0answers
69 views
Same (poor) results with linear and non-linear classifiers
I am doing classification of "text quality" using four classes and using 30 features with 1300 samples. I am using the following classifiers:
LDAC based on linear discriminant analysis from mlpy.
...
1
vote
3answers
478 views
How to evaluate results of linear regression
I have a linear regression problem. In short, I have a dataset, I divided it into two subsets. One subset is used to find the linear regression (training subset), another is used to evaluate it ...
0
votes
0answers
40 views
binomial approximation - precision errors
For an estimation problem, i have to compute binomial probabilities given high n,x and low p.
Specifically, I compose a matrix A, whose entries are 0 if i >j , and dbinom(x=j, size=i, p) if i<= ...
1
vote
1answer
65 views
Multiple cross validation consistently performing worse than chance
I am calculating accuracy results from a single feature using LDA. I am very curious why a certain feature consistently performs at less than chance when calculating accuracy using a stratified ...
2
votes
3answers
283 views
What makes a GLM estimate the means differently from the actual sample means?
Disclaimer: I'll happily admit to not really knowing what I'm doing when it comes to General Mixed Models but hopefully this thread can bring me a little bit closer to some kind of understanding. If, ...
1
vote
2answers
3k views
What is the “root mse” in stata?
I have a question that has been confusing me ever since I took econometrics last year. What does the "root MSE" mean in stata output when you regress a OLS model?
I know that it translates into "root ...
1
vote
1answer
49 views
Linear contrast OLS - Simultaneous
I fit the following linear model using OLS, where $X_{5}$ and $X_{6}$ are both dummy variables and the rest are continuous:
$Y=\beta_{0} + \beta_{1}X_{1}+ \beta_{2}X_{2} + \beta_{3}X_{3} + ...
1
vote
1answer
115 views
Trend of a few time points
Suppose we are given a matrix of many rows (different genes for example) and few columns (different time points) and we want to identify the top rows (genes) that are following a trend, like ...
2
votes
1answer
395 views
When do we use the Durbin-Watson test?
Isn't this test for the determination of auto-correlation of residuals only necessary when time is some sort of a factor in the observed variables?
As it is I had a data-set that had one dependent ...
3
votes
5answers
275 views
Conversion between units of measurement
I have two different measuring instruments, A and B, both measure the same physical quantity but with different unit of measures: $u_A$ and $u_B$.
A is a reference instrument.
I measured a reference ...
5
votes
2answers
252 views
Why does log likelihood function for a model use SSE/n and not SSE/df?
I'm trying to find out how log-likelihood function works for linear regression. I found the formula here and here. Making some experiments with it (see code below), I was quite surprised that the ...
1
vote
0answers
872 views
Inverse Mills ratio after OLS
tl;dr:
Is it possible to create a dependent variable in the first step of the Heckman Selection model such that it is possible to obtain the values for the calculation of the Inverse Mills Ratio for ...
3
votes
0answers
112 views
Predicting a dichotomous variable
I have a series of descriptors, some continuous, some discrete and an output variable which is dichotomous.
I have several parameters, but for the sake of simplicity let's say my data look like:
...
0
votes
0answers
66 views
Deciding which covarates should be log-transformed, before model is done?
I understand that a good way to see if the covariate should be transformed is to do a linear model and plot the residuals. if there is a pattern, a log-transformation may be needed.
but.. I am right ...
1
vote
0answers
100 views
How exactly are standard errors of coefficients (factor levels) computed in a two-way ANOVA?
In one-way ANOVA ($Y_{ij} = A_i + \epsilon_{ij}$) the standard errors of coefficients are computed pretty easily:
$$SE_{\widehat{A_{i}}} = \sqrt{\text{mean residual sum of squares} \over \text{size ...
3
votes
1answer
160 views
ANCOVA/ANOVA: Testing different models against each other
I have a dependent variable $Y$, an independent variable $X$, and a categorical independent variable T (which can take 2 levels).
Now I have 4 models:
MI: different slopes, different intercepts
...
1
vote
1answer
144 views
How to determine the best relationship (linear, log, etc.) between input predictor variable(s) and output variable for multiple linear regression?
I am trying to determine the most accurate relationship between two variables (each predictor versus the output eventually). I want to know if the relationship is linear, or log-linear, or log-log, or ...
-1
votes
2answers
232 views
Trendline formula in R [closed]
I have created a simple scatterplot to analyse the correlation between two variables, x and y. Using the abline command, I added a trendline to this data. I now want to know the formula for this ...
4
votes
3answers
250 views
Simple introduction to linear models in R
It was hard for me to understand linear models in R. There are a lot of documents for the case, but many of them are technical manuals rather than teaching the concept.
I found this article really ...
0
votes
2answers
351 views
Implementation and Interpretation of Fixed versus Random Effects
I am reading an article which uses a simple least squares model to measure the effect of a prevention campaign on methamphetamine use (http://www.ncbi.nlm.nih.gov/pubmed/20638737). In its second ...
3
votes
2answers
103 views
Is matching only for treatment effects with selection bias?
Propensity score matching (and other matching techniques) are used, as far as I have seen, exclusively for identifying causal effects of a treatment (intervention) and particularly where there is a ...
1
vote
0answers
133 views
Doubt on Covariance Matrix in Weighted Least Square Estimation
This is a page from the book linear algebra, geodesy and gps by Gilbert Strang. The page explains about the justification of the inverse of the of the variance-covariance matrix of measurement ...
3
votes
3answers
1k views
Does lm() use partial correlation - R Squared Change?
I come from an SPSS background and am attempting to move to R for it's superior flexibility and data manipulation abilities. I have some concerns however as to ...
1
vote
1answer
81 views
learning R-understanding calculations for specific variable [closed]
Clarify why the prediction for a female in the example below in related topics (taken from a question by @MsSnowy) do we use the new calculations and not the original lm:
...
4
votes
1answer
247 views
What is the equivalent of a standard deviation when considering a least squares fit line?
I am calculating a tolerance interval following http://www.itl.nist.gov/div898/handbook/prc/section2/prc253.htm but this says to multiply the k value by the standard deviation of the sample. I have a ...
3
votes
2answers
350 views
Why does $\overline{y} = \hat \beta_{0} + \hat \beta_{1} \overline{x}$ in simple linear regression?
Today, once again, I observed that the dependent variable was predicted to be its mean when the independent variable was set to its mean in simple linear regression.
Let $(\hat{y},\hat{x})$ be ...
