Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

learn more… | top users | synonyms

1
vote
1answer
17 views

Linear discriminant analysis posterior not giving expected values in R

I have two normal distributions fg and bg with mean (mu) and standard deviations (sd) as follows: ...
1
vote
0answers
12 views

Impact of spurious regressors on out of sample prediction error

The true DGP is \begin{equation} y=\alpha_0 + \alpha_1 x_1 + \dots + \alpha_k x_k +\epsilon, \quad \epsilon\sim \mathcal{N}(0,1)\label{eq:1} \end{equation} but we instead estimate \begin{equation} y=...
3
votes
1answer
42 views

What are periodic version of splines?

In this What's wrong to fit periodic data with polynomials? post, I tried to use Fourier basis expansion and Polynomial basis expansion to fit a toy periodic data (daily temperature data set). I ...
0
votes
2answers
20 views

Effect of quadratic term when variable's range is negative

I am running a Linear Model where I want to include a quadratic term. The dependent and the explanatory variables are all in logarithmic terms. Further, due to Log transformation the range of X is ...
1
vote
1answer
30 views

Relation linear regression and ARMA models

I have data from a time series which I am currently fitting with a linear model. For that Im using the data as cross-sectional data, where each response corresponds to the value of each variable on ...
-2
votes
0answers
76 views

What are common ill-conditioned matrices in linear system? [closed]

Matrix condition number is very important because many problems are ill-conditioned and cannot be reliably solved using double precision computer systems. Here is what I know that could happen in ...
0
votes
0answers
21 views

regression of a level vs a change variable

This question has likely been asked already but due to the lack of proper terminology I might not have been able to find google up the relevant questions. We have data for several years: 2000, 2001, ....
0
votes
0answers
13 views

Using differences or ratios in regression

Is it always wrong to use ratios in linear regression? For example, If I am trying to fit a linear model and I have a predictor given by: average age of team A / average age of team B should i ...
1
vote
0answers
20 views

Help understanding Linear Model in ESL book

Also known as "Nate slowly deciphers ESL to conceptual understanding/plainer language", part two (see part one) Help me understand this (bullets added) The term $\hat{β}_0$ is the intercept, also ...
0
votes
0answers
11 views

Missing rows ANOVA in R [closed]

Why does the S:x1 column disappear (presumably S:x1 goes into ID but I dont know why)? S is a factor, x1 is a covariate and ID is a factor. ...
1
vote
0answers
13 views

Linear model for testing a ratio of ratios

Our experimental design is as follows: For each of two genotypes (wt and ko), we perform two different gene expression assays (Assay1 and Assay2), and do 4 replicates of each assay. We are interested ...
0
votes
1answer
23 views

Performing limma on residuals or including batch and other covariates in the model?

I am trying to analyse a gene expression dataset of about 240 samples (Illumina microarray). There are multiple questions I need to ask (healthy vs diseased, effect of treatment in diseased, ...
0
votes
1answer
30 views

Design/Contrast Matrices and Unestimable Coefficients

I am trying to analyse microarray data from samples with the following characteristics: one of two genotypes, a procedure either carried out or not carried out, and, in the case that the procedure is ...
4
votes
1answer
49 views

Constrained optimization algorithm in linear regression

I am interested in the following constrained parameter estimation in linear regression, $$ \min_\beta\sum_{i=1}^{n}(y_i-x_i\beta)^2 + \lambda \sum_{j}^{p}f(\beta_j) $$ where the model is $y=x\beta+e$, ...
8
votes
3answers
457 views

Residuals in a linear model are independent but sum to zero; isn't it a contradiction?

The sum of the residuals in a linear model equals zero. The residuals in a linear model are independent. Isn't it a contradiction?
2
votes
1answer
71 views

How do I build a regression model with integer constraints on parameters?

My question is similar to: How do I fit a constrained regression in R so that coefficients total = 1? except that I am interested in a solution to the following constraints on the parameters: All $\...
4
votes
3answers
114 views

In linear regression, is there any meaning for the term $X^Ty$?

Recently, I was wondering about this question. In a standard linear regression problem ($y=X\beta$ and we solve for $\beta$), the solution is $\beta = X^{-1}y$ when $X$ is square and invertible, and $...
1
vote
1answer
28 views

ReLUs and Gradient Descent for Deep Neural Nets

I understand that ReLUs are used in Neural Nets generally instead of sigmoid activation functions for the hidden layer. However, many commonly used ReLUs are not differentiable at zero. Gradient ...
7
votes
1answer
73 views

What is the relationship between the function $\mathbb{E}(Y \mid X = x)$ and linear regression?

Consider the function $$ r(x) = \mathbb{E}(Y \mid X = x) $$ This has been called the regression function in a textbook I'm using. I'm trying to figure out the relationship between this function ...
1
vote
1answer
51 views

Circularity in Linear Regression: Independent variable used as dependent in the same model

I have a dataset with at Customer-Date level. I want to fit a line on the data estimating spend of a customer on a certain date. One of the covariates I am using in the model is historical sales of ...
2
votes
2answers
68 views

Linear “self” regression, terminology and references?

Suppose that $X_i, i=1,\ldots,n$ are some random variables. I'd like to do multiple linear regression to learn to predict any of these variables from the others. My model for the reconstructed ...
0
votes
1answer
59 views

Predict results of Elections

I have information on the votes in my town and in the country. I want to predict the results in the country's elections knowing the results in my town. What methods I can use? I have thought of ...
0
votes
0answers
21 views

Regression with some observations having more than one factor level

I have data I want to analyze using multiple regression or machine learning: the response is cells for which I measured viability (a continuous response) and the independent variables are the genes in ...
2
votes
0answers
25 views

Correlation of 2 categorical variables in linear model

I have this dataframe with two categorical variables (Sex and Ethnicity (only Asian or European)) and I need to fit a linear model to estimate the weight of a fetus given the day of the echography and ...
1
vote
0answers
25 views

What is a good way to know if a variable adds value to an existing regression model without its components

Suppose someone gives you the fitted values to a regression model with $k$ terms, along with the fitted coefficients. If this is all you have, and you are investigating an additional "term" or set of ...
0
votes
0answers
5 views

Orthogonal projection onto additive subspace (normal linear model)

It is known that in a normal linear model $X \sim N(\xi, \sigma^2 I) \in \mathbb{R}^N$, if we group our observations using a factor $F$, i.e we claim that $\xi$ lies in the subspace $L_F$ where ...
1
vote
0answers
18 views

Covariance-residual technique for linear regression feature selection

When doing forward feature selection for linear regression, it is a well known trick that to select the next feature to add, we can compute the covariance of each candidate feature against the current ...
0
votes
1answer
43 views

Regression with a ratio as an independent variable

I'm regressing a response versus a ratio between two measurements as an independent variable. I'm getting a significant positive effect and I'd like to test whether the contribution of the increase in ...
1
vote
1answer
67 views

Fixed Prediction Interval

I want to place a Multiple Regression model into a production system and use the Prediction Interval as a threshold for anomalies. I've seen how I can calculate the Prediction Interval two ways: $$...
0
votes
2answers
51 views

Why should we not perform linear regression to predict ordinal dependent variables?

I am reading An Introduction to Statistical Learning. In this book, section 4.2, page 130 (text given below) mentions that linear regression would not be useful to predict ordinal variables because ...
1
vote
0answers
46 views

False discovery rate for across independent models with the same explanatory variables

I have a dataset of >10000 gene expression profiles and I would like to test the effect of 3 explanatory variables and their interactions on each gene expression profile using information criterion (...
0
votes
1answer
38 views

How does alpha relate to C in Scikit-Learn's SGDClassifier?

I'm trying to get the same linear SVM classifier model by using Scikit-Learn's SVC, LinearSVC and ...
0
votes
0answers
68 views

In linear regression, how do I extrapolate parameters obtained using preprocessed data?

Where $h_{\theta} = \theta_{0} + \theta_{1}x$, I am trying to minimize $J(\theta) = \frac{1}{2m}\sum_{i = 1}^{m}(h_{\theta}(x^{(i)}) - y^{(i)})^{2}$ I first transform every sample in the feature (...
0
votes
0answers
12 views

Can I prepare multiple linear model for a single problem by dividing independent variable into multiple clusters?

I have a single problem of predicting the expenses of customers? For that I have a linear model which is performing good for middle range of Income of family. This model has limitation at higher ...
0
votes
0answers
14 views

Reduction of models

In a normal linear model, I have working with 3 categorical factors, $A, B, C$, and my main model is the interactions model $A*B*C$. My question is as follows: If i wanted to see how far I can ...
3
votes
1answer
41 views

How to prove that $Cov(\hat{\beta},\bar{Y}) = 0 $ using given covarience properties

To quote: It is well known that, if $W_1, ..., W_n, Z_1, ..., Z_m$ are random variables and $a_1, ..., a_n, b_1, ..., b_m$ are constants, then $Cov ( \sum_{i=1}^n a_iW_i, \sum_{j=1}^m b_jZ_j) = \...
0
votes
0answers
25 views

Using confidence intervals with Simple Linear Regression

So simple linear regression is performed on 3000 data points, and 1000 data points are withheld. How can we use confidence intervals, along with the withheld data points, to assess the predictive ...
0
votes
1answer
17 views

Effect in linear model versus effect in mixed model

Consider a dataset with 3 observations pertaining to 5 patients. This can be modeled in several ways, two of which are that $$ X_{ij} = \xi_i + Y_j + \epsilon_{ij}, $$ $i = \{1,..,3\}$, $j = \{1,...,...
0
votes
2answers
29 views

Which of the 3 cases should my data matrix belong to ideally?

I found this question, and while useful, I wanted to ask something more spcific: I am trying to get a good handle/intuition for the two types of data dimensionalities (number of data samples, and the ...
2
votes
1answer
26 views

Linear mixed model with partially crossed effects

I'm new to Linear Mixed Models and I'm not sure if I'm specifying the right model. I'd appreciate any feedback that confirms / disproves my model. Here's some background about my data: I have a list ...
0
votes
1answer
24 views

How to apply linear regression to one sensor so that it will match readings from better sensor? [closed]

I have one sensor which has the best accuracy and the other sensor which I want to calibrate using some linear regression (or something else?) - by modifying the software. How to calculate that linear ...
0
votes
1answer
21 views

Linear regression summary in R: Standard vs car.Anova

I am running some linear regressions in R. I am dealing with a linear dependent and linear as well as categorical independent variables using lm. So far, I have looked at the output that ...
0
votes
1answer
31 views

Can I iteratively transform a variable with log10 until it fits a linear model?

I have a response variable, $Z$, for which I'm trying to make a linear model. Here are some of the fit diagnostics plots: From the fan-like shape of the residual-vs-predicted value plots, I ...
1
vote
1answer
18 views

The prediction at the average of the covariates is different from the average of the predictions

I read in Stata manual : "The prediction at the average of the covariates is different from the average of the predictions" after a logistic regression. If I compute predictions for a linear ...
1
vote
0answers
16 views

Which analysis is best for my data with repeated measures and 2 treatment groups?

I'm working in R. I have a data set of 21 fish, roughly half in each of 2 treatments. I measured their behaviour over 10 minutes and want to analyse this to look for changes over time (gradient) and ...
1
vote
1answer
14 views

Expression for correlation in terms of hat matrix H

In a linear regression model $Y = Xβ +ε$ with $E(ε) = 0$ and $E(εε^ T ) = σ^ 2$ I, let $e_1,..., e_n$ be the residuals obtained from the least squares fit. Derive an expression for the correlation ...
1
vote
0answers
12 views

Leverages and effect of leverage points

I just got some question about the hat matrix in linear models. My first question is Why in a balanced one-way layout $(n_1=...=n_c=n_0)$, all leverages $h_{ii}$ have the same value $\frac{1}{n_0}$? ...
2
votes
1answer
27 views

Prove that $Var(\hat {Y_i})=\sigma^2h_{ii}$

I just got a simple question. In general linear model, we have $$\hat Y=HY$$ where $H=X(X^TX)^{-1}X^T$ and the residual $$E=Y-\hat Y.$$ Now I want to prove that $$Var(\hat {Y_i})=\sigma^2h_{ii}$$ ...
0
votes
0answers
48 views

Why are the both of two models' AIC the same?

I would like to ask a question of AIC when we use Generalized Linear Model with R. I show you 4 my models. "x" is continuous variable. "f" is categorical variable and has two levels, C and T. "x*f" ...
1
vote
1answer
32 views

Regression technque to use for continuous data behaving like ordinal

I am trying to create a model to explain/predict fulfillment ratio of a product by a store i.e orders placed divided by orders delivered.The QQ-plot of the fulfillment ratio is: The QQ-plot of the ...