Questions tagged [linear]

For statistical topics which involve the assumption of linearity, for example, linear regression or linear mixed models, or for the discussion of linear algebra as applied to statistics.

388 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
7
votes
0answers
108 views

Is multivariate Cauchy stable?

I am trying to prove (if possible), for given $A_{n\times n}$ and $B_{n\times n}$, there exists a $C_{n\times n}$ satisfying $$A\pmb{X}_1 + B\pmb{X}_2 \stackrel{D}{=} C\pmb{X},$$ where $X_1, ~X_2$, ...
6
votes
0answers
595 views

Which theorem in Cover's 1965 paper is actually referred to as Cover's Theorem?

Cover's Theorem is stated on Wikipedia (and similarly elsewhere) as A complex pattern-classification problem, cast in a high-dimensional space nonlinearly, is more likely to be linearly separable ...
6
votes
0answers
893 views

How to test a linear relationship between log odds and predictors before performing logistic regression?

In case of a linear regression, it's easy to test a linear relationship between a continuous dependent variable and each independent variable. For example, I can plot a scatter plot between the ...
6
votes
0answers
3k views

Is the Naive Bayes family of classifiers linear?

There are a lot of places where you'll see the proof that Naive Bayes classifiers are linear, like this and this. But they always assume a special case of the family of Naive Bayes classifiers which ...
5
votes
0answers
487 views

Is lasso always outperformed by adaptive lasso?

I have been reading some papers and I understood that adaptive lasso has the Oracle properties which lasso lacks. Does that mean adaptive lasso always better than lasso (let's focus on the simple ...
4
votes
0answers
102 views

Method to forecast correlate univariate time-series (with trend, seasonality) via regression

I have two univariate time-series with seasonality and trend--dt1 and dt2. I believe that dt1 and dt2 are strongly correlated, both through a few statistical test (see below) and that in my field dt2 ...
4
votes
0answers
283 views

Do results from the lme-function require adjustment of p-values?

If I run a linear mixed model with the lme() function and get results like these (comparing the score of 4 treatment groups against a placebo group): ...
4
votes
0answers
362 views

Analytical linear regression vs cost function

I'm trying to fit data to a line $y=ax+b$ (you can download it in this gist). You can see a fit in the next image: To do so, I'm using analytical methods of minimization of the cost function, defined ...
4
votes
1answer
135 views

Linear and Nonlinear Components of a Time Series

I am starting to develop a hybrid ARIMA-ANN model for forecasting. Most of the journals I read mention mostly a linear component for ARIMA and a nonlinear for ANN. How can you know which components ...
4
votes
1answer
94 views

Decomposing a locally stationary covariance matrix

Say I have a non-stationary Gaussian Process with a square exponential covariance whose shape varies throughout space. The covariance entries are: $$ K_{ij} = N(|x_i-x_j|,\sigma_i^2+\sigma_j^2) $$ ...
4
votes
0answers
140 views

Adjust linear regression penalty for under/over-estimations

Basically I have a case where under-predictions are worse than over-predictions. Is there a way to penalize the linear regression model during training according to some predefined ratio? E.g. I want ...
3
votes
0answers
44 views

What is the significance of “orthogonal” vectors in statistics?

So I am reading What does orthogonal mean in the context of statistics?, and there are contradictory answers. The most upvoted answer says that "Therefore, orthogonality does not imply ...
3
votes
0answers
38 views

log transform in linear regression

Assume we have a data set and the theory suggests to model $Y \sim X$. We apply a simple linear regression and get the following: Next, let us make a log transform of both $X$ and $Y$. The result is ...
3
votes
1answer
167 views

Clustered standard errors and time dummies in panel data

Assume a simple linear regression model, I have $i$ firms and $t=17$ periods $$Y_{it}=\alpha + \beta_2 T_2 + \beta_3 T_3 + \cdots + \beta_{16} T_{16} + \gamma_i + \varepsilon_{it}$$ In this case, $t=...
3
votes
0answers
25 views

How to set up contrasts in a linear regression models that involve averaging over levels of another factor?

For simplicity, assume we have a linear model which looks like this: Outcome = beta0 + beta1*Treatment + beta2*Time + beta3*Treatment*Time + error where ...
3
votes
0answers
44 views

Linear Regression done simultaneously vs in parts

So I have a factor model which looks like: $x_{i}(t)=\beta_{i0}F_{0}(t)+\sum_{n=1}^{N}\beta_{in}F_{n}(t) + \epsilon_{i}(t)$ I know that $F_{0}(t)$ is linearly orthogonal/uncorrelated to $F_{n}(t)$ ...
3
votes
0answers
56 views

In linear regression, is there any relationship for beta and the beta of its rotated version?

In a linear regression problem, we have: $$ y = x\beta_1 + \epsilon $$ For the convenience of discussion, let's just assume this is a univariate regression problem. (i.e. both $x$ and $y$ are vectors ...
3
votes
0answers
160 views

Cross-validation and testing for linear regression on small heteroscedastic data sets

I would like to perform simple linear regression on a data set ($y_i = a x_i + b + \epsilon_i$) with $N \approx 50$. However, my residuals $\epsilon_i = \epsilon_i(x_i)$ exhibit heteroscedasticity as ...
3
votes
1answer
69 views

Linear mixed model inner working

My question is about the basics of linear mixed model and it may be trivial for some of you. Basically, I followed from this tutorial http://www.bodowinter.com/tutorial/bw_LME_tutorial2.pdf on how to ...
3
votes
0answers
50 views

How Stochastic Binary Neurons works with rectified linear (or Linear threshold) case?

I'm taking a course in coursera about neural networks. I understand the relation and difference between "Sigmoid Neurons" and "Stochastic Binary Neurons", but I don't how could you adapt "Rectified ...
3
votes
0answers
557 views

How appropriate is it to fit a regression line through median values?

I have a data set of tone separation (ranging from 1-3 octaves, plotted on the x-axis) vs. subject performance (y axis) through which I am fitting a regression line. The problem is that I have many ...
3
votes
0answers
559 views

When do we need to use one overall P-value for a categorical variable in linear regression analysis?

I have seen a previous interesting post about how to test the significance of a categorical variable in linear regression How to test the statistical significance for categorical variable in linear ...
3
votes
0answers
177 views

Linear model with proportions as the dependent variable

I have a linear model where the dependent model is a proportion (or a probability). I know I should probably use another type of model, and I am. I would still like to use also a linear model because ...
3
votes
0answers
555 views

How to check variables changeable dependency in R?

I would be very thankful if somebody could help me and explain the answer in simple terms. I have y, x, z variables. They are countinuous, no missing values. y is dependent variable, x and z ...
3
votes
0answers
453 views

repeated measures ANCOVA; covariate main effect and post-hoc tests

let's say I run a repeated measures ANCOVA, with valence (negative and neutral) as repeated measure and age as a covariate. what I want to know is how age affects my data. 1)I get a significant ...
2
votes
2answers
53 views

Linear model with both additive and multiplicative effects

In linear regression, the independent variables have an additive effect on the response (level-level regression): $y=\beta_0+\beta_1x+\epsilon$ In a log-level regression, the independent variables ...
2
votes
0answers
22 views

Test for statistically significant difference in means across different conditions

I've never had a formal statistics course, so I hope these questions aren't too basic or using incorrect terminology. I have samples of bivariate data across different scales of 3 kinds, which I ...
2
votes
1answer
28 views

Checking the constant variance assumption for residuals vs fitted plots: What about for the same fitted values?

For a residuals vs fitted plot, we use the fitted values $\hat{Y} = \beta_0 + \beta_1 + \cdots + \beta_p x_p$ on the horizontal axis and the residuals on the vertical axis, and then compare the ...
2
votes
0answers
86 views

Definition of a simple linear regression model

A while ago I was trying (not entirely successfully) to figure out the definition of a regression model. Now I am narrowing it down to a simple linear regression and trying to identify (loosely ...
2
votes
0answers
21 views

What are the available method that can alleviate the overfitting problem in traditional OLS problem, but still can get a linear fitting?

Recently, I have read the paper https://static1.squarespace.com/static/56def54a45bf21f27e160072/t/5a0d0673419202ef1b2259f2/1510803060244/The_Sampling_Error_in_Estimates_of_Mean-...
2
votes
0answers
19 views

How to fit points to piecewise linear model where all slopes must have the same absolute value?

The current methodology for the genomic data I have involves fitting a spline to multiple points. However, the underlying biology does not support that the fit should be curved at any points. In fact, ...
2
votes
4answers
79 views

Why does linear non-logistic regression work as a linear classifier? What classification error does it minimize?

Suppose the data has two attributes and a label -1 or 1. So, we have a three-column matrix $X$ (two attributes and a column of ones for convenience of working with matrix notation) and a column vector ...
2
votes
1answer
32 views

Does the size of the design matrix change for estimation vs. predictions?

Say I have the model $y = \beta_0 + \beta_1 x_1 + \cdots \beta_p x_p + \epsilon $. Using $n$ observations of data I formed the system of equations $\mathbf{y} = X\beta + \epsilon$ for least squares ...
2
votes
0answers
54 views

Maximization of quotient of quadratic forms in linear regression

I would like to find maximum of the following function: $$I = \max_{a\in \mathbb{R}^p} \frac{(a'\hat{\beta})^2}{S^2a'(X'X)^{-1}a},$$ where $X$ is a design matrix and of course $Y$ is normally ...
2
votes
0answers
86 views

Conjuage priors for linear combination

My knwoledge of statistics in general and Bayesian statistics in particular is limited. With that in mind, I would sincerely appreciate if somebody could help me with the following problem that I have ...
2
votes
0answers
21 views

Is it reasonable to include both independent variables and their interaction terms in a linear predictive model?

Suppose i have outcome variable O (effective/not effective), drug dosage A and variables B&C, and i want to train a linear model to predict O. B&C can only influence O by modifying metabolism ...
2
votes
0answers
28 views

How to interpret the learning curve of a LinearRegression for sparse data?

I have a dataset of shape ca.(4800, 350). Both the dataset X as well as the response y is very sparse (ca. 3500 samples with y=0). I wanted to take a look at the learning curve to estimate the bias-...
2
votes
1answer
61 views

Cross-validation on small dataset

I have a very small dataset (n=42) and use simple bivariate linear regression to predict the target variable. As i understood from several posts on this webpage, in case of small datasets one should ...
2
votes
2answers
693 views

Why perceptron is linear classifier?

It is said that perceptron is linear classifier, but it has a non-linear activation function f = 1 if wx - b >= 0 and f = 0 otherwise If i will use some non-linear function on linear combination of ...
2
votes
0answers
18 views

Assumption testing for a large amount of individual regression models and averaging R-squared?

I am running the following model for my thesis, a simple regression: \begin{equation} y_{i,t} = \alpha_t+\beta_tx_{p,t}+\varepsilon_{i,t} \end{equation} where Y is an observed variable (...
2
votes
1answer
31 views

Linear Regression in stats model OLS

After removing the insignificant variables(p-value >.05), I fitted the OLS model again. I found there are still many variables which had p-value < .05 earlier have p-value > .05 now. Do I need to ...
2
votes
0answers
86 views

How to deal with correlated regressors in a multiple regression model?

i currently try to estimate the effect of different task parameters (IV) on neuronal activation (DV). Some predictors in my design matrix(trials x features) are moderately correlated (r~=.3) and I ...
2
votes
0answers
87 views

Linear Mixed Model for evaluation of students

My dataset about students (n=74) contains one outcome variable (exam points/integer) and eight predictor variables: 2 categorical: gender [F,M] study years [1,2,3] 6 continuous variables: age [in ...
2
votes
2answers
4k views

Linear Regression For Binary Independent Variables - Interpretation

I have a dataset where I want to predict inflow (people joining a platform) but my all independent variables are binary categorical (0,1). Whereas I want to predict continuous variable (inflow -- ...
2
votes
0answers
47 views

significance in linear regression with constraints

I have a problem which is similar to linear regression, but differs in two main points: 1) the number of regressors is equal to the number of observations and 2) I have constraints on the regressors. ...
2
votes
0answers
78 views

Residualizing in OLS

Consider a linear regression $$ y_i = A_i' \theta + B_i' \psi + \epsilon_i $$ There's a "trick" to find the parameters $\theta$ and $\psi$ in a stepwise fashion. First minimizing squared error wrt....
2
votes
1answer
87 views

Data Transformation to achieve Linearity

One assumption of OLS regression is Linearity. To check whether the assumption holds, you can plot component + residual plots or partial residual plots. When a linear relationship is apparent, is's ...
2
votes
0answers
269 views

Linear Prediction and Linearity of CEF

I am revisiting the basic notions of linear regression and stumbled upon the following idea in Cameron and Trivedi's Microeconometrics book: However, for the conditional mean to be linear in x, so ...
2
votes
0answers
216 views

Produce MSE by using cv.glm() in multiple linear regression, how about a transformed y variable

I applied 10-fold cross validation by using cv.glm() function in the linear regressions. I am able to obtain the MSE in this way, mse=cv.glm(data,model,K=10)$delta However, if I applied ...
2
votes
0answers
38 views

Definition of stable distribution

In some places, I find the following definition of stable distribution: A distribution is said to be stable if a linear combination of two independent random variables with this distribution has ...

1
2 3 4 5
8