Questions tagged [linear]

For statistical topics which involve the assumption of linearity, for example, linear regression or linear mixed models, or for the discussion of linear algebra as applied to statistics.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

Composite Score System and Linear Regression

In order to analyse the influence of a treatment in recovery quality after general anesthesia we used a composite score system to appreciate recovery quality (from 11 to 100 points). Now in order to ...
2
votes
1answer
49 views

Linear trend when $y = \infty$ if $x = 0$

If I'm trying to establish a linear relationship between effort (watts produced or speed, for example) and time-taken. How can I account for the fact that if effort is zero (speed equal zero or watts ...
0
votes
0answers
20 views

Piecewise linear regression notation

I want to write out the following regression in $X\beta + \epsilon$ form. $Y = \begin{cases} \beta_0 + \beta_1 x_1 + \beta_2 x_2 &\text{if high school education}\\ (\beta_0 + ...
1
vote
1answer
45 views

Pearl's Causal Inference In Statistics: Study Question 1.5.1

Problem Statement: Suppose we have the following Structural Causal Model (SCM). Assume all exogenous variables ($U$) are independent identically distributed standard normals. \begin{align*} V&=\{...
0
votes
2answers
34 views

Which formula fits better for this Linear Mixed-Effects Model?

I am currently analyzing a dataset that contains a list of flight simulator tests performed by different pilots. I want to analyze if a certain flight parameter (i.e. amount of input errors during ...
0
votes
3answers
56 views

Showing $Cov(\epsilon, b_0) = 0$ or $Cov(\epsilon, b_1) = 0$

I am not sure where to start with this. But I know for $Y_i = \beta_0+\beta_1X_i+\epsilon$, where $\beta_0,\beta_1,X$ are assumed to be constants and $\hat{Y_i} = b_0+b_1X_i$ is the simple linear ...
0
votes
0answers
10 views

Re-randomizing training/CV/test sets for each prediction?

I'm currently randomizing my training, cross-validation, and test sets to calculate the appropriate parameters for a regression model. Although, I was curious to know, if I optimise my parameters, ...
1
vote
1answer
19 views

Comparison of Models for Margin and Win Prediction

I recently created two models for predicting the outcomes of matches for a particular sport, one is a linear regression model that predicts the margin of the match, and the other is a logistic ...
0
votes
0answers
25 views

what's the difference between these two residual plots?

I have a table of x,y data points in a text document called "data.txt" ...
1
vote
1answer
43 views

Is decision boundary of penalized logistic regression linear?

The decision boundary of SVM is a straight line. If we use e.g. RBF kernel, decision boundary is linear in hilbert space, but it the original space it is non-linear. I assume that the logistic ...
4
votes
1answer
118 views

Including indicator variables in linear regression

I have a specific question about indicator/ dummy variables in a model. Right now, I have a set of data over about a year, with various variables such as temperature and operational units. Also in ...
1
vote
1answer
51 views

Algorithm for simple linear regression that is efficient and numerically stable

I'm developing an application that is fed with continuous data while older data is discarded. I'm using some algorithms to compute simple linear regression on these data with Perl. Basically that ...
1
vote
0answers
76 views

Normalization for solving linear equations

Suppose I want to solve a linear equation system in the form of $$A x = b$$ to get $x$, where $A$ are $n$ by $n$ matrix and $b$ is $n$ by 1 vector. Is there any normalization procedure necessary ...
1
vote
0answers
38 views

Linear regression of line in 3-dimensions

I am trying to regress a line (not a surface) in 3-dimensions and having a total brain freeze for how to do this. What I have is a data set that (I think) defines a line in 3-dimensions. So in my ...
0
votes
0answers
35 views

Minimizing RSS in linear regression

Assume that I want to estimate the coefficients in a linear regression model by minimizing the RSS for the first $p$, where $p<P$, regressors: $$ \sum_{i=1}^n \left( y_i - \beta_0 - \sum_{j=1}^p \...
3
votes
1answer
40 views

Linear mixed effect model in statsmodel package

I try to use linear mixed effect model in Python statsmodels package. However, I have no idea how to conduct and interpret the result. Group 1 (20 people) : base line & follow up Group 2 (20 ...
1
vote
0answers
17 views

Breusch-Pagan-Godfrey and estimation of sigma

From what I know, we usually estimate $\hat{\sigma}^2 = \sum_{i=1}^n \frac{(y_i - \hat{y})^2}{n-p-1}$. For the Breusch-Pagan-Godfrey test, my teacher told us to use $\hat{\sigma}^2 = \sum_{i=1}^n \...
2
votes
1answer
109 views

Can non-linearly separable data always be made linearly separable?

A data set that is linearly separable is a precondition for algorithms like the perceptron to converge. It's well-known that we can project low-dimensional data to a higher dimension using kernel ...
0
votes
0answers
31 views

Bayesian Linear Regression: pyMC3 vs. analytical solution

I was re-poducing the plots by Bishop 2006 p.155 and thought it might be interesting to compare both, the analytical solution provided by Bishop and an approximated MCMC soultion with ...
4
votes
1answer
39 views

What algorithm to use for fitting several different lines

I have a unique problem I'm not sure how to approach. I have some data. The data was generated by a function that's basically $k$ different lines ($k$ may or may not be given). Example: However, ...
2
votes
1answer
83 views

calculate $R^2$ given $\hat{\sigma}_{\hat{\beta}_{i}}$

Statistics newbie here. I have the following situation: where $\sigma^{2}$ is the variance of the disturbance for the model $y_{t}=\beta_{1}x_{t,1}+\beta_{2}x_{t,2}+\beta_{3}x_{t,3}$ $\hat{\sigma}^{...
2
votes
1answer
31 views

Multiplying bivariate gaussians by a constant

Say I have the following : $$ (X, Y) \sim N_2(\mu, \Sigma) $$ Then what would be the distribution of $(2X,2Y)$ ? Let $\Sigma = \begin{pmatrix} \sigma_1^2 & \rho\sigma_1\sigma_2\\ \rho\sigma_1\...
2
votes
0answers
86 views

Definition of a simple linear regression model

A while ago I was trying (not entirely successfully) to figure out the definition of a regression model. Now I am narrowing it down to a simple linear regression and trying to identify (loosely ...
2
votes
1answer
46 views

Why the weight vector is a linear combination of the inputs and the outputs in the Perceptron

I was studying Support Vector Machines and I've got stuck with this relation regarding the weight vector of the hyperplane. $w=\sum\limits_{i\in I}^{} y_i x_i$ For reference, I'm studying from the ...
1
vote
0answers
56 views

Derivation of beta hat 1 from the simple linear regression equation

I have a no-intercept relationship: $$y_{i} = \beta_{1}x_{i} + \varepsilon_{i}$$ where $\varepsilon_{i} \sim \text{ iid } \mathcal{N}(0, \sigma^{2})$, and $i = 1, \dots, n$. How do I derive $\hat{\...
0
votes
0answers
8 views

Matlab - Financial Modeling, Linear Regression with Prior

Am trying to implement this equation from the book Doing Data Science Straight Talk from the frontline, In chapter 6, page 161, equation below: From what i can tell it is pretty much an enchanced ...
0
votes
0answers
23 views

multiple time series but index with progression rate

I have a dataset with many object, each object is identified by an id, and many observations indexed by progression rate, like bellow My objective is to find which algorithm use to ...
1
vote
0answers
136 views

standard error on the slope from a linear regression

Suppose you are running an experiment where you measure a (positive) variable $A$ of a given physical system at multiple times, say $t = \{ 0, 10, 20, 45, 90, 120, 180 \} \ min$, and calculate $y(t) = ...
4
votes
2answers
337 views

How to prove $\beta_0$ has minimum variance among all unbiased linear estimator: Simple Linear Regression

Under the condition of simple linear regression model ( $Y_i = \beta_0 + \beta_1X_i + \epsilon_i$) ordinary linear estimators($\hat{\beta_0}$ and $\hat{\beta_1}$) have minimum variance among all ...
1
vote
1answer
81 views

How to express the PCA principal components as a linear combination of input data?

In this paper in equation 1 it shows that the principal component vectors are the eigenvectors of the covariance matrix and gives the following equation $$\lambda_l\psi_l=C\psi_l$$ where $C$ is the ...
1
vote
1answer
48 views

In making scatterplot for correlations between two continuous variables, can we use the choice cubic instead of linear choice [closed]

In making scatterplot for correlations between two continuous variables, can we use the choice cubic instead of linear choice in "create a fit line at total", as shown in the copied Figure, please? ...
0
votes
0answers
14 views

My training and cross validation set aren't matching

I'm new to machine learning and currently I'm doing linear regression. I split my training set into the cross validation set, the test set and the training set. The hypothesis that you see here, is of ...
3
votes
1answer
81 views

Modeling low-cardinality dependent variable continuous linear regression

What problems, if any, would exist if I were to treat a dependent variable with relatively low cardinality (e.g. 10 distinct values) as continuous versus binary (the latter requiring that I create ...
1
vote
0answers
19 views

Colinearity counter example in Linear Regression [closed]

I am trying to understand the colinearity assumption for linear regression. I have produced this counter example which I can't explain: Suppose we are modelling the sales of a shop by the sea and on ...
1
vote
0answers
30 views

Regression F and t statistic

My understanding was that, to make a hypothesis test of a linear combination of regression parameters (e.g. of the type $\beta_1+2\beta_2=0$), you should use a decision rule of the form $|T| = \left| ...
1
vote
1answer
34 views

When using Lasso and calling coefficients (.coef_) which is the coefficient of the constant? [closed]

By calling .coef on the Lasso model built, there are only numbers corresponding to the coefficients. These coefficients are supposed to match, say, the columns of the pandas dataframe given as input. ...
1
vote
1answer
19 views

Bias in P-value of MM-type estimators or Cochrans Q Penalized Regression

There are a number of linear regression methods designed to limit the influence of outliers on estimates: For example, Cochrans Q Penalised regression as described in [1] will do an initial linear ...
0
votes
0answers
13 views

What to use: linear glm wls etc?

This is probably a basic enough question. What I want to achieve is a regression analysis of lapse rates on savings type policies. Say in one year person A withdraws 10 out of 100, and person B ...
1
vote
1answer
33 views

Can I use the interaction between a dummy variable and the variable it was derived from?

I am trying to make a multiple linear regression model. I have a hypothesis that $x$ is a significant predictor of $y$ but only when $x > 0.5$ ($x$ ranges from -2 to + 2). Is it acceptable to ...
1
vote
1answer
56 views

Will a regression model be linear if cross term is included?

I am reading a book on multiple linear regression by using MATLAB. The example shows a case when a cross term is included as $$ Y = \beta_0 + \beta_1XT + \beta_2X^2$$ In MATLAB, we rewrite the ...
1
vote
1answer
29 views

Linear model and confidence level issues in R [closed]

Please again accept my apologies for my little knowledge in R. I', trying to get better! you help me so much, but im a biologist and my statisc knowledge is sadly low I have the following data set: <...
6
votes
0answers
595 views

Which theorem in Cover's 1965 paper is actually referred to as Cover's Theorem?

Cover's Theorem is stated on Wikipedia (and similarly elsewhere) as A complex pattern-classification problem, cast in a high-dimensional space nonlinearly, is more likely to be linearly separable ...
0
votes
0answers
47 views

Why the correlation between intercept and slope becomes zero when x is centered in Bayesian linear regression

I'm learning Statistical Rethinking, Chapter 4 - page 99, on linear regression. The example is simple, fitting a univariate linear model with \begin{align*} y_i & \sim Normal(\mu_i, \sigma) \\ \...
1
vote
0answers
16 views

How does uncertainty of observations propagate through linear regression fits [duplicate]

I'm quite new to statistics, so please bear with :) I'm trying to estimate the uncertainty of a variable which is predicted using a linear equation. The linear equation is estimated with a series of ...
2
votes
0answers
21 views

What are the available method that can alleviate the overfitting problem in traditional OLS problem, but still can get a linear fitting?

Recently, I have read the paper https://static1.squarespace.com/static/56def54a45bf21f27e160072/t/5a0d0673419202ef1b2259f2/1510803060244/The_Sampling_Error_in_Estimates_of_Mean-...
1
vote
2answers
54 views

In NN, the way we have different nonlinear activations, can we have different linear activations?

I am just curious to understand if we can have different linear activations other than $WX+b$? I understand the necessity of weights and biases, but is this the only way out neural net's propagation ...
0
votes
1answer
62 views

Interpretation of (simultaneous) confidence band against fitted values in multiple regression

In a homework question, I am asked to interpret a figure of the confidence band and simultaneous confidence band of 95% confidence level plotted against predicted values. The confidence bands are ...
0
votes
1answer
30 views

Comparing linear regression models under violation of independence assumption

the basic setup is as follows: I have a continuous dependent variable (DV, 7 observations) and two continuous independent variables (IV1 & IV2). I would like to evaluate whether adding IV2 as ...
0
votes
0answers
63 views

Approximate known non-linear function using linear regression

Consider the following model: $$ y_{i}=f\left(\boldsymbol{x}_{i};\theta\right)+\varepsilon_{i} $$ where $y_{i}$ is the dependent variable, $\boldsymbol{x}_{i}$ is a vector of explanatory variables, $...
0
votes
1answer
94 views

derivation of objective function in linear regression

In linear regression, we have a very simple task. This is to measure a distance between Y and y_hat, where y_hat for sake of simplicity is multiplication of X and w. So we can say: Error = Y-y_hat = ...

1 2 3
4
5
20