Questions tagged [linear]

For statistical topics which involve the assumption of linearity, for example, linear regression or linear mixed models, or for the discussion of linear algebra as applied to statistics.

Filter by
Sorted by
Tagged with
1
vote
1answer
36 views

In making scatterplot for correlations between two continuous variables, can we use the choice cubic instead of linear choice [closed]

In making scatterplot for correlations between two continuous variables, can we use the choice cubic instead of linear choice in "create a fit line at total", as shown in the copied Figure, please? ...
0
votes
0answers
16 views

Is it possible to plot a hypothesis for multivariate linear regression? [closed]

I'm new to machine learning and recently I implemented my first linear regression with multiple variables. I have certain questions, 1)Is it possible to plot a hypothesis for multivariate linear ...
0
votes
0answers
9 views

My training and cross validation set aren't matching

I'm new to machine learning and currently I'm doing linear regression. I split my training set into the cross validation set, the test set and the training set. The hypothesis that you see here, is of ...
0
votes
0answers
30 views

Cross validation set answers don't match [closed]

I am new to ML and MATLAB. My cross validation set answers don't match. I have a good hypothesis and with every iteration the value of my cost function decreases too. What should I do?
0
votes
0answers
16 views
+50

Modeling low-cardinality dependent variable continuous linear regression

What problems, if any, would exist if I were to treat a dependent variable with relatively low cardinality (e.g. 10 distinct values) as continuous versus binary (the latter requiring that I create ...
1
vote
0answers
18 views

Colinearity counter example in Linear Regression [closed]

I am trying to understand the colinearity assumption for linear regression. I have produced this counter example which I can't explain: Suppose we are modelling the sales of a shop by the sea and on ...
-1
votes
0answers
11 views

How to make a more accurate predict [closed]

How can I make the program better predict? When you enter the numbers 771, 322, 344, 632, 10, the program predicts 234168, but I need it to be 200000-210000. In linear regression, more than 1000 ...
1
vote
0answers
25 views

Regression F and t statistic

My understanding was that, to make a hypothesis test of a linear combination of regression parameters (e.g. of the type $\beta_1+2\beta_2=0$), you should use a decision rule of the form $|T| = \left| ...
1
vote
1answer
21 views

When using Lasso and calling coefficients (.coef_) which is the coefficient of the constant? [closed]

By calling .coef on the Lasso model built, there are only numbers corresponding to the coefficients. These coefficients are supposed to match, say, the columns of the pandas dataframe given as input. ...
1
vote
1answer
19 views

Bias in P-value of MM-type estimators or Cochrans Q Penalized Regression

There are a number of linear regression methods designed to limit the influence of outliers on estimates: For example, Cochrans Q Penalised regression as described in [1] will do an initial linear ...
0
votes
0answers
12 views

What to use: linear glm wls etc?

This is probably a basic enough question. What I want to achieve is a regression analysis of lapse rates on savings type policies. Say in one year person A withdraws 10 out of 100, and person B ...
1
vote
1answer
31 views

Can I use the interaction between a dummy variable and the variable it was derived from?

I am trying to make a multiple linear regression model. I have a hypothesis that $x$ is a significant predictor of $y$ but only when $x > 0.5$ ($x$ ranges from -2 to + 2). Is it acceptable to ...
1
vote
1answer
46 views

Will a regression model be linear if cross term is included?

I am reading a book on multiple linear regression by using MATLAB. The example shows a case when a cross term is included as $$ Y = \beta_0 + \beta_1XT + \beta_2X^2$$ In MATLAB, we rewrite the ...
1
vote
1answer
29 views

Linear model and confidence level issues in R [closed]

Please again accept my apologies for my little knowledge in R. I', trying to get better! you help me so much, but im a biologist and my statisc knowledge is sadly low I have the following data set: <...
1
vote
0answers
21 views

Which theorem in Cover's 1965 paper is actually referred to as Cover's Theorem?

My question is bolded below, some of my reading notes are also shared for those interested. Cover's Theorem is stated on Wikipedia (and similarly elsewhere) as A complex pattern-classification ...
0
votes
0answers
15 views

Why the correlation between intercept and slope becomes zero when x is centered in Bayesian linear regression

I'm learning Statistical Rethinking, Chapter 4 - page 99, on linear regression. The example is simple, fitting a univariate linear model with \begin{align*} y_i & \sim Normal(\mu_i, \sigma) \\ \...
1
vote
0answers
13 views

How does uncertainty of observations propagate through linear regression fits [duplicate]

I'm quite new to statistics, so please bear with :) I'm trying to estimate the uncertainty of a variable which is predicted using a linear equation. The linear equation is estimated with a series of ...
2
votes
0answers
18 views

What are the available method that can alleviate the overfitting problem in traditional OLS problem, but still can get a linear fitting?

Recently, I have read the paper https://static1.squarespace.com/static/56def54a45bf21f27e160072/t/5a0d0673419202ef1b2259f2/1510803060244/The_Sampling_Error_in_Estimates_of_Mean-...
1
vote
2answers
53 views

In NN, the way we have different nonlinear activations, can we have different linear activations?

I am just curious to understand if we can have different linear activations other than $WX+b$? I understand the necessity of weights and biases, but is this the only way out neural net's propagation ...
0
votes
1answer
7 views

Interpretation of (simultaneous) confidence band against fitted values in multiple regression

In a homework question, I am asked to interpret a figure of the confidence band and simultaneous confidence band of 95% confidence level plotted against predicted values. The confidence bands are ...
0
votes
1answer
22 views

Comparing linear regression models under violation of independence assumption

the basic setup is as follows: I have a continuous dependent variable (DV, 7 observations) and two continuous independent variables (IV1 & IV2). I would like to evaluate whether adding IV2 as ...
0
votes
0answers
37 views

Approximate known non-linear function using linear regression

Consider the following model: $$ y_{i}=f\left(\boldsymbol{x}_{i};\theta\right)+\varepsilon_{i} $$ where $y_{i}$ is the dependent variable, $\boldsymbol{x}_{i}$ is a vector of explanatory variables, $...
0
votes
1answer
37 views

derivation of objective function in linear regression

In linear regression, we have a very simple task. This is to measure a distance between Y and y_hat, where y_hat for sake of simplicity is multiplication of X and w. So we can say: Error = Y-y_hat = ...
1
vote
0answers
9 views

How to exploit orthogonalization procedure in forward stepwise linear regression?

Forward Stepwise linear regression allows to build up a subset of features starting from the intercept. At each step the predictor that most improves the fit is added to the subset. In the book, it ...
3
votes
1answer
85 views

General expression for a single coefficient $\hat{\beta_1}$ in a multiple linear regression? [duplicate]

Suppose I am trying to estimate a multiple linear regression with $k$ regressors and I have $n$ observations $$Y = X\beta + \epsilon$$ Where $\beta \in \mathbb{R}^k$ and $X \in \mathbb{R}^{k \times ...
0
votes
0answers
8 views

Systematic Error Propagation with a Linear Fit

I am doing a Physics experiment in Atomic Spectroscopy. I am trying to find the wavelengths of certain spectral lines. The wavelength readout on our monochromator is not correct, so we must calibrate ...
1
vote
1answer
20 views

Conditional variance [closed]

I'm struggling to understand why the following is true: MLR5 assumption in multiple regression $\text{var}(u|x_1,\ldots,x_n)=\sigma^2$ implies that $\text{var}(u|x_i)=\sigma^2$ for every $i$, ...
0
votes
1answer
25 views

Interpretation of interaction term between 2 continuous variables

I'm running a multiple linear regression of revenue (rev) on identity (ID), an index which measures a customers identity towards ...
1
vote
0answers
12 views

Substituting correlated predictors with an interaction term

Assume you have a linear model with two predictors $y = \beta_0 + \beta_1x_1 + \beta_2x_2$ and you find out that $x_1$ and $x_2$ are correlated and VIFs are over 5. I read that you can remove the co-...
0
votes
0answers
19 views

Does this allow conclusions about an interaction in a linear mixed model?

I am analysing an experiment where groups received either one type of verum treatment (there are two different types of verum treatment) or a placebo treatment. I want to find out whether the ...
0
votes
1answer
45 views

How to interpret this SPSS output table? [closed]

How should I interpret the following results? My thesis takes a long at the underlying values of political preference and the consumption quantities of meat-replacements products (such as vegetarian ...
2
votes
0answers
13 views

How to fit points to piecewise linear model where all slopes must have the same absolute value?

The current methodology for the genomic data I have involves fitting a spline to multiple points. However, the underlying biology does not support that the fit should be curved at any points. In fact, ...
0
votes
0answers
23 views

Creating linear models in R

So I have this data set t.dat with 31 observations and two groups (V5): I'm interested in testing where we can say that the mean of $(V2,V3,V4)$ where we only ...
2
votes
1answer
75 views

Linear Model with Radial Basis Function Transform - What's wrong?

The Maths behind the problem Suppose I have a training matrix $X$ with $n$ observations and $d$ features $$ X = \begin{pmatrix} x_{11} & x_{12} & \ldots & x_{1d}\\ x_{21} & x_{22} &...
0
votes
1answer
24 views

R: LMM with covariate

I know there have been a number of posts on lmer, but I am struggling to find my answer through research and am hoping to get your help. I am analyzing data from a study with the following data: 1) ...
0
votes
2answers
19 views

Multiple regression with predictors for different “directions”?

I expect predictor A to negatively predict my dependent variable, and predictor B to positively predict the dependent variable. Can I include both predictors in a (linear) multiple regression model ...
2
votes
4answers
60 views

Why does linear non-logistic regression work as a linear classifier? What classification error does it minimize?

Suppose the data has two attributes and a label -1 or 1. So, we have a three-column matrix $X$ (two attributes and a column of ones for convenience of working with matrix notation) and a column vector ...
2
votes
1answer
23 views

Does the size of the design matrix change for estimation vs. predictions?

Say I have the model $y = \beta_0 + \beta_1 x_1 + \cdots \beta_p x_p + \epsilon $. Using $n$ observations of data I formed the system of equations $\mathbf{y} = X\beta + \epsilon$ for least squares ...
2
votes
0answers
39 views

Maximization of quotient of quadratic forms in linear regression

I would like to find maximum of the following function: $$I = \max_{a\in \mathbb{R}^p} \frac{(a'\hat{\beta})^2}{S^2a'(X'X)^{-1}a},$$ where $X$ is a design matrix and of course $Y$ is normally ...
1
vote
0answers
16 views

Is there any advantage to using $\chi^2$ for checking linear relationship $y(x)$?

At my university (physics major), we are often forced to use $\chi^2$ for checking if $y(x)$, where $y$ and $x$ are measured, is linear. We fit a linear regression $f(x)$ and then use $\chi^2 = \sum_{...
1
vote
1answer
40 views

Is always true that in the simple linear regression the width of prediction interval corresponding to new observationx=xo increases linearly with xo

Is it always true that in the simple linear regression model the width of the prediction interval corresponding to a new observation x=xo increases linearly with xo? Thanks in advance
0
votes
1answer
17 views

Describe the graphs linear association between these two variables height and width

I have written the code below to create a scatterplot to visualize whether the two variables are linearly associated but I am not sure how you would describe this output. I would say it is not ...
0
votes
1answer
53 views

Multiple Linear Regression on NBA Player Salaries

I've recently learnt the basics of regression as I progress through R and statistical modelling and approached a simple project of predicting NBA player salaries based off Age, Assists, Blocks, ...
0
votes
0answers
12 views

How to use regression for an overall estimate of increasing rate of different products over time

My data are as follows: For each product, I have measured a specific feature over time. That is let's say every week. Therefore, for each time point (week 1, week 2 etc.), I have measured the specific ...
0
votes
1answer
38 views

Why do we use PCs instead of transformed data as new variables in linear regression?

PCA and SVD Using the SVD on matrix X (column as features) We have X = U\sigmaV* where V contains the PCs and U\sigma would be the transformed data PCA with linear regression The ...
8
votes
2answers
491 views

Linearity assumption of linear regression

According to this website, if the scatter plot follows a linear pattern (i.e. not a curvilinear pattern) then linearity assumption is met. Here is an example where the assumption is not met. But ...
0
votes
0answers
20 views

Covariance/correlation matrix for linear regression

Just a simple question on interpreting coefficients given by vcov(model) (and also cov2cor), say we have the model: $ y_{i} $ = $\beta_{0} + \beta_{1}x_{1i} + \beta_{2}x_{2i}+\epsilon_{i}$, would ...
0
votes
0answers
43 views

In a regression model fitting, should we should check for the normality of the response at each level of X, but not collectively for X?

My questions stems from this answer: https://stats.stackexchange.com/a/390839/37540 I understand that we should check normality of residuals. But regarding dependent variable, am I understand ...
0
votes
0answers
10 views

Interpretation magnitude regression coefficients

I'm running a linear regression model in which my dependent variable is standardized while my regressors are dummy variables. The coefficients I obtain are all statistically significant yet I was ...
1
vote
2answers
43 views

Is this a linear model? [closed]

mpg = mileage per gallon and hp = horsepower Why is this model a linear model despite having a square of horsepower in it?