Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [linear]

For statistical topics which involve the assumption of linearity, for example, linear regression or linear mixed models, or for the discussion of linear algebra as applied to statistics.

1
vote
0answers
10 views

Examples of features construction for linear methods in Reinforcement Learning

I am referring to page 210 of of Sutton and Barto book on Reinforcement Learning available here: book Linear function approximation for state-value functions are of the form $$\hat{v}(s,\mathbf{w}) = ...
1
vote
1answer
18 views

How to Interpret p-value for categorical variable in multiple linear regresion? [duplicate]

I have a query on how to interpret the result for multiple regression with categorical variables. I have categorical variable called Stay_In_current_city_years which has 5 levels. After running the ...
5
votes
1answer
78 views

Need help understanding what a natural log transformation is actually doing and why specific transformations are required for linear regression [duplicate]

I’m taking an online “Intro to AI” course for which I’m doing some azure machine learning labs. This course is largely about how to apply azure ML solutions and, while there is an “essential math for ...
0
votes
1answer
20 views

Confidence Interval for non-smooth term in gam (mgcv)

I fitted a gam model in mgcv package and now want to get the confidence intervals for the non-smooth term. ...
1
vote
1answer
20 views

Random draw from a linear model

I'm building an individual-based model where one component includes increasing winter temperature over time (years). For each iteration of the model (say 1000 iterations that have 25 years each) I ...
0
votes
0answers
18 views

When to use variance stabilizing method?

Let's suppose we want to estimate $p$ from $m$ independant realisations of $X\sim Bin(n,p)$: $x_1,x_2,\dots,x_m$, with respective size $n_i$ $i$ in $\{1,\dots,m\}$. Let $p_i$ be $p_i:=x_i/n_i$. To ...
1
vote
1answer
34 views

How to deal with predictions if taking log of dependent variable

I have a very basic question about linear regression. I have a dataset where the response variable is largely skewed to the right -- if I take a log of it, the distribution becomes a lot closer to ...
1
vote
1answer
15 views

Dealing with negative values in a positive regression

What is the best way to deal with two sets of independent variables that have two different value ranges? I have one data set with values ranging from -1 to 1 and another data set ranging from 0 to 1....
4
votes
1answer
46 views
+50

How to solve an adaptive lasso model?

Assuming we are working with a linear regression model, lasso penalization solves: \begin{equation} \min_{\beta}\left\{\left\lVert y-X\beta\right\rVert_2^2+\lambda\sum_{j=1}^p \left\vert \beta_j\...
1
vote
1answer
27 views

Meaning of residual maker matrix

Suppose that $M_1$ is the residual maker for a unity vector (i.e. a vector made of $n$ 1's). I am told that this matrix, when premultiplying a variable, transforms the variable "into deviations from ...
0
votes
1answer
30 views

Why do we use “Sum of Squared Errors” as loss function in linear regression? [duplicate]

What is a loss function? How can we relate the slope of Linear Regression with Sum of Squared Errors?
0
votes
1answer
42 views

Is the equation is Linear Regression? [closed]

Employees Salary = 3000 + x(Employee Age)^2, is this a Linear Regression?
0
votes
0answers
11 views

What formula do i use to scale these numbers

If x=51 then y=0 and if x=255 then y=2000 Is there such a formula to find y by using an x value between 51-255
0
votes
2answers
31 views

Main effects are not significant anymore after adding interaction terms in my linear regression

I have two models: $levelOfCreativity = \alpha Extravert + \beta Woman + \gamma hasACollegeDegree + \zeta isOlderThan25$ $levelOfCreativity = \alpha Extravert + \beta Woman + \gamma ...
0
votes
0answers
10 views

Chossing between high number of components in PCR vs linear regresion

Let's say my original data set has 18 variables. If the result of the cross-validation error is lowest on the 17 components of PCR is that a good indication that you most likely choose the ...
0
votes
0answers
13 views

In what cases (if any) does r^2 remain unchanged on adding a new variable? [duplicate]

Given that the r^2 changes with the addition of a new variable, are there circumstances where r^2 is unchanged? I can think of the case where the new variable is perfectly correlated with the one of ...
0
votes
1answer
30 views

Dummy regression - p-value interpretation

Suppose I want to predict the quality of an essay as a function of how many essays a person produces in a year. Something like this: $quality = m_{0}\ (quantity/year) + k$ in which, $m_0$ is the ...
1
vote
0answers
12 views

Does multiplying a regression variable by a random variable with mean 1 affect the estimates

I'm dealing with a regression of the form $\log(Z)=\alpha\log(X) + \beta\log(Y) + u$, but instead of observing $Z$, I observe $Z'=aZ$, where $a$ is a random variable with mean 1. Does using these ...
0
votes
0answers
19 views
11
votes
6answers
948 views

Linear regression when Y is bounded and discrete

The question is straightforward: Is it appropriate to use linear regression when Y is bounded and discrete (e.g. the test score 1~100, some pre-defined ranking 1~17)? In this case, is it "not good" to ...
1
vote
0answers
21 views

How to deal with correlated regressors in a multiple regression model?

i currently try to estimate the effect of different task parameters (IV) on neuronal activation (DV). Some predictors in my design matrix(trials x features) are moderately correlated (r~=.3) and I ...
0
votes
0answers
16 views

About linear regression and polynomial functions [duplicate]

Can I apply linear regression to any function to the x's. For example: Polynomials with one variable: y = w_0 + w_1 * x + w_2 * x2 + ... + w_D * xD Polynomials with multiple variables (let's say ...
13
votes
5answers
2k views

Why Normality assumption in linear regression

My question is very simple: why we choose normal as the distribution that error term follows in the assumption of linear regression? Why we don't choose others like uniform, t or whatever?
1
vote
0answers
12 views

Comparison effects of regressors for several dependent variables

I want to study the effect of two explanatory variables $X$ and $Z$ on four different outcomes (which are binary variables, i.e. equal 1 if satisfied and 0 if disatisfied): $Y_1$, $Y_2$, $Y_3$ and $...
1
vote
1answer
17 views

How to show that the error variance of the best linear predictor is inferior to the proportional predictor?

Let's consider the 1D case. How do we prove that the error variance of the Best Linear Predictor (BLP) is inferior than the Proportional Predictor (i.e. the Linear Predictor without the intercept)? ...
0
votes
0answers
26 views

Explanation of covariance matrix of polynomial parameters [duplicate]

I'm asked to find the covariance matrix of $\alpha$, $\beta$, and $\gamma$ for: $$y_i=\alpha+\beta(x_i-\bar{x})+\gamma[(x_i-\bar{x})^2-\zeta^2]+\epsilon_i$$ where all the errors have equal variance $...
0
votes
1answer
12 views

Linear Discriminant Analysis vocabulary question

I am doing a descriptive LDA on a dataset with two (known, easily separable) classes and many features (and many more observations). I intend to use the latent variable values as a dimensionally-...
1
vote
0answers
23 views

Log results in linear regression

I am just starting working with regression in R. I used some variables and their logs as well in the same regression equation. Unexpectedly, the results show significance of both, variables and their ...
0
votes
1answer
40 views

Linear regression - confidence interval for expected difference in $Y$ with respect to unknown values of $X$

Suppose I am given all of the necessary parameters about some linear model, but not the data itself. Namely, I am given $\hat{\beta}_1,\hat{\beta_0},\bar X, S_e, r^2$, etc. Also, I know that $X_1,\...
1
vote
0answers
22 views

Linear Regression Expected Value [duplicate]

Whiles reading An Introduction To Statistical Learning under linear regression (Chapter 3), I found: $$E(Y - \hat{Y})^2 = |f(X) - \hat{f}(X)|^2 + Var(ε)$$ where $E(Y - \hat{Y})^2$ represents the ...
2
votes
1answer
72 views

Use $X^{-1}Y$ instead of $(𝑋^T𝑋)^{−1}𝑋^T Y$ to calculate $\beta$ when $X$ is already a square matrix in the least square problem

In the least squares problem $X\beta = Y$, the solution is $\hat{\beta} = (𝑋^T𝑋)^{−1}𝑋^TY$. I learned that two facts: $𝑋^T𝑋$ is square matrix so that the definition of matrix inversion is ...
4
votes
0answers
26 views

Method to forecast correlate univariate time-series (with trend, seasonality) via regression

I have two univariate time-series with seasonality and trend--dt1 and dt2. I believe that dt1 and dt2 are strongly correlated, both through a few statistical test (see below) and that in my field dt2 ...
1
vote
1answer
38 views

Forecasting or solving system of linear equations

I am looking for advice on the best approach for solving a system of equations with imperfect and incomplete data, such that standard approaches including: Guassian elimination, Gauss-Jordan ...
0
votes
0answers
20 views

Cosine similarity matrix of linearly transformed inputs

Given a matrix $\mathbf{C}$ which contains pairwise cosine similarities between rows of a matrix $\mathbf{A}$, linearly transformed by matrix $\mathbf{U}$: $$ \mathbf{C} = K(\mathbf{UA}, \mathbf{UA}) $...
2
votes
0answers
20 views

How can I get RMSE from RMSLE [duplicate]

Is that correct If I take e^RMSLE will the resulting value give me RMSE or I am missing sometihing . I want to interpret RMSE from RMSLE
0
votes
0answers
52 views

Interpretation of interaction terms

I try to improve my statistical knowledge and right now I am working on a regression that contains interaction terms. I am currently investigating the effects of different variables ($a_1$, $a_2$ etc.)...
2
votes
0answers
63 views

Linear Mixed Model for evaluation of students

My dataset about students (n=74) contains one outcome variable (exam points/integer) and eight predictor variables: 2 categorical: gender [F,M] study years [1,2,3] 6 continuous variables: age [...
0
votes
0answers
7 views

Effect of Time Events on Short Term Price Elasticity

I'm trying to build a model to understand units purchased over time for a product whose price changes frequently. Would it be possible to use a multivariate regression where short term week over week ...
3
votes
0answers
16 views

Time series with continuous predictors and outcomes

I am trying to carry out a multivariate regression model where my main predictor is a continuous variable that changes over time, and my dependent variable is also a continuous variable that changes ...
0
votes
1answer
61 views

Find a model for two continuous predictors of a single categorical DV

Suppose I ask subjects to place a value on two cups: A and B; on a scale from -10 to +10. She says A is worth -4 and B is worth 9. Now I say 'ok pick one'. She picks one. Now I ask 300 people the ...
1
vote
1answer
36 views

What are some good robust loss functions for binary classification using LDA?

I am doing a project where I use LDA for binary classification. I want to know how it performs when there are outliers. What are some good robust loss functions for binary classification?
1
vote
0answers
46 views

How to assess linearity in multiple linear regression?

I have a question about how to check if the relationship between the independent variable, yt , and the explanatory variables t and t^2 is linear? I fitted the linear regression with the AUTO-...
2
votes
1answer
57 views

Unequal number of repeated measure for random factor and lmer

I'm doing a linear mixed model such as: model1<-lmer(Y~factor(X1)*X2+(1|species)) as I have repeated measures of the triplet (Y,X1,X2) per species depending ...
1
vote
2answers
90 views

Standard Error and confidence interval for multiple linear regression in matrix notation

Lets say we are trying to fit a normal linear model to our data as follows: $y = \beta _0 + x_1\beta_1 + x_2\beta_2 + ... + x_p\beta_p + \epsilon$. My question is that how we can derive the ...
0
votes
0answers
28 views

Multivarite Linear Mixed model in R

I want to fit the following multivariate mixed effects linear model in R but am failing miserably: $y_{ijk}$ = $({\textbf{x}}_{ijk}'{\beta}_{k})$+$(\gamma_{ik})$+$(a_{0k}+a_{1k}t_{ij}+ a_{2k}{t_{ij}}^...
1
vote
2answers
29 views

In linear regression, is finding the minimum of the parameters the same as gradient descent?

I'm taking a course on ML and have just began. Given a loss function, $$L = \frac{1}{N}\sum^N_{n=1}(t_n - w_0 + w_1x_n)^2$$ I am confused between the difference of using gradient descent (and maybe ...
0
votes
1answer
33 views

Why isn't the least squares predictor $\Phi(\Phi^\top\Phi)^{-1}\Phi^\top$ simply the identity matrix? [duplicate]

Given target vector $y$. Want to predict it using linear regression $h(w) = w^Tx$ Let $\Phi$ be the least squares matrix, i.e., $\Phi = \begin{bmatrix} x_1^\top \\ \vdots\\ x_n^\top \end{bmatrix}$ ...
1
vote
0answers
22 views

Why would the r^2 values of the same data set but separated be lower?

I have a set of data on one year(11 months, janv-nov), for each day I have the "load" and the "prod hours". I try to predict the prod hours thanks to the load. So i'm fitting a linear regression in my ...
1
vote
1answer
37 views

Why is the conditional expectation the best predictor but only if we have the joint distribution?

If we want to predict one variable $Y$ based on another $X$, the best predictor is apparently $\mathbb{E}[Y \mid X = x]$. However, this apparently assumes two things: The distribution is symmetric. ...
0
votes
1answer
41 views

Linear Regression For Binary Independent Variables - Interpretation

I have a dataset where I want to predict inflow (people joining a platform) but my all independent variables are binary categorical (0,1). Whereas I want to predict continuous variable (inflow -- ...