Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

For statistical topics which involve the assumption of linearity, for example, linear regression or linear mixed models, or for the discussion of linear algebra as applied to statistics.

0
votes
0answers
3 views

Interpretation of correlation in endogenous regression model

Suppose you have a linear regression with an endogenous regressor $x$ that can be represented as follows: $x = z'\delta + \epsilon_1$ $y = \beta x + w'\gamma + \epsilon_2$ where $\begin{pmatrix}\...
0
votes
0answers
2 views

Is residual in a SRF, an estimate of error in PRF? (Regression Question Series - Part 2)

PRF: Given a sample set $(X,Y)$ we hypothesize underlying population has a regression line as follows. $$\begin{aligned} & Y = \beta_0 + \beta_1x + \varepsilon & \text{Prediction (1)} ...
0
votes
2answers
19 views

Statistical measure for linear regression with two distinct clusters of points

In the following plot, I have a linear regression of 30 points, representing 10 treatments with three replicates each. As you can see, the r-squared value is quite strong (0.83) and the p-value is ...
1
vote
0answers
32 views

Linear least squares algorithms

I have stumbled across these two questions and accepted answers: (1) Do we need gradient descent to find the coefficients of a linear regression model? (2) Why use gradient descent for linear ...
1
vote
0answers
68 views
+50

Which is the dependent variables?

I was looking at this Data Science question on TestDome. The problems is stated as the following: Implement the desired_marketing_expenditure function, which returns the required amount of money ...
1
vote
1answer
17 views

Linear regression of dependent variable squared & retransformation

I have performed linear regression of a dependent variable squared, & my statistics package produced least squares means for each level of categorical variables that I would like in original units....
1
vote
0answers
25 views

What is the difference between taking a transformation of a response variable to then apply linear regression and a GLM?

From what I've studied so far, GLM's are to be used when the error term of a response variable is not assumed to be normally distributed. However, I also read that sometimes a transformation of a ...
0
votes
0answers
16 views

user friendly interpretation of linear model covariance output

Given the following scenario: I have 3 parameters used in a linear model to estimate the 4th. all three are positively and significantly correlated to the predicted parameter slope (~0.2-0.4) (p<0....
0
votes
1answer
21 views

Converting relative effect to absolute effect in log model

I have the following model; log(daily sales) = intercept + B1*(event dummy) + error My response variable(daily sales) is basically a daily time series and 'event dummy' is an indicator variable ...
0
votes
0answers
11 views

Scale-Location Graph question

This graph is confusing me. I am assuming there is no equal variance because the residuals are spread out so much; however I am not sure. Please verify this for me.
1
vote
0answers
21 views

Pearson and R^2 Correlation between three variables

Get it from someone else but don't quite know how to answer. If $\rho_{X,Z}=0.4$, $\rho_{Y,Z}=0.3$, what is the range of $\rho_{X,Y}$? Here $\rho$ is the Pearson correlation coefficient. We run a ...
1
vote
0answers
14 views

Linear Unconditional X-Y, Non-Linear Conditional X-Y

Intuitively, I can imagine that an unconditional (i.e., unadjusted for any covariates) Y~X relation can present as a linear relation, whereas a conditional Y~X|Z relation can present as a non-linear ...
0
votes
0answers
8 views

linear graph for 2 datasets one with 100s legend and the other with millions

I ran into an issue that I have to display the correlation between 2 lines in 1 graph chart, the problem is 1 line values are within hundreds, the other line values are millions. I do not care about ...
0
votes
0answers
51 views

Linear Regression Using a Variable and Standard Deviation of the Variable

UPDATED: Sorry for the confusion! I changed parameters to regressors, and added in example data (not real data). I am currently doing something like Day ~., but using ANOVA to find the best regression ...
0
votes
2answers
17 views

Unsupervised Classification of Linear Trends

I have a data set which results in a series of non-parallel linear trends on a scatter plot. I'm trying to find a way to classify each data point into its closest corresponding linear trend. There ...
0
votes
0answers
19 views

Proving that $S_{Y_x}^2 = \frac{S_{xy}^2}{n-2}(1-r_{xy}^2)$

Exercise : Show that for the simple linear model, it is : $$S_{Y_x}^2 = \frac{\sum(y_i-\hat{y}_i)^2}{n-2} = \frac{1}{n-2}\bigg(S_{yy}-\frac{S_{xy}^2}{S_{xx}}\bigg) = \frac{S_{xy}^2}{n-2}(1-r_{xy}^...
0
votes
0answers
40 views

Proving that $V(\hat{y}_{x_0}) = \sigma^2\bigg[\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}\bigg]$ [duplicate]

Exercise : Prove that the variance of $\hat{y}_{x_0} = \hat{b_0} + \hat{b_1}x_0$ is : $$\text{Var}(\hat{y}_{x_0}) = \frac{\sigma^2\sum x_i^2}{n\sum(x_i-\bar{x})^2}+\frac{\sigma^2x_0^2}{\sum(x_i-\...
2
votes
2answers
76 views

Showing that $\sum_{i=1}^n (y_i-\hat{y_i})(\hat{y_i} - \bar{y}) = 0$ for the generalized linear model [closed]

Exercise : Prove that for the generalized linear model, it is : $$\sum_{i=1}^n (y_i-\hat{y_i})(\hat{y_i} - \bar{y}) = 0$$ Question : How would one proceed with proving that for the generalized ...
2
votes
2answers
46 views

Degrees of freedom for linear regression

I'm reading on a text book about linear regression, and when I thought I finally understood degrees of freedom, I found a statement that made me doubt what I know so far. Well it's in the context of a ...
1
vote
0answers
22 views

Converting linear variable's coefficient to log scale

I have a linear regression model with some variables log transformed: Y = Beta1.Log(X1) + Beta2.Log(X2) + Beta3.X3 Y is a percentage variable (A credit card companies market share) and X3 is Premium ...
2
votes
0answers
33 views

Linear Prediction and Linearity of CEF

I am revisiting the basic notions of linear regression and stumbled upon the following idea in Cameron and Trivedi's Microeconometrics book: However, for the conditional mean to be linear in x, so ...
0
votes
1answer
52 views

Functional form of f

I am reading An Introduction to Statistical Learning with Applications in R by G. James, D. Witten, T. Hastie and R. Tibshiran 2013 after taking a basic statistics course a little while ago. On ...
1
vote
1answer
32 views

How can I remove the effect of one independent variable so I can standardize and compare the values of my dependent variable?

I have a table of television viewership data with each row being one series and the columns being various data about that series, e.g. name, time the series is on TV, length of an episode, how many ...
1
vote
0answers
25 views

Produce MSE by using cv.glm() in multiple linear regression, how about a transformed y variable

I applied 10-fold cross validation by using cv.glm() function in the linear regressions. I am able to obtain the MSE in this way, mse=cv.glm(data,model,K=10)$delta However, if I applied ...
1
vote
0answers
23 views

Linear Regression Coefficient changes with additional variables

Folks, In linear regression, I am looking to understand why the coefficients of a given independent variable (HS_ENGL in this example) would change as other independent variables are added (SAT_VERB ...
0
votes
0answers
50 views

How do you prove $E(\text{MSPE})=\sigma^2$

I started with: $$E(\text{MSPE})= E\big(\text{SSPE}/(n-c)\big)= E\bigg(\sum_{i=1}^{c}\sum_{j=1}^n(y_{ij}-\bar{y_i})^2/(n-c)\bigg),$$ but I am not sure where to go from here.
0
votes
0answers
20 views

Finding Linear relationships in multiple output per input dataset

I have a dataset with many outputs for a given input. In the data there are several linear relationships that are easy to see when the data is plotted. The linear relationships are all of interest to ...
-1
votes
0answers
5 views

Drop insignificant columns with categorical values in R [closed]

From what I have understood from this video: https://www.youtube.com/watch?v=qst0QGBntxc , if a field has a high P value, you should drop that column. But what if the fields in a columns have ...
1
vote
0answers
22 views

Linear Regression - Which Features Should We Apply a Polynomial Transform to and Why? [closed]

In which situations would a feature have a polynomial transformation appropriately applied to it, and why would we do this; what ultimate impact does this have on the data. Supposing we select the ...
1
vote
1answer
43 views

What is standardization's effect on p values?

When you standardize variables (prior to linear regression), is it the case that it will always increase the p value of your intercept term close to or to 1? Or, is it the case that your p value on ...
0
votes
0answers
20 views

Centering the design matrix in multiple linear regression

What happens to the ordinary least squares (OLS) (multiple) regression estimates when one centers the explanatory variables in either of the following cases: Including an intercept: assume that the ...
3
votes
1answer
68 views

Intriguing properties of neural networks

In a paper by Ian GoodFellow1, on page 3, what is meant by: Our experiments show that any random direction $v ∈ \mathbb R^n$ gives rise to similarly interpretable semantic properties. More formally,...
0
votes
0answers
30 views

linear regression or time series analysis, combining multiple datasets

I am interested in a relationship between A and B, while controlling for C-F. The relationship I wish to establish is how outside air temperature impacts (5) stock markets. The hypothesis is that ...
2
votes
1answer
33 views

Can I use as dependent variable a percentage in a linear multiple regression model?

I'm trying to model percentage of recovery (pain intensity) after surgery. The median is higher than 50% and the upper quartile 89%. If normality of residuals and linearity between variables along ...
1
vote
1answer
27 views

Is Elastic Net my best choice for finding sparse linear models in correlated features?

I have a linear regression problem, 1000 data points, but with 36 correlated features, those features are very highly correlated. And I know the ground truth must be linear. I know Lasso would give ...
1
vote
1answer
30 views

Assumptions of linear fit; linearity and homoscedasticity

I'm reading about the assumptions of taking a linear fit between two variables from here, and that source says: For diagnosing non-linearity: nonlinearity is usually most evident in a plot of ...
0
votes
0answers
23 views

Linear regression, getting probabilities for different outcomes

I ran a linear regression (i got standard error as well) it has the form of y= a+b*x if i plug in 30 for x I get y=5 The question is, what is the probability given x that y is at least 2. I have ...
0
votes
1answer
23 views

Correlation coefficient & Regression: Intercept

Key points I have in mind, and then followed by my question: 1. I have a regression linear model with a set of attributes and their coefficients. 2. I also ran a correlation analysis on these ...
0
votes
1answer
87 views

Why, oh why, does R automatically eliminate variables from my dataset when running lm? [duplicate]

First of all, forgive my moronosity (new word I made up). I am running R on data from trees in burn areas. Simple, straightforward dataset. And yet, R for some reason hates chestnut oak. My species ...
0
votes
0answers
9 views

Regression for combination of categorical and continuous dependent variables?

I am dealing with a model that has several categorical independent variables, 4 continuous dependent variables, and 2 categorical dependent variables. My goal is to examine how the IV's affect the DV'...
1
vote
0answers
52 views

Partitioned simple linear regression

What I got was $\text{Var}(\beta) = \sigma^2(X'X)^{-1}$ $\text{Var}(\beta_1) = \sigma ^ 2(X_1'X_1)^{-1}$ $\text{Var}(\beta_2) = \sigma ^ 2(X_2'X_2)^{-1}$ $\text{Var}(\tilde{\beta}) = \frac{1}{4}\left(...
0
votes
0answers
19 views

ACF, detrend and first difference

Does anyone know how to interpret these plots?
2
votes
1answer
32 views

Does regularization in regression help with numerics when the data matrix is not full rank?

I am trying to get some intuition around regression when the data matrix $A$ is not full rank in the following regression/least squares problem: $$y=Ax+b$$ where $y \in \mathbb{R}^n$, $A \in \mathbb{...
1
vote
0answers
28 views

Why StatsModels shows log likelyhood function decreases as the fit gets better?

I was going through https://www.statsmodels.org/stable/examples/notebooks/generated/ols.html and as you see from the example (the output of sm.OLS) Log-Likelihood increases as R-squared decreases. ...
1
vote
1answer
29 views

Estimating point of flat behavior of curve?

I am working with discrete data that falls along a decay-like curve, and I want to estimate a $k$ such that for $i > k$, the y value doesn't change much at all. Here's simulated data that looks ...
1
vote
0answers
22 views

Role of rank condition in identification of 2SLS - matrix algebra

I could write down all the steps for identification of the 2SLS estimator but my question is really a matrix algebra question which is required in the last step for finding out what the beta vector is ...
0
votes
0answers
19 views

How does this proof assume the existence of an inverse?

I am reading the proof of theorem 2 here https://www.tandfonline.com/doi/full/10.1080/03610926.2016.1183786 Part of this proof says that If $AX \perp BX$ then $AX \perp f_B(BX) = X^T BB^- BX = X^T B ...
0
votes
0answers
14 views

R/S-Analysis: Hurst Coefficient Using Least Squares Method

The application I am working on (it is an image processing program) needs to calculate, given an m-by-m matrix of integers, the so-called Hurst coefficient for that matrix, considering it as a time-...
0
votes
1answer
27 views

Condition Index and Variance Proportions - Multiple Linear Regression

I am a postgraduate student. I would like to ask if there is a matter of multicollinearity, serious or not, when in a multiple linear regression the condition index in a dimension is below 15 (12....
1
vote
1answer
26 views

Difference between linear mixed model and linear mixed-effect model?

Is there a difference between linear mixed model and linear mixed-effect model?