Questions tagged [leverage]

Leverage is a measure used in regression to highlight observations which are outlying in the space of the predictors.

Filter by
Sorted by
Tagged with
0
votes
0answers
20 views

Understanding leverage and influence

there is a reference to the $i^{th}$ diagonal entry in $H$ where $H=X(X^TX)^{-1}X^T$ in the definitions of leverage and cook's distance. See: https://en.wikipedia.org/wiki/Leverage_(statistics) and ...
0
votes
0answers
34 views

PRESS from the hat matrix and numerical stability from statsmodels ols.fit()

Leave one out cross validation in the context of ordinary least squares regression can be done via the hat matrix: The "hat" or projection matrix $$ H = X(X^T X)^{-1} X^T $$ many fit ...
0
votes
0answers
7 views

Why does high leverage point in SLR have unusual covariate values

As stated above and also wondering what a covariate value means, is it just covariance? IF not, then what it is and what is the difference between covariance and covariate
0
votes
0answers
27 views

Leverage Statistic (h) for Multiple Predictors [duplicate]

I'm working through Introduction to Statistical Learning and understand the leverage statistic for a simple linear regression. However the text says "There is a simple extension of h_i to the ...
1
vote
0answers
57 views

How to decide between different robust standard errors?

Specifying my model I ran into some very mild heteroscedasticity problems. Given its superior small-sample properties (my dataset contains 79 observations) I used the HC3 specification of the White ...
1
vote
0answers
20 views

Interpreting results from glm.diag.plots (package 'boot')

I'm having hard time interpreting results from glm.diag.plots (package 'boot'). I read through some questions and answers (this one was very helpful, thank you! Interpreting glm.diag.plots) but didn't ...
0
votes
0answers
21 views

Condition number and leverage

Condition number is defined using the eigenvalues of $X^\top X$ while leverage values are the diagonal elements of the projection matrix. How are the two related? How can rescaling help, because aren'...
0
votes
0answers
26 views

Leverage after Tobit Regression

In STATA, after running a TOBIT regression, I'm trying to calculate the leverage and Cook's Distance values. When I run: predict c, cooksd I get the error: 'option ...
0
votes
1answer
136 views

leverage() diagnostic test not supported for glmmTMB models in r

I am using a glmmTMB to look at the effect of numerous variables on how far individuals travel (distance). The example below is ...
0
votes
0answers
25 views

N size for not being considerated leverage

Suposse I have a dataset with two variables x and y, with the purpose to run a linear regression y ~ x. We have all x values equal and y varying between 1 and 10. For example (in R code): ...
0
votes
0answers
22 views

Relation between dyi/dei and its leverage (hii)

Althought I have tried in different ways, I have not been able to show that $$\frac{\mathrm{d}Y_i}{\mathrm{d}e_i} = \frac{1}{1 - l_{ii}}$$ $e_i$ is equal to $l_{ii}$ is the $i-th$ element of the ...
0
votes
0answers
8 views

How can I manually draw the contour plot for the leverages?

Please look at Figure 7.7 here at https://books.google.co.jp/books?id=PI2FJzBh2vMC&pg=PA161&lpg=PA161&dq=contour+of+leverage&source=bl&ots=3zwgq4dIbM&sig=...
1
vote
1answer
168 views

Simple Linear Regression: Hat-Value $h_i$

I'm trying to finish proving that in simple-regression analysis, $h_i = \frac{1}{n} + \frac{(X_i - \bar{X})^2}{\sum_{j=1}^{n}(X_j - \bar{X})^2}$, where $h_i := h_{ii} = \sum_{j=1}^nh_{ij}^2$, the ...
3
votes
1answer
147 views

Individual significance of data points in correlation

In this question on stackoverflow, I asked about how it is possible to find the individual significance of each correlation coefficient of each node. I answered the question myself later stating that ...
3
votes
1answer
2k views

Proof about the diagonal element of the hat matrix

I'd really appreciate it if you could help me find the proof for the following formula: $$h_{ii}=1/n + \frac{(x_{i}-\bar{x})^2}{\sum(x_{j}-\bar{x})^2},$$ where $j=1,\ldots,n$. I don't really know ...
2
votes
0answers
139 views

Outliers and influential observations in elastic net logistic regression

My dataset has many biomarkers and the boxplots of these variables show the presence of many outliers. However, these 'outliers' are real data and not misread observations. I want to use elastic net ...
5
votes
1answer
84 views

Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
1
vote
1answer
533 views

Leverage and Influence

Is it possible that an outlier is neither influential nor does it have high leverage? Or can it happen that an observation with high leverage is not an outlier and is neither influential?
0
votes
0answers
49 views

How outliers influence your results? and what are good and bad leverage points? [duplicate]

I am confused between outliers and leverage points. And the difference between good and bad leverage points in time series analysis. Can somebody help me?
1
vote
1answer
81 views

Numerically Distinguish Between Real Correlation and Artifact

I'm looking at correlation for a large number of vectors, and many (about 3000) of these pairwise comparisons appear to have a significant correlation even after Bonferroni correction. Plotting these ...
1
vote
1answer
346 views

Why divide by 1-leverage?

I'm reading about resampling methods, and specifically leave-one-out cross-validation. I understood the method, and how to calculate the estimate of the test MSE (Mean squared error): In the setup ...
2
votes
0answers
30 views

Name for spurious linear Regression Plots [duplicate]

Yesterday I was at a medical conference in which a lot of plots of Point Clouds with linear fits were shown. In many cases the fit seemed (at least to me and colleagues) to be influenced mostly by ...
1
vote
1answer
454 views

Why uncentered hat matrix can be used to measure the distance from the center of data?

This question is motivated from here :Why leverage measure the distance of the ith observation from the center of the x space? which is the question related to the wonderful answer of this link :...
6
votes
2answers
854 views

Identifying outliers in the data

Sample data ...
2
votes
1answer
2k views

Influential observations and Outliers in Linear Regression model

What is the difference between influential observations and outliers in linear regression model?
2
votes
1answer
106 views

Is there a specific standard error for an individual residual?

On page 97 of Introduction to Statistical Learning book, there is a paragraph on studentized residuals within the context of looking for outliers. But in practice, it can be difficult to decide how ...
0
votes
0answers
89 views

News Impact curve shift

I computed the NIC for both vanilla GARCH and EGARCH model specifications (with t(4)-student distributed innovations). As we can see from the plot, no asymmetries are present, but the curve has ...
0
votes
0answers
94 views

How to solve a well fitted model - Model Misspecification

I am currently writing a paper, analysing the impact of goldprice movements on the capital structure of gold mining firms. My basic model is a simple OLS model with (y=leverage and x=ln(goldprice)). ...
1
vote
0answers
137 views

Sensitivity Measures for GEE Model

Is there any method (e.g. like Cooks D) implemented in R to identify leverage points for GEE Models? I used geepack to fit my models and would like to do a sensitivity analysis now. However, I don't ...
2
votes
1answer
814 views

Leverage formula/derivation and Hat matrix

1) So I know that $h_{ii}$ is just the ith row ith column of $H=X(X^TX)^{-1}X^T$. Intuitively, why is this the case? I understand that H is the projection matrix and leverage is measuring how far away ...
5
votes
2answers
12k views

Cook's distance vs. hat values

What exactly does Cook's distance measure? And how is this different from what hat values measure? I know hat values measure how distant a point it form its corresponding fitted point. I also know ...
3
votes
1answer
1k views

Leverage Statistic - equation explanation

I came across this equation for the leverage statistic in ISL. Does the denominator translate to the summation of all the squares of $(x_i - mean(x))^2$ excluding the current observation of x? In ...
6
votes
1answer
543 views

What are the leverage values for Ridge regression?

In linear least squares the parameter estimates are: $\hat{\beta} = \left(X^{\top}X\right)^{-1}X^{\top}y$. In Ridge regression the standardized parameter estimates are given by $\hat{\beta}_{\Gamma} = ...
0
votes
0answers
95 views

which residual vs fitted plot is looking better here?

This is a data analysis project we have at school where we try different ways to correct a simple linear regression model. My first attempt is just trying to transform the response variable by taking ...
1
vote
0answers
56 views

How to Adapt Leverage vs Residual plot for Multi-variate Regression

After reading about What is the best way to identify outliers in multivariate data? I noticed that some heuristics are hard to adapt to mult-variate analysis. I'm hoping there is a way to at least ...
1
vote
0answers
23 views

How to check if overall results from a survey are significantly affected by participation from a large company(ies)?

Suppose there is some sort of survey where people from various companies answer some questions. Is it possible to determine whether overall results from the survey are significantly affected by ...
2
votes
1answer
997 views

Is it OK if I have some high leverage points in my experimental plots and they weren't outliers?

I've analyzed effects of temperature and sample thickness on drying process via response surface methodology. I've faced to four high leverage points (my four axial runs) that stick together on normal ...
1
vote
1answer
30 views

Using low coverage predictors in models?

The conventional wisdom I run up against is to drop predictors with low coverage without given them much consideration. By low coverage predictors: I meant predictors whose values are mostly missing. ...
0
votes
1answer
67 views

Deleting Influential Points Leading to Non-constant Variance

http://archive.ics.uci.edu/ml/datasets/Wine+Quality Using this data set, I am regressing chlorides on ten predictors (all except quality). After Box-Cox transforming chlorides, I get the following Q-Q ...
0
votes
0answers
63 views

Linear model and mean value of covariates

Suppose a linear model $$Y_i=\beta_0+x_{i1}\beta_1+\ldots+x_{ip}\beta_p+e_i,\quad e_i\sim N(0,\sigma^2),\quad i=1,\ldots,n,$$ and its hat matrix $P=X(X^TX)^{-1}X^T$, where $Y=X\beta+e$. Let $\bar{x}...
1
vote
0answers
198 views

Influence function for glm?

I would like to calculate the influence of each sample on the coefficient under a logistic regression model. The R built-in function influence() is suppose to do ...
0
votes
1answer
649 views

How to extract/compute leverage and Cook's distances for Generalised Additive models

Similar to the following question, How to extract/compute leverage and Cook's distances for linear mixed effects models Is there any method in R to determine influential points in a GAM?
7
votes
1answer
3k views

What is .hat in regression output

The augment() function in the broom package for R creates a dataframe of predicted values from a regression model. Columns created include the fitted values, the ...
3
votes
0answers
320 views

leverage.plot() R - CAR package

I'm trying to complete a homework regarding added-variables and leverage plots using the CAR package. In the documentation of the ...
1
vote
1answer
406 views

Linear regression question on Idempotent matrix and leverage points

I am considering a linear regression model $Y_i = X_i^T \beta + \epsilon_i, i = 1,2,\dots,n$. where $X_i \in \mathbb{R}^p$. $\epsilon_i$'s are independent copies of random error $\epsilon \in \mathbb{...
3
votes
1answer
353 views

Leverages and effect of leverage points

I just got some question about the hat matrix in linear models. My first question is: Why in a balanced one-way layout $(n_1=...=n_c=n_0)$, all leverages $h_{ii}$ have the same value $\frac{1}{n_0}$? ...
2
votes
0answers
75 views

Why leverage measure the distance of the ith observation from the center of the x space? [duplicate]

I know the definition of leverage points in regression, that is $h_{ii}=x_{i}'(X'X)^{-1}x_{i}. $ In many places and text books, they always say that leverage is a standardized measure of the distance ...
2
votes
1answer
152 views

How does centering affect leverage?

For a linear regression model $Y = β_0 + β_1X$, consider the matrix $X$ and $X_c$ with centering the mean. How do you use algebra to show directly that centering does not change the leverage? I have ...
11
votes
1answer
18k views

Hat matrix and leverages in classical multiple regression

What is Hat matrix and leverages in classical multiple regression? What are their roles? And Why do use them? Please explain them or give satisfactory book/ article references to understand them.
1
vote
1answer
209 views

variance decreases when x gets farther from the average x?

I just read the description of Studentized residual on Wikipeida. I'm confused about what it says about variance, it says that "the residuals, unlike the errors, do not all have the same variance ...