Questions tagged [leverage]

Leverage is a measure used in regression to highlight observations which are outlying in the space of the predictors.

Filter by
Sorted by
Tagged with
0
votes
0answers
42 views

Identification High Leverage for Logistic Regression

As seen below from Piet De Jong, Generalized Linear Models for Insurance Data, for linear model, we can identify high leverage, if the value of leverage>2p/n (or hii > 2p/n) then the indicates ...
0
votes
0answers
11 views

Identification High Leverage for Logistic Regression [duplicate]

As seen below from Piet De Jong, Generalized Linear Models for Insurance Data, for linear model, we can identify high leverage, if the value of leverage>2p/n (or hii > 2p/n) then the indicates ...
0
votes
0answers
12 views

How to deal with leverages when modelling data?

I am currently modelling my data using a negative binomial. When plotting my leverages, I have one point above my cut off line at $\frac{3p}{n}$. However, when I remove the influential observation and ...
1
vote
0answers
32 views

Mistake about Covariance on Wikipedia Page for Leverage?

Wikipedia's page for leverage defines the design matrix $X \in \mathbb{R}^{n \times p}$ where $n$ is the number of data and $p$ is the dimensionality of each datum, and then, under "Relation to ...
1
vote
1answer
17 views

Developing Leverage Statistics Manually in R

I've asked this question on Stack Overflow but think it might be better here. I currently have the below data frame df with the regression equation ...
0
votes
0answers
15 views

Outlier detection/influential observations multinomial logit in R

I'm using the mlogit package to fit models using a multinomial likelihood. Does anyone know how to check for outliers or influential observations when fitting multinomial models with this package? ...
0
votes
0answers
124 views

Cook's distance, not plotting an observation with leverage one. What is this?

I am doing some regression diagnostics in R. I use plot() function and look at the four graphs. However, when I reach Cook's distance graph, I receive a warning ...
2
votes
0answers
32 views

Selecting a subset of observations that retains the most information

Is there a "best" way to select a subset of data (observations, not variables) to use in a multiple regression? The problem: A colleague of mine is planning a project for this field season. ...
2
votes
2answers
153 views

Logistic regression with rare events, all events have large residuals and are influential points

I am running a logistic regression model where the outcome is relatively rare (250 out of 5000). My main interest is to see if there are differences between age groups, sex, educational levels, income ...
3
votes
1answer
48 views

Why (or when) can we neglect residuals mutual correlation?

I've found in several books so far only a waving of hands explanation on why one can simply neglect the residuals mutual correlation, in a linear model context, and plot a QQ-graph to do a qualitative ...
2
votes
0answers
510 views

Understanding Residuals vs Leverage plot in terms of meeting regression assumptions

Can someone help me understand the Residuals vs Leverage plot in terms of meeting the assumption of independence/influence for multiple linear regression models? My understanding is that the ...
0
votes
0answers
18 views

Is it possible to run several linear regressions and use the leverage effect to improve quality?

I have created a for-loop which takes pairs of values from a table and calculates 100 different linear regressions. There is already a condition, namely that exactly one pair of values from a number (...
0
votes
0answers
118 views

PRESS from the hat matrix and numerical stability from statsmodels ols.fit()

Leave one out cross validation in the context of ordinary least squares regression can be done via the hat matrix: The "hat" or projection matrix $$ H = X(X^T X)^{-1} X^T $$ many fit ...
0
votes
1answer
13 views

Why does high leverage point in SLR have unusual covariate values

As stated above and also wondering what a covariate value means, is it just covariance? IF not, then what it is and what is the difference between covariance and covariate
1
vote
0answers
70 views

How to decide between different robust standard errors?

Specifying my model I ran into some very mild heteroscedasticity problems. Given its superior small-sample properties (my dataset contains 79 observations) I used the HC3 specification of the White ...
1
vote
0answers
47 views

Interpreting results from glm.diag.plots (package 'boot')

I'm having hard time interpreting results from glm.diag.plots (package 'boot'). I read through some questions and answers (this one was very helpful, thank you! Interpreting glm.diag.plots) but didn't ...
0
votes
1answer
232 views

leverage() diagnostic test not supported for glmmTMB models in r

I am using a glmmTMB to look at the effect of numerous variables on how far individuals travel (distance). The example below is ...
0
votes
0answers
27 views

N size for not being considerated leverage

Suposse I have a dataset with two variables x and y, with the purpose to run a linear regression y ~ x. We have all x values equal and y varying between 1 and 10. For example (in R code): ...
0
votes
1answer
156 views

How to check for outliers and leverage values in SPSS when conducting a multinomial logistic regression?

I am conducting a multinomial logistic regression in SPSS. I want to check for the presence of outliers and high leverage values. How do I do that in SPSS?
0
votes
0answers
26 views

Relation between dyi/dei and its leverage (hii)

Althought I have tried in different ways, I have not been able to show that $$\frac{\mathrm{d}Y_i}{\mathrm{d}e_i} = \frac{1}{1 - l_{ii}}$$ $e_i$ is equal to $l_{ii}$ is the $i-th$ element of the ...
1
vote
1answer
285 views

Simple Linear Regression: Hat-Value $h_i$

I'm trying to finish proving that in simple-regression analysis, $h_i = \frac{1}{n} + \frac{(X_i - \bar{X})^2}{\sum_{j=1}^{n}(X_j - \bar{X})^2}$, where $h_i := h_{ii} = \sum_{j=1}^nh_{ij}^2$, the ...
3
votes
1answer
304 views

Individual significance of data points in correlation

In this question on stackoverflow, I asked about how it is possible to find the individual significance of each correlation coefficient of each node. I answered the question myself later stating that ...
4
votes
1answer
3k views

Proof about the diagonal element of the hat matrix

I'd really appreciate it if you could help me find the proof for the following formula: $$h_{ii}=1/n + \frac{(x_{i}-\bar{x})^2}{\sum(x_{j}-\bar{x})^2},$$ where $j=1,\ldots,n$. I don't really know ...
2
votes
0answers
190 views

Outliers and influential observations in elastic net logistic regression

My dataset has many biomarkers and the boxplots of these variables show the presence of many outliers. However, these 'outliers' are real data and not misread observations. I want to use elastic net ...
6
votes
1answer
108 views

Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
1
vote
1answer
737 views

Leverage and Influence

Is it possible that an outlier is neither influential nor does it have high leverage? Or can it happen that an observation with high leverage is not an outlier and is neither influential?
0
votes
0answers
52 views

How outliers influence your results? and what are good and bad leverage points? [duplicate]

I am confused between outliers and leverage points. And the difference between good and bad leverage points in time series analysis. Can somebody help me?
1
vote
1answer
103 views

Numerically Distinguish Between Real Correlation and Artifact

I'm looking at correlation for a large number of vectors, and many (about 3000) of these pairwise comparisons appear to have a significant correlation even after Bonferroni correction. Plotting these ...
1
vote
1answer
532 views

Why divide by 1-leverage?

I'm reading about resampling methods, and specifically leave-one-out cross-validation. I understood the method, and how to calculate the estimate of the test MSE (Mean squared error): In the setup ...
2
votes
0answers
31 views

Name for spurious linear Regression Plots [duplicate]

Yesterday I was at a medical conference in which a lot of plots of Point Clouds with linear fits were shown. In many cases the fit seemed (at least to me and colleagues) to be influenced mostly by ...
1
vote
1answer
722 views

Why uncentered hat matrix can be used to measure the distance from the center of data?

This question is motivated from here :Why leverage measure the distance of the ith observation from the center of the x space? which is the question related to the wonderful answer of this link :...
6
votes
2answers
968 views

Identifying outliers in the data

Sample data ...
2
votes
1answer
3k views

Influential observations and Outliers in Linear Regression model

What is the difference between influential observations and outliers in linear regression model?
3
votes
1answer
211 views

Is there a specific standard error for an individual residual?

On page 97 of Introduction to Statistical Learning book, there is a paragraph on studentized residuals within the context of looking for outliers. But in practice, it can be difficult to decide how ...
0
votes
0answers
165 views

News Impact curve shift

I computed the NIC for both vanilla GARCH and EGARCH model specifications (with t(4)-student distributed innovations). As we can see from the plot, no asymmetries are present, but the curve has ...
0
votes
0answers
112 views

How to solve a well fitted model - Model Misspecification

I am currently writing a paper, analysing the impact of goldprice movements on the capital structure of gold mining firms. My basic model is a simple OLS model with (y=leverage and x=ln(goldprice)). ...
1
vote
0answers
184 views

Sensitivity Measures for GEE Model

Is there any method (e.g. like Cooks D) implemented in R to identify leverage points for GEE Models? I used geepack to fit my models and would like to do a sensitivity analysis now. However, I don't ...
2
votes
1answer
1k views

Leverage formula/derivation and Hat matrix

1) So I know that $h_{ii}$ is just the ith row ith column of $H=X(X^TX)^{-1}X^T$. Intuitively, why is this the case? I understand that H is the projection matrix and leverage is measuring how far away ...
6
votes
2answers
15k views

Cook's distance vs. hat values

What exactly does Cook's distance measure? And how is this different from what hat values measure? I know hat values measure how distant a point it form its corresponding fitted point. I also know ...
3
votes
1answer
1k views

Leverage Statistic - equation explanation

I came across this equation for the leverage statistic in ISL. Does the denominator translate to the summation of all the squares of $(x_i - mean(x))^2$ excluding the current observation of x? In ...
6
votes
1answer
703 views

What are the leverage values for Ridge regression?

In linear least squares the parameter estimates are: $\hat{\beta} = \left(X^{\top}X\right)^{-1}X^{\top}y$. In Ridge regression the standardized parameter estimates are given by $\hat{\beta}_{\Gamma} = ...
0
votes
0answers
130 views

which residual vs fitted plot is looking better here?

This is a data analysis project we have at school where we try different ways to correct a simple linear regression model. My first attempt is just trying to transform the response variable by taking ...
1
vote
0answers
69 views

How to Adapt Leverage vs Residual plot for Multi-variate Regression

After reading about What is the best way to identify outliers in multivariate data? I noticed that some heuristics are hard to adapt to mult-variate analysis. I'm hoping there is a way to at least ...
1
vote
0answers
23 views

How to check if overall results from a survey are significantly affected by participation from a large company(ies)?

Suppose there is some sort of survey where people from various companies answer some questions. Is it possible to determine whether overall results from the survey are significantly affected by ...
2
votes
1answer
1k views

Is it OK if I have some high leverage points in my experimental plots and they weren't outliers?

I've analyzed effects of temperature and sample thickness on drying process via response surface methodology. I've faced to four high leverage points (my four axial runs) that stick together on normal ...
1
vote
1answer
30 views

Using low coverage predictors in models?

The conventional wisdom I run up against is to drop predictors with low coverage without given them much consideration. By low coverage predictors: I meant predictors whose values are mostly missing. ...
1
vote
1answer
75 views

Deleting Influential Points Leading to Non-constant Variance

http://archive.ics.uci.edu/ml/datasets/Wine+Quality Using this data set, I am regressing chlorides on ten predictors (all except quality). After Box-Cox transforming chlorides, I get the following Q-Q ...
0
votes
0answers
63 views

Linear model and mean value of covariates

Suppose a linear model $$Y_i=\beta_0+x_{i1}\beta_1+\ldots+x_{ip}\beta_p+e_i,\quad e_i\sim N(0,\sigma^2),\quad i=1,\ldots,n,$$ and its hat matrix $P=X(X^TX)^{-1}X^T$, where $Y=X\beta+e$. Let $\bar{x}...
1
vote
0answers
278 views

Influence function for glm?

I would like to calculate the influence of each sample on the coefficient under a logistic regression model. The R built-in function influence() is suppose to do ...
1
vote
1answer
747 views

How to extract/compute leverage and Cook's distances for Generalised Additive models

Similar to the following question, How to extract/compute leverage and Cook's distances for linear mixed effects models Is there any method in R to determine influential points in a GAM?