Questions tagged [explainable-ai]
The explainable-ai tag has no usage guidance.
14
questions
0
votes
0
answers
15
views
What do negative feature attributions in machine learning mean?
Maybe this is not a particular smart question, but I fail to understand negative feature attributions (importance scores) from algorithms such as IntegratedGradients in explainable (interpretable) ...
2
votes
0
answers
46
views
SHAP values under multicolinearity/feature dependence
My task is to explain individual predictions, but having read the original paper and sifted through the internet, I am still unsure whether using something like TreeSHAP can help me with the situation ...
- 21
3
votes
1
answer
332
views
The best interpretable regression model currently
I'm seeking recommendations for explainable regression models for tabular data. I'm looking for approaches that offer a balance between complexity and interpretability – something more sophisticated ...
- 31
0
votes
0
answers
135
views
Shap values associated to missing features
I have a model trained with Xgboost on some training data X_train, described by 10 features (x$_{1}$,...,x$_{10}$) and some of them might exhibit some missingness, i.e. some x$_{i}$ = NaN. This is ok ...
- 3
0
votes
0
answers
34
views
Can SHAP values across different models in the same family of models trained on different datasets be aggregated?
Let's say I have time - series data for 100 products in a particular store. I fit 100 regression models to generate 1-step forecasts for these products.
Let's say that all features are common across ...
- 109
8
votes
1
answer
551
views
Why can a model's SHAP values change on a new dataset?
Background
I'm validating a model and as part of the process I've been calculating SHAP values for different validation datasets.
I've calculated SHAP values for every sample in each dataset taken ...
- 645
3
votes
1
answer
141
views
How to obtain the last convolutional layer of a model in torchvision for applying grad cam?
I'm using efficient net b0 from torchvision for training a classifier for cifar10. I would like to apply grad cam for generating saliency maps for explaining the predictions. However, I'm not sure ...
- 1,008
0
votes
0
answers
124
views
SHAP for text input: aggregating values across instances
I am working on a binary text classification task, where the input is text (~100 words long, but length varies). I am using a fine-tuned BERT-based model. My goal is to get insights about which ...
- 1
5
votes
2
answers
247
views
Is there a way to know if a machine learning model is extrapolating at inference time?
I am working on a regression model, trained with a finite dataset. The algorithm that I use is a LightGBM, but I think the solution I am looking for would be algorithm-agnostic in essence.
I suspect ...
- 51
0
votes
0
answers
104
views
Shapley Values: What is the permutation of a set?
Apologies if this is a very simple question--I am very new to Shapley values!
I am reading through an article on Shapley values written by some of the original authors of the SHAP paper. While their ...
- 115
0
votes
1
answer
140
views
Feature importance/model exploration in a large survival model
I've built very-well performing survival model (weibull proportional hazards model for interval censored data, modelled with IcenReg) with many covariates, some ...
- 85
1
vote
0
answers
75
views
Explainable AI - Noise in gradients and embeddings of large language models
I am doing experiments related to explainable AI. I have two BERT models - the standard bert-base-cased and a distilled ...
- 11
0
votes
1
answer
1k
views
which one between XGBoost and neural networks is more interpretable? [closed]
I have heard people say, "One of the disadvantages of neural networks is that they are generally less interpretable". But I wonder, how is another model, such as XGBoost, more interpretable ...
- 3
2
votes
1
answer
105
views
Distribution of Sobol's Indices
Some background: Given a linear regression model (or any other GLM), we all know how to test the null hypothesis $\hat\beta_i=0$. The lm function in ...
- 4,017