3
$\begingroup$

I see that we have a concept called expected value being used in machine learning (ML) models. For example, SHAP has a concept called Expected value. It means when all input features are 0, we can consider model to output the expected value (baseline value).

Similarly, in linear regression, we have a term called intercept. Intercept also means the same, which is expected model output when all input features are zero.

While, I understand that in real world we may not encounter this scenario of all input features being zero.

Why do we consider this base performance? Is it because, we assume that all models will have some inherent power to predict output (on random). So, we have that base value?

Like a student who doesn't prepare for exams, can still get some marks (>0). Is that the same understanding here?

$\endgroup$

1 Answer 1

7
$\begingroup$

It is the simplest (or amongst the simplest) possible model(s). If your explanatory data (feature list) is $\mathcal D$ and the target variable is $Y$, what models usually try is predicting the conditional expectation, i.e. $\mathbb E[Y|\mathcal D]$. But, in the absence of data/features, you're left with unconditional expectation, i.e. $\mathbb E[Y]$. It is the simplest you can do without looking at the data/features.

$\endgroup$
4
  • 2
    $\begingroup$ +1 I often refer to the $\hat y_i=\bar y$ model as the “most sensible naïve model”. $\endgroup$
    – Dave
    Mar 26 at 18:33
  • $\begingroup$ thanks upvoted. But why can't the expected value be zero? And predictrd value can be purely based on input variables. In absence of inpit variables, let the output be zero but why some baseline value? $\endgroup$
    – The Great
    Mar 27 at 0:25
  • $\begingroup$ That wouldn't be intuitive in the first place. When you have a target variable that comes from a distribution having a non-zero mean, and you don't look into the data, $0$ would be a terribly bad estimator; e.g. average age of a person w/o looking into any features would best be the sample mean in the dataset. How would $0$ ages look like as a prediction in that case? $\endgroup$
    – gunes
    Mar 27 at 10:38
  • $\begingroup$ @gunes - Can I seek your help regarding a follow up question on this subject posted here - stats.stackexchange.com/questions/569985/… $\endgroup$
    – The Great
    Apr 1 at 11:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.