Hot answers tagged

1

When zero is a valid value for a variable (i.e., not missing) then use it in the analysis. However, for home price, when zero is replaced for price since it's missing, delete the zeroes, since most software will recognize the missing fields as missing. You don't ever want to (e.g.) calculate average house price by including home with price values of zero, ...


1

You’ll use two new binary features and use one hot encoding. For example, for Dr. A your features will be [1,0], and for Dr. B your features will be [0,1]. Assigning arbitrary numbers to each doctor is not the correct approach because it induces an implicit ordering, i.e. normally you don’t have Dr. A < Dr. B but depending on your assigned numbers, you’ll ...


1

Welcome to the site, @Nick, and good luck with your job application. I will wade in regarding your forth question. Whenever data are collected from scratch, which seems to be the case here, you have to understand first why the data are collected. Let's say in this case the data are collected to help the company predict which customers are likely to opt ...


1

$x\sim D$ is not a typical notation if $D$ is a dataset. Normally the notation is used as $x\sim p(x)$ where $p(x)$ represents a probability distribution or the population. Abusing this notation, $x\sim D$ can mean $x$ is sampled from the dataset $D$, where every sample is iid, i.e. we get a random sample from the dataset. This dataset can be a usual dataset ...


1

what exactly is the bias that is referred to in the validation and testing datasets? That bias refers to systematically under- or overestimating the predictive performance of the model. In other words, the figure of merit (e.g. some error) you calculate is systematically off. In this particular situation (hyperparameter tuning) we're concerned with ...


1

Bias refers to one component of the error of your model. The other component is variance, and there is a trade-off between the two. Bias comes essentially from the assumptions you (or your model) make about the data. For example, in linear regression you assume linearity, normal distribution of errors, and homoscedasticity. Variance, on the other hand, ...


1

Yes, the bias lies in the model's success metrics, biased because of the potential for overfitting. The data you observe are shaped not only by the substantive relations you are attempting to model but by a range of other factors specific to the conditions under which this dataset was collected. Your modeling may exploit those peculiarities but attribute the ...


Only top voted, non community-wiki answers of a minimum length are eligible