If we reduce size of training dataset does it decreases bias? I'm a newbie and learning ML. I've a doubt, normally we know we should increase the size of training dataset or should add more data to reduce variance (fairly understood why). Now variance has inverse relationship with bias, so it means when we're adding more data, we're reducing variance - or we're increasing bias. Then, why this is not possible to reduce bias by reducing the number of training samples. Could someone please explain me.
 A: 
Now variance has an inverse relationship with bias

Not necessary. A picture is worth a thousand words, so let me use the image below. (Check also the Intuitive explanation of the bias-variance tradeoff? thread.)

Imagine your model is an oracle that perfectly predicts the target, it will have no bias and no variance.

Then, why is not possible to reduce bias by reducing the number of training samples.

Imagine a model that always predicts the same constant (say, $42$), it will be biased regardless of how much data would you use because the result is independent of the data. The example is abstract, but not as abstract as you may think, for example, this would be the case for a Bayesian model with a very strong prior, or using an incorrect model for the job (e.g. image classification using a model that was designed for natural language processing), such models are doomed to make bad predictions regardless of the data.
A: You can only speak of 'inverse relation' when you change model complexity (including, to some extent, feature selection vs feature addition).

*

*As the number of samples grows, variance drops, bias is unchanged.

*Increasing the model complexity decreases bias but increases variance.

*Dropping irrelevant features decreases variance, adding relevant features decreases bias.

A: The variance bias relationship is a causal relationship.
For certain types of bias the variance can reduce when the bias increases. That means that the other way around does not work. If you reduce/increase the variance (by some other means than bias, for instance sample size) then you do not increase/reduce the bias.
A clear example is a shrinkage estimator for estimating the mean of a normal distribution. The typical unbiased estimator is the sample mean $\bar{x}$, which has a variance $\frac{\sigma^2}{n}$. We could also use a biased estimate by multiplying the sample mean with a factor $c \cdot \bar{x}$, which has a variance $c^2\frac{\sigma^2}{n}$. By changing the bias $c$ we can decrease the variance, but by changing the variance, for instance by having different $n$, we do not change $c$.
In a more complex situation we could have an interaction between bias and variance. In the previous example the bias is dependent on $c$. There are settings in which we change this parameter based on the observed variance. But often the relationship is in the other direction. The bias is optimised to reduce the mean squared error, and if we have less variance, then we need less bias.
