What do we exactly mean by a biased estimate? I am not able to understand the exact meaning of bias. The least squares is said to give us unbiased estimates during linear regression but still we refer linear regression to have a high bias because of its assumption ? 
 A: Bias is a relative term, meaning approximately

How far on average is the estimated thing from the truth.

Depending on what we are assuming the word "truth" means, we have different conceptions of bias.  You are experiencing that two of those conceptions are relevant for linear regression, and they can come to opposite conclusions about the model.

The least squares is said to give us unbiased estimates during linear regression

When we say this, we are assuming that the truth has a specific structure
$$ y \mid X \sim \beta_0 + \beta_1 X_1 + \cdots + \beta_k X_k + \epsilon $$
(where $\epsilon$ is a random noise term which does not depend on $X$) and we are using the model to try to uncover something about the numbers $\beta_0, \beta_1, \ldots, \beta_k$.  The unbiasedness of linear regression in this scenario says that on average, when we use linear regression to estimate the $\beta$s, we get the correct answer.
The assumption here is strong, we need to be willing to accept that, in truth, the conditional mean of $y$ is a linear function of $X$.  If we weaken this assumption...

but still we refer linear regression to have a high bias because of its assumption

Here we are making much weaker assumption about what the truth looks like.  We are just assuming that there is some function $f$ such that
$$ y \mid X \sim f(X) + \epsilon $$
Since the only shapes our fit model can assume is a line, but it is possible in this case that $f$ is very not a line, it is impossible for our fit linear regression to assume, on average, the correct shape.  In this setup, we may say that linear regression is biased (*).
(*) Note though, in the case where $f$ really is a linear function, we would not say that linear regression is biased, even in the second setup.
A: I don't have enough points to comment so here goes:

*

*I think there may be a confusion between linear regression as an unbiased model of the data, and a linear model's estimation method, e.g. Ordinary Least Squares (OLS), being an unbiased estimator for the real coefficients.

*I think OP may be referring to the second definition of bias, and "The least squares is said to give us unbiased estimates during linear regression" is a sentence he would have typically found in a book describing OLS under Gauss-Markov assumptions.

*If that's what OP means, then I would point to the difference between the linear regression's coefficient estimates and a linear relationship as an 'estimate' of the true relationship.

*

*The coefficient estimator can be unbiased using OLS or other estimators depending on the assumptions

*While the linear model is often biased because the relationship between the dependent variable and its predictors is not linear (so the predicted value is a biased estimate of the real target value).



