Putting it simply: If the data is not uniformly distributed in $x$ you get a wiggly estimate!
In effect your estimate at point $x^*$, $\mu(x= x^*)$ is still unbiased but it suffers it terms of variance. Think of it as the weighted regression estimate around point $x^*$ but lacking any close neighbours. You still estimate something but you are more uncertain about it.
The weight for an individual point in the particular region (sparsely or densely populated is irrelevant) in not proportional to the number of neighbours it has. It is only determines by the kernel and the bandwidth used. Commonly it has the form $K(\frac{x - x^*}{b})$, ie. you just care how far you are from all the other points and you normalize that by the bandwidth $b$. (Kernel functions are symmetric in the sense $K(-u)=K(u)$ so don't get boggedd down by the sign).
The obvious implication when using a smoothing parameter is that if your bandwidth is too small you will base your estimate $\mu(x^*)$ in a small number of points. Would you believe a regression with 3 data points for example? In addition some kernel functions (most commonly the Gaussian kernels) have infinite support. That means that you will always be able to get some minute information from distant points even if you have a relatively small bandwidth. The Epanechnikov or the rectangular (uniform) kernel functions have finite supports; practically, information outside the bandwidth range is ignored. You might be OK with that if you have a slightly dense design but if you have irregularly sampled data you might end up some holes in your estimates.
I think a decent rule of thumbs is a $k$-NN (nearest neighbour) approach. You use the smallest bandwidth that guarantees at least $k$ points at each point of estimation $x^*$ (usually this is used to find the minimum bandwidth). You can then CV your way to better bandwidth's $b$!