Conceptual questions on Estimation theory: Beginner level I have just started to study Estimation theory and following the book "Fundamental of Statistical Signal Processing" by Steven M. Kay Vol 1. Most of the examples are based on the assumption that data follows a Gaussian distribution. COnsider, the example of estimating an unknown parameter $A$ from the relation: $x[n] = A + w[n]$ where w[n] is said to be $i.i.d$ random gaussian noise of known variance and zero mean. I have few conceptual questions and would appreciate intuitive notion behind them
Q1: Performing, $x[n]-w[n] = A$ would yield the unknown parameter A since w[n] is known. Then why does one need to consider the estimation using the estimator $\hat{A} = \frac{1}{N}\sum_{n=0}^{N-1}x[n]$ (Pg 12, Sec 2.3 Unbiased Estimators)
Q2: In general, does the number of samples, n play any role in estimation? Say n= 10 and n=1000, would this consideration affect the possibility of finding better estimates?
Q3: In general, why do we consider the samples and the noise to be $i.i.d$ ?? Is it ever possible that the samples are not iid? If so then when are they correlated?
Thank you
 A: Q1: you know the distribution of $w_n$, you don't know the value of $w_n$ for a give $n$. For a stupid example, you know that when you toss a coin $P(\text{head})=50\%$, but you don't know what the result of the next toss will be.
Q2: of course. $\hat{A}=\frac{1}{N}\sum_{n=1}^N x_n$, so its expected value is:
$$E\left[\hat{A}\right]=E\left[\frac{1}{N}\sum_{n=1}^N x_n\right]=\frac{1}{N}\sum_{n=1}^N E[x_n]\overset{iid}{=}\frac{1}{N}\sum_{n=1}^N E[x]=\frac{1}{N}NE[x]=E[x]$$
but
$$\text{Var}\left(\hat{A}\right)=\text{Var}\left(\frac{1}{N}\sum_{n=1}^N x_n\right)\overset{ind}{=}\frac{1}{N^2}\sum_{n=1}^N\text{Var}(x_n)\overset{id}{=}\frac{1}{N^2}N\text{Var}(x)=\frac{\text{Var}(x)}{N}$$
I.e. $\hat{A}$ is a good (unbiased) estimate, but its uncertainty (variance) decreases as $N$ increases. For another stupid example, if you get 2 heads in three tosses ($\hat{A}=2/3$) you can't say that the coin is unbalanced, but if you get 2000 heads in 3000 tosses ($\hat{A}$ is still $2/3$)... it's another matter!
Q3: because the iid assumption makes all simpler ;-) Samples may be not iid (e.g. in multistage sampling) and $x_n$ may be correlated to $A$. In such cases you have to use more complex estimation strategies, but learning the simpler ones at first is better.
