What exactly are moments? How are they derived? We are typically introduced to method of moments estimators by "equating population moments to their sample counterpart" until we have estimated all of the population's parameters; so that, in the case of a normal distribution, we would only need the first and second moments because they fully describe this distribution. 
$E(X) = \mu \implies \sum_{i=1}^n X_i/n = \bar{X}$
$E(X^2) = \mu^2 + \sigma^2 \implies \sum_{i=1}^n X_i^2/n$
And we could theoretically compute up to $n$ additional moments as:
$E(X^r) \implies \sum_{i=1}^nX_i^r /n$
How can I build intuition for what moments really are? I know they exist as a concept in physics and in mathematics, but I find neither directly applicable, especially because I don't know how to make the abstraction from the mass concept to a data point. The term seems to be used in a specific way in statistics, which differs from usage in other disciplines.
What characteristic of my data determines how many ($r$) moments there are overall?
 A: This is a bit of an old thread, but i wish to correct a misstatement in the comment by Fg Nu who wrote "Moments are parameterized by the natural numbers, and completely characterize a distribution".
Moments do NOT completely characterize a distribution. Specifically, knowledge of all infinite number of moments, even if they exist, does not necessarily uniquely determine the distribution.
Per my favorite probability book, Feller "An Introduction to Probability Theory and Its Applications Vol II" (see my answer at Real-life examples of common distributions ), section VII.3 example on pp. 227-228, the Lognormal is not determined by its moments, meaning that there are other distributions having all infinite number of moments the same as the Lognormal, but different distribution functions.  As is widely known, the Moment Generating Function does not exist for the Lognormal, nor can it for these other distributions possessing the same moments.
As stated on p. 228, an essentially nonzero random variable $X$ is determined by its moments if they all exist and
$$\sum_{n=1}^{\infty} (\mathbb{E}[X^{2n}])^{-1/(2n)}$$
diverges. Note that this is not an if and only if.  This condition does not hold for the Lognormal, and indeed it is not determined by its moments.
On the other hand, distributions (random variables) which share all infinite number of moments, can only differ by so much, due to inequalities which can be derived from their moments.
A: A corollary to Glen_b's remarks is that the first moment, the mean, corresponds to the center of gravity for a physical object, and the second moment around the mean, the variance, corresponds to its moment of inertia.  After that, you're on your own.
A: It's been a long time since I took a physics class, so let me know if any of this is incorrect.
General description of moments with physical analogs
Take a random variable, $X$. The $n$-th moment of $X$ around $c$ is:
$$m_n(c)=E[(X-c)^n]$$
This corresponds exactly to the physical sense of a moment. Imagine $X$ as a collection of points along the real line with density given by the pdf. Place a fulcrum under this line at $c$ and start calculating moments relative to that fulcrum, and the calculations will correspond exactly to statistical moments.
Most of the time, the $n$-th moment of $X$ refers to the moment around 0 (moments where the fulcrum is placed at 0):
$$m_n=E[X^n]$$
The $n$-th central moment of $X$ is:
$$\hat m_n=m_n(m_1) =E[(X-m_1)^n]$$
This corresponds to moments where the fulcrum is placed at the center of mass, so the distribution is balanced. It allows moments to be more easily interpreted, as we'll see below. The first central moment will always be zero, because the distribution is balanced.
The $n$-th standardized moment of $X$ is:
$$\tilde m_n = \dfrac{\hat m_n}{\left(\sqrt{\hat m_2}\right)^n}=\dfrac{E[(X-m_1)^n]}
{\left(\sqrt{E[(X-m_1)^2]}\right)^n}$$
Again, this scales moments by the spread of the distribution, allowing for easier interpretation specifically of Kurtosis. The first standardized moment will always be zero, the second will always be one. This corresponds to the moment of the standard score (z-score) of a variable. I don't have a great physical analog for this concept.
Commonly used moments
For any distribution there are potentially an infinite number of moments. Enough moments will almost always fully characterize and distribution (deriving the necessary conditions for this to be certain is a part of the moment problem). Four moments are commonly talked about a lot in statistics:


*

*Mean - the 1st moment (centered around zero). It is the center of mass of the distribution, or alternatively it's proportional to the moment of torque of the distribution relative to a fulcrum at 0.

*Variance - the 2nd central moment. Interpreted as representing the degree to which the distribution of $X$ is spread out. It corresponds to the moment of inertia of a distribution balanced on its fulcrum.

*Skewness - the 3rd central moment (sometimes standardized). A measure of the skew of a distribution in one direction or another. Relative to a normal distribution (which has no skew), positively skewed distribution have a low probability of extremely high outcomes, negatively skewed distributions have a small probability of extremely low outcomes. Physical analogs are difficult, but loosely it measures the asymmetry of a distribution. As an example, the figure below is taken from Wikipedia.


*Kurtosis - the 4th standardized moment, usually excess Kurtosis, the 4th standardized moment minus three. Kurtosis measures the extent to which $X$ places more probability on the center of the distribution relative to the tails. Higher Kurtosis means less frequent larger deviations from the mean and more frequent smaller deviations. It is often interpreted relative to the normal distribution, which has a 4th standardized moment of 3, hence an excess Kurtosis of 0. Here a physical analog is even more difficult, but in the figure below, taken from Wikipedia, the distributions with higher peaks have greater Kurtosis.

We rarely talk about moments beyond Kurtosis, precisely because there is very little intuition to them. This is similar to physicists stopping after the second moment.
A: A binomial tree has two branches each with a probably of 0.5. Actually, p=0.5, and q=1-0.5=0.5. This generates a normal distribution with an evenly distributed probability mass. 
Actually, we have to assume that each tier in the tree is complete. When we break data up into bins, we get a real number from the division, but we round up. Well, that's a tier that is incomplete, so we don't end up with a histogram approximating the normal. 
Change the branching probabilities to p=0.9999 and q=0.0001 and that gets us a skewed normal. The probability mass shifted. That accounts for skewness. 
Having incomplete tiers or bins less than 2^n generate binomial trees with areas that have no probability mass. This gives us kurtosis. 

Response to comment:
When I was talking about determining the number of bins, round up to the next integer. 
Quincunx machines drop balls that come to eventually approximate the normal distribution via the binomial. Several assumptions are made by such a machine: 1) the number of bins is finite, 2) the underlying tree is binary, and 3) the probabilities are fixed. The Quincunx machine at the Museum of Mathematics in New York, lets the user dynamically change the probabilities. The probabilities can change at any time, even before the current layer is finished. Hence this idea about the bins not being filled. 
Unlike what I said in my original answer when you have a void in the tree, the distribution demonstrates kurtosis. 
I'm looking at this from the perspective of generative systems. I use a triangle to summarize decision trees. When a novel decision is made, more bins are added at the base of the triangle, and in terms of the distribution, in the tails. Trimming subtrees from the tree would leave voids in the distribution's probability mass. 
I only replied to give you an intuitive sense. Labels? I've used Excel and played with the probabilities in the binomial and generated the expected skews. I have not done so with kurtosis, it doesn't help that we are forced to think about probability mass as being static while using language suggesting movement. The underlying data or balls cause the kurtosis. Then, we analyze it variously and attribute it to shape descriptive terms like center, shoulder, and tail. The only things we have to work with are the bins. Bins live dynamic lives even if the data can't. 
A: 
How can I build intuition for what moments really are?

In the context of computer vision, recognizing two-dimensional shapes such as letters or geometric objects from pixel data, a classic article exploiting moments is:
Ming-Kuei Hu, "Visual pattern recognition by moment invariants," in IRE Transactions on Information Theory, vol. 8, no. 2, pp. 179-187, February 1962.
