# Tag Info

26

Both answer how far your values are spread around the mean of the observations. An observation that is 1 under the mean is equally "far" from the mean as a value that is 1 above the mean. Hence you should neglect the sign of the deviation. This can be done in two ways: Calculate the absolute value of the deviations and sum these. Square the deviations and ...

15

Today, statistical values are predominantly calculated by computer programs (Excel, ...), not by hand-held calculators anymore . Hence, I would posit that calculating "mean deviation" is no more cumbersome than calculating "standard deviation". Although standard deviation may have "... mathematical properties that make it more useful in statistics", it is, ...

11

Let's suppose we could find a (measurable) function $\chi$ defined on the real numbers with the property that $$\chi(a + b) = \chi(a)\chi(b)$$ for all numbers $a$ and $b$ and for which there is a finite positive number $M$ for which $|\chi(a)| \le M$ for all $a$. Notice how $\chi$ relates addition (which is the fundamental operation appearing in a ...

9

I think you want to have a look at what the text-mining people call smoothing. A simple smoothing technique is to add one to every word count, so no word has a zero probability estimate - essentially pretend that every word occurs once more than it does in reality. Generalized, this is sometimes called "Laplace smoothing" or "additive smoothing" - it's a ...

9

They both measure the same concept, but are not equal. You are comparing $\frac{1}{n} \sum |x_i-\bar{x}|$ with $\sqrt{\frac{1}{n} \sum (x_i-\bar{x})^2}$. They aren't equal for two reasons: Firstly the square-root operator is not linear, or $\sqrt{a+b} \neq \sqrt{a} + \sqrt{b}$. Therefore the sum of absolute deviations is not equal to the square root of ...

8

Your histograms need to not just have the same number of bins, they need to have the same bins. Set your histograms relaive to the max and min of your whole dataset, not just relative to each gender. Designing a good histogram is more art than science. The heuristic you used to determine the number of classes is fine as a rule of thumb, but really you should ...

8

This kind of plot could be generated with geom_rect. Your data: names <- read.csv("http://samswift.org/files/app_c.csv") sum50 <- tapply(names$count, (seq_along(names$count)-1) %/% 50, sum) First, we need additional variables: The cumulative sum: cum <- rev(cumsum(rev(sum50))) Put all into a data frame. The variables start and stop indicate ...

8

A chi-square statistic gets bigger the further from expected the entries are. The p-value gets smaller. Very small p-values are saying "If the null hypothesis of equal probabilities were true, something really unlikely just happened" (the usual conclusion is then usually that something less remarkable happened under the alternative that they're not equally ...

8

Welcome to CV! Do the # of classes only have to be the same? Not necessarily. Do the histograms both have to start at the same point (0 or 20)? Highly recommended, and the axes should also end at the same number, and better if they are of the same length as well. More babbling: It depends what do you mean by "comparing." From just the two histograms I ...

8

@itsols, I'll add to Kasper's important notion that The mean deviation is rarely used. Why is standard deviation considered generally a better measure of variability than mean absolute deviation? Because arithmetic mean is the locus of minimal sum of squared (and not sum of absolute) deviations from it. Suppose you want to assess the degree of altruism. ...

7

It is hard to answer your needs without more detail. In text analysis, word frequencies are replaced by tf*idf which stands for "term frequency times inverse document frequency". This is an empirical score that corrects for the occurrence of terms that are frequent in the corpus and thus do not discriminate documents. It is widely used to compare texts, in ...

7

A common misunderstanding is the term "frequency". To some, it seems to be the count of objects. But usually, frequency is a relative value. TF/IDF usually is a two-fold normalization. First, each document is normalized to length 1, so there is no bias for longer or shorter documents. This equals taking the relative frequencies instead of the absolute ...

7

Most likely you have two seasonal periods: 48 (number of intervals per day) and 48x5 (number of intervals per week assuming a 5-day week). The tbats() function from the forecast package in R will handle multiple seasonal periods. For example (where x is the data): library(forecast) x <- msts(x, seasonal.periods=c(48, 48*5)) fit <- tbats(x) fc <- ...

7

One thing worth adding is that the most likely reason your 30-year-old textbook used the absolute mean deviation as opposed to standard deviation is that it is easier to calculate by hand (no squaring / square roots). Now that calculators are readily accessible to high school students, there is no reason not to ask them to calculate standard deviation. ...

7

Both measure the dispersion of your data by computing the distance of the data to its mean. the mean absolute deviation is using norm L1 (it is also called Manhattan distance or rectilinear distance) the standard deviation is using norm L2 (also called Euclidean distance) The difference between the two norms is that the standard deviation is calculating ...

7

You can deal with the unequal number of days in a month by using the concept of an exposure. A good reference for this is Data Analysis Using Regression and Multilevel Models chapter 6.2. Here's an overview. Let's attempt to estimate, instead of the number of events per month, the number of events per day. This is better, because a day is always a fixed ...

7

There are many metrics. They are best used in conjunction with visualizing the data appropriately. Among the solutions worth considering are to compare the distributions of the frequencies (regardless of time) to your reference distribution, the uniform one. Theory suggests that the deviations from perfect uniformity--the residuals--should be about the ...

6

Correspondence Analysis is a method to visualize a contingency table, such as frequency cross-table. I presume that the table in your case would be 5 Brands X 11 Attributes and the entries are frequencies (counts) of 1s: attribute is characteristic of a brand. And you want the analysis to produce a biplot wherein points-brands are acommpanied by points-...

6

I don't think this is conceptually possible. I don't know how Eviews can claim to do this (@user1493368's answer). It sounds incredibly dangerous. Apart from anything else, if you have 22 observations, that is all you have - you cannot somehow pretend you have 88 and then use them in a model as though you had four times as many data points as you do. ...

6

They are similar measures that try to quantify the same notion. Typically you use st. deviation since it has nice properties, if you make some assumption about the underlying distribution. On the other hand the absolute value in mean deviation causes some issues from a mathematical perspective since you can't differentiate it and you can't analyse it ...

6

I would treat this as one observation of, assuming you started at midnight on Dec. 31, 2000, 4 years, and another observation censored at 12.956 years. In the case of the exponential distribution, this is a pretty easy problem to solve: $$p(x_1, x_2 | \lambda) = \left(\frac{1}{\lambda}e^{-x_1/\lambda}\right)\left(e^{-c_2/\lambda}\right)$$ where the first ...

5

This is an empirical version of the "probability integral transform." Suppose you have $n$ data values. Sorting them in ascending order, index them as $$x_1 \le x_2 \le \cdots \le x_n.$$ Replacing each $x_i$ by $i$ gives as uniform a distribution as possible. That's all there is to it. If you would like the replacement values to lie within a given ...

5

First I would like to say that I'm not sure if I fully understand what you ask in your actual question. The whole concept of statistics (or more rigorously inference theory) is to draw conclusions from the given data, so yes it is possible to use statistics to make predictions about a single case as long as you have any data to base you prediction on. You ...

5

No, larger sample sizes have more power. If you have something that occurs 1 in 100,000 times then, unless you have a very large number of trials, you are unlikely to see any events. And, if you see one event, you will have a very poor idea of how likely events are. The problem is that you don't know (prior to the test) how likely events are, exactly. ...

5

The example table shows how many times each length was observed. For example: length 1 was observed once, length 2 was observed 15 times, etc. The typical purpose of a histogram here would be to show how often each length is observed. If you simply plotted the numbers in the table with length on the x axis and count on the y axis, it would be a proper ...

4

This question calls for a modification of the solution to a sequence counting problem: as noted in comments, it requests a cross-tabulation of co-occurrences of values. I will illustrate a naive but effective modification with R code. First, let's introduce a small sample dataset to work with. It's in the usual matrix format, one case per row. x <- ...

4

You have a discrete variable with, in your example, very small frequencies. It would be fine to show relative frequencies so long as you made clear the small sample size. I suggest that it would be nevertheless be simpler to show sample frequencies directly: In this case we hope that people are able to look at the graph and think (e.g.) 3 out of 7 were on ...

4

Your title (and question body) ask about computing the mean of a sample. The sample mean is perfectly well-defined and doesn't depend in any way on the distribution from which the data were drawn (as long as it makes sense to actually calculate a mean; you can hardly do that if the values are nominal categories like $\{\text{red},\text{blue},\text{yellow}\}\... 4 This follows from the fact that the local effect of the function$R\to R^3$is to expand values more and more as$R\$ increases. We don't actually need any ideas of Calculus to demonstrate this, though: by formulating the situation in the right way, it comes down to demonstrating a simple inequality. Just don't lose site of the meaning of that inequality. ...

3

That would work. Or just use 365.25/7, the average number of weeks in each year. The difference between leap and non-leap years is so small it shouldn't matter much.

Only top voted, non community-wiki answers of a minimum length are eligible