Questions tagged [frequency]

The frequency measures the number of times an event occurs in a data set.

Filter by
Sorted by
Tagged with
54
votes
7answers
28k views

Period detection of a generic time series

This post is the continuation of another post related to a generic method for outlier detection in time series. Basically, at this point I'm interested in a robust way to discover the periodicity/...
35
votes
11answers
110k views

Mean absolute deviation vs. standard deviation

In the text book "New Comprehensive Mathematics for O Level" by Greer (1983), I see averaged deviation calculated like this: Sum up absolute differences between single values and the mean. Then ...
14
votes
3answers
2k views

How to measure the statistical “distance” between two frequency distributions?

I am undertaking a data analysis project which involves investigating website usage times over the course of the year. What I would like to do is compare how "consistent" the usage patterns are, say, ...
7
votes
2answers
3k views

What's a good approach to estimate the probability of word frequencies?

I have a document corpus and I want to estimate the probability of occurrence of a certain word $w$. Simply calculating the frequencies and use such a number as an estimation is not a good choice. Is ...
7
votes
2answers
1k views

Why does multiplication in the frequency domain equal convolution in the time domain?

This question came in the context of understanding how to get a distribution of a sum of two iid random variables. I'm working through the top answer to this question Consider the sum of $n$ uniform ...
7
votes
3answers
3k views

How to figure out what numbers often appear together in a dataset?

I have a lottery style dataset we produce internally (example below). I am trying to figure out which numbers appear most frequently together. Example questions: What are the top 10 pair of numbers ...
7
votes
2answers
913 views

Frequency jump detection

It is generally known that 'jumps' in frequency data are difficult to estimate. In the current literature, many different techniques for estimating such jumps have been tested and often with ...
7
votes
2answers
721 views

chi-squared to test if two variables have the same frequency distribution

I want to test if x and y have the same frequency distributions using chi-squared. In my code below, I've concluded that because ...
6
votes
4answers
586 views

How do I quantify the uniformity of sampling time?

My datasets cover a long time, say 20 years, and the data are sampled at different time. For example, some are obtaiend at 1:00(stored as 100), some are 23:00(2300). If I plot the histogram of the ...
6
votes
1answer
4k views

Chi-squared distribution for dice not returning expected values?

I'm trying to verify that my dice are reasonably fair and balanced. I'm not sure I'm doing it right, though, since I'm basically teaching myself stats without references. To achieve this, I've settled ...
6
votes
1answer
100 views

Most significantly frequent category

I am measuring the frequency of consonants in a sample lexicon of a language. The results are as follows: ...
6
votes
2answers
2k views

Statistical comparison of two signals

I need to develop an algorithm that will compare two signals and generate some metric(s) to describe changes between them. Signal processing and analysis isn’t my strong point so I would appreciate ...
5
votes
1answer
1k views

Visualization of binned frequency distribution in R

I am trying to visually represent the distribution of last names in the US. Specifically, I am trying to show that the distribution is such that the most common names (say the top 50) are very common ...
5
votes
2answers
401 views

How to estimate the Poisson distribution with one event occurrence?

Let us say we observed one event on 1 Jan 2005, and we were observing the events since 2000. What's the best estimate of Poisson distribution? One approach is to set intensity $\lambda=1/18$ per year....
5
votes
2answers
2k views

Word entropy / frequency in human speech

I am wondering how to best approximate the information value of a word $x$ in general human speech. By information value I literally mean its entropy: $H(X) = \mathbb{E}_{X} [I(x)] = -\sum_{x \in \...
5
votes
1answer
110 views

Frequency ratio test with correlation structures

I have some data about two machines, each machine has 10-15 operation logs, and inside the log are the time-stamped status checks that takes on the value of 1 if ...
5
votes
0answers
1k views

How to measure the period of an oscillation with least error?

I have a question about analyzing the data from a coupled pendulum. I have measured the amplitude $\psi(t)$ which is expected to be a beat and I want to measure the period. The ideal plot would be ...
4
votes
2answers
1k views

Am I breaking the assumptions of the Poisson distribution?

The Poisson distribution arises when events are counted within a specified interval. I've recorded the number of events each month (I'll not discuss what these events represent). This appears to meet ...
4
votes
1answer
10k views

Frequency of time series in R

I have 10 minutes-intervaled wait-times data for a coffee shop for 4 weeks between 9am-5pm. I use R's ts for my analysis. What should be the frequency parameter? Is it (48=# of intervals per day) or ...
4
votes
1answer
12k views

Normalizing Term Frequency for document clustering

I have a problem understanding the normalization of Term Frequency weight in document Vector Space Model for clustering. Let's say that for document d I have counted occurences of all terms. I ...
4
votes
1answer
8k views

(Multiple) Correspondence Analysis for count data entered as binary variables

I have a data set, 1014 cases and 55 variables which are binary and is in the form of ...
4
votes
1answer
98 views

Frequency distribution of volumes of spheres is positively skew if the radii are symmetrically distributed

Suppose the frequency distribution of radii of $n$ spheres is symmetric. Show that the frequency distribution of the volumes of these spheres is positively skew. I am trying to show this using Bowley'...
4
votes
4answers
3k views

Econometric Model and deciding the frequency of data collection

I am looking to build an econometric model and I am wondering if using annual data vs monthly or quarterly data is going to produce a less accurate model. If the dependent variable is affected by ...
4
votes
2answers
229 views

Is it possible to obtain more accurate annual extremes predictions from sub-annual data?

I'm looking at various extreme climate variables, such as 50-year or 500-year maximum daily precipitation, using a generalized extreme value (GEV) distribution. The problem with this is that there are ...
4
votes
4answers
365 views

How to estimate a probability of an event to occur based on its count?

I have a generator of random symbols (single act of generation produces exactly one symbol). I know all the symbols that could be generated and for each symbols I would like to estimate the ...
4
votes
1answer
286 views

Frequency/harmonic analysis of accents

I am looking to find out how accents could/would be detected by a programme, I haven’t as of yet undergone an experiment into detecting or observing anything noticeable about recorded accents. I am ...
4
votes
2answers
406 views

Calculate mean from cumulative percentage table

Is it possible to calculate mean from cumulative percentage table below? Scenario: Bob is a manager. He is trying to estimate the sales for the next year, and from his experience, he knows the sales ...
4
votes
1answer
222 views

How to test for a break in a time series cycle

I've been scratching my head over this issue and would appreciate some help. I have a time series from 1920-2011 which I've used a Baxter Kings filter on to detrend. I would like to test whether the ...
4
votes
1answer
846 views

Poisson Rate Regression: Offset? [duplicate]

I'm comparing two groups on the # of hospitalizations they have in a 5 year period. Length of hospitalization stay is highly variable. For example, a patient might have 3 hospitalizations that in ...
4
votes
0answers
111 views

Statistics question related to the Charleston shooting

Note: This will be a long post, and I apologize for that. There's been a lot of parsing of President Obama's words with regards to the "frequency" of shootings in other countries. I contend he's ...
4
votes
2answers
3k views

Does Sampling Frequency matter for Time Series analysis?

I am given two time series of prices between 2009 and 2013. Price series A is weekly data, series B is monthly data. I would like to compare some basic descriptive statistics of these two time series ...
3
votes
6answers
341 views

How do I find the mean of counts of reported events when the last category is “5 or more”

I know that we can find mean (No. of errors per patient) from 0*5+1*30+2*56+3*15+4*10 But for 5 or more, what do I need to do?
3
votes
3answers
7k views

Comparing histograms

The data I have are the ages male or female won an award (79 data points, ages, for each gender). When constructing a freq table I used to find the number of classes, $$ n = 79 $$ $$ \frac{\log(n)}{\...
3
votes
2answers
1k views

Why do small sample sizes give more significant p-values for rare events?

Maybe I'm getting a bit confused, but I have a single binomial variable with very low frequency (e.g. 1 event in 100,000 trials), and I'm trying to figure out what sample size is required to show a ...
3
votes
1answer
231 views

Why is “using counts of a frequency table as data” for a histogram a “mistake”?

The textbook, The Practice of Statistics, states the following as a "common mistake" when making histograms: Don't use counts (in a frequency table) or percents (in a relative frequency table) as ...
3
votes
2answers
72 views

How many counts for good frequency estimate (binomial)?

This is a pretty basic question, I think, but I'm finding it difficult to locate an answer or decent treatment of it on the web. For a binary dataset (binomial) with some unknown parameter p (...
3
votes
3answers
115 views

Distribution of *conditional* frequencies when frequencies follow a Dirichlet distribution

Context: we have a large number of individuals characterized by two binary traits; call these $T$ with values $\{0,1\}$, and $T'$ with values $\{0',1'\}$. So there are four types of individuals: $00'$,...
3
votes
3answers
3k views

How to compare frequency distributions

I have one categorical variable that can assume 5 possible values: 1, 2, 3, 4, 5. I observe the value of the variable in two samples of different size. Now I want to compare the frequency distribution ...
3
votes
2answers
200 views

select words randomly in proportion to their frequencies

I have a large text and I need to select words randomly in proportion to their frequencies. A raw method is to put all the words in an array (N indexes) and to use a random (uniform) generator to ...
3
votes
2answers
162 views

Comparison of frequency rates

Patients with chronic schizophrenia will take some antipsychotic drug during some period of time. The frequency of schizophrenia for each patient will be recorded before and after that period. I ...
3
votes
1answer
996 views

Prevalence conversion to effect size in meta-analysis

I'm used to thinking about meta-analyses in terms of evaluating the impact of an IV on an outcome of interest (ex: relationship between a teacher's level of self-confidence and student outcomes), and ...
3
votes
1answer
139 views

Significance of most frequent answer

I have the following data: I have participants who answered on their favorite out of 6 positions of various information types on a display (e.g. 1 = bottom left, 2 = top left etc.). I can now say for ...
3
votes
1answer
49 views

Categorical Significance Test

I'm working on analyzing a dataset with the following structure: ...
3
votes
1answer
180 views

Calculating expected number of zero-crossings of a white noise process

I am curious about how to calculate the expected number of zero-crossings of a white noise process over some time $t$. In particular I would like to be able to deduce the expected frequency of zero-...
3
votes
1answer
342 views

How to get a Standard Error for a Poisson Variable with Exposure?

I'm looking over some work performed by a consultancy and I'm unsure whether the standard error formula they have used is correct, and the subsequent conclusions they have drawn are erroneous. They ...
3
votes
0answers
24 views

Granger causality tests when series are measured at different frequencies

As a research exercise I should perform Granger causality tests on pairs of national series (one describing output gap and the other an index of consumers' confidence). However, I face a situation ...
3
votes
0answers
60 views

Probability of bad milk

Assume there are (always) $N$ bottles of milk on the grocery shelf. Each day, a deliveryman randomly selects a fixed $M \lt N$ bottles of milk from the shelf and replaces them with new bottles of milk....
3
votes
0answers
60 views

How do I compare frequency data of more than two groups?

I am comparing the frequencies of a disease presenting in particular season. The expected proportion would be 0.25 and the observed frequencies for spring, summer, autumn, and winter were 7, 9, 11, ...
3
votes
0answers
164 views

How to determine significant differences between frequency spectra?

I want to determine whether the mean of frequency spectra are statistically different from each other. I have a set of trials in different drug conditions and I want to test whether a drug had an ...
3
votes
0answers
217 views

Discrete Fourier Transform and uneven sampling

In this blog article an example is given of one can use the DFT to detect frequencies much higher than the sample rate. In the comments sections I asked how it was done, since DFT normally requires ...