Questions tagged [frequency]
The frequency measures the number of times an event occurs in a data set.
353
questions
64
votes
13
answers
191k
views
Mean absolute deviation vs. standard deviation
In the text book "New Comprehensive Mathematics for O Level" by Greer (1983), I see averaged deviation calculated like this:
Sum up absolute differences between single values and the mean. Then
...
62
votes
7
answers
39k
views
Period detection of a generic time series
This post is the continuation of another post related to a generic method for outlier detection in time series.
Basically, at this point I'm interested in a robust way to discover the periodicity/...
33
votes
5
answers
4k
views
Why should the frequency of heads in a coin toss converge to anything at all?
Suppose we have any kind of coin. Why should the relative frequency of getting a heads converge to any value at all?
One answer is that this is simply what we empirically observe this to be the case, ...
18
votes
3
answers
10k
views
How to measure the statistical "distance" between two frequency distributions?
I am undertaking a data analysis project which involves investigating website usage times over the course of the year. What I would like to do is compare how "consistent" the usage patterns ...
9
votes
4
answers
2k
views
How do I quantify the uniformity of sampling time?
My datasets cover a long time, say 20 years, and the data are sampled at different time. For example, some are obtaiend at 1:00(stored as 100), some are 23:00(2300). If I plot the histogram of the ...
8
votes
2
answers
174
views
Best plot in R for count data with a broad range and lots of low frequency data
I have a data set that looks like this, in reality there are about twenty categories with a count of 1:
...
8
votes
2
answers
4k
views
What's a good approach to estimate the probability of word frequencies?
I have a document corpus and I want to estimate the probability of occurrence of a certain word $w$. Simply calculating the frequencies and use such a number as an estimation is not a good choice. Is ...
8
votes
2
answers
2k
views
Why does multiplication in the frequency domain equal convolution in the time domain?
This question came in the context of understanding how to get a distribution of a sum of two iid random variables. I'm working through the top answer to this question Consider the sum of $n$ uniform ...
8
votes
3
answers
5k
views
How to figure out what numbers often appear together in a dataset?
I have a lottery style dataset we produce internally (example below). I am trying to figure out which numbers appear most frequently together. Example questions:
What are the top 10 pair of numbers ...
8
votes
2
answers
1k
views
Frequency jump detection
It is generally known that 'jumps' in frequency data are difficult to estimate. In the current literature, many different techniques for estimating such jumps have been tested and often with ...
7
votes
1
answer
1k
views
Visualization of binned frequency distribution in R
I am trying to visually represent the distribution of last names in the US. Specifically, I am trying to show that the distribution is such that the most common names (say the top 50) are very common ...
7
votes
1
answer
163
views
Most significantly frequent category
I am measuring the frequency of consonants in a sample lexicon of a language.
The results are as follows:
...
7
votes
1
answer
1k
views
chi-squared to test if two variables have the same frequency distribution
I want to test if x and y have the same frequency distributions using chi-squared. In my code below, I've concluded that because ...
6
votes
2
answers
1k
views
How to estimate the Poisson distribution with one event occurrence?
Let us say we observed one event on 1 Jan 2005, and we were observing the events since 2000. What's the best estimate of Poisson distribution?
One approach is to set intensity $\lambda=1/18$ per year....
6
votes
2
answers
106
views
Statistical test to assess significant difference in landcover selection
I have GPS collar data that I have used to extract values to points through ArcGIS Pro on MODIS landcover data. From this, I have a record of the number of times the animal has been in each landcover ...
6
votes
2
answers
2k
views
Word entropy / frequency in human speech
I am wondering how to best approximate the information value of a word $x$ in general human speech. By information value I literally mean its entropy:
$H(X) = \mathbb{E}_{X} [I(x)] = -\sum_{x \in \...
6
votes
2
answers
4k
views
Statistical comparison of two signals
I need to develop an algorithm that will compare two signals and generate some metric(s) to describe changes between them. Signal processing and analysis isn’t my strong point so I would appreciate ...
5
votes
1
answer
12k
views
Frequency of time series in R
I have 10 minutes-intervaled wait-times data for a coffee shop for 4 weeks between 9am-5pm. I use R's ts for my analysis. What should be the frequency parameter?
Is it (48=# of intervals per day) or ...
5
votes
1
answer
10k
views
(Multiple) Correspondence Analysis for count data entered as binary variables
I have a data set, 1014 cases and 55 variables which are binary and is in the form of
...
5
votes
1
answer
19k
views
Normalizing Term Frequency for document clustering
I have a problem understanding the normalization of Term Frequency weight in document Vector Space Model for clustering. Let's say that for document d I have counted occurences of all terms. I ...
5
votes
3
answers
191
views
Distribution of *conditional* frequencies when frequencies follow a Dirichlet distribution
Context: we have a large number of individuals characterized by two binary traits; call these $T$ with values $\{0,1\}$, and $T'$ with values $\{0',1'\}$. So there are four types of individuals: $00'$,...
5
votes
4
answers
2k
views
How to estimate a probability of an event to occur based on its count?
I have a generator of random symbols (single act of generation produces exactly one symbol). I know all the symbols that could be generated and for each symbols I would like to estimate the ...
5
votes
4
answers
5k
views
Econometric Model and deciding the frequency of data collection
I am looking to build an econometric model and I am wondering if using annual data vs monthly or quarterly data is going to produce a less accurate model. If the dependent variable is affected by ...
5
votes
1
answer
336
views
Frequency/harmonic analysis of accents
I am looking to find out how accents could/would be detected by a programme, I haven’t as of yet undergone an experiment into detecting or observing anything noticeable about recorded accents. I am ...
5
votes
1
answer
2k
views
Calculating confidence intervals for each class in a frequency distribution for a binomial distribution
This is a question from the book "Statistical Methods (6e) by Snedecor, Cochran":
10 balls, numbered 0 to 9, are placed in a box. For our sample, a ball is drawn from the box and then ...
5
votes
2
answers
258
views
Is it possible to obtain more accurate annual extremes predictions from sub-annual data?
I'm looking at various extreme climate variables, such as 50-year or 500-year maximum daily precipitation, using a generalized extreme value (GEV) distribution. The problem with this is that there are ...
5
votes
1
answer
117
views
Frequency ratio test with correlation structures
I have some data about two machines, each machine has 10-15 operation logs, and inside the log are the time-stamped status checks that takes on the value of 1 if ...
5
votes
0
answers
2k
views
How to measure the period of an oscillation with least error?
I have a question about analyzing the data from a coupled pendulum.
I have measured the amplitude $\psi(t)$ which is expected to be a beat and I want to measure the period. The ideal plot would be ...
5
votes
2
answers
6k
views
Does Sampling Frequency matter for Time Series analysis?
I am given two time series of prices between 2009 and 2013. Price series A is weekly data, series B is monthly data. I would like to compare some basic descriptive statistics of these two time series ...
4
votes
3
answers
10k
views
Comparing histograms
The data I have are the ages male or female won an award (79 data points, ages, for each gender). When constructing a freq table I used to find the number of classes,
$$ n = 79 $$
$$ \frac{\log(n)}{\...
4
votes
1
answer
155
views
Can I use logistic regression if one category of the dependent variable has a very low frequency?
An example of 100 subjects. Let's say I wish to study the impact of literacy levels (0 as limited and 1 as adequate), anxiety levels (0 as no anxiety and 1 as severe anxiety) and sex (0 for female and ...
4
votes
2
answers
2k
views
Am I breaking the assumptions of the Poisson distribution?
The Poisson distribution arises when events are counted within a specified interval.
I've recorded the number of events each month (I'll not discuss what these events represent). This appears to meet ...
4
votes
2
answers
2k
views
Why do small sample sizes give more significant p-values for rare events?
Maybe I'm getting a bit confused, but I have a single binomial variable with very low frequency (e.g. 1 event in 100,000 trials), and I'm trying to figure out what sample size is required to show a ...
4
votes
1
answer
2k
views
What time-series frequency should I use? Weekly, monthly, quarterly
First, I read the Q&As here, but it's not entirely clear for me what to do given my situation.
I have more than 1700 different time series, each expressed as weekly data. My objective is to ...
4
votes
2
answers
1k
views
select words randomly in proportion to their frequencies
I have a large text and I need to select words randomly in proportion to their frequencies.
A raw method is to put all the words in an array (N indexes) and to use a random (uniform) generator to ...
4
votes
1
answer
238
views
Frequency distribution of volumes of spheres is positively skew if the radii are symmetrically distributed
Suppose the frequency distribution of radii of $n$ spheres is symmetric. Show that the frequency distribution of the volumes of these spheres is positively skew.
I am trying to show this using Bowley'...
4
votes
2
answers
136
views
Comparing frequencies
I'm trying to find the right test to use to compare 3 frequencies.
Basically, in my research, 3 separate user interface prototypes were given to the same sample (10 people) and particular tasks needed ...
4
votes
3
answers
4k
views
How to compare frequency distributions
I have one categorical variable that can assume 5 possible values: 1, 2, 3, 4, 5. I observe the value of the variable in two samples of different size. Now I want to compare the frequency distribution ...
4
votes
2
answers
183
views
Comparison of frequency rates
Patients with chronic schizophrenia will take some antipsychotic drug during some period of time. The frequency of schizophrenia for each patient will be recorded before and after that period. I ...
4
votes
2
answers
1k
views
Calculate mean from cumulative percentage table
Is it possible to calculate mean from cumulative percentage table below?
Scenario: Bob is a manager. He is trying to estimate the sales for the next year, and from his experience, he knows the sales ...
4
votes
1
answer
236
views
How to test for a break in a time series cycle
I've been scratching my head over this issue and would appreciate some help. I have a time series from 1920-2011 which I've used a Baxter Kings filter on to detrend. I would like to test whether the ...
4
votes
1
answer
1k
views
Poisson Rate Regression: Offset? [duplicate]
I'm comparing two groups on the # of hospitalizations they have in a 5 year period. Length of hospitalization stay is highly variable. For example, a patient might have 3 hospitalizations that in ...
4
votes
0
answers
201
views
Fisher Information for frequency estimation under non-circular complex Gaussian noise
Consider a non-circular complex Gaussian noise $v$, which is given as $v = v_r + iv_i$. Here, real and imaginary components are independent and are Normal random variables such as $v_r \sim \mathcal{N}...
4
votes
0
answers
255
views
How to determine significant differences between frequency spectra?
I want to determine whether the mean of frequency spectra are statistically different from each other. I have a set of trials in different drug conditions and I want to test whether a drug had an ...
3
votes
6
answers
415
views
How do I find the mean of counts of reported events when the last category is "5 or more"
I know that we can find mean (No. of errors per patient) from 0*5+1*30+2*56+3*15+4*10
But for 5 or more, what do I need to do?
3
votes
3
answers
586
views
Doubt around Covariance definition
Given the general definition of Covariance between two random variables $x$ and $y$:
\begin{equation*}
\text{Cov}(x,y)=\frac{1}{n}\sum_{i=1}^n(x_i-\bar{x})(y_i-\bar{y})
\end{equation*}
Does the above ...
3
votes
2
answers
353
views
Goodness-of-fit test for very skewed data [closed]
I am working with two quite large datasets (9000 obs and 3800 obs). Each observation has been grouped into 1 of 12 categories and I am looking to test if the frequency of categories in the smaller ...
3
votes
3
answers
4k
views
How to compare two samples of frequencies with categorical x values where one is subset of the other
I am trying to compare two arrays of frequencies. The second is a subset of the first. I want to know if the second is a representative of the first. The arrays are:
...
3
votes
1
answer
1k
views
Variable frequency redistribution
I know that there is a way to "redistribute the frequencies of a variable" as stated here:
Slide number 14 and 15 about redistribution
and here:
Dorian Pyle book chapter 7 section 2 paragraph 3 (7....
3
votes
1
answer
4k
views
Why, in a grouped discrete data starting with zero, does the lower boundary start by -0.5?
My statistics book states that:
Special care must be taken with class boundaries when drawing a histogram to represent grouped discrete data.....a continuous scale is used.
Consider the group 0-9. ...