# All Questions

187,875 questions
Filter by
Sorted by
Tagged with
9k views

### Mathematical Statistics Videos

A question previously sought recommendations for textbooks on mathematical statistics Does anyone know of any good online video lectures on mathematical statistics? The closest that I've found are: ...
5k views

### Is it valid to aggregate a time series to make it look more meaningful?

Another question about time series from me. I have a dataset which gives daily records of violent incidents in a psychiatric hospital over three years. With the help from my previous question I have ...
4k views

### How does random forest generate the random forest

I am not an expert of random forest but I clearly understand that the key issue with random forest is the (random) tree generation. Can you explain me how the trees are generated? (i.e. What is the ...
960 views

### Good line color for "threshold" line in a time-series graph?

We're plotting time-series metrics in the context of network/server operations. The data has a 5-minute sample rate, and consists of things like CPU utilization, error rate, etc. We're adding a ...
61k views

### Always Report Robust (White) Standard Errors?

It has been suggested by Angrist and Pischke that Robust (i.e. robust to heteroskedasticity or unequal variances) Standard Errors are reported as a matter of course rather than testing for it. Two ...
173k views

### What is your favorite "data analysis" cartoon?

Data analysis cartoons can be useful for many reasons: they help communicate; they show that quantitative people have a sense of humor too; they can instigate good teaching moments; and they can help ...
36k views

### What book would you recommend for non-statistician scientists?

What book would you recommend for scientists who are not statisticians? Clear delivery is most appreciated. As well as the explanation of the appropriate techniques and methods for typical tasks: ...
9k views

### Why isn't RANSAC most widely used in statistics?

Coming from the field of computer vision, I've often used the RANSAC (Random Sample Consensus) method for fitting models to data with lots of outliers. However, I've never seen it used by ...
19k views

### Introduction to statistics for mathematicians

What is a good introduction to statistics for a mathematician who is already well-versed in probability? I have two distinct motivations for asking, which may well lead to different suggestions: I'd ...
13k views

### Motivation for Kolmogorov distance between distributions

There are many ways to measure how similar two probability distributions are. Among methods which are popular (in different circles) are: the Kolmogorov distance: the sup-distance between the ...
442 views

### How to approximate measurement uncertainty?

At the moment I use standard deviation of the mean to estimate uncertainty: where N is in hundreds and mean is a time series (monthly) mean. I present it then like this: for each element (month) in ...
6k views

### What best practices should I follow when preparing plots?

I usually make my own idiosyncratic choices when preparing plots. However, I wonder if there are any best practices for generating plots. Note: Rob's comment to an answer to this question is very ...
223 views

### How to tell if something happened in a data set which monitors a value over time

I have a data set where a series of measurements are being taken each week. In general the data set shows a +/- 1mm change each week with a mean measurement staying at about 0mm. In plotting the data ...
479 views

### Testing (and proving) the randomness of numbers [duplicate]

Possible Duplicate: Testing random variate generation algorithms What's a good way to test a series of numbers to see if they're random (or at least psuedo-random)? Is there a good statistical ...
2k views

### What are pivot tables, and how can they be helpful in analyzing data?

What are pivot tables, and how can they be helpful in analyzing data?
6k views

### The Monty Hall Problem - where does our intuition fail us?

From Wikipedia : Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who ...
2k views

### What are the key statistical concepts that relate to data mining?

What topics in statistics are most useful/relevant to data mining?
886 views

### Working through a clustering problem

Say I've got a program that monitors a news feed and as I'm monitoring it I'd like to discover when a bunch of stories come out with a particular keyword in the title. Ideally I want to know when ...
16k views

### What is the single most influential book every statistician should read?

If you could go back in time and tell yourself to read a specific book at the beginning of your career as a statistician, which book would it be?
127k views

### What is the difference between the Shapiro-Wilk test of normality and the Kolmogorov-Smirnov test of normality?

What is the difference between the Shapiro-Wilk test of normality and the Kolmogorov-Smirnov test of normality? When will results from these two methods differ?
1k views

### The trinity of tests in maximum likelihood: what to do when faced with contradicting conclusions?

The Wald, Likelihood Ratio and Lagrange Multiplier tests in the context of maximum likelihood estimation are asymptotically equivalent. However, for small samples, they tend to diverge quite a bit, ...
962 views

### Bias towards natural numbers in the case of least squares

Why do we seek to minimize x^2 instead of minimizing |x|^1.95 or |x|^2.05. Are there reasons ...
28k views

### What is a good algorithm for estimating the median of a huge read-once data set? [duplicate]

I'm looking for a good algorithm (meaning minimal computation, minimal storage requirements) to estimate the median of a data set that is too large to store, such that each value can only be read once ...
18k views

### kNN and unbalanced classes

Do you think that unbalanced classes is a big problem for k-nearest neighbor? If so, do you know any smart way to handle this?
4k views

### Resources for learning about the Statistical Analysis of Financial Data

I realize that the statistical analysis of financial data is a huge topic, but that is exactly why it is necessary for me to ask my question as I try to break into the world of financial analysis. As ...
4k views

### Good introduction into different kinds of entropy

I'm looking for a book or online resource that explains different kinds of entropy such as Sample Entropy and Shannon Entropy and their advantages and disadvantages. Can someone point me in the right ...
4k views

### How does gentle boosting differ from AdaBoost?

There is a variant of boosting called gentleboost. How does gentle boosting differ from the better-known AdaBoost?
150 views

### What approach could be used for modelling virus infections?

I'm a physics graduate who ended up doing infosec so most of the statistics I ever learned is useful for thermodynamics. I'm currently trying to think of a model for working out how many of a ...
34k views

### When conducting a t-test why would one prefer to assume (or test for) equal variances rather than always use a Welch approximation of the df?

It seems like when the assumption of homogeneity of variance is met that the results from a Welch adjusted t-test and a standard t-test are approximately the same. Why not simply always use the Welch ...
425k views

### In linear regression, when is it appropriate to use the log of an independent variable instead of the actual values?

Am I looking for a better behaved distribution for the independent variable in question, or to reduce the effect of outliers, or something else?
2k views

### Resources for learning Stata

I know of Cameron and Trivedi's Microeconometrics Using Stata. What are other good texts for learning Stata?
3k views

### Estimating beta-binomial distribution

Suppose that I culture cancer cells in n different dishes g₁, g₂, … , gn and observe the number of cells ni in each dish that look different than normal. The total number of cells in dish gi is ti. ...
5k views

### What is the difference/relationship between method of moments and GMM?

Can someone explain to me the difference between method of moments and GMM (general method of moments), their relationship, and when should one or the other be used?
86k views

### What is a "saturated" model?

What is meant when we say we have a saturated model?
272 views

### How to deal with the effect of the order of observations in a non hierarchical cluster analysis?

When a non-hierarchical cluster analysis is carried out, the order of observations in the data file determine the clustering results, especially if the data set is small (i.e, 5000 observations). To ...
13k views

### Spatial statistics models: CAR vs SAR

When would one prefer to use a Conditional Autoregressive model over a Simultaneous Autoregressive model when modelling autocorrelated geo-referenced aerial data?
10k views

### How large should a sample be for a given estimation technique and parameters?

Is there a rule-of thumb or even any way at all to tell how large a sample should be in order to estimate a model with a given number of parameters? So, for example, if I want to estimate a least-...
7k views

### Poisson regression with large data: is it wrong to change the unit of measurement?

Due to the factorial in a poisson distribution, it becomes unpractical to estimate poisson models (for example, using maximum likelihood) when the observations are large. So, for example, if I am ...
406k views

### What is the difference between a population and a sample?

What is the difference between a population and a sample? What common variables and statistics are used for each one, and how do those relate to each other?
9k views

### How do I calculate if the degree of overlap between two lists is significant?

If I have two lists A and B, both of which are subsets of a much larger list C, how can I determine if the degree of overlap of A and B is greater than I would expect by chance? Should I just ...
3k views

### What is the best out-of-the-box 2-class classifier for your application? [closed]

Rules: one classifier per answer vote up if you agree downvote/remove duplicates. put your application in the comment
11k views

### What is the easiest way to create publication-quality plots under Linux?

We may assume that we have CSV file and we want a very basic line plot with several lines on one plot and a simple legend.
9k views

### How does boosting work?

What is the easiest way to understand boosting? Why doesn't it boost very weak classifiers "to infinity" (perfection)?
727 views

### Variance components

I have a set of $N$ bodies, which is a random sample from a population whose mean and variance I want to estimate. A property of each body is being measured $m_i$ times ($m_i>1$) and different for ...
884 views

### Using time series analysis to analyze/predict violent behavior

This is a bit of a flippant question, but I have a serious interest in the answer. I work in a psychiatric hospital and I have three years' of data, collected every day across each ward regarding the ...
791 views

### Why is the average of the highest value from 100 draws from a normal distribution different from the 98th percentile of the normal distribution?

Why is the average of the highest value from 100 draws from a normal distribution different from the 98% percentile of the normal distribution? It seems that by definition that they should be the ...
2k views

### Recommended visualization libraries for standalone applications

Which visualization libraries (plots, graphs, ...) would you suggest to use in a standalone application (Linux, .Net, Windows, whatever). Reasonable performance would be nice as well.
361 views

### Intro to statistics for an MD?

I have a friend who is an MD and wants to refresh his Statistics. So is there any recommended resource online (or offline) ? He did stats ~20 years ago.
If $X_1, ..., X_n$ are independent identically-distributed random variables, what can be said about the distribution of $\min(X_1, ..., X_n)$ in general?