# All Questions

140,433 questions
11k views

### Good sources for learning Markov chain Monte Carlo (MCMC)

Any suggestions for a good source to learn MCMC methods?
2k views

### What is the probability of picking p primes from n random numbers?

The probability of that a number between 1 to x is prime is $\frac{1}{\ln{x}}$ as per prime number theorem and also the total number of prime numbers between $1$ to $x$ will be $\frac{x}{\ln{x}}$. But ...
429 views

### How to best estimate the time remaining for a variable-length questionnaire?

The greatest gain of the statistics classes in both school and university seems to be that I now have an inkling of which QA site to use for this question. :) I'm a programmer and I'm making a ...
915 views

### Experiment design for proportion

I would like to test a hypothesis that a database contains data that are correct in 95% or more of the samples. I found this Hypothesis test for a proportion experiment design that looked appropriate....
270 views

### The range of non-identically distributed binomial variables

Lets say we have three independent variables: \eqalign{ X_{1}\sim &B(n,\frac{1}{2}+\beta) \cr X_{2}\sim &B(n,\frac{1}{2}) \cr X_{3}\sim &B(n,\frac{1}{2}-\beta). } I'm looking for ...
168 views

2k views

### Statistical significance (chi squared) for percentages over 100%?

So, working in web analytics, it's common to use a chi-square test for A/B tests to test statistical significance. I often use a calculator like this: http://www.usereffect.com/split-test-calculator , ...
732 views

### Good text for resampling?

Can the group recommend a good introduction text/resource to applied resampling techniques? Specifically, I am interested in alternatives to classical parametric tests (e.g. t tests, ANOVA, ANCOVA) ...
1k views

### What is the connection between Kernel Logistic Regression and Smoothing Splines?

Working on probabilistic outputs of kernel methods I found the formulation of the SVM as a Penalized Method using the Binomial Deviance (described for example in "The Elements of Statistical Learning ...
2k views

### Cluster data points by distance between clusters

I have a load of point data in 3D, and I wish to cluster them like so: Each cluster contains points all of which are at most distance $d$ from another point in the cluster. All points in two ...
1k views

### Annotating graphs in R [closed]

This is more of a "how to use R" question than an actual hardcore statistics question, but I think the concentration of R masters here makes this a good forum for it. I'm refreshing a time series ...
705 views

### First passage time distribution in a irreducible transient discrete-time Markov chain (DTMC)

In a Markov chain, a state $j$ is transient if $f_{jj}<1$ ($f_{jj}$ is probability of ever visiting state $j$ starting from state $j$ ). Suppose, I have an irreducible transient DTMC (means all ...
640 views

### War stories where wrong decisions were made based on statistical information?

I think it is fair to say statistics is an applied science so when averages and standard deviations are calculated it is because someone is looking to make some decisions based on those numbers. ...
2k views

### How do I calculate and compare incidence of infection between age-groups

We work on cervical cancer detection and we started accumulating data in 2006 by performing HPV testing on all our samples. We have accumulated a data set with almost 400,000 women. Data enters the ...
9k views

### Knot selection for cubic regression splines [duplicate]

I was wondering if anybody had experience in how to set the knot points when using cubic regression splines. Some background: I have a response and predictor variable, and I want to determine the ...
4k views

### Kernel density estimate takes values larger than 1 [duplicate]

I am using scipy.stats.gaussian_kde to estimate a pdf for some data. The problem is that the resulting pdf takes values larger than 1. As far as I understand, this ...
609 views

### What is my model statement for mixed models (nlme) in R?

I want to know if a covariate for each subject interacts with three types of trials, and the difficulty of those trials. My dependent measures are accuracy and response times (RT). For this question, ...
314 views

### Help in setting up and solving a transportation / traffic problem [closed]

Traffic light synchronization nowadays is not a tedious project, Image: For a personal research project I'm trying to build a statistical model to solve such problems in two and higher dimensions. ...
194 views

### Demo for bioinformatics

I am a beginner level programmer preparing for the interview in medical research company. Job sounds damn interesting and I would like to get there. To show my skills and interest, I want to write a ...
6k views

### Checking factor / covariate independence in ANCOVA

One of the ANCOVA assumptions, as I read it here, says that: Independent variables orthogonal to covariates. In traditional ANCOVA, the independents area assumed to be orthogonal to the factors. If ...
1k views

### Leave-one-out cross validation and boosted regression trees

Colleagues of mine recently presented a work where they calibrate boosted regression trees (BRT) models on small data sets ($n= 30$). They validated the models using leave-one-out cross validation (...
237 views

### Is it possible to predict the likelihood of an order of random events?

Sorry if this is a n00b question, I'm just trying to wrap my head around the problem. I am trying to refute conventional wisdom here, so any help is greatly appreciated. And now for the question: ...
350 views

### Is there a bias correction for effect size in a data mining context?

Given $K$ possible 'treatments' of some kind, and independent observations of some response under those treatments, say $X_{i,k}$ for $i=1,\ldots,n_k$ and $k=1,\ldots,K$, I am faced with the classical ...
6k views

### R-square from rpart model

How can I extract the R-square from a fit rpart model? rsq.rpart(fit) plots the two graphs, but I simply want to extract the R-square value for the full tree. ...
2k views

### Alternatives to using Coefficient of Variation to summarize a set of parameter distributions?

Background I have a model with 17 parameters, and I currently use the coefficient of variation ($\text{CV}=\sigma/\mu$) to summarize the prior and posterior distributions of each parameter. All of ...
3k views

### How to tell if the slope of a line is 0 or there is just no relationship?

I am attempting to examine the change in slope between a predictor and response over two years. In year 1, it is definitely positive. (Linear regression, the 95% CI of the slope doesn't overlap 0). ...
2k views

### Which kernel method gives the best probability outputs?

Recently I have used Platt's scaling of SVM-outputs to estimate probabilities of default-events. More direct alternatives seem to be "Kernel logistic Regression" (KLR) and the related "Import Vector ...
32k views

### Variance of a function of one random variable

Lets say we have random variable $X$ with known variance and mean. The question is: what is the variance of $f(X)$ for some given function f. The only general method that I'm aware of is the delta ...
601 views

### Odds of winning 2 trials in a row

There are 8 people playing poker. So, the odds of winning the entire round = 1/8 2 rounds are played, and Bill wins both rounds. What are the odds this was random? (Hypothesis test?) NullH = Bill ...
134k views

### Can principal component analysis be applied to datasets containing a mix of continuous and categorical variables?

I have a dataset that has both continuous and categorical data. I am analyzing by using PCA and am wondering if it is fine to include the categorical variables as a part of the analysis. My ...
836 views

### Multiple comparisons on nested subsets of data

Suppose I am doing some experimental procedure on two treatment groups. The procedure has several stages, each of which may fail. Failure at any stage halts the experiment. If all stages are passed ...
1k views

### Can I test the effect of Block in a split plot design in R?

I have been confused with the problem for a very long time, and hope that somebody here can help me out. I have a experiment installed in split-plot design, with 3 temporal groups as blocks, and 2 ...
1k views

### Measure of closeness

Given a list of numbers, is it possible to find out (or in other words, is there a statistical measure to tells the) the closeness of the numbers (do note that i am not talking about correlation - ...
483 views

### What logistic regression is best to use?

My project is about whether early language scores (on a reading task) can predict later scores on a social attribution task. I have to conduct a logistic regression for my project but I am stuck on ...
2k views

### When is a deviation statistically significant?

In a double-blind study, when are deviations from the control group considered statistically significant? And is this related to the number of samples? I realise that every experiment is different ...
20k views

### Look and you shall find (a correlation)

I have several hundred measurements. Now, I am considering utilizing some kind of software to correlate every measure with every measure. This means that there are thousands of correlations. Among ...
159 views

### When is it better to average observations at the same abscissa?

If you want to regress y on x, where multiple y's are observed at each x, is it ever better to instead take the mean at each x, and the use those means for the regression? Does it depend on the ...
22k views

### If A and B are correlated with C, why are A and B not necessarily correlated?

I know empirically that is the case. I have just developed models that run into this conundrum. I also suspect it is not necessarily a yes/no answer. I mean by that if both A and B are correlated ...
636 views

### Optimal stopping from an unknown distribution

The Secretary problem has an algorithm for fixed N and immediate accept/reject (that is, reject reject ... accept one, stop). There are several variants; in mine, secretaries or samples come from a ...
258 views

### How can I calculate the probability of model $g_i$ given a set of $n$ models and AIC values?

Background I have a data set $Y$: ...
5k views

### Is there any library like LOESS or ARIMA in Java/C# or Python?

I try to implement a time series data analysis project, but I have to do in Java, C# or Python, is there any good libary such like LOESS, ARIMA in R you can recommend? Many thanks
4k views

### Generate random multivariate values from empirical data

I'm working on a Monte Carlo function for valuing several assets with partially correlated returns. Currently, I just generate a covariance matrix and feed to the the ...
868 views

### Why “supremum”, not “maximum” in Kolmogorov-Smirnov test?

Why the test function in Kolmogorov-Smirnov test takes the supremum of the set of differences, not maximum? When maximum (greatest element) of a set does not exist then supremum may exist, but if ...
9k views

### Reducing false positive rate

I am running a model for a problem in insurance domain. The final results show some false positive x and some false negative y. I am using SAS Enterprise Miner for this. Can somebody suggest me how to ...