All Questions

3
votes
1answer
744 views

Elasticities Using GLM

The coefficient on a logged explanatory variable when the dependent variable also is in log form is an elasticity (or the percentage change in the dependent variable if the explanatory variable ...
0
votes
1answer
142 views

What statistical model is used to calculate the test results for GWO?

Google Website Optimizer (GWO) is a tool provided by Google to do A/B and MVT experiments on websites. This has been an unanswered question for a long time so I thought I'd ask it here and see if I ...
1
vote
1answer
501 views

Regression output and Fisher-Snedecor distribution

I'm working on regression models in STATISTICA application and I need to know what is Fisher-Snedecor distribution for and how to analyze my regression model in this distribution. What the ...
15
votes
5answers
4k views

Comparing the variance of paired observations

I have $N$ paired observations ($X_i$, $Y_i$) drawn from a common unknown distribution, which has finite first and second moments, and is symmetric around the mean. Let $\sigma_X$ the standard ...
0
votes
2answers
3k views

R lrm model with no predictors

I am creating multiple logistic regression models using lrm from Harrell's Design package in R. One model I would like to make is the model with no predictors. For example, I want to predict a ...
6
votes
3answers
10k views

What is AIC? Looking for a formal but intuitive answer

I've heard that AIC can be used to choose among several models (which regressor to use). But i would like to understand formally what it is in a kind of "advanced undergraduated" level,which i think ...
27
votes
5answers
8k views

Any suggestions for making R code use multiple processors?

I have R-scripts for reading large amounts of csv data from different files and then perform machine learning tasks such as svm for classification. Are there any libraries for making use of multiple ...
0
votes
2answers
137 views

How can I link item by relevance?

I am building a web application for used book trading and I am adding a feature to propose other book that would be interesting when they view an offer. Currently the data that I store are the ...
9
votes
2answers
2k views

What is the difference between functional data analysis and high dimensional data analysis

There are a lot of references in the statistic literature to "functional data" (i.e. data that are curves), and in parallel, to "high dimensional data" (i.e. when data are high dimensional vectors). ...
5
votes
3answers
6k views

When should normalization never be used?

Lately, there have been numerous questions about normalization What are some of the situations where you never ever ever should normalize your data, and what are the alternatives?
2
votes
2answers
936 views

How do I calculate the SE for PPV, NPV, and DOR?

I am attempting to calculate the standard error (SE) for the positive predictive value (PPV), negative predictive value (NPV), and diagnostic odds ratio (DOR) that I have obtained using the rates of ...
74
votes
8answers
144k views

Calculating optimal number of bins in a histogram

I'm interested in finding as optimal of a method as I can for determining how many bins I should use in a histogram. My data should range from 30 to 350 objects at most, and in particular I'm trying ...
-3
votes
2answers
1k views

Normalization of series [closed]

I have data compiled by someone else where score averages have been computed over time- averages range from 0-100. The original scores have negative values in many cases and the average would have ...
4
votes
2answers
4k views

Why (or when) to use the log-mean?

I am looking at a scientific paper in which a single measurement is calculated using a logarithmic mean 'triplicate spots were combined to produce one signal by taking the logarithmic mean of ...
9
votes
7answers
2k views

Normal distribution and monotonic transformations

I've heard that a lot of quantities that occur in nature are normally distributed. This is typically justified using the central limit theorem, which says that when you average a large number of iid ...
9
votes
4answers
5k views

Operations research versus statistical analysis?

What is the difference between operations research and statistical analysis?
6
votes
5answers
323 views

Is it possible to use machine learning as a method for learning stats, rather than vice-versa?

During every machine learning tutorial you'll find, there is the common "You will need to know x amount of stats before starting this tutorial". As such, using your knowledge of stats, you will learn ...
2
votes
2answers
298 views

What types of data analysis do not count as statistics?

When does data analysis cease to be statistics ? Are the following examples all applications of statistics ?: computer vision, face recognition, compressed sensing, lossy data compression, signal ...
13
votes
3answers
5k views

Linear Mixed Effects Models

I have commonly heard that LME models are more sound in the analysis of accuracy data (i.e., in psychology experiments), in that they can work with binomial and other non-normal distributions that ...
4
votes
2answers
785 views

Use of Bayesian Search Theory in geological interpretation

I was having a look round a few things yesturday and came across Bayesian Search Theory. Thinking about this theory led me to think about a problem I was working on a few years ago regarding ...
238
votes
148answers
126k views

Famous statistical quotations

What is your favorite statistical quote? This is community wiki, so please one quote per answer.
6
votes
4answers
1k views

Suggested R packages for frontier estimation or segmentation of hyperspectral images

An hyperspectral image is a multidimensional image with more than 200 spectral bands i.e. an image for which each pixel is a vector of dimension 200 (most often it is a sampled spectral curve that is ...
10
votes
3answers
5k views

How can I test whether my clustering of binary data is significant

I'm doing shopping cart analyses my dataset is set of transaction vectors, with the items the products being bought. When applying k-means on the transactions, I will always get some result. A random ...
2
votes
1answer
2k views

Dataset for multi class perceptron

I am developing a multi-class perceptron algorithm and was wondering if there are any datasets that could be used to test a multi-class perceptron? - A dataset where the classes are linearly separable ...
13
votes
4answers
5k views

Data anonymization software

Is anyone aware of good data anonymization software? Or perhaps a package for R that does data anonymization? Obviously not expecting uncrackable anonymization - just want to make it difficult.
9
votes
2answers
367 views

How do I determine if a survival model with missing data is appropriate?

Oversimplifying a bit, I have about a million records that record the entry time and exit time of people in a system spanning about ten years. Every record has an entry time, but not every record has ...
-5
votes
1answer
387 views

Lies, Damn Lies and Statistics [closed]

Is there something about statistics that lends itself to this sort of saying, or is it just that people will say anything to support their case, and this includes citing irrelevant or incomplete ...
34
votes
8answers
13k views

What is Bayes' theorem all about?

What are the main ideas, that is, concepts related to Bayes' theorem? I am not asking for any derivations of complex mathematical notation.
106
votes
21answers
68k views

What's the difference between probability and statistics?

What's the difference between probability and statistics, and why are they studied together?
8
votes
2answers
1k views

Automating statistical correlation between “texts” and “data”

I am collecting textual data surrounding press releases, blog posts, reviews, etc of certain companies' products and performance. Specifically, I am looking to see if there are correlations between ...
17
votes
12answers
32k views

Best books for an introduction to statistical data analysis?

I bought this book: How to Measure Anything: Finding the Value of Intangibles in Business and Head First Data Analysis: A Learner's Guide to Big Numbers, Statistics, and Good Decisions What ...
12
votes
4answers
5k views

Best ways to aggregate and analyze data

Having just recently started teaching myself Machine Learning and Data Analysis I'm finding myself hitting a brick wall on the need for creating and querying large sets of data. I would like to take ...
30
votes
7answers
3k views

How do you convey the beauty of the Central Limit Theorem to a non-statistician?

My father is a math enthusiast, but not interested in statistics much. It would be neat to try to illustrate some of the wonderful bits of statistics, and the CLT is a prime candidate. How would you ...
5
votes
2answers
6k views

Where is a good place to find survey results?

Sites like eMarketer offer general survey results about internet usage. Who else has a big set of survey results, or regularly releases them? Preferably marketing research focused. Thanks!
9
votes
4answers
2k views

Calculating ratio of sample data used for model fitting/training and validation

Provided a sample size "N" that I plan on using to forecast data. What are some of the ways to subdivide the data so that I use some of it to establish a model, and the remainder data to validate the ...
4
votes
4answers
1k views

Incorporating boolean data into analysis

I have a data set of about 3,000 field observations. The data collected is divided into 20 variables (real numbers), 30 boolean variables, and 10 or so look up variables and one "answer" variable ...
51
votes
4answers
24k views

Standard deviation of standard deviation

What is an estimator of standard deviation of standard deviation if normality of data can be assumed?
3
votes
1answer
392 views

Forecasting handbooks

In engineering, we usually have Handbooks that pretty much dictate the state of the practice. These books are usually devoid of theory and focus on the applied methodology. Is there a forecasting ...
55
votes
2answers
21k views

What is the difference between a partial likelihood, profile likelihood and marginal likelihood?

I see these terms being used and I keep getting them mixed up. Is there a simple explanation of the differences between them?
37
votes
11answers
8k views

Open Source statistical textbooks?

There have been a few questions about statistical textbooks, such as the question Free statistical textbooks. However, I am looking for textbooks that are Open Source, for example, having an Creative ...
59
votes
8answers
32k views

Is PCA followed by a rotation (such as varimax) still PCA?

I have tried to reproduce some research (using PCA) from SPSS in R. In my experience, principal() function from package psych ...
26
votes
1answer
2k views

How can one empirically demonstrate in R which cross-validation methods the AIC and BIC are equivalent to?

In a question elsewhere on this site, several answers mentioned that the AIC is equivalent to leave-one-out (LOO) cross-validation and that the BIC is equivalent to K-fold cross validation. Is there ...
7
votes
2answers
632 views

Why prediction of a predicted variable from a discriminant analysis is imperfect

I am puzzled by something I found using Linear Discriminant Analysis. Here is the problem - I first ran the Discriminant analysis using 20 or so independent variables to predict 5 segments. Among the ...
4
votes
2answers
301 views

Heuristics for optimizing ν-SVM?

Do you know any good heuristics for finding optimal value of ν in case of ν-SVM classification? In this particular problem I have a radial basis kernel, if it helps.
2
votes
2answers
685 views

Something like E-M for discriminative models?

E-M provides a way to improve the estimation of a generative model with unannotated data. Is there anything out there that works the same way for discriminative models (e.g. perceptrons)? For example,...
16
votes
1answer
554 views

E-M, is there an intuitive explanation?

The E-M procedure appears, to the uninitiated, as more or less black magic. Estimate parameters of an HMM (for example) using supervised data. Then decode untagged data, using forward-backward to '...
3
votes
1answer
308 views

To what extent can we call a Geometric Distribution a Geometric Density

In some papers, for example in "The Geometric Density with Unknown Location Parameter" by Klotz, a Geometric Distribution is called a Geometric Density. For me, this claim looks erroneous, however ...
17
votes
2answers
18k views

What are the differences between the Baum-Welch algorithm and Viterbi training?

I am currently using Viterbi training for an image segmentation problem. I wanted to know what the advantages/disadvantages are of using the Baum-Welch algorithm instead of Viterbi training.
209
votes
10answers
117k views

Is there any reason to prefer the AIC or BIC over the other?

The AIC and BIC are both methods of assessing model fit penalized for the number of estimated parameters. As I understand it, BIC penalizes models more for free parameters than does AIC. Beyond a ...
20
votes
4answers
18k views

Post-hocs for within subjects tests?

What is the preferred method for for conducting post-hocs for within subjects tests? I've seen published work where Tukey's HSD is employed but a review of Keppel and Maxwell & Delaney suggests ...

15 30 50 per page