1
vote
1answer
183 views

Continuous-states hidden Markov chain

How to deal with HMC that has continuous states? Any papers, links, materials that explain the solution?
8
votes
2answers
877 views

Math behind multivariate testing for website optimization

I am looking for theoretical resources (books, tutorials, etc.) to learn about making sound statistical inferences given (plenty of) multivariate website conversion data. I'm after the math involved, ...
8
votes
2answers
291 views

Light bulb color problem

Please first have a look at the following little problem: There are two indistinguishable light bulbs A and B. A flashes red light with prob .8 and blue with prob .2; B red with .2 and blue .8. ...
8
votes
3answers
2k views

How do I handle predictor variables from different distributions in logistic regression?

I am using logistic regression to predict y given x1 and x2: z = B0 + B1 * x1 + B2 * x2 y = e^z / (e^z + 1) How is logistic regression supposed to handle cases ...
2
votes
3answers
2k views

Correlation versus cause-effect regression

I know correlation does not imply causation. I have read it nth time. (i.e. weight does not cause height etc. etc.) However, to find the effect of a moderator variable on X-Y relationship, a ...
2
votes
2answers
1k views

How to derive a distance function based on multiple variables for cluster analysis?

I am not a statistician, so please excuse my lack of statistics knowledge/terminology. I have bunch of network nodes that I want to run cluster analysis on and identify clusters. So as far as I ...
3
votes
1answer
92 views

How to deal with information of variable length?

This is the question arised from my previous question. Basically, I am trying to build a prediction model for movie's rating. So I have to deal with numeric attribute such as ...
2
votes
1answer
367 views

How to collapse ordinal variable values into significant different values?

I have an ordinal variable related to an outcome that is comprised of many levels and I´d like to collapse the number of ordinal values as much as possible. ...
3
votes
0answers
249 views

Pointwise mutual information for tag generation in Twitter

I came across the following link. http://stackoverflow.com/questions/2764116/tag-generation-from-a-small-text-content-such-as-tweets The question is about how do we assign tag to tweets ...
9
votes
6answers
9k views

Where to find a large text corpus? [closed]

I am looking for large (>1000) text corpus to download. Preferably with world news or some kind of reports. I have only found one with patents. Any suggestions? Thanks in advance.
0
votes
1answer
1k views

AR process invertibility and stationarity conditions

I am studying the stationarity and invertibility conditions of AR processes. Concerning the stationarity condition of the simplest form of an AR(1) process (without constant, drift). Let us say that ...
7
votes
1answer
267 views

Combining n-grams

In text mining, if we've computed n-gram counts, for say $n=1\ldots4$, is there a principled way to combine them, other than just concatenating the $tf-idf$ matrices for each one? (equivalent to an ...
0
votes
0answers
1k views

SPSS Cross tabulation error messages when analysing a 3x2 contingency table

I would like some clarification on what I initially thought was relatively simple analysis. A single survey was given to 100 german show jumpers (horses) to complete What we would like to know is ...
15
votes
0answers
689 views

Practical thoughts on explanatory vs predictive modeling [duplicate]

Possible Duplicate: Practical thoughts on explanatory vs. predictive modeling This question has been bugging me for some time, and I was going to write a blog post about it. However, I ...
0
votes
2answers
129 views

Missing daily data for one month

I have a two years of daily data. The data for one month is missing as it was not recorded due to some problems at the company. How should I treat the data with no values for a month.
38
votes
1answer
11k views

Bagging, boosting and stacking in machine learning

What's the similarities and differences beetween this 3 methods: bagging, boosting, stacking? Which is the best one? And why? Can you give me an example for each?
5
votes
2answers
789 views

Understanding MANOVA in case of a single predictor

I'm trying to understand the statistical analysis I saw in a clinical study. They measured performance of 3 groups of subjects with a series of performance measures (A, B, C, ..., N). The objective ...
7
votes
1answer
6k views

How to calculate a confidence interval for Spearman's rank correlation?

Wikipedia has a Fisher transform of the Spearman rank correlation to an approximate z-score. Perhaps that z-score is the difference from null hypothesis (rank correlation 0)? This page has the ...
0
votes
1answer
1k views

Likelihood ratio tests

Is likelihood ratio test ($F$-test) of significance of difference of two linear models the same as chi-square test of difference of $-2\log L$? SAS PROC GLM ...
4
votes
1answer
493 views

Why is semipartial correlation cited so seldom?

In OLS regression, I find the semipartial correlation (a.k.a. part correlation) to be a very useful indicator. When squared, it shows each predictor's unique contribution to explained variance in the ...
0
votes
3answers
4k views

How do I calculate projected figures for the next year based on past performance?

I'm a SQL/C++ developer who recently has been asked to generate a report from our database to predict some future performance based on historical data; the problem is that I don't have much experience ...
8
votes
3answers
6k views

When should one consider using GMM?

One of the things which makes econometrics unique is the use of the Generalized Method of Moments technique. What types of problems make GMM more appropriate than other estimation techniques? What ...
3
votes
1answer
5k views

How to interpret coefficients from a GARCH model?

Recently I have opened a question here to understand the output of a GARCH model. My goal is to understand if the series I'm checking is heteroscedastic or not. I'm using the ...
0
votes
1answer
182 views

How best to represent a web app's usage?

I've been gathering data for the usage of a web app. For each day at the same hour and over several months I have: (A) total number of members number of online members and its % to (A) number of ...
3
votes
0answers
241 views

“Brute force” expected deviance for logistic regression?

A commonly used goodness of fit statistic for logistic regression is the deviance. This is also known as the likelihood ratio chi-square statistic. It is defined as: $$D=\sum_{i=1}^{N}d_i^2$$ ...
6
votes
1answer
793 views

What is the intuition behind the score function?

Wikipedia tells us that the score plays an important role in the Cramér–Rao inequality. It also phrases out the definition: $$V = \frac{\partial}{\partial \theta} \log{L(\theta; X)}$$ However, I ...
2
votes
2answers
244 views

Graphing versus further testing interactions

Following the question here, I have come across different (textbook) recommendations. Some authors suggest that the effect of the moderating variable is best understood in a graphic form. Others ...
4
votes
2answers
683 views

How to deal with identical p-values with the Benjamini-Hochberg method for correcting for multiple testing

I'm testing for mutations in DNA and I'm using the Benjamini-Hochberg method to modify the threshold I'm testing my p-values against. The method is basically to rank the p-values and compare them to a ...
8
votes
4answers
474 views

Why aren't all tests scored via item analysis/response theory?

Is there a statistical reason why item analysis/response theory isn't more widely applied? For instance, if a teacher gives a 25 question multiple choice test and finds that 10 questions were answered ...
0
votes
0answers
94 views

Understanding summations in COV formula for time series

I am looking through Time Series Analysis: With Applications in R (my first exposure to time series) and refreshing summations. I. When given the following rule: COV[$\sum_{i=1}^{m} ...
1
vote
1answer
292 views

Reusing past MCMC samples from a similar distribution

I have two graphical models that have significant overlap in the random variables they are modeling. Is it possible to use samples from one graphical model to avoid sampling as much from the other ...
3
votes
2answers
217 views

Draw surface graph

Does any know how to draw a graph like this: Which programs can do that? I'm a developer so I've no problem on formatting a complex files so that a software can handle it.
16
votes
1answer
2k views

Is cross validation a proper substitute for validation set?

In text classification, I have a training set with about 800 samples, and a test set with about 150 samples. The test set has never been used, and waiting to be used until the end. I am using the ...
4
votes
1answer
828 views

What are subjective interestingness measures?

I came across the word subjective interestingness measures in my book, where author says that: Subjective interestingness measures are based on user belief in the data. These measures find ...
4
votes
1answer
383 views

Issues with the Huynh-Feldt values using either Anova or ezANOVA in R?

Is there an issue with the output of the Huynh-Feldt correction when running an ANOVA for both between and within factors (mixed design) in R? My issue is that when I run a 'within-subject (only)' ...
2
votes
0answers
85 views

H-L Test: Looking at the data

Can you just eyeball the Hosmer-Lemeshow test (e.g. just look at observed and expected values in each group and see if they agree more or less)? Even if the Hosmer-Lemeshow statistic is ...
4
votes
2answers
332 views

How to test for association between proportions with multiple variables?

I have tested 100 bacterial isolates for resistance to 6 antibiotics (tetracycline, erythromycin, kanamycin, streptomycin, gentamicin, chloramphenicol). The result for any antibiotic could be either ...
5
votes
2answers
5k views

“Moderation” versus “interaction”?

I have come across these two terms which are used interchangeably in many contexts. Basically, a moderator (M) is a factor that impacts on the relationship between X and Y. Moderation analysis is ...
1
vote
0answers
89 views

Is there any stochastic method to approximate the likelihood?

I am looking for a list of stochastic algorithms to approximate likelihoods.
34
votes
2answers
36k views

When (and why) to take the log of a distribution (of numbers)?

Say I have some historical data e.g., past stock prices, airline ticket price fluctuations, past financial data of the company... Now someone (or some formula) comes along and says "let's take/use ...
2
votes
2answers
625 views

How to obtain the inverse of the F cumulative distribution based on the F cumulative distribution?

I am currently implementing a application that needs to obtain the inverse of the F (Fisher-Snedecor) cumulative distribution. I already have a library that contains the F distribution and I can ...
0
votes
0answers
184 views

Describe the distribution - too few data points

I'm struggling slightly with part of a homework problem. We are asked to "describe the distribution of weights, at the school level" There are n=220 students in total There are p=38 schools I have ...
4
votes
0answers
423 views

How to predict a binary outcome with unbalanced repeated measures data?

I’m somewhat lost as to how to approach a problem I have and was hoping someone can suggest the most appropriate method. The problem: I have a data set consisting of concentration values for 8 ...
0
votes
0answers
93 views

What is the average expectation value of two specific objects in a queue?

I need some hand to solve this problem. Suppose a queue can be filled by three type objects (M, F and A). Depending on the number of each objects the line length can be different. We are looking ...
9
votes
3answers
5k views

Where to find raw data about clinical trials?

I wish to use raw data about clinical trials for the end year examination of my master students. These data could deal with any kind of molecule as long as the trials were complete (Phase 1 to 4). Do ...
2
votes
1answer
217 views

Scoring new observations after cross-validation

I have some doubts about cross validation and scoring a new set of observations. Let's say I want to predict $y=b_0 + b_1x_1$ and have built a 10-fold cross-validation data set, run a regression ...
6
votes
1answer
720 views

Incremental IDF (Inverse Document Frequency)

In a text mining application, one simple approach is to use the $tf-idf$ heuristic to create vectors as compact sparse representations of the documents. This is fine for the batch setting, where the ...
5
votes
4answers
1k views

How to treat holidays when working with time series data?

I am analysing hourly demand data for electricity. To make my forecasts more accurate, how should I treat the national holidays in the data? In particular, how should I treat them in R?
6
votes
2answers
4k views

Increasing number of features results in accuracy drop but prec/recall increase

I am new to Machine Learning. At the moment I am using a Naive Bayes (NB) classifier to classify small texts in 3 classes as positive, negative or neutral, using NLTK and python. After conducting ...
5
votes
1answer
666 views

Presenting marginal effects of logit with fixed effects

I have a traditional logit model with a dichotomous dependent variable and several independent variables. One of the IVs is categorical and in my R code, I treat it as a factor. In the past, I've ...

15 30 50 per page