# All Questions

414 views

### Weka J48 decision tree problem

I have a CSV dataset which contains mean (Numeric), spread (Numeric), review (string), ...
56 views

### (Nominal) raters with no gold standard

A friend of mine took a document and broke it up into parts, then asked 5 subject matter experts to classify each part into nominal category A, B, C, D, or E. (I'm not sure yet, but D may be "All of ...
258 views

### Visualize survival analysis with time dependent covariates

As a follow-up to Model suggestion for a Cox regression with time dependent covariates is the Kaplan Meier plot accounting for the time dependent nature of pregnancies. In other words the dataset is ...
108 views

### Is it reasonable to calculate AIC of a subset of the data set which was used to fit the model?

There is a factor variable called "Treatment" in my data set. This factor consists of two levels, "C" and "H". I want to test whether there is there any significant difference between two levels. I ...
95 views

### About tail distribution of a sum

Do we know anything about the tail distribution of sum of squares of a limited number of i.i.d exponentially distributed random variables? I'm looking for a good bound.
67 views

### Program yourself or use data mining toolkit?

I'm quite comfortable using Python and additional ML/Visualization packages for ML and data mining. How does the free data mining software RapidMinder, KNIME, etc. compare to this approach and does it ...
94 views

### Regarding the sampling procedure in Adaboost algorithm

The AdaBoost algorithm states that it is to train a classifier based on the training data according to a weight vector. Assume the size of training data is N, the weight vector is of dimension N as ...
159 views

### Logistic regression model for analysis of many IVs with a relatively small sample size

I'm trying to determine the influence (direction and relative strength) of certain attributes of incoming students to an academic program on their successful completion of the program. My sample size ...
103 views

### Visualizing probability of event over time, based on epoch time

I have a list of epoch/unix times at which an event happened. My hypothesis is that there are certain times during the week when this event might happen more frequently. How can I visualize/determine ...
44 views

### Inferring speed from non-I.I.D. GPS data

I have a collection of GPS log of many vehicles. Each row has timestamp, location, speed and vehicle number. I'd like to infer to distribution of speed, or at the minimum, the portion of time they are ...
357 views

### How to correct for heteroskedasticity in fixed effects panel regression with correction for clustered standard error?

I am a student at RSM and I have a question regarding my regression analysis for my thesis as I have encountered issues I do not know how to deal with. I have performance data (dependent variable) of ...
55 views

### How to pick from a set of random variables the one with the highest mean using a fixed number measurements?

Suppose we have $N$ approximately normally distributed continuous random variables $X_1, X_2, X_3, \ldots,X_N$, each with an unknown mean and variance. I'd like to find the random variable with the ...
385 views

### Strange pattern in residual plot from mixed effect model

I've run a mixed effect model in R by using lme. The explanatory (Temp_Diff & Distance) and responsive (LF_Diff) factors are continued variables. ...
87 views

### How to perform exploratory factor analysis on associative network?

In an article by Teichert and Schontag ("Exploring Consumer Knowledge Structure Using Associative Network Analysis", 2010), the authors perform (page 387) an exploratory factor analysis (EFA) on an ...
20 views

### Average of ongoing durations

Suppose I run a community center and I want to find the average length of time that someone is a rec center member. It is straightforward to do this with people who have already cancelled their ...
162 views

### Bayesian hypothesis testing and Bayes factors

Supposing a parameter of interest is assumed to follow an exponential distribution with rate $\lambda$. Two competing prior hypotheses are formed such that the first prior is given by ...
137 views

### How do you show explained variation in cforest?

I'm trying to run a cforest model in R with continuous and categorical variables. When I tried this in randomForest, the ...
69 views

### Classifiers with post-training constraints on the prediction space

I'm familiar with using tools like SVMs and decision trees for discrete classification problems. But one detail that I have not encountered in that domain is: what do you do if your classifier must ...
82 views

### Deducing a single variance from a covariance matrix

I use linear discriminant analysis and I want to compare different assumptions about the covariance matrix. I want to explore what happens if I use a covariance matrix proportional to the identity ...
210 views

### Why is correlation between y and $\hat{y}$ in a model with and without intercept equal?

I consider a very simple example with two models like this; ...
80 views

### Can I validly expand the odds ratio analogously to the relationship between relative risk & the incidence rate ratio?

I am involved in a nested Case-Control study that involves cohorts of cases and controls entering a program, with the outcome of failure by six months. Cases and controls are individually matched on ...
307 views

### Is Xorshift RNG good enough for Monte Carlo approaches? If not what alternatives are there?

I recently stumbled across an article on pseudorandom numbers in Java which mention potential weaknesses in the default algorithm, called linear congruential generator (LCG), and gives some ...
148 views

### Computation of log-likelihood in semi-supervised naive bayes

I have the following 2 questions about log-likelihood computation in semi-supervised Naive Bayes. I have read on several documents online that, in every EM iteration of the semi-supervised Naive ...
179 views

### Had statisticians predicted 2008 financial crisis?

Are there any statistical or econometric studies before 2008 that predicted 2008 financial crisis? Note that there are some publications that attemp to predict contagion between markets using copula ...
135 views

### How to predict the probability of a performance rank for a test taker on a future test based on previous test scores?

Four students sit a series of tests, each test contains 10 questions. Students are ranked 1-4 in each test depending upon who got the most questions correct. ...
111 views

### Relationship between LASSO T and LARS number of steps k

We can see on the figure (cf Least Angle Regression p30, Efron, Hastie, Johnstone, Tibshirani - link: Least Angle Regression) that there is a direct relationship between: LASSO T absolute norm of ...
172 views

### Predicting time interval based on statistics

Let's say, we have some random event. We also have a hist of time intervals between two events, based on statistical data. For example, a frequency distribution: ...
108 views

### Spatial interpolation of vectors in vector fields

Statistical modeling is new to me and I would appreciate some thoughts on my project. I am trying to model the spatial (and possibly temporal as well) relationships within vectors in vector fields. ...
66 views

### Regressing on many outcomes in a spatial model

I'm measuring many microbe populations in a 'river model' ecosystem by sampling at two points. I want to test the hypothesis that key species upstream are the best predictors for any given population ...
53 views

### Comparing performance after CV over a single dataset : does it violate the independence assumption?

Cross validation lets us get multiple estimates of the generalization performance of an algorithm when the performance cannot be estimated for each example independently. (As with an F1-score, we ...
417 views

### Why does a finite, irreducible and aperiodic Markov chain with a doubly-stochastic matrix P have a uniform limiting distribution?

The theorem is "If a transition matrix for an irreducible Markov chain with a ﬁnite state space S is doubly stochastic, its (unique) invariant measure is uniform over S." If a Markov Chain has a ...
48 views

### Help with designing the analysis of a radio telemetry study

I was wondering if there are any spatial ecologists out there who can provide some insight on the trouble I'm having with my Masters research. I am looking at the association a bat species to high ...
114 views

### When can I leave counterbalancing factors (between participants) in an ANOVA out of a write up?

The following is a problem that I keep encountering in different forms. For now, I give one example, but I would also be interested in the answers to the more general question. I have a design with a ...
3k views

### The relationship between the gamma distribution and the normal distribution

I recently found it necessary to derive a pdf for the square of a normal random variable with mean 0. For whatever reason, I chose not to normalise the variance beforehand. If I did this correctly ...
90 views

901 views

### What is the correct way to calculate the explained variance of each EOF as calculated from a gappy data set?

I am trying to determine the correct amount of variance explained by each mode of an Empirical Orthogonal Function (EOF) analysis (similar to "PCA") as applied to a gappy data set. (i.e., containing ...
134 views

### Estimating parameters of an unknown PID controller

Say that I have your standard PID controller at work. To keep it extremely simple imagine I have a target $x^*$ on the variable $x$. Then the controller is: \$y(t) = K_p ( x^* - x_t) + K_i \int_0^t ...
161 views

### Specifying the linear predictor, survival function and PDF of a log-normal survival distribution

Continuing on my exploration of the log-normal distribution, I'm working on reimplementing some code originally written for a Weibull/Exponential model for a log-normal model. Among the things it does ...
42 views

### Analysing when tunes are alike: testing ordinal and categorical similarity

I'd like to compare some Jazz tunes. Jazz tunes are characterized by a succession of chords. The interval between each succeeding chord (number of tones) and the order in which they appear matter. ...
154 views

### Derivation of prediction intervals for a normally distributed population with unknown population standard deviation

I have via the ISO standard 16269 found the solution to a problem that I've been working on. Based on a couple of independent samples from a normally distributed population, I would like to determine ...
59 views

### Convergence of MCMC for ill-behaved functions

We are doing MCMC sampling over an n-dimensional space with a population of MCMCs. The goal is to have them stop when they approximately converge. The problem is that the convergence, which is so ...
45 views

### Estimating a time frame for interventions or judging the extent to which events become “determined” as time goes on

In certain arenas, it's valuable to be able to intervene early on to prevent problems from getting worse, because after a certain point there's not much you can do. Two examples might be public health ...
60 views

### How to define and model consumption bundles?

Imagine an a la carte buffet with n different rooms. On entering the buffet you pick a room (let's say American, Mexican or Italian food) where you stay for the duration of your visit. Once in a ...