1
vote
1answer
165 views
Observation symbols for training a set of HMMs
If we are to classify 2 separate classes/actions using HMMs, we design 2 separate HMMs (one for each class). Do they share same set or a different set of observations-symbols for each of the HMM?
If ...
0
votes
1answer
58 views
How to determine if distribution in timeseries has shifted?
Is it possible to determine if distribution of time-series has shifted significantly without assuming any particular underlying data distribution.
Furthermore, would it be possible to a identify ...
2
votes
2answers
63 views
How do you explain 'co-integration' to determine spurious regression to a fairly new time series student?
I would appreciate it if someone could explain the concept of spurious regression intuitively / without it being too technical.
So far the student understands that spurious regression is basically ...
19
votes
8answers
1k views
What is the meaning of “All models are wrong, but some are useful”
"Essentially, all models are wrong, but some are useful."
--- Box, George E. P.; Norman R. Draper (1987). Empirical Model-Building and Response Surfaces, p. 424, Wiley. ISBN 0471810339.
What ...
1
vote
1answer
18 views
comparing AIC (or BIC or whatever) between different SETS of models
Suppose I have $m$ competing models. Suppose also that I could classify these models into $s$ sets. For example, I could classify models of migration behavior conditional on climatic conditions by the ...
2
votes
2answers
39 views
Testing Spearman's rho between-groups
I am analysing ordinal, non-normally distributed dependent measures using Spearman's rho. Specifically, we have an independent groups design, and I have found a significant relationship between the ...
1
vote
2answers
111 views
4
votes
2answers
63 views
Understanding bootstrap method for confidence interval of correlation coefficients
Please correct me where I'm wrong:
My understanding of bootstrapping is that it is a way to estimate the distribution of some statistic (mean, standard error, Pearson's correlation coeff, etc), ...
1
vote
1answer
363 views
compare means of of multiple dependent variables between two groups
I have two independent groups who travel with different modes. If I want to compare the distance traveled (which is a continuous variable) by different modes for these two groups. How do I compare ...
3
votes
1answer
62 views
Interpretation of a Durbin-Watson test?
I have calculated a Durbin-Watson test and got as far as
$$\eqalign{
d&=2.207551844, \\
dL&= 1.6164, \\
dU&= 1.7896.
}$$
I want to test
$$H_0 \gt 0,\ H_1 \le 0.$$
However, I do not ...
0
votes
0answers
51 views
Selection of lme models using AIC & appropriate random effects & variance structure
I am using three categorical predictor variables X1, X2, X3 and one continuous dependent variable Y, and I want to treat X3 as a random effect.
The simplest model I could come with:
...
2
votes
1answer
115 views
+100
Select best set of binary variables for clustering known sample labels
I have a set of samples, for which I know the "true groups". For this samples I have about 200 binary variables, I would like to know a method to select the subset of variables, that gives me a ...
3
votes
2answers
37 views
Use Random Forest model to make predictions from sensor data
say I have a sensor that measures temperature, pressure ++, and want to use this data to predict some quantity "A". If I use multivariate regression, I can simply implement a model of the form ...
2
votes
1answer
51 views
Iterative proportional fitting in R
The mission
I am trying to find a way to do Iterative Proportional Fitting in R. The logic of the procedure is like this: one has a table with e.g. sample distribution of some variables. Let us say ...
0
votes
0answers
42 views
Is there any Python equivalent of R's biglm?
I have used biglm in R and found it very useful. Now I need the same type of functionality in python. Any ideas? I have seen that patsy/statsmodels has an ...
1
vote
0answers
30 views
Maximum likelihood parameter estimates for Kalman filter with time-varying state transition
I'm currently studying Kalman filter, and the model I'm trying to fit is a uni-variate regression:
$Y_t = \beta_t \mathbf{X_t} + v_t$
$\beta_t = \beta_t-1 + w_t$
where $v$ and $w$ are both ...
1
vote
1answer
39 views
How to plot a “differential word cloud” in R?
I have a questionnaire with one "open" text box, and 20,000 responses. I'm not planning to read all of them, but I'm thinking of interesting things to do with them. A general tag cloud would not be ...
0
votes
1answer
36 views
Does the regression line have equicontinuity? [closed]
I was reading this wikipedia article. The article is about equicontinuity. And my first thought was the regression line. THis question is to more help me understand and to check my understanding. ...
0
votes
1answer
26 views
adf.test returning p-value > 0.99 on series bounded by -1 and 1
I have a pair of time series that I differenced and now take on values {-1, -0.5, 0, 0.5, 1}. My goal is to test them for Granger causality and discover any lead/lag relationships. Since this ...
2
votes
1answer
68 views
What does R mean by “unbalanced design”?
I think this means an unequal sample in different conditions. But it seems to mean something else. . .
I have a data set like below
...
2
votes
1answer
126 views
Analyzing age-dependent variable with age-at-death as a predictor. Collinearity problems
Let's say I'm analyzing the incidence of pathological lesions, observed during necropsy. No information is available about when the lesion occurred before death. So, I'm using a general linear model ...
1
vote
1answer
40 views
How to fit a model with lagged variables
I am trying to fit a model with lagged variables.
The problem is: In a big classroom with windows open, the outside temperature, humidity and solar activity will lead to variation in the ...
0
votes
2answers
85 views
Drawing inferences from word counts
I'm trying to figure out if there is a good model that can connect word counts to classifying whether or not a person might go to an event. Here's the description: I have users, and events. 100 ...
0
votes
0answers
35 views
How to deal with a lot of plots in R [migrated]
I have a for loop which produces 60 plots. I would like to save all this plots in only one file.
If I set par(mfrow=c(10,6)) it says : ...
0
votes
0answers
10 views
Import Voozanoo data into R [migrated]
Voozanoo allows exporting the data in various formats. What's the best way to import the data into R?
I tried the CSV export, it worked, but extracting the codes and values of categorical variables ...
0
votes
0answers
10 views
What's the difference between a feedback loop and a bidirectional path?
I had always thought that the relationship between x and y here was a bidirectional path. However, I note that Kenny at http://davidakenny.net/cm/iv.htm describes it as a feedback loop.
That makes ...
3
votes
1answer
25 views
How to interprete TraMineRs results of disstreedisplay and seqtreedisplay
I am not quite sure what the differences are between the outputs and results of disstreedisplay and seqtreedisplay. Maybe you ...
4
votes
2answers
95 views
The way an MA(q) model works
I am trying to understand the way MA(q) models work.
For this purpose I have created a simple data set with only
three values. I then adapted a MA(1) model to it. The results
are shown below:
...
1
vote
1answer
92 views
Simulate arima by hand
I was working on arima in r and I am trying not to use library 'forecast' as much as possible. I have a code for finding the best arima model, but it is showing some warning messages, please help.
...
2
votes
1answer
36 views
Can agresti-coull binomial confidence intervals be negative?
According to http://www.itl.nist.gov/div898/handbook/prc/section2/prc241.htm
agresti-coull intervals cannot be negative, however using the formula from wikipedia as well as the binom.confint function ...
1
vote
0answers
20 views
MR with dummy variables
I need to calculate MR but my data consists of many dummy variables. I had a question in my survey that asked for the budget/month. But instead of giving the option to fill in a value, I offered ...
2
votes
1answer
385 views
Why does a paired t-test (when appropiate) result in better variance?
I didn't quite get the wikipedia explanation here:
http://en.wikipedia.org/wiki/Paired_difference_test#Use_in_reducing_variance
I agree that both the unpaired and paired means are the same...then I ...
1
vote
0answers
21 views
Assessing Significance in Neural Networks
I'm trying to determine statistical significance of inputs on the dependent variable using neural networks. I'm trying to test the null hypothesis that the partial derivatives are equal to zero. I ...
3
votes
1answer
54 views
Statistical tests when each variable in a sample is a percentage
I am performing a temperature preference test in mice. Mice are placed into a chamber with one half of a chamber having a cold floor and the other half having a warm floor. Mice spend a specific total ...
1
vote
1answer
45 views
Bootstrapping to find confidence intervals. Very small sample size
I am trying to generate confidence intervals for estimates of numbers of fish observed with a very small number of surveys. Most of the numbers of fish observed are very small as well.
The estimate ...
1
vote
0answers
19 views
Pre- and post-intervention effect, what to use?
We have data from 2006-2010 on heart attacks and smoking, we want to measure what was the effect on a smoking ban made in 2008 on heart attack rates.
What is the best statistical analysis to use?
...
0
votes
0answers
6 views
Topic Extraction using Scikit. Get documents for the Topics
I was trying to get a NNF model work for Topic Extraction.
SCIKIT code http://scikit-learn.org/dev/auto_examples/applications/topics_extraction_with_nmf.html# works fine. But what i wanted was ...
1
vote
1answer
137 views
Understanding “unbiased ridge regression”
Could someone please explain what "$J$" consists of in this paper, equation 1.5.
$$
J \sim N(\beta, \sigma^2 I/k)
$$
What's $\beta$ here? What's $N$?
Also, why are they putting that much effort in ...
3
votes
1answer
57 views
Outline of benefits/costs of using Bayesian rather than Frequentist OLS and time-series?
I'm reading up on Bayesian techniques for Linear models and time series. While the texts are great at teaching the theory I would like to get a better handle on the pro's/con's of Bayesian analysis vs ...
1
vote
0answers
45 views
Floating point issues when transforming an arbitrary correlation matrix to positive semi-definite
I'm following Peter Jackel's book "Monte Carlo Methods in Finance", where an algorithm for transforming malformed correlation matrices into acceptable correlation matrices (positive semi-definite) to ...
3
votes
1answer
107 views
Is the exponential distribution a good model for this data?
I'm trying to determine if the exponential distribution is a good model for a data set that I'm exploring. It doesn't have to be precise. I'm using the data for capacity planning (if it's a good fit) ...
0
votes
1answer
31 views
Confusion related to minimization of a gaussian likelihood function
I have this confusion related to minimization of gaussian likelihood function. The negative of the log likelihood of gaussian distribution is
$-\log \det(Q) + \text{tr}(SQ) + \lambda||Q||_{1}$ where ...
1
vote
1answer
46 views
statistical error on difference f(x) - x
Suppose I've a random variable $X$ and a sample of it with size $N$. I count how many element of the sample fall in a specific range $a<x<b$ and I found $n$ entries, so if I use Poisson ...
0
votes
0answers
12 views
deviding a storing a file as different vectors [closed]
I have a file with 3 columns how can i read the file and then save the third column’s values in different vectors depends on the first and second columns. for example in this example i need to save ...
1
vote
1answer
45 views
Can I do a correlation between a likert scale and a dichotomous scale with the same list of items
I am trying to do a correlation between a 4 level likert scale with 11 items and a dichotomous present-not present scale of the same 11 items. Can I do this?
0
votes
0answers
26 views
Scikit-learn reports memory error when fitting Gaussian process model
I want to fit a Gaussian Process with about 50,000 training examples and 130 features using Scikit-learn. Right now, I only have 1 theta hyperparameters as I run the process with all defaults. But I ...
0
votes
1answer
82 views
Which variables have too much variation?
So I have 1000 variables (different lakes), each with 12 observations (each lake was tested once a year for 12 years). Some of the lakes are really small, so we think they have way too much variation. ...
1
vote
1answer
71 views
Unit root test specifications?
I am puzzled as to what specification I should include in my unit root tests of the following data: .
I will use ADF, KPSS and ZA tests. I can see there is a break in trend at observation 9. However, ...
0
votes
0answers
20 views
Upper bound for the Cardinal of the Active Set in Lasso
I am searching for a proof to:
With probability tending to one, we have that the cardianl of the active set in the Lasso estimator is bounded by $C\frac{n}{\lambda_n^2}$, ($C$ is some constant) i.e.;
...
1
vote
1answer
105 views
Generating a log-normally distributed pseudorandomly generated data set
The feedback I received from my initial post seemed to indicate that my question was ill-posed. Hence, I would like to clarify what I am doing and how I hope to achieve it.
I'm running some ...
