All Questions

2 views

2x3 repeated measures for skewed data

I am using 2x3 repeated measures ANOVA. I am analyzing three years of test data and comparing two different groups. However, my data is negatively skewed. What is the best way to transform ...
2 views

ARIMA + day dummy model

I am trying to fit an ARIMA model with weekdays dummy. I have managed to fit the model using the following command: modelfitsample <- read.table(file="C:/Users/em9224/Desktop/nan_data_ts.txt",...
7 views

How to determine discount rate

Suppose you have two products that are the same except that they are made in different time periods: $P$ and $P'$ (older product). How do you determine the discount on $P'$?
5 views

Using posterior predicted values in estimation in Stan

I have some data on prices that I would like to predict as a function of covariates. One of those covariates is the predicted rating of that item as estimated from totally separate item rating data. I ...
8 views

R Studio - clustering [migrated]

I am trying to do the clustering on a set of data using the easy: fit <- kmeans(my data, 2) command The problem is, when I run it, I get: Error in do_one(nmeth) : NA/NaN/Inf in foreign ...
8 views

Can I work out the regression coefficients from the betas and the t-values?

I have t, and beta, but I didn't write down the coefficients- or the s.e. How can I work out the unstandardised coefficients?
13 views

Gamma distribution and applications

I'm looking for references to read about gamma distribution and applications in industry or in quality control. I had a look at statistical methods for reliability data (Meeker and Escobar). It has ...
8 views

Time Series Prediction using Machine Learning

I am trying to predict the request arrive time of some objects in network traffic. I have few features of the object like their type, size, previous arrival time, etc. So I was think that I should use ...
14 views

Correlation when one variable has both positive and negative values

This is probably ridiculously simple but because I have to include the word 'negative' in my searches I'm constantly finding only posts and websites that discuss negative correlation which isn't what ...
16 views

I'm using R to create a time series - having some difficulty understanding frequency

I'm working with a dataset of electrical submeter readings taken once per minute for 47 months. I'm trying to create a time series that covers meter readings from the kitchen during the week of ...
3 views

How to analyse multiple and different rankings by user to same objects?

I have a number of products with types ranging from type 1 to type 5. Now a number of customers are allowed to rank the best types they like for all the product. The response I get consists of ratings ...
3 views

In MATLAB we have to set spread constant for RBF network. Documentation says something like "there is more active RBF neurons with higher number of spread constant but its not good to active all RBF ...
8 views

Non-parametric one-way random effects ANOVA?

I have repeated measures (GPS points) from 6 individuals randomly selected from a population of squirrels. I calculated duration of time spent in three different habitat types (open, shrub, tree) and ...
6 views

How can I get lid out outliers in Repeated Measures ANOVA?

I'm a person who has barely statistic knowledge. Anyway, I'm analyzing my repeated measures data. Firstly, I did repeated measures ANOVA using "ez" package in R. However, the results don't have ...
6 views

Viterbi algorithm for profile hidden Markov models

the difference of pHMM is that they have silent deletion states. I was wondering if there was an adaptation of Viterbi, forward-backward algorithms for these models, and it turns out that there is, ...
10 views

Fisher's information for multiple binomials

I am trying to quantify error of the MLE for the following model using Fisher's information: $Y_{j} \sim Binomial(n_{j}, p_{j})$ $logit(p_{j}) = \eta + \gamma_{j}$ where the $n_{j}$'s and $\gamma_{j}$'...
6 views

Treat confidence table in supervised learning as a confusion matrix

I have a lookup table with predicted and actual classes (like confusion matrix), but I use it to store confidence about a certain choice (there is a feedback telling whether the choice is right or ...
15 views

What models are well suited to varying-length feature vectors?

I'm trying to classify a bit of a strange-looking data set where the input vectors have a variable length. The actual data is pretty boring, but imagine a related problem where you're trying to ...
7 views

Random generation from multivariate kernel density

Bernard W. Silverman in Density Estimation for Statistics and Data Analysis describes algorithm for random generation from kernel densities: Step 1 Choose $I$ uniformly from $\{1,\dots,n\}$ Step ...
6 views

RBF network normalization, standardization in MATLAB

First of all, is it good to do normalization and standardization for Radial basis neural network? Does MATLAB do that automatically for RBF? I've read that MATLAB does normalization and ...
44 views

Why does this distribution look completely different after splitting into groups?

I've got some data that represents two groups, with a before and after result for both groups. When I plot the data together, it looks as though the after is much tighter around the mean. However, ...
8 views

Time series object for hourly datat with daily seasonal in R

I have got hourly data from 2012.01.01 to 2016.06.13 and the data shows daily seasonal. I am trying to create a time series object with only daily seasonal in R. ...
14 views

Sparsity as missing data + MLE

I just had this "funny" idea: what about a classifier that not only tries to learn weights for predicting y but actually works with "deleted" data (as in sparse ...
6 views

Can I split a table through the correlation of its 2 columns?

I've got a 2 column table (X, Y). Every row of the table says that the X on that row use the Y specified in the same row. This are the sample data. https://jpst.it/L6I5 I'd like to cluster the table ...
2 views

How do apriori based algorithms avoid finding spurious relationships in their rules?

I'm taking a class on data mining and of course, we went over the Apriori algorithm (Fast Algorithms for Mining Association Rules). Something that I want to know is how do apriori based ...
28 views

Contradiction between the Law of Large Numbers and the Gambler's Fallacy?

If I flip a fair coin 10 times, and all of them land on heads, according to the Law of Large Numbers I am more likely to get tails on the next flip. However, this is clearly the Gambler's Fallacy. ...
13 views

Importance of workflow visualization in a statistical software package?

From a blog post ... The important thing is why they’re missing. Gartner excluded them this year, “…due to not satisfying the [new] visual composition framework [VCF] inclusion criteria.” VCF is ...
8 views

Impact of spurious regressors on out of sample prediction error

The true DGP is $$y=\alpha_0 + \alpha_1 x_1 + \dots + \alpha_k x_k +\epsilon, \quad \epsilon\sim \mathcal{N}(0,1)\label{eq:1}$$ but we instead estimate y=...
12 views

What is the effect of training a model on an imbalanced dataset & using it on a balanced dataset?

When evaluating a model, for example a binary classifier, should the train and test set have 50% + and 50% - label distribution or could the distribution be random? If the distribution is biased in ...
23 views

Test for No Correlation [on hold]

I'm wondering: Is there a statistical test for determining "no correlation"? I would not want to use Fisher's transformation.
14 views

If I'm running a test to determine whether two variables are NOT correlated: what is my hypothesis?

I want to show that two variables are NOT correlated. When I run spearman on it, I get a very large p-value and a correlation of about -0.1. So, I accept the null ...
5 views

Determining Optimal Markdown price for Items

Suppose you have a set of items. Some items are old and some items are new. Naturally, the demand for old items will diminish while the demand for new items will increase. As a result, old items will ...
9 views

QDA vs EM with Gaussian likelihoods

QDA (quadratic discriminant analysis) assumes that the K different classes are generated by K different multivariate Gaussians, each with potentially different mean vector and covariance matrix. If ...
11 views

Reasoning behind reciprocal link function for exponential regression

The reciprocal link function in exponential regression doesn't constrain to positive values, whilst on the other hand it does thoroughly space out points close to zero when fitting the linear ...
13 views

Properly Computing Effect Size For Comparing Two Proportions

I am highly confused about computing Effect Size for comparing two proportions. Question 1: How does the below formula make sense? Why is it that all I need to pass in is two proportions (without ...
7 views

Transform my data or use the original data in my multiple regression analysis?

I have three independent variables and one dependent variable (two separate questions which I want to analyse seperately against the tree independent variables). I want to look at the relationship ...
9 views

Does ReLU layer work well for a shallow network?

I am currently working on training a 5-layer neural network, and I got some problems with tanh layer and would like to try ReLU layer. But I found that it becomes even worse for ReLU layer. I am ...
17 views

Variable importance

I fitted a random forest with the h2o package (but I have the same problem with other models with h2o) in a dataset with multinomial target. I find out the variable importance of each variable. Can I ...
6 views

Escobar and West Sampler for Dirichlet Process Parameters

I am reading Escobar&West paper and in particular am interested in their Gibbs sampler for the concentration parameter of Dirichlet Process. The issue I have is at the end of their section 6, ...
12 views

FIML using Mplus

I am trying to translate some logistic regressions from SAS to Mplus. Some are fairly straightforward, e.g., no random effects, and others are mixed models with random intercepts. The example here ...
9 views

Does it make sense to compute the KL divergence of two KDEs?

Does it? I'm not exactly certain. I fit two KDE PDFs on two datasets. I want to measure the disimilarity between them.
6 views

Kullback-Leibler divergence with sample data likelihood

I'm trying to get my head around the KB divergence in the context of the sample likelihood under two competing hypotheses, one optimal $H_0$ and one suboptimal $H_1$. Roughly speaking, I want to see ...
3 views

Self-organizing maps using Tensorflow?

So, I'm trying to sort out a problem from some data I've collected. Any ideas of how to start dealing with this problem is helpful. I just started to use Tensorflow, and will probably stick with it. ...
401 views

Why isn't variance defined as the difference between every value following each other?

This may be a simple question for many but here it is: Why isn't variance defined as the difference between every value following each other instead of the difference to the average of the values? ...
17 views

Paired T test vs Wilcoxon signed-rank

I am trying to determine whether a paired T test or Wilcoxon ranked sum test is more appropriate to test for significance of improvement in the following: Patients were surveyed for pain scores ...
26 views

McNemar's test p-value NA

I have paired data from 9 individuals. The response variable is ordinal with levels 1-5. As the sample is very small and the data are categorical, I thought the appropriate test was McNemar's test. ...
27 views

Prove Causation [on hold]

Could anyone share me some good material to start to read How one can prove causation or the steps to prove it using statistical test via code e.g python or R? Thanks in advanced.
11 views

KL divergence vs Absolute Difference between two distributions?

Why should I use KL divergence over just giving the abs difference from two PDFs?