0
votes
0answers
2 views

2x3 repeated measures for skewed data

I am using 2x3 repeated measures ANOVA. I am analyzing three years of test data and comparing two different groups. However, my data is negatively skewed. What is the best way to transform ...
0
votes
0answers
2 views

ARIMA + day dummy model

I am trying to fit an ARIMA model with weekdays dummy. I have managed to fit the model using the following command: modelfitsample <- read.table(file="C:/Users/em9224/Desktop/nan_data_ts.txt",...
-1
votes
1answer
7 views

How to determine discount rate

Suppose you have two products that are the same except that they are made in different time periods: $P$ and $P'$ (older product). How do you determine the discount on $P'$?
0
votes
0answers
5 views

Using posterior predicted values in estimation in Stan

I have some data on prices that I would like to predict as a function of covariates. One of those covariates is the predicted rating of that item as estimated from totally separate item rating data. I ...
0
votes
0answers
8 views

R Studio - clustering [migrated]

I am trying to do the clustering on a set of data using the easy: fit <- kmeans(my data, 2) command The problem is, when I run it, I get: Error in do_one(nmeth) : NA/NaN/Inf in foreign ...
0
votes
0answers
8 views

Can I work out the regression coefficients from the betas and the t-values?

I have t, and beta, but I didn't write down the coefficients- or the s.e. How can I work out the unstandardised coefficients?
0
votes
0answers
13 views

Gamma distribution and applications

I'm looking for references to read about gamma distribution and applications in industry or in quality control. I had a look at statistical methods for reliability data (Meeker and Escobar). It has ...
0
votes
0answers
8 views

Time Series Prediction using Machine Learning

I am trying to predict the request arrive time of some objects in network traffic. I have few features of the object like their type, size, previous arrival time, etc. So I was think that I should use ...
0
votes
1answer
14 views

Correlation when one variable has both positive and negative values

This is probably ridiculously simple but because I have to include the word 'negative' in my searches I'm constantly finding only posts and websites that discuss negative correlation which isn't what ...
0
votes
1answer
16 views

I'm using R to create a time series - having some difficulty understanding frequency

I'm working with a dataset of electrical submeter readings taken once per minute for 47 months. I'm trying to create a time series that covers meter readings from the kitchen during the week of ...
0
votes
0answers
3 views

How to analyse multiple and different rankings by user to same objects?

I have a number of products with types ranging from type 1 to type 5. Now a number of customers are allowed to rank the best types they like for all the product. The response I get consists of ratings ...
0
votes
0answers
3 views

Radial basis function network MATLAB spread constant

In MATLAB we have to set spread constant for RBF network. Documentation says something like "there is more active RBF neurons with higher number of spread constant but its not good to active all RBF ...
0
votes
0answers
8 views

Non-parametric one-way random effects ANOVA?

I have repeated measures (GPS points) from 6 individuals randomly selected from a population of squirrels. I calculated duration of time spent in three different habitat types (open, shrub, tree) and ...
0
votes
0answers
6 views

How can I get lid out outliers in Repeated Measures ANOVA?

I'm a person who has barely statistic knowledge. Anyway, I'm analyzing my repeated measures data. Firstly, I did repeated measures ANOVA using "ez" package in R. However, the results don't have ...
0
votes
0answers
6 views

Viterbi algorithm for profile hidden Markov models

the difference of pHMM is that they have silent deletion states. I was wondering if there was an adaptation of Viterbi, forward-backward algorithms for these models, and it turns out that there is, ...
0
votes
1answer
10 views

Fisher's information for multiple binomials

I am trying to quantify error of the MLE for the following model using Fisher's information: $Y_{j} \sim Binomial(n_{j}, p_{j})$ $logit(p_{j}) = \eta + \gamma_{j}$ where the $n_{j}$'s and $\gamma_{j}$'...
0
votes
0answers
6 views

Treat confidence table in supervised learning as a confusion matrix

I have a lookup table with predicted and actual classes (like confusion matrix), but I use it to store confidence about a certain choice (there is a feedback telling whether the choice is right or ...
-1
votes
0answers
15 views

What models are well suited to varying-length feature vectors?

I'm trying to classify a bit of a strange-looking data set where the input vectors have a variable length. The actual data is pretty boring, but imagine a related problem where you're trying to ...
0
votes
0answers
7 views

Random generation from multivariate kernel density

Bernard W. Silverman in Density Estimation for Statistics and Data Analysis describes algorithm for random generation from kernel densities: Step 1 Choose $I$ uniformly from $\{1,\dots,n\}$ Step ...
1
vote
0answers
6 views

RBF network normalization, standardization in MATLAB

First of all, is it good to do normalization and standardization for Radial basis neural network? Does MATLAB do that automatically for RBF? I've read that MATLAB does normalization and ...
2
votes
1answer
44 views

Why does this distribution look completely different after splitting into groups?

I've got some data that represents two groups, with a before and after result for both groups. When I plot the data together, it looks as though the after is much tighter around the mean. However, ...
0
votes
0answers
8 views

Time series object for hourly datat with daily seasonal in R

I have got hourly data from 2012.01.01 to 2016.06.13 and the data shows daily seasonal. I am trying to create a time series object with only daily seasonal in R. ...
0
votes
0answers
14 views

Sparsity as missing data + MLE

I just had this "funny" idea: what about a classifier that not only tries to learn weights for predicting y but actually works with "deleted" data (as in sparse ...
0
votes
0answers
6 views

Can I split a table through the correlation of its 2 columns?

I've got a 2 column table (X, Y). Every row of the table says that the X on that row use the Y specified in the same row. This are the sample data. https://jpst.it/L6I5 I'd like to cluster the table ...
0
votes
0answers
2 views

How do apriori based algorithms avoid finding spurious relationships in their rules?

I'm taking a class on data mining and of course, we went over the Apriori algorithm (Fast Algorithms for Mining Association Rules). Something that I want to know is how do apriori based ...
1
vote
1answer
28 views

Contradiction between the Law of Large Numbers and the Gambler's Fallacy?

If I flip a fair coin 10 times, and all of them land on heads, according to the Law of Large Numbers I am more likely to get tails on the next flip. However, this is clearly the Gambler's Fallacy. ...
1
vote
0answers
13 views

Importance of workflow visualization in a statistical software package?

From a blog post ... The important thing is why they’re missing. Gartner excluded them this year, “…due to not satisfying the [new] visual composition framework [VCF] inclusion criteria.” VCF is ...
1
vote
0answers
8 views

Impact of spurious regressors on out of sample prediction error

The true DGP is \begin{equation} y=\alpha_0 + \alpha_1 x_1 + \dots + \alpha_k x_k +\epsilon, \quad \epsilon\sim \mathcal{N}(0,1)\label{eq:1} \end{equation} but we instead estimate \begin{equation} y=...
1
vote
0answers
12 views

What is the effect of training a model on an imbalanced dataset & using it on a balanced dataset?

When evaluating a model, for example a binary classifier, should the train and test set have 50% + and 50% - label distribution or could the distribution be random? If the distribution is biased in ...
0
votes
2answers
23 views

Test for No Correlation [on hold]

I'm wondering: Is there a statistical test for determining "no correlation"? I would not want to use Fisher's transformation.
0
votes
0answers
14 views

If I'm running a test to determine whether two variables are NOT correlated: what is my hypothesis?

I want to show that two variables are NOT correlated. When I run spearman on it, I get a very large p-value and a correlation of about -0.1. So, I accept the null ...
0
votes
0answers
5 views

Determining Optimal Markdown price for Items

Suppose you have a set of items. Some items are old and some items are new. Naturally, the demand for old items will diminish while the demand for new items will increase. As a result, old items will ...
0
votes
0answers
9 views

QDA vs EM with Gaussian likelihoods

QDA (quadratic discriminant analysis) assumes that the K different classes are generated by K different multivariate Gaussians, each with potentially different mean vector and covariance matrix. If ...
0
votes
1answer
11 views

Reasoning behind reciprocal link function for exponential regression

The reciprocal link function in exponential regression doesn't constrain to positive values, whilst on the other hand it does thoroughly space out points close to zero when fitting the linear ...
0
votes
0answers
13 views

Properly Computing Effect Size For Comparing Two Proportions

I am highly confused about computing Effect Size for comparing two proportions. Question 1: How does the below formula make sense? Why is it that all I need to pass in is two proportions (without ...
1
vote
0answers
7 views

Transform my data or use the original data in my multiple regression analysis?

I have three independent variables and one dependent variable (two separate questions which I want to analyse seperately against the tree independent variables). I want to look at the relationship ...
2
votes
0answers
9 views

Does ReLU layer work well for a shallow network?

I am currently working on training a 5-layer neural network, and I got some problems with tanh layer and would like to try ReLU layer. But I found that it becomes even worse for ReLU layer. I am ...
0
votes
0answers
17 views

Variable importance

I fitted a random forest with the h2o package (but I have the same problem with other models with h2o) in a dataset with multinomial target. I find out the variable importance of each variable. Can I ...
0
votes
1answer
6 views

Escobar and West Sampler for Dirichlet Process Parameters

I am reading Escobar&West paper and in particular am interested in their Gibbs sampler for the concentration parameter of Dirichlet Process. The issue I have is at the end of their section 6, ...
0
votes
0answers
12 views

FIML using Mplus

I am trying to translate some logistic regressions from SAS to Mplus. Some are fairly straightforward, e.g., no random effects, and others are mixed models with random intercepts. The example here ...
0
votes
0answers
9 views

Does it make sense to compute the KL divergence of two KDEs?

Does it? I'm not exactly certain. I fit two KDE PDFs on two datasets. I want to measure the disimilarity between them.
0
votes
0answers
6 views

Kullback-Leibler divergence with sample data likelihood

I'm trying to get my head around the KB divergence in the context of the sample likelihood under two competing hypotheses, one optimal $H_0$ and one suboptimal $H_1$. Roughly speaking, I want to see ...
0
votes
0answers
3 views

Self-organizing maps using Tensorflow?

So, I'm trying to sort out a problem from some data I've collected. Any ideas of how to start dealing with this problem is helpful. I just started to use Tensorflow, and will probably stick with it. ...
5
votes
7answers
401 views

Why isn't variance defined as the difference between every value following each other?

This may be a simple question for many but here it is: Why isn't variance defined as the difference between every value following each other instead of the difference to the average of the values? ...
0
votes
1answer
17 views

Paired T test vs Wilcoxon signed-rank

I am trying to determine whether a paired T test or Wilcoxon ranked sum test is more appropriate to test for significance of improvement in the following: Patients were surveyed for pain scores ...
1
vote
2answers
26 views

McNemar's test p-value NA

I have paired data from 9 individuals. The response variable is ordinal with levels 1-5. As the sample is very small and the data are categorical, I thought the appropriate test was McNemar's test. ...
0
votes
1answer
27 views

Prove Causation [on hold]

Could anyone share me some good material to start to read How one can prove causation or the steps to prove it using statistical test via code e.g python or R? Thanks in advanced.
3
votes
1answer
11 views

KL divergence vs Absolute Difference between two distributions?

Why should I use KL divergence over just giving the abs difference from two PDFs?
3
votes
1answer
34 views

What are periodic version of splines?

In this What's wrong to fit periodic data with polynomials? post, I tried to use Fourier basis expansion and Polynomial basis expansion to fit a toy periodic data (daily temperature data set). I ...
0
votes
0answers
8 views

Outliers detection methods for RNA-Seq data

I have a rna-seq dataset with normalization in RPKM. The dataset have 1 gene per row with 4 different experiment condition. I need a detect de outliers values in this dataset. I used de weka filter ...

15 30 50 per page