All Questions
214,248
questions
1
vote
0
answers
59
views
What statistical analysis to use?
What statistical analysis should I use if I have 2 groups of subjects (high and low comprehenders) which are divided into 3 subgroups according to text genre (poetry, short story, drama) and subjected ...
3
votes
1
answer
278
views
Illustrating power-law properties with Chinese Restaurant Process (PYP)
I often read that Pitman-Yor Process has power-law properties. Let's say I am interested in modelling English word's distribution (which follows power-law). Using CRP metaphor, words come and get ...
- 1,462
1
vote
1
answer
2k
views
About stepwise regression and correlation
I am trying to fit a model to some observed data with the least squares method. Now, I am at the stage where I have run a stepwise regression (traditional), with Entry level $=0.025$ and Stay level $=...
- 1,035
24
votes
3
answers
7k
views
Uniform random variable as sum of two random variables
Taken from Grimmet and Stirzaker:
Show that it cannot be the case that $U=X+Y$ where $U$ is uniformly distributed on $[0,1]$ and $X$ and $Y$ are independent and identically distributed. You should not ...
- 3,395
0
votes
0
answers
2k
views
lowess: R (& python-statsmodels) vs MATLAB (& biopython)
There seems to be two different interpretations of what LOWESS really means: one from R (also used by python-statsmodels), and one from MATLAB (also used by biopython), see comparison below.
Could ...
- 121
2
votes
1
answer
684
views
estimating the population standard deviation from the sample standard deviation
Is there a way to estimating the population standard deviation, $\sigma$, from the sample standard deviation, $s$, if the size of the finite population is $N$ and the size of the sample is $n$?
Is ...
- 123
15
votes
2
answers
23k
views
Output of Scikit SVM in multiclass classification always gives same label
I am currently using Scikit learn with the following code:
...
- 445
1
vote
0
answers
139
views
Compute effect size based on simulation using different lists of different length
For my research purpose I wrote a small simulation to compute an empirial p-value based on an input gene list and a reference gene list.
The idea is to check for each gene in the input gene list, the ...
- 564
2
votes
1
answer
4k
views
After Clustering, how can I evaluate which features had the biggest impact?
I've just performed unsupervised clustering (using DBSCAN) on a dataset for which I have no expert knowledge on.
I'm interested in working out which features had the greatest impact on my clustering.
...
- 37
6
votes
1
answer
225
views
Unrealistically high significance when marginalizing over large number of parameters
The setup
I've tried to reduce the problem to a self-contained subset for this question, but it still ended up being pretty long. Sorry about that!
I have a set of observations of a set of objects ...
- 163
3
votes
1
answer
114
views
Selecting variables using SAS and R - all effects are significant even when shuffling the data
Dear all: I need to test which effects I should include in my model for genetic evaluation of cows. I was using the following code in R:
...
- 51
0
votes
1
answer
268
views
Full effects from Poisson GLM
I am running a Poisson GLM with count data as response variable and both continuous and categorical variables as predictors. I made use of the following (dispersion is OK):
...
- 1
0
votes
1
answer
189
views
identify nature of missingness for categorical variables
could you please give me some hints for identifying the nature of missingness for categorical variables' missing value? I mean, I gave a fast search on google scholar but I didn't find anything ...
- 133
1
vote
0
answers
497
views
repeated measures ranking data
I have a repeated measures design where 180 participants rank their preferences for 12 items at time1, and then undergo an intervention, and then they rank their preferences for the same 12 items ...
- 11
1
vote
0
answers
772
views
Sampling from a multivariate distribution
Given $X = [X_1 X_2 \ldots X_n]^{\top}$ is a vector of independent bernoulli random variables and $A \in \{0, 1\}^{m \times n}$ is an arbitrary boolean matrix. Define a random variable $Y$ as
$$Y = AX$...
- 3,771
1
vote
0
answers
36
views
How do I find corresponding clusters in independent samples?
Lets suppose you believe that observations in your data come from K natural but not directly observable categories and you wish to identify these categories with minimal prior assumptions, so you find ...
- 2,354
2
votes
1
answer
87
views
Study Length Over-Estimating Hazard Ratio?
Are cox model studies over too long a time-scale at risk of over estimating (or under estimating) a covariate's effect on hazard?
I'm studying inbreeding in a captive animal population. Some ...
- 95
3
votes
1
answer
698
views
Does the Mann-Whitney U-test require the groups to have the same distribution?
If one of my data sets is normally distributed and the other is not, can I do a Mann-Whitney U-test on them, or would they both have to be non-normal?
- 77
5
votes
1
answer
2k
views
estimating the upper bound on a uniform distribution from max order statistic
I have a question.
Suppose that $X_1,\ldots,X_n$ are iid $U(0,\lambda)$ and let $X(n)$ denote the nth order statistic.
Suppose $\lambda$ is unknown and should be estimated from the sample.
Take $...
- 751
2
votes
1
answer
5k
views
Machine learning algorithm to predict next user's destination
I'm searching for a way to formulate my problem as a machine learning problem.
Suppose I have a history of user's locations, and I want to predict his next location, similar to how Google Now does it ...
- 123
1
vote
1
answer
442
views
Fitting a mixture model to spatially correlated data
When the data are spatially correlated, is the usual GMM likelihood function overweighted?
The data. Scattering experiment, sensor is like a CCD. Can't see individual events, only density estimate ...
- 1,002
8
votes
1
answer
2k
views
Why does a goodness of fit test use the chi square distribution rather than the hypothesised distribution? [duplicate]
This is a homework(-like) question. I know the answer but don't understand it.
If we were to answer the question "Are 5 different groups of mammals equally common?", what test would you perform?
...
- 349
2
votes
0
answers
222
views
Mean of an interval censored random variable
It is likely that I am incorrect terminology here, but I am trying to compute a "mean" of an interval censored random variable.
Here is an example where:
1. a random sample from the standard normal ...
- 8,722
6
votes
1
answer
7k
views
What is the limiting distribution of the sample mean?
My question is relatively simple: what is the limiting distribution of the sample mean? But there are some technicalities I want to discuss.
context: I was asked this problem in an exam, and I feel ...
- 163
3
votes
2
answers
1k
views
Reproduce Cross Correlation results in Python
I'm trying and failing to reproduce the results found in this paper, where the cross correlation between two distributions is calculated:
...
- 951
10
votes
1
answer
7k
views
What is the difference between R hat and psrf?
In convergence diagnosis in WinBUGS/JAGS/Stan, there are different statistics reported for each variable. In WinBUGS/Stan, Rhat ($\hat{R}$) is reported. In JAGS with the ...
- 6,193
0
votes
1
answer
149
views
Why do two identical feature vectors (distance score 0) get different labels in DBSCAN?
I have two identical feature vectors. They have a distance score of 0.
I perform DBSCAN Clustering (using sci-kit) and they get different labels.
Is this expected behaviour?
- 37
2
votes
2
answers
308
views
Backward feature selection with CV model selection
I am thinking about doing the following to a data set with $N$ samples and $m$ features
1) Train using semi-supervised learning and cross validate on labeled data using LOO-CV to select the best ...
user45299
0
votes
1
answer
919
views
High leverage Points In Logistic Regression in R [closed]
Do You Know Any Good R Package For This Model?
- 9
1
vote
1
answer
705
views
Survival analysis with competing risks
I am analysing the effect of an intervention to reduce length of hospital stay (LOHS) after surgery. The main outcome is LOHS and the intervention is the main exposure. Death while in hospital is an ...
- 117
4
votes
1
answer
272
views
Using quadratic programming to fit a piecewise linear model plus seasonality
I am reading this paper on fitting an L1TF model to data using quadratic programming. Section 7.4 states how one could add seasonality to the model however it doesn't go very far into it. I am trying ...
- 1,143
12
votes
3
answers
10k
views
The best way for clustering an adjacency matrix
I've had a hard time interpreting resulting clusters of an adjacency matrix. I have 200 relatively big matrices representing subjects that contains partial correlations (z scores) of time series (...
- 145
0
votes
1
answer
2k
views
Comparison of predictive models
I am trying to compare the predictive ability of various models in predicting survival in patients. I would like to examine the predictive performance of each model using 4 tests: squared Pearson ...
4
votes
2
answers
7k
views
t test with log transformation
One of my variables to be compared in a t-test is normally distributed, while the other is non-normally distributed. What test should I use? I thought I should do a reflect log10 transformation on the ...
- 77
2
votes
0
answers
59
views
Item Information Distorting Scale
I'm refining a psychometric questionnaire and working to estimate (2PL) IRT parameters for the items in order to score the items. It's a small scale consisting of 5 questions. The MAP and VSS-1 ...
- 749
3
votes
1
answer
812
views
Explain: Latent Variable (e.g for Latent Dirichlet Allocation)
I am trying to understand the Latent Dirichlet Allocation but therefore I need the basic understanding about what exactly a latent variable is in that sense.
I know that the basic idea of a latent ...
- 277
4
votes
1
answer
61
views
Meta-analyses for variance rather than means
What are the main complications / differences when conducting a meta-analyses where the metric of focus is not effect size (i.e., means) per se, but instead estimates of variances from models?
Nods ...
- 1,007
2
votes
0
answers
148
views
Is There A Machine Learning Algorithm For Textual Data With Thousands Of Classifiers?
I've been asked to migrate this from StackOverflow to CrossValidated.
I have a problem that I think Machine Learning can solve but am having a very hard time determining which ML Algorithm to use and ...
- 21
1
vote
0
answers
403
views
Regression Analysis with String values in column of DataFrame [duplicate]
I have a dataset which is having multiple columns where some column has string values with comma seperated
...
1
vote
0
answers
1k
views
Can Chebyshev inequality be used to bound the error of the sample mean?
Can the probability of error of the sample mean, i.e., $\Pr(|\bar{X}-E[X]| \geq \epsilon)$, be bounded using Chebyshev inequality (or something similar)? $X$ is a discrete random variable with an ...
- 41
0
votes
0
answers
46
views
As a rule of thumb, difference in amount of administered items: classically scored vs adaptive (irt-based) tests?
I am thinking of developing a new adaptive test. One of the main goals is to shorten existing, classically scored tests. Is there a rule of thumb about how many items are needed to administered ...
- 605
1
vote
2
answers
707
views
How many features can we use to avoid overfitting the classification?
We have a classification problem: classify type A tumour from type B tumour. In total we have 50 patient cases (25 A and 25 B cases). We use texture or shape analysis to generate features we can ...
- 1,739
2
votes
2
answers
277
views
Compare differences in dependent frequencies
I would like to compare two error rates (proportion of incorrect answers in questions with dichotomous response-format) for two different items, and I am looking for some advice on how to do that.
...
- 663
8
votes
1
answer
23k
views
Normalize variables for calculation of correlation coefficient
I have two vectors (arrays) of values. One vector represents a variable whose values are between 0 and 1 (ratio-type variable). The other vector represents a variable whose values are continuous float ...
- 505
5
votes
1
answer
6k
views
Linear mixed effects models: what to do when the residual QQ-plot looks non-normal?
I have four linear mixed effect models of similar structure:
...
1
vote
1
answer
294
views
The linearity of perceptrons
I am having quite a dilemma whether multi-layer perceptrons are linear in nature or not.
In this wikipedia article, it is said that:
If a multilayer perceptron has a linear activation function in ...
- 111
0
votes
0
answers
635
views
How to name the concept of the mean shift divided by the standard deviation
In the context of a paper dealing with Statistical Quality Control, I am defining and using the concept of the mean shift divided by the standard deviation of a (normally distributed one-dimensional) ...
- 799
3
votes
1
answer
3k
views
Selecting an appropriate VAR model
I would like to receive critical comments on an idea explained below.
Suppose I have variables $x_1$ through $x_K$, and this is a time series setting.
My aim is forecasting.
I know that all the ...
- 67.5k
1
vote
2
answers
424
views
Likert Scale 3 2 1 0 1 2 3 possible?
In my questionnaire I want to ask people to score on a dimensional Likert-type scale their attitude toward a specific target group.
E.g.
People with .... are
stupid.... intelligent.
I want to use ...
- 11
2
votes
1
answer
181
views
Frequency significantly different from zero
I've got a few frequencies, and I'm trying to work out if each of the frequencies is significantly different from zero. Most of the frequencies I can eyeball, but I wanted to have some for of ...
- 779