# All Questions

2 views

### Bayesian estimation of multivariate Gaussian from noisy observations with known error variances

I have a dataset $\mathbf{D} = \{ (\tau_i, \Gamma_i) : 1 \le i \le n \}$ of observations $\tau_i = X_i + \epsilon_i$ from a $p$-dimensional Gaussian $X_i \sim \mathcal{N}(\mu, \Sigma)$ contaminated by ...
8 views

### Categorize continuous data effectively (taking into account a response variable)

I wonder what are the better approaches to categorize continuous data (e.g. age) than dividing them with the use of quantiles and cut function (in ...
38 views

### Topic modeling for grouped data

Is there a variation of Latent Dirichlet Allocation (LDA) for grouped data? As an example, let us consider corpus of all Yahoo Q&A (where for simplicity we consider a question lumped together with ...
14 views

### Linearized exponential regression by lm() vs. non-linear nls() regression

Disclaimer I am new to this site, relatively new to R (two weeks of learning), have just a really basic knowledge in statistics so sorry if I'm doing a dumb mistake there or asking bad question or ...
11 views

### Analysing grade distributions: difference in proportions across years

I have data on grades obtained in a course over three consecutive years. The grade distribution includes the number of students who received a fail, pass, credit, distinction and high distinction. I ...
21 views

### Probability in hidden Markov model

I'm considering a Hidden Markov Model as follows: $X_{n+1} = F_n(X_n,\Theta)$ $Y_n = G_n(X_n,\Theta)$ where the $Y_n$ are the observations, the $X_n$ the hidden states and $\Theta$ the parameters. ...
54 views
+50

### Transition from pure mathematics to statistics via operational research

I am currently studying an M.Sc. in pure mathematics (my dissertation focuses on category/set theory)). However recently I have decided that I wish to transition to a much more applied field. During ...
60 views

### central limit theorem

can I do my statistics work based on the central limit theorem? I need to perform a t-test, ANOVA and multiple regression. my outcome variable is highly not normally distributed (Highly positively ...
6 views

### Help understand how to interpret a confidence interval - What happens when the sample mean is outside the interval?

I understand that the 95% confidence interval is supposed to tell us that if we were to repeatedly take samples (of the same size) from the population, and compute the interval in the same way, then ...
963 views

### Fast linear regression robust to outliers

I am dealing with linear data with outliers, some of which are at more the 5 standard deviations away from the estimated regression line. I'm looking for a linear regression technique that reduces the ...
3 views

### Fast diagonalization for coordinate descent

IN order to perform efficient coordinate descent, I am using SVD to diagonalize my system $$y=Ax$$ to $$y=USV^Tx\\ U^Ty=y_{new}=S(V^Tx)=Sx_{new}\\ y_{new}=Sx_{new}$$ where $S$ is the diagonal matrix ...
10 views

### Show that the Geometric Mean $g$ can be expressed as $\log g=a+\dfrac{k}{n}\sum_{i=1}^r f_i(i-1)$

In a frequency distribution the upper boundary of each class interval has a constant ratio to the lower boundary. Show that the Geometric Mean $g$ can be expressed as \log ...
3 views

### Why use D prime instead of proportion (or percentage) correct?

Apart from the fact that D prime is in Z units making it comparable regardless of the orginal units of measurement I can't see what the advantage in analysing D prime instead of proportion correct ...
69 views

### How to exclude interaction effect in linear regression modelling?

I am working with biological dataset. I am looking for significantly differentially expressed genes that stand out under effect of 3 different factors (e.x. gut microbiota level has/not, tissue, ...
15 views

### An econometrics question

(1) Conventional firms have code (or ID) numbers (given by FIRM) less than 100, while the code (or ID) numbers for worker cooperatives exceed 99. COOP = a dummy variable for a worker cooperative. ...
57 views

### How to use Viterbi algorithm for multiple chain HMM?

I need to build a HMM with two chains: the structure is as the figure below. How can I use the Viterbi algorithm for this model?
176 views

### Conservativeness of tests based on a discrete random variables

For discrete test statistics, the distribution of the corresponding $p$-value is discrete and stochastically larger than the uniform distribution. Hence the corresponding hypothesis test based on the ...
87 views

### Difference between using ARMA and GARCH to model volatility?

I just went through the following work http://epublications.bond.edu.au/cgi/viewcontent.cgi?article=1199&context=ijbf and was wondering what is the difference between model (9) on page 12 which ...
19 views

### What is the null hypothesis in the definition of Sensitivity and Specificity?

I have hard time to understand that what is the null hypothesis in the definition of Sensitivity and Specificity here? I'm student of Statistics and from my little knowledge in this field I can see ...
170 views

### Data split into training and test

I am implementing an EEG classifier with 15 subjects (patients), specifically a support vector machine classifier. I randomly choose the training and testing sets, but I was faced by a question "how ...
15 views

...
310 views

### Modelling a mixed model in JAGS/BUGS

I am currently in the process of implementing a model for soccer result prediction in JAGS. Actually, I have implemented several, but I have reached my most difficult challenge yet: A model described ...
12 views

...
171 views

### Bayesian and Fisher's approaches to linear discriminant analysis

I know 2 approaches to do LDA, the Bayesian approach and the Fisher's approach. Suppose we have the data $(x,y)$, where $x$ is the $p$-dimensional predictor and $y$ is the dependent variable of $K$ ...
10 views

### Calculating the expected number of zeros under Poisson and Negative Binomial distributions in R

I know the maths, but how to calculate the expected number of zeros under Poisson and Negative Binomial distributions in R? Is there any function to do so?
11 views

...
21 views

### Drop / collapse rows [migrated]

Say I have a dataset like the one below: x y z 1 9 3 3 0 3 4 0 2 1 4 1 5 1 9 4 2 3 9 0 1 I would like to to drop / collapse all the rows that are repeated based ...
6 views

### Clustering email with mixed types of attribute

I am looking to cluster thousands of emails in one's mailbox. Different from traditional analysis with emphasis on email body, the attachments will play a big role in my work. The data set contains ...
24 views

### Compare the variances of several groups seprately

I asked this question: Compare the variances of several groups The Levene's method only test all groups. How can I identify the groups that have a different variances?
378 views

### Unknown p-value calculation

I was recently debugging an R script and I found something very weird, the author defined their own p-value function ...
302 views

### A serious in-depth problem of probabilities for flipping coins

Lets say I'm doing 10,000 flips of a coin. I would like to know the probability of how many flips it takes to get, 4 or more consecutive heads in a row. The count would work as the following, you ...
27 views

### Questions on bag of features methods

I've been studying the bag of features (BoF) methods. I understand the basics and how it's done but I think I still am not good at the details. So here's my understanding of such methods: Suppose you ...
18 views

### Multiple covariates in an ANCOVA design

I have found a group difference (3 groups) for my dependent variable and now I am supposed to find out if possible covariates influence this finding. What is the best way to "correct" for 5 ...
20 views

### Warning message in auto.arima

I am using auto.arima() for prediction, and getting the following warning message. I want to know if I can ignore this warning message or if I should be worried. ...
13 views

### Chances of someone finding your lost phone and/or authentication device and having the same fingerprint

I was going through the FAQ page of the FIDO alliance, a consortium to implement a next generation authentication system. To answer to the question, How secure are fingerprints? They have said, ...
30 views

### Example of CLT when moments do not exist

Consider $X_n = \begin{cases} 1 & w.p (1 - 2^{-n})/2\\ -1 &w.p~ (1 - 2^{-n})/2\\ 2^{k} &w.p~ 2^{-k} \text{ for } k > n\\ \end{cases}$ I need to show that even though this has ...
46 views

### How to calculate correlation between multiple?

I do have some 10-12 independent variable, I want to know how these 10-12 variables are related to a single variable(X). so I can decide that some variables are positively correlated to X and some are ...
17 views

### Expectation / minimizing variance of weighted sum of means

(I'm assuming the second $\bar x$ should be a $\bar y$), but I'm mostly confused how to solve this problem because it seems like since $\bar x$ and $\bar y$ are values, not random variables, $W$ is ...
168 views

### computing slope on heavy tailed lognormal distribution

I have a heavy tailed logNormal distribution and i want to know if it's possible to make a linear regression in order to compute R² and slope for this data. I make my linear regression on computed ...
18 views

### How to compute for Bivariate Logistic Distribution

This is the logistic distribution of single random variable (taken from Wikipedia). $x$ = random variable $\mu$ = mean of all random variables $s$ = variance. Now, I want to do a Bivariate ...
113 views

### Understanding Differences of Mean between Two Groups

I have a variable test score that is measured for an organization consisting of 2050 people. The goal is to test if men outperform women (or vice versa). ...
16 views

### Cannot replicate odds ratio

I am trying to replicate some odds ratio calculations which are reported in this paper. For example in the paper on page 3 in the Strome et al. data for tonsil: it states there are 52 cases and 48 ...
40 views

### Time series prediction: visualising path uncertainty region

I am predicting a time series' future evolution and am evaluating the path uncertainty using bootstrapping. Is there a good way to visualise the uncertainty that goes beyond simply plotting a pair of ...
144 views

### What's the best way to visualize the effects of categories & their prevalence in logistic regression?

I need to present information about the main predictors of a candidate's votes using a public opinion survey data. I have run a logistic regression using all the variables that I care about, but I ...
123 views

### Finding events that occur together

I'm interested in computing hashtag correlation; that is, given one hashtag, which other hashtags are likely? I know how to do correlation when the data is in $(x, y)$ format, but this is in ...
23 views

### Standardising the weights generated from feedforward back propagation NN

I have binary data input to NN. The weights generated by NN are not normalized, i.e., the weights are not the same for every run of the algorithm. How to standardize these weights?
38 views

### Run time analysis of the clustering algorithm (k-means)

I was reading some notes on ML and clustering and it claimed that the run time of clustering was O(kn) where k is the number of clusters and n is the number of points. I was wondering why this was ...
24 views

### Good algorithms for feature extraction from images?

I am searching for some algorithms for feature extraction from images which I want to classify using machine learning . I have heard only about scale-invariant feature transform (SIFT), I have images ...