# All Questions

2answers
12 views

### Error bars using median absolute deviation

I have tried to find a solution to my question on Google, but I can’t seem to find much information about error bars and median absolute deviation and I do not know all too much about statistical ...
0answers
28 views

### Multiple imputation for missing values

I would like to use imputation for replacing missing values in my data set. I have some constraints , for example I dont not want imputed variable x1 be less than ...
1answer
23 views

### Training a convolutional neural network

Based on my research on convolution neural networks, every other layer in such a network has a subsampling operation, in which the resolution of the image is reduced so as to improve generalization of ...
0answers
4 views

### Finding statistical significance between two binary tests

I have two sets of data. One with a sample size of 82 with 53 "hits" and 29 "Misses." And a second sample of 105 with 67 "hits" and 38 "misses" Given that the second set of data is a control, is ...
1answer
44 views

### a guide to using the phi coefficient

This one is straightforward. I am trying to find a good documentation of the phi correlation. There is a Chi-Square Statistic to test the significance of the the correlation coefficient, I am hoping ...
1answer
19 views

### Approximate function observed through noise

I have a function $f : [a,b] \rightarrow \mathbb{R}$ that I can observe through some noise, i.e. I can only directly measure $f(x) + \eta$ where $\eta$ is some random noise with mean 0. I have a ...
1answer
9 views

### classification performance and the feature set selection

I am now working on a classification problem. The generated feature set can be separated into two group. I did a comparison study: use all of the features; use the features of group 1 only; and use ...
1answer
48 views

### Calculate variance explained by each predictor in multiple regression using R

I have run a multiple regression in which the model as a whole is significant and explains about 13% of the variance. However, I need to find the amount of variance explained by each significant ...
0answers
41 views

### How to convert LS means into arithmetic means after SAS PROC GLIMMIX?

SAS PROC GLIMMIX output contains series of statistics for LS Means: . How to convert given LS means, their differences and other statistics into arithmetic mean, their differences and other ...
1answer
158 views

### Comparison between MDL and BIC

I'm currently studying Hidden Markov Models. There's a set of observations from which I need to determine the optimal number of states. After having found the maximum likelihood using Baum-Welch, I ...
2answers
93 views

### K-means as limit of Soft K-means algorithm

I encountered the following exercise Show that as the stiffness $\beta$ goes to $\infty$, the soft $K$-means algorithm becomes identical to the original hard $K$-means algorithm, except for the ...
1answer
52 views

### Random effect on the intercept in models with categorical predictor

I'm doing a mixed model analysis of a model with a categorical predictor (using glmer, binomial model). To my surprise, the variance I obtain for the intercept does not change when I change the ...
0answers
4 views

### getting the variance of a set composed of a group of subsets

I have a data set $\mathcal{S}$, which is composed of several subset as $\mathcal{S}=\{S_1,...S_i,...S_n\}$ Each $S_i$ has its own mean $\mu_i$, variance $\sigma_i$, and size $\|S_i\|$. Is there a ...
1answer
69 views

### Using Random Number Table

In a random number table , the random numbers are given as first thousand, second thousand, third thousand, and so on. Every thousand has 25 rows and 1 to 4, 5 to 8 up to 37 to 40 columns. Suppose ...
1answer
14 views

### Good book for Statistical and Probability Theory

I have taken one introductory statistics class and am finishing up an econometrics class, but these classes are more about applications and just applying formulas without any explanations as to how ...
0answers
6 views

### Getting predictions after rfImpute

I'm doing some modelling using package randomForest. The rfImpute function is very nice for handling missing values when fitting the model. However, is there a way ...
0answers
7 views

### Are there other ways to represent time series part-to-whole data in a chart besides an area chart?

Are there other ways to represent time series part-to-whole data instead of using an area chart like the one in the example below? I want to highlight changes in population year to year. And I have ...
1answer
194 views

### Assumption of Gaussian distribution of acceleration

I have a data set consisting of noisy position values of a trajectory of a human hand. I want to estimate a generative model of these trajectories, and the obvious choice is a Kalman Filter/linear ...
3answers
48 views

### Why does statistical significance increase with data, BUT the effects may not be meaningful?

I have read that "When you get more and more data, you can find statistically significant differences wherever you look" My question is: Why is this case? (any intuitive examples that show this ...
1answer
24 views

### Test for Statistical Significance in the Accuracy of a Machine Learning System

I have what I imagine is an elementary question about evaluating statistical significance, but while I know a lot about probability I can't t-test my way out of a paper bag. From here I'm hoping to ...
1answer
52 views

### Regression analysis in R using text field?

I'm working in R. I'd like to run a regression analysis for predicting price against terms in a text field. I have a dataset of jewellery auction listings, with price paid, date, and an unstructured ...
0answers
40 views

### Poisson Approximation to Binomial

I have a Poisson approximation to binomial question, posted below. I'm not too sure if I'm using the proper formula: $$P(x) = e^{-np}(np)^x/x!$$ . Of anyone can tell me if I'm doing this ...
2answers
82 views

### How exactly is sparse PCA better than PCA?

I learnt about PCA a few lectures ago in class and by digging more about this fascinating concept, I got to know about sparse PCA. I wanted to ask, if I'm not wrong this is what sparse PCA is: In ...
0answers
11 views

### Using the sample weights in regression

I have dataset A where households are interviewed each quarter for one year. I am focussing only on the households who have data for all quarters. With this imposition, observations are reduced by ...
1answer
164 views

### What are finite window effects?

I'm reading a paper that uses a Poisson process to model real world events. The authors mention "finite window effects". What are finite window effects? Here is quote from the paper where the ...
2answers
52 views

### Maximum number of independent variables in Logistic Regression

Is there a measure in logistic regression that maybe penalizes you for having too many independent variables like in multiple regression with the adjusted R squared? That is, does having too many ...
0answers
31 views

### A measure of distance that accounts for probabilities

I have a couple of vectors (each vector represents a student, each coordinate, a question) that I want to compare, to say if they are similar or not. I am using the cosine distance. I would like to ...
0answers
51 views
+50

### Correlation between two Decks of cards?

I've written a program to simulate an overhand card shuffle. Each card is numbered, with suit going from CLUBS, DIAMONDS, HEARTS, SPADES and the rank from Two ...
1answer
272 views

### Unequal Sample Sizes for Two-way Repeated Measures

I have 3 different treatments with the observations of 6 different repetitions. However, all these different treatments (groups) have different numbers of patients. The first group has 26 patients, ...
0answers
29 views

### one sample t-test: result is 'infinity' - what do I report?

I have been comparing the outcomes from a number of groups to an expected value. The outcomes are a simple scale with three possible options: -1, 0, +1 The expected value is the '0' and one of my ...
2answers
50 views

### How can I choose the 10 variables that explain the most variation in a wealth index?

I have household survey data with 32 questions about assets the household has or doesn't. I assume that taken together the answers to these asset questions (e.g. how many televisions does the ...
1answer
31 views

### standard error of the estimated p-values from simulations

This might be a question in general: due to computational burden, I have to use a subset of my complete data (say, 1,000 out of the complete 10,000 observations) to get a p-value of a test. The test ...
2answers
27 views

### Repeated Measures GLM / restricted permutations

I have a question regarding repeated measures and GLMs: Suppose i have counted the abundance of some species in lakes at different time points - every lake received a different treatment at time = 0. ...
1answer
61 views

### Best way to visualize predictions from a linear model

Let's say I'm doing some predictive analytics and am trying to predict US GDP per month using a two or three month lag. After every month, I generate new predictions and am able to compare my ...
0answers
18 views

### Does ARIMA require normally distributed errors or normally distributed input data?

I have two questions related to time series forecasting with ARIMA: Does ARIMA require normally distributed errors or normally distributed input data ? Are there any assumptions on input time series ...
2answers
40 views

### Designing a probability function with a desired kurtosis and skewness

Is it possible to design a probability function P(x) such that the distribution will have a specified kurtosis, K, and skewness, S? I am not otherwise interested in any other properties of this ...
2answers
84 views

### Incorporating averaging models from AIC and still using k-fold cross validation?

I've a county/district that I've divided into ~300 grids that are 15km^2 in size attributed with various habitat and economic variables that have been summarized and standardized. I then have 2 types ...
0answers
7 views

### Multilevel exploratory factor analysis

We are creating a scale to measure the 'strength of tie' between two people. We conducted a survey where each person was asked to name a number of other people they know. They were then asked to ...
0answers
63 views

### Is it possible to test for a Gaussian distribution?

Suppose that I have $n$ independent samples from a probability distribution. I believe that the distribution is Gaussian ($G$) but I don't know; then my question is: does there exist a test such that ...
2answers
59 views

### Trade-off between using explicit inputs and using fewer inputs for neural networks

Suppose you have 2 distinct features x1 and x2 with discrete values between 1 and 10. When building a neural network, I have 2 options: Create a new feature x3 = 10*x1 + x2 and use this as the only ...
1answer
55 views

### How to calculate a mean and standard deviation for a lognormal distribution using 2 percentiles

I am trying to calculate a mean and standard deviation from 2 percentiles for a lognormal distribution. I was successful in performing the calculation for a normal distribution using ...
0answers
23 views

### Weighted least squares for energy data

I am trying to do a weighted multivariate least squares regression on utility data. Basically I have utility bills where I use : Days billed, Consumption. For the same billing period I calculate ...
1answer
20 views

### Mixed Effects Model with Nesting

I have data collected from an experiment organized as follows: Two sites, each with 30 trees. 15 are treated, 15 are control at each site. From each tree, we sample three pieces of the stem, and ...
0answers
16 views

### Time series (stochastic process) estimating parameters using characteristic function

I have a time series of assets ${A_1, A_2, ..., A_n}$, which is described by a sophisticated distribution having the following characteristic function: $\phi(u; t;\theta)$, where $\theta$ is a vector ...
1answer
199 views

### Survival Analysis data preparation for time dependent covariates in Stata

I am trying to do a survival analysis with time dependent covariates in Stata. The time dependent covariates change at very specific times. I do not know how to do the stset in this case (spli. I ...
0answers
19 views

### Evaluating correlation: which test to choose

I want to evaluate the correlation between two samples of data (300 samples, 2 parameters). I've done some researches and I reached these conclusions (correct me if I'm wrong): Pearson's correlation ...
0answers
15 views

### Gibbs sampling for LDA — does a small Dirichlet concentration parameter make a difference?

I'm using a Gibbs sampler for Latent Dirichlet allocation as described by Griffiths and Steyvers (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC387300/). The sampling of a new topic $j$ for word $i$ is ...
2answers
453 views

### Binomial GLM and different sample sizes

I have a data set which consists of binomial proportions, let's say the success rate of converting a customer depending on the advertisement, the customer age, and various other factors. For some ...
0answers
15 views

### Model checking and generalized multilinear regression

In my research I am analyzing the quality of game player softwares by making them play multiple times against a certain opponent and counting the number of victories. It seems to be very clear that ...
2answers
158 views

### Markov chain separation element and message

From Bishop's book I know an observed variable in a Markov Chain does work as d-separation element. What I am more uncertain about is what happens to the message itself. In the sum-product algorithm, ...

15 30 50 per page