# All Questions

2 views

### Abnormally Distributed Data - What Can I do with the z scores?

layman here... I have some sets of several hundred scores that are not distributed normally. I want to normalize these scores onto a percentile scale. The mean-median-mode for these look like this: ...
18 views

...
52 views

### Ways to simplify a neural network in R for interpretation

I would like to try to make some sense of a neural network. The neural network has a single hidden layer and is used on 30-40 attributes, which are used to classify the probability that the ...
47 views

### Calculating CIs for $\eta^2$ via Z scores - sample size?

In the thread Confidence Interval for $\eta^2$ it was proposed that if only limited statistics are available (in my case, F, df1, df2, means), one could calculate the 95% CI for $\eta^2$ by: ...
2 views

### How can I plot “intcox” interval cox proportional survival curve?

I'm doing survival analysis. I read intcox package manual but it doesn't provide R code for plotting survival curve. Anyone knows how to plot it? Many thanks ...
8 views

### In plain language, why is there no VIF for binary outcome regression models?

As far as I know, the variance inflation factor is not computed with pseudo-$R^{2}$ or generalized $R^{2}$ in binary outcome models (e.g. logistic regression). Are there other measures of ...
20 views

### Combining categorical and contuous variables to calculate a factor

I have income-categorical, area, no. of bed rooms and no of cars-continuous variables. How can I form a single factor from these variables? In other words I want to combine all these variables to get ...
54 views

### Panel data visualization

Does anyone know how to graph changes in a dependent variable over time, based on changes in several independent variables (i.e., graphs for panel data regressions)?
14 views

### HPD interval for the mean

Suppose we have iid observation with the following model $Y_t \sim \mathcal{N}(\mu,1/\mu) , t=1,2,..T$ The question is " Assuming a flat prior on $(0 ,\infty )$ find a 95% HPD interval for $\mu"$ ...
81 views

### Discriminative vs. Generative Models

This has be asked before, but I still have not grasped it completely. I know that generative models model the feature distribution and that this includes modelling the P(x|y) and P(y), which are not ...
39 views

### Does an exponential model fit my data?

I am measuring accumulation of a fluorescent-tagged protein at a particular location within a cell over time. In previous experiments that I have performed, I see a standard exponential distribution ...
547 views

### Estimate multinomial probit model with mlogit (R package)

From the document and help, probit model is supported by mlogit. But when I tried it with these R scripts, the estimation takes much longer time to run (than the logit verion) and the result is quite ...
237 views

### Confidence Interval in Monte Carlo integration

I want to integrate $\int_{\mathbb{R}_+}\mathbb{1}_A(x) d\mathbb{P}(x)$, in other words I am interested in $\mathbb{P}(A)$. I did this numerically with two Monte Carlo steps. First, I drew, say a ...
18 views

### ANOVA and Count/Percent Data

I'm just starting to work with some count data and I'm still trying to understand some of the complexities of it, so any help would be greatly appreciated. First the simple version, and then the ...
65 views
+100

### The meaning of “positive dependency” as a condition to use the usual method for FDR control

Benjamini and Hochberg developed the first (and still most widely used, I think) method for controlling the false discovery rate (FDR). I want to start with a bunch of P values, each for a different ...
3 views

### Create grid and color cells with average values of scatterplot using ggplot2 [migrated]

Given a numeric dataset $\{(x_i, y_i, z_i)\}_{i = 1}^N$, one can create a scatterplot by drawing a point $P_i = (x_i, y_i)$ for each $i = 1, \dots N$ and color each point with an intensity depending ...
3k views

### Libsvm one-class svm: how to consider all data to be in-class

I am using Libsvm for Matlab. I would like to construct the model for a full circumscription of all training data (in the higher SVM-space). For this I assume all my training data is correct and has ...
9 views

### about the definition of bayesian network

In this PDF http://people.csail.mit.edu/yks/documents/classes/mlbook/pdf/chapter2.pdf page 5 says: given a set of function $f(x_i,pa(x_i))$ non negative and sum to 1, we define a joint probability as ...
457 views

### Hidden Markov Model to predict the next state

I am learning to use HMM and I am trying to solve the following problem. There is a robot moving around the nodes in graph. The robot can move to adjacent nodes with certain probabilities. Each time ...
1k views

### If you run OLS regression on cross sectional data, should you test for autocorrelation in residuals?

I have a set of observations, independent of time. I am wondering whether I should run any autocorrelation tests? It seems to me that it makes no sense, since there's no time component in my data. ...
91 views

### How to calculate correlation between multiple?

I do have some 10-12 independent variable, I want to know how these 10-12 variables are related to a single variable(X). so I can decide that some variables are positively correlated to X and some are ...
2 views

### How can I counteract the effect of a degenerate classifier in an OVA Model?

Suppose I build a OVA classification model for classification with more than 2 possible classes (a model of sub-models, where each submodel predicts the probability of a data point belonging in a ...
10 views

### Usage of rma.glmm() for random effect single group study

Below are first few rows of my data that need to be analysed. I would need to understand the relation between number of years and survival rate. My statistical teacher advised on using rma.glmm() ...
4 views

### How to determine the demographics of consumers of a product, given demographic information and consumption rates

Okay, so I'm going to admit right away that I'm very new to statistical analysis, so please go easy on me. I have a 50+ different groups of people and for each group, I have the number of people ...
20 views

### Curve-fitting and variance-stabilizing transforms

I'm working with data corrupted by a mixed Poisson-Gaussian noise model (for example with images gathered in astronomy or electron microscopy), and have been using the generalized Anscombe ...
41 views

### Avoiding numerical error propagation

I simulated a MA(1) process $y_t = \mu + \epsilon_t + \theta \epsilon_{t-1}$ in R and tried to recover the error terms $\epsilon_t$ just by computing them successively (supposing I know the parameters ...
2k views

### Is Fisher's LSD as bad as they say it is?

When we perform experiments (on small sample sizes (usually the sample size per treatment group is about 7~8)) on two groups, we use a t-test to test for difference. However, when we perform an ANOVA ...
44 views

### PyMC – Calculate Evidence (Bayes' Theorem denominator)

I have one simple model and one more complex model (both are PDEs that describe displacement, when different kind of forces and thermal effects take place). From those, given some data, I calculate ...
104 views

### Weighted average of measurements with unequal errors

Suppose I have the numbers below. Consider them as results of some measurement. ...
24 views

### Why is Lasso regression for high dimensional data better than Stepwise AIC?

I know Lasso eventually set some parameters to zero, acting like variable selection. I also read from paper talking about automated variable selection method like Stepwise AIC can be troublesome. So ...
72 views

### How should I set up my data for classification when there is a time component?

I am assuming there is an optimal why to set up my data to achieve my goal of predicting who will retire next year. I can think of two methods. Which do you think is the most appropriate and why? Or ...
38 views

### Design of matrix of contrasts in R

I am doing some post-hoc comparisons (in lme4, but here I'll just present a simple linear model), and I am having a hard time making sure that I am building the right matrix of contrasts to test ...
11 views

### Multiple comparisons when investigating consistency of a device

I am trying to compare the performance of two devices designed to cool anaesthetised patients. This is a substudy analysis and the patient allocation between the groups is not even. Once cooling was ...
7 views

### How do you test for trend in a generalized linear model, particularly for odd numbers of levels of a variable?

I'm trying to test for a trend across levels of a categorical variable using a contrast statement in a model and can not figure out how to do it when there are odd numbers of levels. Looking at SAS's ...
6 views

### Conceptual questions on semi-blind identification

In performing system identification with a driven signal, we need to select the driving signal. In blind system identification, we only have the information about the observed signal. In semi-blind ...
62 views

### Recommendations for a short, yet theoretically rigorous, seminar or course on principal components analysis

I am looking for a short course (1 week or less) that teaches the theoretical and applied aspects of principal components analysis in a mathematically rigorous way. I've seen university courses on ...
102 views
+50

### Likelihood and estimates for mixed effects Logistic regression

First let's simulate some data for a logistic regression with fixed and random parts: ...
34 views

### Is my learnings in Machine Learning fine? [on hold]

I am a self learner in Machine Learning. I am from humanities background. My goal was to know bit of Machine Learning and to develop few applications based on few algorithms. To get feel of the ...
22 views

### statistical analysis with interval data

I have been assigned the task of helping someone who has collected some survey associated data related to some kind of government facilities provided to people. The data corresponding to small scale ...
67 views

### Sample size calculation for paired ordinal data

What formula should be used for sample size calculations for paired ordinal data? Imagine we have an intervention and we want to measure the effect based on before and after measurements of an ordered ...
485 views

### R: product of dummy variables when using factor in a linear regression

I have data in a dataframe with the following columns: date, time, symbol, price I am attempting to run the following model in R ...
69 views

### How does the inclusion of an intercept change the variability of the residual?

I want to use the variability of the residual as a measure M and then test whether M is higher or lower after some event. However, I estimate separate regression before and after the event to obtain ...
8 views

### Correlation or clustering of continuous score and discrete variable states

I have an experiment that produces a decimal score representing quality, and a bunch (5-30) of variables that each take on one of a set of discrete states. - The states are not meaningfully ...
17 views

### Average Growth Rate for Year 1 across 5 groups

I have a question that pertains to time series or more likely pertains just to simple math. Lets suppose that I am measuring the number of online visitors to 5 websites on a monthly basis, so I have ...
34 views
+50

### Linear post-treatment of nonlinear regression

I have often found in practice, using nonlinear regression techniques such as feedforward neural nets or random forests, that the resulting actual-vs-fitted plot (on training set) seems obviously ...
154 views

### Similarity between objects based on tags (binary features)

I have five millions of objects each of them having one or more tags. How do I compute statistically sound similarity score between each pair of the objects taking into account that: There are 100 ...
47 views
+100

### Library routine for rolling window lag 1 autocorrelation?

I am looking for a library routine that will calculate the lag 1 autocorrelation of a time series with a rolling window; meaning "slide a window of size N points along the time series, calculate the ...
34 views

### How to perform multilevel interaction [on hold]

Which model is correct and can be applied? $$Y=x_{1} + x_{2} + \left(x_{1}x_{2}2013\right)+ \varepsilon$$ $$Y=x_{1} + x_{2} + \left(x_{1}2013\right) + \left(x_{1}x_{2}2013\right)+ \varepsilon$$ ...