0
votes
0answers
2 views

Abnormally Distributed Data - What Can I do with the z scores?

layman here... I have some sets of several hundred scores that are not distributed normally. I want to normalize these scores onto a percentile scale. The mean-median-mode for these look like this: ...
2
votes
1answer
18 views
0
votes
1answer
52 views

Ways to simplify a neural network in R for interpretation

I would like to try to make some sense of a neural network. The neural network has a single hidden layer and is used on 30-40 attributes, which are used to classify the probability that the ...
3
votes
1answer
47 views

Calculating CIs for $\eta^2$ via Z scores - sample size?

In the thread Confidence Interval for $\eta^2$ it was proposed that if only limited statistics are available (in my case, F, df1, df2, means), one could calculate the 95% CI for $\eta^2$ by: ...
0
votes
0answers
2 views

How can I plot “intcox” interval cox proportional survival curve?

I'm doing survival analysis. I read intcox package manual but it doesn't provide R code for plotting survival curve. Anyone knows how to plot it? Many thanks ...
1
vote
0answers
8 views

In plain language, why is there no VIF for binary outcome regression models?

As far as I know, the variance inflation factor is not computed with pseudo-$R^{2}$ or generalized $R^{2}$ in binary outcome models (e.g. logistic regression). Are there other measures of ...
0
votes
2answers
20 views

Combining categorical and contuous variables to calculate a factor

I have income-categorical, area, no. of bed rooms and no of cars-continuous variables. How can I form a single factor from these variables? In other words I want to combine all these variables to get ...
2
votes
1answer
54 views

Panel data visualization

Does anyone know how to graph changes in a dependent variable over time, based on changes in several independent variables (i.e., graphs for panel data regressions)?
3
votes
0answers
14 views

HPD interval for the mean

Suppose we have iid observation with the following model $ Y_t \sim \mathcal{N}(\mu,1/\mu) , t=1,2,..T$ The question is " Assuming a flat prior on $(0 ,\infty )$ find a 95% HPD interval for $\mu"$ ...
1
vote
1answer
81 views

Discriminative vs. Generative Models

This has be asked before, but I still have not grasped it completely. I know that generative models model the feature distribution and that this includes modelling the P(x|y) and P(y), which are not ...
4
votes
1answer
39 views

Does an exponential model fit my data?

I am measuring accumulation of a fluorescent-tagged protein at a particular location within a cell over time. In previous experiments that I have performed, I see a standard exponential distribution ...
4
votes
2answers
547 views

Estimate multinomial probit model with mlogit (R package)

From the document and help, probit model is supported by mlogit. But when I tried it with these R scripts, the estimation takes much longer time to run (than the logit verion) and the result is quite ...
1
vote
1answer
237 views

Confidence Interval in Monte Carlo integration

I want to integrate $\int_{\mathbb{R}_+}\mathbb{1}_A(x) d\mathbb{P}(x)$, in other words I am interested in $\mathbb{P}(A)$. I did this numerically with two Monte Carlo steps. First, I drew, say a ...
2
votes
1answer
18 views

ANOVA and Count/Percent Data

I'm just starting to work with some count data and I'm still trying to understand some of the complexities of it, so any help would be greatly appreciated. First the simple version, and then the ...
6
votes
2answers
65 views
+100

The meaning of “positive dependency” as a condition to use the usual method for FDR control

Benjamini and Hochberg developed the first (and still most widely used, I think) method for controlling the false discovery rate (FDR). I want to start with a bunch of P values, each for a different ...
0
votes
0answers
3 views

Create grid and color cells with average values of scatterplot using ggplot2 [migrated]

Given a numeric dataset $\{(x_i, y_i, z_i)\}_{i = 1}^N$, one can create a scatterplot by drawing a point $P_i = (x_i, y_i)$ for each $i = 1, \dots N$ and color each point with an intensity depending ...
1
vote
1answer
3k views

Libsvm one-class svm: how to consider all data to be in-class

I am using Libsvm for Matlab. I would like to construct the model for a full circumscription of all training data (in the higher SVM-space). For this I assume all my training data is correct and has ...
0
votes
0answers
9 views

about the definition of bayesian network

In this PDF http://people.csail.mit.edu/yks/documents/classes/mlbook/pdf/chapter2.pdf page 5 says: given a set of function $f(x_i,pa(x_i))$ non negative and sum to 1, we define a joint probability as ...
1
vote
1answer
457 views

Hidden Markov Model to predict the next state

I am learning to use HMM and I am trying to solve the following problem. There is a robot moving around the nodes in graph. The robot can move to adjacent nodes with certain probabilities. Each time ...
6
votes
1answer
1k views

If you run OLS regression on cross sectional data, should you test for autocorrelation in residuals?

I have a set of observations, independent of time. I am wondering whether I should run any autocorrelation tests? It seems to me that it makes no sense, since there's no time component in my data. ...
1
vote
2answers
91 views

How to calculate correlation between multiple?

I do have some 10-12 independent variable, I want to know how these 10-12 variables are related to a single variable(X). so I can decide that some variables are positively correlated to X and some are ...
0
votes
0answers
2 views

How can I counteract the effect of a degenerate classifier in an OVA Model?

Suppose I build a OVA classification model for classification with more than 2 possible classes (a model of sub-models, where each submodel predicts the probability of a data point belonging in a ...
0
votes
0answers
10 views

Usage of rma.glmm() for random effect single group study

Below are first few rows of my data that need to be analysed. I would need to understand the relation between number of years and survival rate. My statistical teacher advised on using rma.glmm() ...
0
votes
0answers
4 views

How to determine the demographics of consumers of a product, given demographic information and consumption rates

Okay, so I'm going to admit right away that I'm very new to statistical analysis, so please go easy on me. I have a 50+ different groups of people and for each group, I have the number of people ...
3
votes
0answers
20 views

Curve-fitting and variance-stabilizing transforms

I'm working with data corrupted by a mixed Poisson-Gaussian noise model (for example with images gathered in astronomy or electron microscopy), and have been using the generalized Anscombe ...
5
votes
2answers
41 views

Avoiding numerical error propagation

I simulated a MA(1) process $y_t = \mu + \epsilon_t + \theta \epsilon_{t-1}$ in R and tried to recover the error terms $\epsilon_t$ just by computing them successively (supposing I know the parameters ...
9
votes
3answers
2k views

Is Fisher's LSD as bad as they say it is?

When we perform experiments (on small sample sizes (usually the sample size per treatment group is about 7~8)) on two groups, we use a t-test to test for difference. However, when we perform an ANOVA ...
1
vote
1answer
44 views

PyMC – Calculate Evidence (Bayes' Theorem denominator)

I have one simple model and one more complex model (both are PDEs that describe displacement, when different kind of forces and thermal effects take place). From those, given some data, I calculate ...
1
vote
1answer
104 views

Weighted average of measurements with unequal errors

Suppose I have the numbers below. Consider them as results of some measurement. ...
0
votes
0answers
24 views

Why is Lasso regression for high dimensional data better than Stepwise AIC?

I know Lasso eventually set some parameters to zero, acting like variable selection. I also read from paper talking about automated variable selection method like Stepwise AIC can be troublesome. So ...
0
votes
1answer
72 views

How should I set up my data for classification when there is a time component?

I am assuming there is an optimal why to set up my data to achieve my goal of predicting who will retire next year. I can think of two methods. Which do you think is the most appropriate and why? Or ...
2
votes
1answer
38 views

Design of matrix of contrasts in R

I am doing some post-hoc comparisons (in lme4, but here I'll just present a simple linear model), and I am having a hard time making sure that I am building the right matrix of contrasts to test ...
3
votes
2answers
11 views

Multiple comparisons when investigating consistency of a device

I am trying to compare the performance of two devices designed to cool anaesthetised patients. This is a substudy analysis and the patient allocation between the groups is not even. Once cooling was ...
1
vote
0answers
7 views

How do you test for trend in a generalized linear model, particularly for odd numbers of levels of a variable?

I'm trying to test for a trend across levels of a categorical variable using a contrast statement in a model and can not figure out how to do it when there are odd numbers of levels. Looking at SAS's ...
0
votes
0answers
6 views

Conceptual questions on semi-blind identification

In performing system identification with a driven signal, we need to select the driving signal. In blind system identification, we only have the information about the observed signal. In semi-blind ...
4
votes
1answer
62 views

Recommendations for a short, yet theoretically rigorous, seminar or course on principal components analysis

I am looking for a short course (1 week or less) that teaches the theoretical and applied aspects of principal components analysis in a mathematically rigorous way. I've seen university courses on ...
5
votes
0answers
102 views
+50

Likelihood and estimates for mixed effects Logistic regression

First let's simulate some data for a logistic regression with fixed and random parts: ...
0
votes
0answers
34 views

Is my learnings in Machine Learning fine? [on hold]

I am a self learner in Machine Learning. I am from humanities background. My goal was to know bit of Machine Learning and to develop few applications based on few algorithms. To get feel of the ...
0
votes
1answer
22 views

statistical analysis with interval data

I have been assigned the task of helping someone who has collected some survey associated data related to some kind of government facilities provided to people. The data corresponding to small scale ...
6
votes
1answer
67 views

Sample size calculation for paired ordinal data

What formula should be used for sample size calculations for paired ordinal data? Imagine we have an intervention and we want to measure the effect based on before and after measurements of an ordered ...
1
vote
2answers
485 views

R: product of dummy variables when using factor in a linear regression

I have data in a dataframe with the following columns: date, time, symbol, price I am attempting to run the following model in R ...
1
vote
3answers
69 views

How does the inclusion of an intercept change the variability of the residual?

I want to use the variability of the residual as a measure M and then test whether M is higher or lower after some event. However, I estimate separate regression before and after the event to obtain ...
0
votes
0answers
8 views

Correlation or clustering of continuous score and discrete variable states

I have an experiment that produces a decimal score representing quality, and a bunch (5-30) of variables that each take on one of a set of discrete states. - The states are not meaningfully ...
0
votes
0answers
17 views

Average Growth Rate for Year 1 across 5 groups

I have a question that pertains to time series or more likely pertains just to simple math. Lets suppose that I am measuring the number of online visitors to 5 websites on a monthly basis, so I have ...
3
votes
0answers
34 views
+50

Linear post-treatment of nonlinear regression

I have often found in practice, using nonlinear regression techniques such as feedforward neural nets or random forests, that the resulting actual-vs-fitted plot (on training set) seems obviously ...
1
vote
2answers
154 views

Similarity between objects based on tags (binary features)

I have five millions of objects each of them having one or more tags. How do I compute statistically sound similarity score between each pair of the objects taking into account that: There are 100 ...
0
votes
2answers
47 views
+100

Library routine for rolling window lag 1 autocorrelation?

I am looking for a library routine that will calculate the lag 1 autocorrelation of a time series with a rolling window; meaning "slide a window of size N points along the time series, calculate the ...
0
votes
0answers
34 views

How to perform multilevel interaction [on hold]

Which model is correct and can be applied? $$Y=x_{1} + x_{2} + \left(x_{1}x_{2}2013\right)+ \varepsilon$$ $$Y=x_{1} + x_{2} + \left(x_{1}2013\right) + \left(x_{1}x_{2}2013\right)+ \varepsilon$$ ...
0
votes
0answers
11 views

How can I determine local minima from a Kernel Density Estimation?

I have a fairly large 1-dimensional array that I am trying to cluster. I came across several other questions on this site where the top answer is to use a Kernel Density Estimation and then locate ...
4
votes
2answers
458 views

15 30 50 per page