0
votes
1answer
20 views

Mulitivariable regression?

For my research project on the effect of a specific kind of surgery on clinical outcome, I want to study the correlation between the improvement in clinical outcome scores (ie the difference between ...
0
votes
1answer
11 views

Log transformation for logistic regression

Is it necessary to log transform non-normally distributed variables to perform logistic regression? If so, when is it appropriate?
1
vote
0answers
17 views

Fit and evaluate a second order transition matrix (Markov Process) in R?

I already built one first order discrete state Markov Chain model. It was built with R using the function 'markovchainFit()' in ...
1
vote
1answer
14 views

mixed models: wrong random intercepts and interpretation issues

I have a dataset of individues that participated in several races, but not all individuals did every race. Of every indivual I have their average speed. My goal is to have a measure of how difficult ...
0
votes
0answers
2 views

Multiple logistic regression models and survey data

I have a set of survey data that consists of answers to 10 knowledge questions along with demographic information about the respondent. Our question of interest is to identify which demographics are ...
2
votes
2answers
150 views

Gradient boosting algorithm (steps) question

So, far I have read following regarding boosting: Boosting is an ensemble technique. Train learner sequentially, where early learners fit simple models to the data. Analyze data for errors, that ...
0
votes
0answers
4 views

Intercept, deterministic and stochastic cointegration

I have reading on the Johansen co integration model. I am using the jci test function of MATLAB. I have some a little trouble selecting the 'model.' can somebody explain in layman terms which and what ...
0
votes
0answers
11 views

Efficient/fast Mahalanobis distance computation

Suppose I have $n$ data points $x_1,\dots,x_n$, each of which is $p$-dimensional. Let $\Sigma$ be the (non-singular) population covariance of these samples. With respect to $\Sigma$, what is the most ...
1
vote
1answer
20 views

Random Forests overfitting/unbalanced classes?

Suppose I am using random forests where the classes are highly unbalanced. How do you detect over fitting and what can you do to avoid it? Breiman says in his paper that random forests do not overfit, ...
0
votes
0answers
3 views

Duplicate AICc values for multiple models with interactions

I am going through a model selection process with a mixed-model with 3 variables: A, B, and C. B and C are orthogonal factors. B or C may interact with A, so my full model would be: fixed: ...
2
votes
1answer
70 views

Simplifying P(x|y,z) to P(x|z)

I am looking for the probability of an outcome (x in Win, y in Opponent Actions and z in Opponent Cards) in a poker scenario. I need a mathematical proof to simplify $\Pr (x \mid y, z)$ to $\Pr (x ...
3
votes
1answer
1k views

Using BIC to estimate the number of k in KMEANS

I am currently trying to compute the BIC for my toy data set (ofc iris (: ). I want to reproduce the results as shown here (Fig. 5). That paper is also my source for the BIC formulas. I have 2 ...
5
votes
1answer
232 views

Difference of Prediction between Graph Representation and Data Matrix Representation

In data mining, data can be usually represented in different forms such as records of a matrix, graphs or ordered data. While we find in research different papers addressing methods or solutions for ...
-1
votes
1answer
19 views

Example Bayesian resolution of the Two Envelopes Problem

What is a concrete example of a Bayesian resolution to the Two Envelopes Problem?
0
votes
0answers
7 views

Scale-invariant feature transform explanation

How do I explain the scale-invariant feature transform (SIFT) to a layman?
0
votes
1answer
113 views

How to determine if survey response samples are comparable?

I'm looking for a method to determine if the number of survey response received from a regional office are substantial enough to compare to each other year over year. For example, one year we might ...
5
votes
0answers
28 views

Textbooks on Matrix Calculus?

See this question on Math SE. Short story: I read The Elements of Statistical Learning and got frustrated when I was trying to verify some of the results, e.g., given $$\text{RSS}(\beta) = ...
1
vote
0answers
9 views

Overtraining Neural Networks?

I am currently exploring the training of Neural Networks. I have some toy data and I've trained a NN with 2 hidden layers on it and I get 99 % accuracy on the test set. But the problem is that if I ...
1
vote
0answers
3 views

Error propagation and Standard Deviation

I never quite got the hang of this during my entry stat course and it has been bugging me for a long time now. Lets assume I'm trying to find the focal length of a concave lens. Using a mockup ...
2
votes
1answer
23 views

R Own Implementation of Cross Correlation using Convolution

I try to implement my own cross correlation function in R by translating it as a convolution problem. Part I: So I have two arrays, e.g. two identical arrays, and I want to get the cross correlation ...
6
votes
1answer
109 views

$E[e^{cX}]$ where $c < 0$ and $X$ is lognormally distributed

I am trying to calculate the expectation $$E[e^{cX}]$$ for arbitrary $c<0$ (for $c>0$ the expectation is infinite) if $X$ is lognormally distributed, i.e. $\log(X) \sim N(\mu, \sigma)$. My idea ...
7
votes
2answers
290 views

whether to rescale indicator / binary / dummy predictors for LASSO

For the LASSO (and other model selecting procedures) it is crucial to rescale the predictors. The general recommendation I follow is simply to use a 0 mean, 1 standard deviation normalization for ...
1
vote
0answers
7 views
1
vote
1answer
108 views

Similarity between different length vectors containing related items

I have a vector (V1) with which I need to calculate the similarity of other vectors (ex V2,V3 ... ) which may be of different lengths. The different angle here is that the elements inside the vectors ...
3
votes
0answers
42 views

simulating birth death process with random numbers from negative binomial

I am trying to generate random deviates for the population size at time $t$ for a birth-death process with constant birth and death rates per individual and initial size $N_0 \gt 0$. For the simple ...
0
votes
0answers
5 views

General advice on modeling clustered data

I have biomass data from ~100 sites which I wish to relate to site environmental characteristics, and then based on those relationships predict biomass (gm2) across synopic layers of surrogate data in ...
0
votes
0answers
23 views

Body Analysing Scales - the maths? [on hold]

Wondering if anyone has bothered to gather the data and do the maths necessary to determine which of the common: Total body Water / Fat / Lean Mass regression equations are lurking in their ...
0
votes
1answer
33 views

Correlation between spot and futures

I am Airliner.I want to protect my business from price volatility of jet fuel cost.Jet fuel is not traded in futures market but Crude oil is traded in futures market. I have daily spot prices of jet ...
0
votes
0answers
3 views

Scaling before class rebalancing or after?

I'm training an SVM on my train data, which I'm then trying on my test data to find the accuracy of the algorithm. Would it normally be best to rebalance the class imbalance (2 classes with 60/40 ...
1
vote
1answer
11 views

Clustering data points based on edge strength

I'm looking at a Computer Vision application where I try to analyze the strength of edges a certain set of colors make with another color. For, this I take images of two colors falling on top of each ...
37
votes
13answers
8k views

Expected number of ratio of girls vs boys birth

I have came across a question in job interview aptitude test for critical thinking. It is goes something like this: The Zorganian Republic has some very strange customs. Couples only wish to ...
0
votes
0answers
9 views

How to calculate linearized rate of postoperative complications in % patient/year in r [on hold]

Linearized rate is a method summarise constant hazard function in a very simple way and defined as: total number of observed events divided by total patients-year (person-year). These rates should be ...
3
votes
0answers
12 views

Why is stl function giving significant seasonal variation with random data

I plotted with following code with stl (Seasonal Decomposition of Time Series by Loess) function: plot(stl(ts(rnorm(144), frequency=12), s.window="periodic")) ...
1
vote
0answers
17 views

Bayesian Analysis of Box-Cox Transformation

This problem is problem 5 in Chapter 7 of Bayesian Data Analysis, 3rd edition. Consider the Box-Cox transformation: $y_i^{(\lambda)} \sim \mathcal{N}(\mu, \sigma^2)$ where $y_i^{(\lambda)} = ...
0
votes
0answers
23 views

How to change the null hypothesis of the coefficient in the least squares fitting?

I have a least square fitting like this: fit = lsfit(log10(M), log10(RS), wt) This function lists statistics and p-values for the coefficient considering the ...
0
votes
0answers
17 views

markov chain simulation [on hold]

I keep getting the error when I try to run this simulation could somebody tell me what I'm possibly doing wrong? ...
6
votes
3answers
105 views

ETS() function, how to avoid forecast not in line with historical data?

I am working on an alogorithm in R to automatize a monthly forecast calculation. I am using, among others, the ets() function from the forecast package to calculate forecast. It is working very well. ...
1
vote
1answer
376 views

Can I use all pairwise comparisons at a factor level to perform correlation analysis?

I have 4 data sets, each with thousands of records/instances. There are 2 factors. One factor has 3 instances at each factor level (there are thousands of levels in the data set). The other factor ...
0
votes
0answers
6 views

Correction for multiple comparisons repeated measures MANOVA [on hold]

I am running a repeated measures MANOVA and wanted to know the best way to correct for multiple comparisons. If I use Bonferroni, for example, how would I go about doing that within SPSS?
3
votes
1answer
21 views

Bayesian Probabilistic Matrix Factorization (BPMF) with PyMC3: PositiveDefiniteError using `NUTS`

I've implemented the Bayesian Probabilistic Matrix Factorization algorithm using pymc3 in Python. I also implemented it's precursor, Probabilistic Matrix ...
0
votes
0answers
6 views

Class weights in unbalanced SVM classification

The answer to this question says that class weights for unbalanced SVM classification can be picked so that that sums of the weights for each class are equal. Should this be done before ...
0
votes
0answers
7 views

I observed very different feature scoring from two different classifiers. What does it really mean?

Here what I've done. Given the dataset, I run a Random Forests and Logistic Regression with 5 Fold Stratified Data Sampling. Then I plot the feature importance for Random Forests and Logistic ...
4
votes
1answer
819 views

Interpreting odds ratios less than 1 with 3-category outcome

I have a 3-category ordered outcome (food consumption: 1=no food, 2=less food, 3=more food) and a 3-category ordered predictor (food exposure: 3=no time, 2= less time, 1= more time- whereby 3=no time ...
1
vote
0answers
10 views

Finding multiple records of the same person when the name cannot be trusted

I have a list of 100 of events, and each event has a list of thousands of persons. Some persons went to multiple events, and I would like to select all the events a person went to. I could just look ...
7
votes
2answers
82 views

How to interpret these acf and pacf plots

Following are acf and pacf plots of a monthly data series. The second plot is acf with ci.type='ma': The persistence of high values in acf plot probably represent a long term positive trend. The ...
2
votes
1answer
34 views

Introductory data trend analysis with R [on hold]

I am new to the data analysis world and am struggling to find relevant articles / examples related to what I'm trying to do. I have a data set that is like this, for example how many apples are sold ...
1
vote
1answer
35 views

Can you show that $\bar{X}$ is a consistent estimator for $\lambda$ using Tchebysheff's inequality?

This question was taken from a practice exam in my statistics course. Given a random sample $X_1, X_2, ... X_n$ from a Poisson distribution with mean $\lambda$, can you show that $\bar{X}$ is ...
2
votes
0answers
9 views

Sandwich Estimator in Maximum Likelihood Estimation of Logit

I am estimating a discrete choice model using mixed logit using Halton Draws. So everything is basically numerically done. The code is written in MATLAB. I am using MATLAB's fminunc to do ...
1
vote
1answer
32 views
+50

Compare performance of 2 models

I have a dataset which I have split into 3 parts: a training set, a cross-validation set and a test set. I have used the training set and cross-validation set to train 2 models. For this, I have taken ...

15 30 50 per page