All Questions

3 views

How to detect noisy datasets (bias and variance trade-off)

Studying the bias-variance trade-off: expected loss = bias + variance + noise I understand that we minimize this quantity by finding the "best" balance between ...
9 views

Relative Importance of categorical variables

My question concerns management issues more than statistical theory. I am searching for a way to measure the relative importance/influence of categorical variables in a logistic regression model with ...
7 views

Directed acyclic graphs in regression model

I am using DAGs to select best set of variables for my logistic regression analysis. Assessment of DAG includes one exposure, number of covariates and an outcome variable. I have not found any ...
5 views

lme4: Why won't lsmeans output my fixed effects?

I'm trying to plot confidence intervals for linear mixed effects models trained with lme4 and lmerTest in R. I am using this data file, which I've shared via Google Drive. Here is my trained model. ...
5 views

Smoothing a particle filtered 2D trajectory

I am currently developing a very basic particle filter for a 2D robot localization task. My process is defined by a really simple velocity / steering angle based motion model. I am re-weighting the ...
4 views

Median-computation (“grouped median”). Problem with gmedian-computation in SPSS

In my basic-statistics course I wanted to delve deeper into the problem of the (grouped) median-computation and did some computations using SPSS. The protocol seems to indicate either a different ...
11 views

Imposing a model on a probability

Lets say I have a model $$y=x\beta-+u$$ where $$u\sim NID(0,1)$$ and I want to derive the likelihood function of $\beta$ given $x$ and $y$. The second step is entirely intuitive. Is it correct? If so, ...
5 views

Johansen cointegration test: interpretation of results in EViews

I am not sure whether I am interpreting the cointegration test correctly. This is the test result: Because of the probability of the test I understand that my series are cointegrated of order 2. ...
16 views

Understanding neural networks and classes

I want to know if this argumentation is valid or not of my algorithm. I'm trying to implement a CBIR (Content-Based Image Retrieval) where I've used the basics on CBIRs (colour, texture, shape, ...
23 views

What makes the recommendation problem unable to be solved by traditional machine learning algorithms directly?

We have collaborative filtering and content based algorithms there for recommendation. What stops traditional algorithms from directly being used to find missing values in the Utility matrix ...
15 views

Is continuous inputs an assumption of factor analysis?

Should we use only continuous inputs for factor analysis (FA)? My data is a mix of continuous and categorical inputs: one of the inputs has only 600, 700 and 1000 as values. I found that principal ...
9 views

Lowering C parameter increases the number of support vectors

I know that the C (cost) parameter controls the trade-off between model complexity and misclassification. A large C should increase the error weight, and therefore the model complexity. Nevertheless, ...
15 views

What are the formulae used in R by predict.lm when interval= a) 'none', b) 'prediction', and c) 'confidence'?

The references provided in the R documentation for predict.lm, taken together, actually leave open a number of possibilities for the formulae for confidence and prediction intervals (including the ...
19 views

How do I calculate clustering of symptoms?

I am trying to assess whether a risk factor (A) has an impact on clustering of 3 types of symptoms (B, C, D). I know that A has a significant association with B, C, and D. So first I did a ...
17 views

Double hierarchical GLM reports non-conformable arguments error

I am new modeling and I am trying to analyse my data using the package Double Hierarchical Generalized Linear Models. However, I always get the same sort of error: ...
9 views

Running Latent Dirichlet Allocation (LDA) on word counts

I have difficulties understanding the VB implementation lda-c. In particular, the method expects as input a bag-of-words representation of documents, where distinct words appearing in a document are ...
7 views

How to train a Recurrent Neural Network for Temporal XOR?

I have coded a Elman RNN using BackPropagation Through Time. In order to check my implementation, I have chosen Temporal XOR(a sequence of binary digits with the third being the xor of previous two ...
6 views

Is there an accepted way to compare within all demographic categories (e.g., within males, within females, within whites, etc.) using MANOVA

I have three experimental groups (group A, group B, and group C), and gave all participants the same 50-question questionnaire. I then can run a MANOVA to determine if their responses (quantitative) ...
10 views

Conditional independence conjecture

Suppose there are four random variables (events), $A$, $B$, $C$ and $X$. If we have $X\perp A|B$, saying $X$ and $A$ are conditional independent given $B$, then I want to ask that whether we have ...
11 views

Statistically testing difference between two sets of data in multiple situations

We have been conducting a small study recently that, basically, looks at actual behaviour and self-estimated behaviour in a number of situations and is trying to figure out whether they are similar or ...
8 views

How to evaluate uncertainty estimates in regression?

Some regression algorithms (e.g. Gaussian process regression) can produce uncertainties along with point predictions at test time. These should also be evaluated. How about calculating the Pearson ...
7 views

sufficient number of data points to estimate mean, median for multivariate distribution

I just wonder what is the sufficient number of data points to estimate the reliable mean and median of a multivariate distribution and suppose I know that the distribution is not normal. What is the ...
12 views

How to calculate Accuracy for regression and time series models?

What is the best way to find accuracy for regression? What is the best way to find accuracy for time series models?
14 views

Statistical graphical models in practice

I've followed the probabilistic graphical models course in coursera and now I would like to apply what I've learnt with real data. I want to implement LDA on a large corpus of text and I was ...
22 views

What is the most efficient method to conduct for hypothesis testing?

To gather from the data whether the hypotheses I proposed are confirmed, I calculate the two means between different reform variables to see if the reform have influence on the performance indicators ...
15 views

Machine Learning when missing data is state-dependent (e.g. adaptive questionnaires)

I have the following problem: I am dealing with an adaptive questionnaire, meaning a questionnaire where there are questions that are only asked when a previous question had a specific answer. The ...
6 views

Export contribution of every input in every principal component that explained variance percentage in Principal Component Analysis (PCA) in MATLAB [migrated]

How can I export a report like this in PCA function of MATLAB R2015a? 1 to 10 are PCA ...
9 views

Censored logistic regression

I have the following problem: We have data with a 0-1 outcome which can occur precisely once. It can occur at any time within a certain time period (say 3 years). For this data set, for some ...
17 views

Factor Analysis

I have following data regarding Voice traffic(dependent variable) and some 10-12 other variables which are my independent variables. So, to use factor analysis shall I use voice traffic as one of the ...
27 views

When using SVMs, why do I need to scale the features?

According to the documentation of the StandardScaler object in scikit-learn: For instance many elements used in the objective function of a learning algorithm (such as the RBF kernel of Support ...
40 views

Understanding Sampling with replacement

I am doing Stat110 and in the book, "Introduction to Probability" they give the following definition of sampling with replacement: Theorem 1.4.5 (Sampling with replacement). Consider n objects and ...
26 views

What is Bootstrapping in statistics? How can I use it to determine error in the mean, variance, kurtosis and skewness of a data set?

From what I understood from searching randomly is that it has something to do with resampling. What does this resampling mean? Is it selecting random data from a distribution or is it getting data ...
20 views

Which statistical test to use on mean data?

I have sets of data from two sample A and B. the data in sample A and B are the average mean of the population sample. Examle, sample A has list of data. each data in he sample is a an average value ...
7 views

How to replace agrep with something faster? [on hold]

I am currently using approximate grep (Agrep) to get matches between a list of thousands of short strings vs. a list of millions of longer strings. agrep works fine, but it's too slow for what I have ...
8 views

Comparing performances of distributions

I have a Probability Distribution Function (PDF) of a new distribution given below, g(x)=\frac{1}{\Gamma \left( \alpha ...
8 views

How order of stuctural breakpoints is decided?

Using 'breakpoints' command in R, I got multiple breakpoints. Yet if I limited the number of breakpoints to 1. It gives just a single breakpoint as desirable. How is this 1st breakpoint is decided and ...
63 views

ARIMA model fits what kind of data

What kind of data fits ARIMA model well? If not ARIMA, what are the other good models that can be used to forecast the time series data?
6 views

Non normality and nonlinearity in a generalized mixed model

My thesis is about the encroachment in Arctic regions. My experiment contained 6 plots, of each type (encroached and non-encroached) on three elevations. Elevation and type are fixed factors, plot is ...
12 views

F-test of joint significance vs multiple t-test for regression parameters? [duplicate]

In the context of linear regression, I don't understand why you need to perform an F-test for the H0 that all parameters are zero, instead of just looking at all the t-tests for each parameter. I ...
14 views

Maximum Likelihood Algorithm Implementation

Hello Guys i have to implement Algorithm of Maximum Likelihood using Hyper-spectral data of 170 bands, have to classify 16 classes using ground truth data. The formula for implementation is as follow, ...
13 views

Hazard function of a gamma distribution

The system we are working on is biological, more specifically the distribution of specific events across a chromosome. This can be thought of as 1D array (the chromosome) across which points can be ...
24 views

Derivation of likelihood function for latent variable model made explicit

I am trying to make the steps deriving the likelihood function for the following latent variable model as explicit as possible: $$y^0=X\beta + u$$ where $$u \sim NID(0,\sigma^2).$$ The observed data ...
14 views

How to transform this time series data?

enter image description here Given in these plots above is the US unemply. data since 1948 till recently a month back. I tried using a log and difference transform to make the data look more ...
6 views

Contingency testing in geographic data with the issue of independence?

How can I test contingency between two variables in geographic data with the issue of independence? I have a column of fire frequency and a column of vegetation change but do not think a simple chi ...
16 views

Zooming in on tail of histogram plot in R [on hold]

First off I've been using the truehist() function in R instead of hist() mainly because it looks better (but I'm pretty sure this mainly just because it has a better bin allocation). Currently, I ...
14 views

Relative importance of predictors in the final model

A common question that frequently comes up, while presenting the findings of a predictive model to a Business audience (with non-statistical background) is - Which variable/predictor is most important ...
9 views

What is the shape of the decision surface of a Gaussian Process classifier?

I've got a binary classification problem, which I am trying to solve using a generative classifier. If I use Gaussian Discriminant Analysis, and fit two Gaussian distributions to my two classes, the ...
6 views

How do I test whether there is a deviation in mutual fund investor strategy depending on movement of market indices, etc?

Im fairly new to the world of statistics and econometrics. I have done a years worth of study for both in the UoL international program. Im working on a project this summer to see whether the ...