0
votes
0answers
2 views

Plot large high dimensional dataset

I have a high dimensional dataset (12 variables) with 42000 observations. I made a clustering and i have evidence to choose five cluster.So i want to plot with some tools my dataset to show (if it is ...
0
votes
0answers
3 views

Problem Calculating Standard Error

I have a set of individuals categorized into two classes as follows: ...
0
votes
0answers
2 views

MCC or F-measure, which measure is the best to represent multi-class confusion matrix

I'm building a classifier of multi-class problem. In my context I have to classify vehicles into five categories: cars, vans, trucks, buses and motorcycles. I would like to use some measure to ...
0
votes
0answers
4 views

type 1 and type 2 errors for non-normal residuals in a regression

He guys, I did a regression and performed normality tests on my residuals, which reject the NH that the residuals are normally distributed (the true distribution seems to be flatter with fatter ...
0
votes
0answers
3 views

How to select weights in the F-Measure to be aligned with that used in cost-sensitive SVM training?

I am dealing with a classification problem in which Recall is more important than Precision, and the training dataset is an imbalanced one. The approach I am taking is to use oversampling to mitigate ...
0
votes
0answers
4 views

Using contrasts in Poisson regression

I don't really understand how to use contrasts in regression analysis. We have devided the participants in our sample into 6 groups, according to their body mass index at baseline (3-class variable: ...
0
votes
0answers
9 views

Sequence of probabilities

The probability distribution of a random variable X is $P(X=k)=p_k,k=0,1,2,...$ and $\frac{p_k}{p_{k-1}}=a+\frac{b}{k}$. If $p_0=\frac{1}{9},p_1=p_2=\frac{4}{27}$,what can be said about the sequence ...
0
votes
0answers
6 views

Linear regression (best fit line) on moving averages vs raw data?

I have a series of sets of data over a period of time, with the amount of data available being quite variable between sets. One has points for almost every day but is really quite noisy; another has ...
1
vote
1answer
11 views

Unbiased estimator of poisson parameter

The number of accidents per day is a Poisson random variable with parameter $\lambda$, on 10 randomly chosen days the number of accidents were observed as 1,0,1,1,2,0,2,0,0,1, what will be an unbiased ...
0
votes
0answers
7 views

What is the difference between GAS ( Generalized Autoregressive Score) model and a GARCH?

I am trying to analyze some data about Brent Oil volatility. So far I have managed to fit a GARCH(1,1) model and an EGARCH. However, someone has recommended to use a GAS model, Generalized ...
0
votes
0answers
3 views

Use case for Anomaly Detection using the multivariate gaussian distribution

Given the following real world problem: We have 5000 vehicles of DIFFERENT models (trucks, small cars, large cars) with 100sensors in each car (measuring consumption, distance, averange speed etc for ...
0
votes
0answers
9 views

Consultancy or research scientist in medicine/bioinformatics in West Europe

I'm a mathematician who's working as a postdoc on statistical modeling of biomedical data, combining statistics and differential geometry. I'm not well familiar with machine learning (ML), but gaining ...
0
votes
0answers
2 views

Conflicting intuitions re linear systems in levels vs shares

Suppose I have a set of components that sum to some total value, different in each period. Call the total Y. I have a set of equations, linear in the (sometimes transformed) variables, each ...
0
votes
0answers
5 views

Can I add categorical variables in NARX

Can I add categorical variables as exogenous variables on NARX(in Matlab)? Eg: Type of day, Month
0
votes
0answers
5 views

Is it possible to derive a relation between parameters in Poisson process representation of extremes and parameters in GPD model?

I want to derive the theoretical relation between the parameters in a point process model for extremes and the parameters in the GPD model for extremes. I'm following Coles - An introduction to ...
0
votes
0answers
6 views

Hourly electricity demand data finding AR and MA terms

I am new to time series analysis. I have hourly electricity demand data for five years (having multiple seasonalities as daily,weekly,annually) and I want to guess the number of AR and MA terms using ...
0
votes
0answers
7 views

Neural Networks to Explain User Behavior

In the past I've used logistic regression to not only predict user behavior, but to understand the relative importance of each variable to help the business teams understand which levers they can pull ...
1
vote
0answers
20 views

Unbiased estimator of binomial parameter

Let 4,3,5,2,6 are 5 observations of the binomial(10,p) random variable.What will be an unbiased estimate of $(1+p)^{10}$ ? I have tried to solve the problem in this way. We know that ...
-2
votes
0answers
9 views

Can PCA be applied to normalized features?

I want to perform feature dimension reduction using PCA. The feature is 256 bin normalized histogram of images. I would like to reduce feature dimension to less than 256. I centered the train and test ...
0
votes
1answer
12 views

a hypothesis test for evidence that one thing is dependant on another

I'm not sure what the null hypothesis for this would be/ what the correct symbols are. the data is unpaired and I'm needing to find if there evidence ( at 5% level of significance) that one thing is ...
0
votes
0answers
1 view

Measuring the effect of weather on retail sales

I'm currently working on modeling this as an ad hoc. Sr mgmt want to know how much of our sales growth during the year can be attributed to weather. I chose to investigate "weather" as temp & ...
0
votes
0answers
5 views

How to calculate mean average precision given precision and recall for each class?

I use Pascal VOCdevkit to calculate object detection average precision for each class, but how can I get mean average precision for the whole dataset? Should I average each average precision or should ...
0
votes
0answers
20 views

How do we obtain E(exp(C+By-Y'aY))?

How can I prove this ? I did get $B-2uA$ by identifying the $a+by+cy^2= -1/2 (y-u)^2/\sigma^2$ But, I don't know how to get $(2a+\Sigma^{-1})^{-1} \dots$ Am I missing something? A little help would ...
0
votes
0answers
9 views

hypothesis testing for two sample

I have two samples and I need to do hypothesis testing to see if the mean has changes between these samples and I don't know what data I should use as I have 2 SE mean and standard deviation one for ...
0
votes
0answers
4 views

Convenient plot size detectable difference vs Least Signifcant Difference

I am curious how Hatheway's (1961) Convienient plot size calculation, once rearranged to solve for the detectable difference, relates to a standard LSD calculation: $$ d^2 = \frac{(t_1 + t_2)^2 \times ...
0
votes
0answers
6 views

confidence interval when the variation is known [on hold]

when we calculate the confidence interval for the mean and the variation of the population is known,it is obvious that the population mean is known. Thank you.
0
votes
0answers
5 views

SPSS | Repeated-Measures ANCOVA | Covariate

We are performing a RCT with 4 groups (1 control + 3 experimental), where we measured physical activity (PA) at baseline, post-test and follow-up. Although we have done stratified randomization, the ...
0
votes
0answers
22 views

MDS: Are there measures of stress that are not affected by the number of objects?

This question flows naturally out of a previous question, in which it was explained that Stress-1 is somewhat impacted by the number of objects. You can assume that the number of objects in the ...
0
votes
0answers
5 views

Is it legitimate to use bootstrap to estimate regression parameter with hypothetical sample data?

Consider a simple OLS model: $$ y = \beta_0 + \beta_1x_1 + \beta_2x_2 +\epsilon $$ Suppose $x_2$ is dummy variable which has value either 1 or 0 and the model is successfully fitted with collected ...
1
vote
1answer
23 views

Advantage/acceptability of using contrast codes to conduct pair-wise comparison vs. t-test on subset of data?

I conducted a study with three conditions (A, B, and C) and I want to test the difference between A and C. My default approach would be to run a t-test (or its equivalent) on a subset of the data ...
0
votes
1answer
14 views

Is it possible to train a linear classifier to maximize f1-score?

The usual approach is to maximise f1-score is to train the classifier first and then optimize a threshold value on a validation set. The decoupling of weight learning and threshold searching sounds ...
2
votes
0answers
14 views

How to calculate a confidence interval for a trimmed mean

I try to get the sample size that presents the population, I averaged randomly the 10 and then the 20 readings then I used the formula CI = 1.96 * std/root n, that gave me the CI at any (n), but how ...
0
votes
0answers
8 views

Is data correlation take into account for variance components?

Given a hierarchical model $$y \sim \mathcal{N}(X\beta + Zu, I\sigma^2_e)$$ where $$\begin{align*}\beta \sim \text{Uniform}(-\inf, \inf) \\ u \sim \mathcal{N}(0, I\sigma^2_u) \\\end{align*}$$ we often ...
0
votes
0answers
6 views

Why are these two samples correlated with each other (ISRL passage)?

On pg. 44 of Introduction to Statistical Learning: Here we create two correlated sets of numbers, x and y, and use the ...
0
votes
1answer
26 views

Computing Conditional Probability - $P(A|B^c)$

I'm terribly out of practice in even the most basic rules of probability, any help with the following question is appreciated. Essentially, I am having an issue with the application of the conditional ...
3
votes
0answers
21 views

Logistic Regression: What do people exactly mean by “cell size”? (example: small cell N's but no warning messages and convergence criteria met.)

I've been performing some logistic regressions and found some great results. The problem is I'm unsure if my model is stable or not. This is mostly because I am confused about what people mean by ...
0
votes
0answers
11 views

Model for predicting real value from vector of binary features

I have a population where individuals are described by a set of binary features (about 20 variables); some features are correlated, and some features imply others (i.e., if var A is positive, var B is ...
0
votes
0answers
9 views

t-Test residual analysis and Welch's correction

I'm reading Experimental Design and Analysis by H.Seltman (http://www.stat.cmu.edu/~hseltman/309/Book/Book.pdf) and working on the provided HCI dataset (SPSS format, can be downloaded from page 143). ...
0
votes
0answers
10 views

After calculating inter-rater reliability, which data (e.g. consensus data?) should be used for further analyses?

My study used 2 raters in a fully crossed design to code 1) a categorical variable and 2) a continuous variable (i.e counting the number of times something happened). My question is, after ...
0
votes
0answers
8 views

R dlnm package fitting and interpretation

I'm a first time user of the dlnm package in R for distributed lag models. I'm trying to fit a distributed lag linear model for some data in the following form $Y_t = \alpha + \sum_{i=1}^N ...
0
votes
1answer
17 views

Bayesian errors-in-variables model definition in JAGS and symbolically

I'm fairly new to probability theory and am attempting to understand and implement an errors-in-variables simple linear regression model. I am assuming a model of the form $$ Y=\theta X_a+\epsilon_Y ...
0
votes
0answers
10 views

Estimation of causal effect in experiments with lots of missing values

This is a generic question. Suppose you run an experiment with units randomized between a treatment and a control group. Imagine the response rate is very low in each group, say, below 10% and with a ...
0
votes
0answers
12 views

Any way to make density estimate from the data with importance factors? [on hold]

I have a data with probabilities: $<x_i,p_i>$ which can be used to approximate CDF (build stepwise function and then smooth it). How can I get smooth density estimate from this data in R? All ...
0
votes
0answers
10 views

How to mitigate risks for an oversampled training and validation set for a Zero-Inflated Poisson model?

I have a data set, with training, validation, and test already decided for me. It is a mailing list response project so it has a lot of zeroes representing no response, and some ones representing a ...
0
votes
0answers
15 views

What is a hard data set for support vector machines?

Suppose that: There are only numeric features in the data set. There is a binary target variable (since a general nominal case is resolved by 1vs1 or 1vsTheOthers approaches). I have noticed that ...
0
votes
1answer
23 views

Formulate equation after fitting to log(y)~x) using lm() [duplicate]

Following code was used to fit a function to my data; ...
0
votes
0answers
4 views

Adding a glm regression to a ggplot, where datapoints are colored after groups [migrated]

I am trying to visualize a regression by adding it to the data plot, but I want to color the data points after group (eight different groups) and then ggplot makes a glm line each group instead of one ...
2
votes
0answers
13 views

Difference in Beta random variables

I am having trouble deriving the distribution of the difference of two beta random variables and would like some help verifying the steps I have taken. In particular calculating the bounds. Say I ...
0
votes
0answers
10 views

Scraping data from web application? [on hold]

I am trying to get data that is displayed from what seems to be a javascript app (not sure). For example, consider this page. The graph under the "Finance" tab contains pricing information about some ...
1
vote
1answer
11 views

Parameter n_iter in scikit-learn's SGDClassifier

I have a doubt concerning parameter n_iter in function SGDClassifier from scikit-learn. Hereafter is the definition: n_iter ...

15 30 50 per page