All Questions

185,016 questions
Filter by
Sorted by
Tagged with
2 views

Test pareto against lognormal

I have a set of data that is primarily lognormal distributed and its tail follows a generalized Pareto distribution. I would like to obtain this result: at a threshold value, a p-value tells me that ...
3 views

How to do hypothesis test for overlapping data?

I am now studying the effects of two different drugs on a specific outcome. The problem is some of my subjects could take these two drugs at the same time, which makes it hard to estimate individual ...
3 views

Regression coefficients while minimizing absolute error

I understand that we like working with square error instead of absolute error because it makes the calculus easy. But I was wondering about the parameters of Linear Regression minimized for absolute ...
6 views

Comparing P-Values of two tests doing the same task

There are two statistical tests which do the same thing. For instance, they both tell if two samples are generated from same distribution. KS Test (2 samples) and Anderson Darling Test (K Samples). I ...
2 views

Should I only include hard samples in multi-label classification loss when training CNN

I was just reading about Triplet Loss, and there they typically only update parameters on the "hard" triplets, rather than the easy ones with a loss of zero. Is there a similar strategy in ...
5 views

Neural network prediction gets worse when ground truth is close to the edges of output range

I am trying to use a simple neural network to predict numerical values of certain properties of sensor data (Somehow a regression problem). My network has only 1 output with Tanh activation, so output ...
9 views

Two equivalent ways of sampling with replacement

The original context of this problem is for a derivation of the lookdown model https://projecteuclid.org/journals/annals-of-probability/volume-24/issue-2/A-countable-representation-of-the-Fleming-Viot-...
3 views

Confusion about the time step in LSTM input shape

I learning deep learning, LSTM by experimenting. I do have some confusion about employing the time dimension on the LSTM. I do understand clearly that the input shape goes like (number of data points, ...
10 views

Calculating VAR(Y) given VAR(X) and RMSE

I feel like this is a calculation I've done before but can't recall how, and Google searches don't seem to answer my question. Before I describe the calculation I want to do, let me first set up the ...
5 views

What should I do if only 1 covariate violate the proportionality assumption in Cox PH?

Can I still use Cox PH model? or I will have to find a non-PH model? What are the available models that can be used here? Schoenfeld residuals
12 views

power analysis for multiple logistic regression via simulation vs power.prop.test()

I am using simulation for a power analysis of an experiment tested with a multiple logistic regression model. I find substantial improvements in power for my model with covariate vs without covariate. ...
6 views

Interpreting random effect model (random intercept and slope)

I fitted the following random effect model, where I assumed a correlation between the weights of each individual over time (represented by age), and used the inidividual id as a random intercept and ...
13 views

How to show $Cov(\hat{\beta_1},\hat{\beta_2})=Var(\hat{\beta_0})$

$$\hat{Y}=\hat{\beta_0}+\hat{\beta_1}X_1+\hat{\beta_2}X_2 , \quad\text{X_1, and X_2 are indicator variables}$$ In ANOVA, regression on one factor with 3 levels, how to show the following. Cov(\...
4 views

Predict risk from pre-trained Cox PH model

I came accross this article about AMD (age-related macular degeneration, a kind of eye diseases) recently. The article fitted a Cox PH model to predict risk of disease advancing to advanced stage. The ...
5 views

Apriori Algorithm Visualization & Interpretation in R

I would like to visualize the items that are frequently bought together. Also I would like to know how to interpret the visualization. The dataset is taken from https://www.kaggle.com/sivaram1987/...
5 views

For ARMA processes, are predictions using the equation directly, Durbin-Levinson algorithm, and the innovations algorithm the same?

So there are at least 3 ways of prediction for ARMA processes: (1) simply by writing out the equation and setting the the future error term to zero (e.g. for $X_n = aX_{n-1}+\epsilon_n$, the estimate ...
19 views

Do people use log weighted average?

I have before and after measurements for the same set of subjects. Before After 1 2 25 14 ... ... 3000 3215 I want ...
12 views

Simple question about the use of a random forrest classifier

I have a simple question regarding the proper use of a random forrest classifier. I have 56 independent observations of mice, half of which are male and the other have are female. I have three trials ...
6 views

Can a bayesian neural network with independent normal variational distributions over weights and biases produce a multi-modal posterior?

For a project I am involved in, I am doing surrogate modelling. This means that I simulate data $\mathcal{D}=\{X,Y\}$ that is used to train some probabilistic non-linear regression model. The model is ...
7 views

SMOTE for logistic regression model had a worse result compared to original?

Not sure why using more sample from SMOTE() could lower the overall accuracy: ...
3 views

RSM and steepest ascent for new $x'$ coordinates

In Myers, Montgomery, & Anderson-Cook, C. M. (2016)$^1$ the authors provide (p. 235) a general methodology to find new $x'$ coordinates in the path of steepest ascent based on constrained ...
10 views

GBM Algorithm / Gaussian Loss function

I am working with the gbm() package in R, in which one is required to specify a distribution for the loss function. For regression problems, one is typically required to specify "Gaussian". ...
5 views

Do I need to make one or more predictors covariance stationary?

I'm working with the Chicago Crime Data set. I want my model to use, among other variables, commodities prices to predict a particular type of crime. Since the data represent were collected daily, ...
9 views

Analysing responses across different groups - R Studio

I have data from a survey with responses as categorical data (multi-choice answers). I'm trying to find a way to explore if the % who chose each multi-choice answer change across different types of ...
11 views

Is this K-Fold Cross-validation approach correct?

I've seen that Train-Validation-Test set technique is discussed and there is no consensus. Some people don't differ validation from test. When I was studying this technique and K-fold cross-validation,...
5 views

How to test whether the relationship between an IV and a DV varies depending on levels of two other variables

Let's say I have 2 different conditions participants can be assigned to (stored in the variable condition). Within each condition...
6 views

Feature selection prior to PCA to eliminate multicollinearity?

I have time series with a handful of predictor variables (not relevant in this context) and around 100 depedent variables (texture features of algae cells). I would like to do a PCA to see how the ...
19 views

Why is my 1D CNN so bad?

TLDR: My 1D CNN is doing a really bad job classifying graphs. Here's more context: Note: I've tried adopting the advice listed here and here, but my CNN hasn't stopped overfitting. I've already tried ...
16 views

Is it possible to use xgboost to maximize a sum?

I am just learning about machine learning (no pun intended). I have used xgboost's objective function binary:logistic to find probabilities from labels of 0s and 1s. Is there some way to guide the ...
6 views

Does Weight of Evidence Need to be Monotonic the Whole Time?

I've binned a few of my variables and have plotted them to take a look at the monotonic relationship. A good amount of them dip in the very beginning, then consistently rise until they get to the top. ...
8 views

Determining survey fatige?

I am looking at some survey data and wondering if there are any sound strategies for investigating whether people started to click through the survey as they got toward the end (e.g., answered the ...
15 views

What is target and sample population? the portion of population for whom we have received data or those who were invited to participate

I am new to statistics and I am trying to understand target population and sample data. Suppose a company wants to know the level of satisfaction of all their customers. To this end, it has sent a ...
12 views

Can the sampling of the entire universe/population be considered "random sampling"? [closed]

I have a series of instances with a universe between 1-X and I am required to "randomly sample" 10 occurrences from each instance. Replacement is not allowed for this sample. Is it possible ...
30 views

Probability from Chi-Squared distribution

If I have a random variable called $A$ and $A \sim \mathcal{N}(0,1)$, how can I get the value of $X$ if $P(X<5A^2<6.52)=0.5310$. It seems that the answer is $2.4109$, but I don't know how to ...
6 views

How do you interpret ANOVA with transformed response?

Let's say I have a quantitative variable Y that ranges between 0 and 1. I have a categorical variable X that has 3 levels, A, B, C. Since Y ranges between 0 and 1, I transform all of Y using the ...
7 views

What analysis for longitudinal data with grouping variable?

Study design was longitudinal with multiple waves of the same survey. One dichotomously coded variable captured whether an event had occurred in a person's life (coded yes/no). I want to look at ...
10 views

How do I confirm that HC3 has fixed Heteroscedasticity?

My data is heteroscedastic and cannot be remedied with transformations (I tried boxcox to no avail). I've recently found out about HC3 but am unsure about its effectiveness, as I've run a breusch-...
7 views

Reinforcement learning: is a softmax policy actor-critic expected to work on mountain car?

I am following David Silver's RL course and I'm struggling to apply the Actor Critic concept to the Mountain Car environment. I am using a softmax policy with linear function approximation. I am also ...
38 views

Proof of consistency of Maximum Likelihood Estimator

I would appreciate some help comprehending a logical step in the proof below about the consistency of MLE. It comes directly from Hogg, McKean, Craig, Introduction to Mathematical Statistics, 6th ...
10 views

Continuation of Training for Neural Networks

originally posted in SO Artificial Intelligence, advised to post here instead Background I am developing a GAN model (based off this paper), and am trying to learn how to feed subsequent time series ...
11 views

Definition and Taxonomy of Seasonal Time Series

I want to categorize a large number of time series into non-seasonal and seasonal divide the seasonal ones into a small number of subgroups by type of seasonality Are there any formal definitions/...
4 views

Comparing paired samples in mixed-effects model

I am hoping to determine if there is a statistical difference between test results from samples taken from the same site in the same year. For all samples, they fall in to either two categories: the ...
39 views

What should i do with this kind of data? (price from the ads and actual price)

There are two subsamples in the dataset - on one the target is real(valid), and on the other it is approximate (I don't know how it differs yet, on one sample the real price of an apartment, and on ...
15 views

Cumulative distribution function for Bivariate Compound Poisson-Gamma distribution [closed]

Could you explain to me how can you find the CDF of the bivariate compound Poisson-Gamma distribution if it does exist or explain to me why it does not exist?.
10 views

Fixed and Random Effects / Fixed and Random Factors

What is the difference between a fixed effect and a random effect? Are fixed and random effects different from fixed and random factors?
6 views

Problem: Why are my loadings good in SMARTPLS's report (Outer Loadings) but bad when conducting factor analysis in SPSS? Details: When I conduct Factor Analysis in SPSS and go through all the ...