All Questions

Filter by
Sorted by
Tagged with
1
vote
0answers
7 views

Distribution of the squared length of a multivariate normal vector

Let $\mathbf{X} \sim \mathcal{N}(\mathbf{\mu}, \mathbf{\Sigma})$ be a multivariate Normal variable with $d$ dimensions. I'm interested in the marginal distribution of $\|\mathbf{X}\|^2 = \sum_{i=1}^d ...
0
votes
1answer
2 views

failure to get survreg results- all NaN- Error in coxph.wtest

I was trying to run Interval regression but all I got was this error message Error in coxph.wtest(t(x) %% (wt * x), c((wt * eta + weights * deriv$dg) %% : NA/NaN/Inf in foreign function call (arg ...
0
votes
0answers
6 views

RandomForest , confidence interval for class membership probabilities

I would like to calculate confidence intervals for predicted probabilities of a class membership obtained with randomForest. I know I could use predict.all in randomForest(), which gives me the ...
-1
votes
1answer
15 views

R: Simulate data with interaction given Y and X

I want to simulate an independent variable $Z$ given a dependent variable $Y$ and an independent variable $X$. Specifically, I want to simulate data that fits the following model : $$Y=b_0+b_1 X+b_2 Z+...
0
votes
0answers
5 views

Regression - stratified parameterisation in R

Just had a question in regards to stratified parameterization in R. Suppose we have y ~ x*g, then we get the stratified parameterization by doing y ~ g/x. I just wanted to ask if y ~ g/x is itself a ...
0
votes
0answers
3 views

Efficient implementation of spike removal

I have implemented the moving median absolute deviation (moving MAD) and it seems like bit-exact to Matlab's implementation. Nevertheless, I am sure that it is not efficient. The usual median filter ...
1
vote
1answer
5 views

Relevance of overfitting while examining causal effects

Let's assume that we have a well defined directed acrylic graph (DAG) showing correct causal relationships between variables. And this DAG tells us that we need to adjust for 7 variables to analyse ...
0
votes
0answers
10 views

Drawing the same lottery numbers twince in a row - is it really different?

While for every single lottery draw the chance is always the same, I know that to get the very same result in the next draw, the probability is squared (1/probablity*probability). But I just cannot ...
0
votes
0answers
14 views

Logistic regression on all subsets does not work well

I have approximately 3k data rows. I wanted to get a model which can says whether a row should be labeled A or B. I've used logistic regression and trained model for all subsets of features that I ...
1
vote
0answers
12 views

Repeated-measure design with only three subjects but many replicated observations

I am interested in studying the difference between two therapies ('TH') using data obtained from a computational model. Three subjects are modelled and each subject receives both therapies for each ...
0
votes
0answers
7 views

How to predict future exchange rate of Euro with machine learning model

This daily exchange rate dataset from Central Bank from 25/ 11/ 2014 to 23/8/2020, we aim to apply ML models to predict for the next 3 months. the dataset consists of four columns (sell, buy, currency,...
0
votes
0answers
9 views

Effect size for goodness of fit test

What effect size measure can be used for a goodness of fit test (given data and theoretical proportions)? I read about the Cohen's w (https://en.wikipedia.org/wiki/Effect_size#Cohen's_w) but I also ...
0
votes
0answers
2 views

Recalibration of constants with ordered logit for forecasting

I have estimated an order logit model: I have one utility function (without constant, in the form $\sum_i \beta_i * x_i$) with cuts and I estimated the $\beta_i$ and the cuts on revealed preference ...
0
votes
0answers
3 views

Linear mixed model / cross-lagged panel model for longitudinal relationship between imaging metric and symptoms on continuous scale?

Background I'm working with longitudinal volumetric brain data and psychiatric symptoms, measured simultaneously at two time points during childhood (let's call them brain1, brain2, symptom1, symptom2)...
0
votes
0answers
6 views

Can I use `Y ~ X + X:Z` when `Z` does not follow a linear relationship with `Y`?

I want to model the relationship between Y and X, but I am also interested in knowing if this relationship changes depending on <...
0
votes
0answers
4 views

Highly skewed outcome variable in longitudinal data analysis?

I am making a model between predictor x and outcome y, which are both repeatedly measured over time. My outcome is highly skewed to the left (see picture), which I cannot correct with common ...
0
votes
0answers
7 views

Am I to calculate the 95% confidence interval in this question?

MI5 has finally decided that 007 doesn't have to be a man but they still need to be British and shorter than 175.3 cm. The average height of all British citizens is 160 cm and the standard deviation ...
0
votes
0answers
10 views

What statistical method should I use? (0 element)

After K-means, I am iterating over many files consisting of tables, mostly are 3X3. Some of them with "0" elements inside cells but most of them aren't. I read that chi2_contingency is ...
0
votes
1answer
25 views

Can I conduct a paired t-test to compare regression coefficients?

I want to compare the regression coefficients for two variables from the same sample. My models may look like this: $$ \hat{Y}_1 = \alpha + \beta_1 * V_1 + \beta_2 * V_2\\ \hat{Y}_2 = \alpha + \beta_3 ...
0
votes
0answers
13 views

Hypothesis tests in Machine Learning: Independence assumption violated?

Assuming I have a training set that is used to train 3 different random forest models. The values of the response variable $y$ are the same for all of the 3 models. But the predictors $x$ differ ...
0
votes
0answers
6 views

Which output activation is recommended when predicting a variable with a lower but not an upper bound?

I need to predict something using a neural network. The output values are bound to be non-negative, but there's not really an upper bound. I do know that the output is never going to be higher than a ...
0
votes
1answer
12 views

Probability of drawing two differently colored balls in a partition of equally sized partitions

If I randomly distribute, say, 100 red and 10 blue balls onto a field of 10000 square meters (100*100 m), how would one calculate the chance of having at least one red and at least on blue in the same ...
0
votes
1answer
19 views

When should we use splines in regression? What's the justification?

I am examining the direct causal effect of x on y. Let's assume we a model as follows y ~ x Shows no significant effect of x on y. The reason may be that x ...
0
votes
0answers
7 views

training variational autoencoder - loss is 0 val loss is 0

I am training a VAE in Keras on the colab. The issue that I am encountering is that after the first epoch the loss is 0 and the validation loss is 0 (but the model didn't learn anything). this is the ...
0
votes
0answers
10 views

How to calculate and interpret a marginal treatment effect (local instrumental variable)? (Intuition through simple example.)

I am working on the intuition behind local instrumental variables (LIV), also known as the marginal treatment effect (MTE), developed by Heckman & Vytlacil. I have worked some time on this and ...
0
votes
0answers
4 views

DAG: causal direction unknown

I have a DAG as follows. Interested in the direct effect of E on O. Plus interested in publishing an article with a sound methodology. There definitely (read: highly probably) is a causal direction ...
0
votes
0answers
8 views

Probability of FP in One Dataset

I read the following Notes: how this probability is calculated? (I think FP Rate from these 3 Data on Axis).
3
votes
4answers
71 views

Is there a test that uses $|{\mu_A}-{\mu_B}|\le \delta $ as the null hypothesis?

In the usual t-test, the null hypothesis is that "the difference between the means of the two groups is zero". My Question: Is there a test that uses "the difference between the means ...
0
votes
0answers
6 views

How to control for Co-variate shift in test data set compared to train data for regression task?

I am working on a regression project. But I am facing the problem of covariate shift in features due to time delay.Test data was collected a year later due to which there has been some change in ...
0
votes
0answers
12 views

Changing L2 regularization constant in logistic regression proportionally to the number of columns/rows in the dataset

I'm trying to use scikit LogisticRegression to solve a multiclass text classification problem with variying number of columns (unigrams) in the trainging datasets. From what I understood, L2 ...
0
votes
0answers
4 views

improve detect filter microphone deepspeech [closed]

We saw an error when we connected the microphone to the neural network in deepspeech https://github.com/mozilla/DeepSpeech For example, the user says Hi And software in the output H Typing. What ...
0
votes
0answers
12 views

Obscure Error when Fitting CatBoostClassifier in Python

I am receiving an obscure error when attempting to fit a CatBoostClassifier to training data. I have created a simple dataset below to help illustrate the problem: ...
0
votes
0answers
10 views

Transforming Logistic Regression Model

I have a Logistic Regression Model in R ...
0
votes
0answers
4 views

How to Calculate the Pearson correlation coefficient between the bits of a 32 bit Hash value

I am new to statistics. I am checking the uniformity of the hash functions. There are examples to find the correlation between two values. I don't know how to find the correlation between the bits of ...
0
votes
0answers
4 views

MLE for the number of samples given $k$ largest values

I'm given the $k$ largest of $n$ i.i.d. samples, and I wish to estimate $n$. Let $F(v)=p(X\leq v)$. Then I believe that the probability of having exactly $k$ of $n$ samples greater than $v$ is ...
0
votes
0answers
9 views

General question about oversampling with TSNE or PCA

I have an unbalanced dataset and I am planning to use over-sampling (resampling) and finally, I want to use PCA or TSNE for a better demonstration of the data (the data has more than 50 dimensions). I ...
0
votes
0answers
5 views

random effect with one observation per group improves AIC drastically — explain

Now that I've posted an example (for a different purpose) at How will random effects with only 1 observation affect a generalized linear mixed model? (resulting in creating an account here, and being ...
2
votes
1answer
23 views

Proof that the mutual information I(X;Y) between two random variables X and Y is 0 if and only if X and Y are independent

On the Wikipedia of mutual information it says that $I(X;Y)=0$ if and only if $X$ and $Y$ are independent. I can proof that if $X$ and $Y$ are independent, then $I(X;Y)=0$, because $p(x,y)=p(x)p(y)$. ...
1
vote
0answers
21 views

Does this scatter plot suggest non-linearity?

As part of assumption testing for ordinary least squares (OLS), I am examining the linearity between the variables in my model. I'm using scatterplots between the independent variables (IVs) and ...
1
vote
0answers
5 views

When using Linear Discriminant Analysis and k nearest neighbours in classification, which is better?

I am currently performing classification on the Iris dataset. I used both LDA and kNN methods to classify the data. I found both to be highly accurate and cannot decide which one is more appropriate ...
1
vote
0answers
9 views

Appropriate model for classification problem with monthly observations for same customer

I'm looking for some help understanding what the appropriate method would be for the classification problem(well really I want the probabilities) at hand. The event that is being predicted is whether ...
0
votes
1answer
15 views

Which hypothesis test should I use for this given situation?

Let's say I have lot of users who spend money on various items. Considering Item A: For this item, I want to find whether there is a statistical difference in average spending in the month of October ...
0
votes
0answers
5 views

May Skilling's Nested Sampling Estimate parameters in hierarchical model?

May Skilling's Nested Sampling integration technique Estimate parameters in hierarchical model?
0
votes
0answers
16 views

MLE in Time Series

Can someone describe how MLE is calculated for a time series model? Conceptually I understand that: Likelihood = Conditional Probability = P ( data I theta ) where data are the observations and theta ...
0
votes
1answer
14 views

Is a variable on scale 1-100 considered ratio or interval level?

Is the variable at ratio or interval level if it can take any value between 1 and 100 but not 0? I cannot determine using the definitions of ratio and interval variables.
0
votes
0answers
4 views

Memoryless conditional expectation of shifted function exponential

Related to this, is the following valid: \begin{align} E[f(X-t) \mid X>t] = \int f(y-t) f_{X|X>t}(y) dy = \int f(x) f_{X|X>t}(x+t)dx = \int f(x) f_X(x) dx = E[f(X)] \end{align} where I make ...
3
votes
0answers
35 views

Shape of MSE in polynomial regression

In polynomial regression, an increase in the degree of freedom can cause high variance and low bias. So model overfits on the training set and loses its generalization. On the other hand with a low ...
0
votes
0answers
5 views

Are there problems with using Ordered Quantile (ORQ) normalization for Multivariate Analysis?

This is intended as a general question, as a Google search for -- "Ordered Quantile" "ORQ" normalization "Multivariate Analysis" -- returns nothing. This option is listed ...
0
votes
0answers
19 views

Understanding what could be wrong with Panel regression

Aim: I want to determine the drivers are of defaulted amounts on student loans and also want to be able to project defaulted amounts into the near-term future (1-3 months from last date in my dataset [...

15 30 50 per page
1
2 3 4 5
3383