All Questions

-1
votes
0answers
5 views

Problemas con decimales [on hold]

Tengo un data.frame con datos de valores grandes en €. Existen tres problemas: 1º No me reconoce el "€" como valor, en su defecto me pone €. Lo he omitido con gsub("€", "", nombrevariable) 2º ...
0
votes
0answers
8 views

A little help with probability calculation [on hold]

i have a question, if we know P(A) and P(B),how can we calculate the $P(A) \bigcap P(B)$? for example let say P(A)=2/3 and P(B)=1/6
0
votes
0answers
3 views

How to deal with circular causality

Often in time series and panels, the "dependent" and "causal" variable don't share purely that relationship. There is a fair bit of reverse causality as well. ,e.g. x causes y, but then either y ...
0
votes
1answer
10 views

Conditional Independence

I have a joint probability, which factors as follows: $P(A,B,C,D) = P(A,B) \cdot P(C|A) \cdot P(D|B)$ So I know that $C$ and $D$ are independent given $P(A, B)$ right? I want to infer $P(A,B|C,D)$....
1
vote
1answer
9 views

Vector Autoregression - How do we choose the correct value of p?

I am following this article: https://otexts.com/fpp2/VAR.html#fn24 ...
0
votes
0answers
8 views

Why we get different important factors when performing logistic regression and decision tree?

Why we get different important factors when performing logistic regression and decision tree? When I select features, I perform logistic regression and decision tree separately and wanna see the ...
0
votes
0answers
3 views

In survival analysis, when should we use fully parametric models over semi-parametric ones?

This question is the counterpoint of the other question In survival analysis, why do we use semi-parametric models (Cox proportional hazards) instead of fully parametric models? Indeed, it clearly ...
0
votes
0answers
14 views

Saving plots in folder in R without overlapping [migrated]

I am currently running my own program in R programming. And I am struggling to figure out the solution for my problem. I have two for loops and I m saving the plots into a folder, first for loop have ...
0
votes
0answers
6 views

How to predict how many customers are going to purchase based on population purchase frequency?

I am trying to calculate a metric "frequency of purchase of a customer" to determine when I would expect a customer to then purchase an item next. Say, the distribution of my customer base looks like ...
0
votes
0answers
6 views

Parallel analysis with rotated data

I am trying to do Factor analysis with varimax rotation for my data using R psych package. To determine number of factors I use R paran package. The problem I see is that eigenvalues produced by ...
0
votes
0answers
7 views

Log transformed data [on hold]

When you log transform data what is being tested on the original scale of the data? And why can we use our log transformed data to answer our scientific question? We are looking at data comparing ...
1
vote
0answers
12 views

Wald test for complex-valued data

Let $\mathbf{x} = [x_1,\ldots,x_N] \in \mathbb{C}^N$ be a set of complex-valued data and let $p(\mathbf{x};\alpha)$ their joint probability density function parametrized by the unknown complex scalar $...
3
votes
2answers
127 views

What is the intuitive meaning of having a linear relationship between the logs of two variables?

I have two variables which don't show much correlation when plotted against each other as is, but a very clear linear relationship when I plot the logs of each variable agains the other. So I would ...
0
votes
0answers
8 views

Do generative models have less degrees of freedom than discriminant models?

I've read here that generative models have less degrees of freedom than discriminant ones, so they are more robust and less prone to overfitting. I would like to understand this statement with a ...
0
votes
0answers
9 views

Validation loss oscillates a lot, validation accuracy > learning accuracy, but test accuracy is high. Is my model overfitting?

I am training a model, and using the original learning rate of the author (I use their github too), I get a validation loss that keeps oscillating a lot, it will decrease but then suddenly jump to a ...
0
votes
0answers
7 views

LightGBM custom L1-Loss not working properly [on hold]

I want to implement a custom loss function that penalizes predictions with the wrong sign stronger. The strength of the penalty is controlled by the variable "wrongSignPenaltyFactor" which is set to ...
0
votes
0answers
9 views

How to obtain all the true positives given two sets of precision and recall values?

Given that I have a dataset with some number of 1s (positives) and 0s (negatives), and the following two sets of precision and recall values: Set A: Positive precision: 0.5 Positive recall: 0.8 ...
0
votes
0answers
6 views

ACF and PACF ARMAacf seasonal (s)arima [on hold]

How do you use the ARMAacf() function in R if you have a seasonal arima model?
0
votes
0answers
8 views

A/B testing for binary classification tasks

I work on project where each user has an IoT sensor and an ML algorithm tries to identify a given situation. The user carries the sensor 24/7 but I split the time into 'sessions'. For each session, I ...
0
votes
0answers
7 views

Matern Covariance Function Changes in Smoothness and Alpha

I have a problem doing a simulation study in the case of the Matern, using one dimensional space (0,1). There are 100 equally spaced points in this space. Graphs are produced that show how the log ...
-1
votes
0answers
17 views

Ordinal Logistic Regression In R

I am applying Ordinal Logistic Regression to a dataset. And I have total of 18 variables. Class is my ordinal variable. It gives this error when I execute the code: Error in polr(Class ~ ., data = ...
0
votes
0answers
7 views

Categorical feature importance

I have a dataset of around 12,000 rows. I have a categorical dependent variable which can take a value of 1, 2 or 3. I have a number of independent categorical variables which can take many values. I ...
0
votes
0answers
8 views

prepare a data set to be fed into a machine learning model

so I am pretty new to the whole Machine Learning concept and I'm coming from a complete beginners standpoint. I am trying to detect whether there is steganography present within jpg files using a ...
1
vote
1answer
12 views

Random forest feature importance with max_depth = 1

I am using sklearn to estimate a random forest classifier. Out of curiosity I have set max_features=None and max_depth=1. ...
0
votes
1answer
13 views

Why use Convolution of probability function instead of cross-correlation?

First, let me quickly remind you of the two operations: convolution and cross-correlation between 2 function $f$ and $g$, assuming continuous domain. Cross-Correlation $f \star g$ : $\int f(\tau)g(\...
0
votes
0answers
4 views

Are there any exisitng scales (e.g. in a paper) for measuring perception of a short team task (or general tasks) in a questionnaire?

The task in concern is a one-time 10-20 second game to be played as a team. Scale can be Likert scale or metric scale (if possible). Perception is e.g. whether the game is boring/interesting, ...
0
votes
0answers
12 views

When do these two definitions of KL-divergence match?

Suppose $P$ and $Q$ are two distributions on a space ${\cal H}$ (could be a subset of an infinite dimensional function space) with p.d.fs denoted by the same letter then one can define the $KL$ ...
0
votes
0answers
5 views

Deriving the values of the range around the mean value

This is part of a bigger quantitative reasoning assignment I was working on. My understanding here is that the upper bound and lower bound of the ranges for each of the exercises should be reflected ...
0
votes
0answers
10 views

Should loss function be defined over output or parameters?

In machine learning loss is usually defined over the actual output and the predicted output $L(Y,\hat{Y}(X))$, while in statistics it's defined in the parameter space $L(\theta,\hat{\theta}(X))$. Why? ...
0
votes
0answers
7 views

Create dummy variables out of a text variable?

I'm trying to create dummy variables for a variable that has text data in rows. Data in 1st row is: ...
0
votes
1answer
29 views

Sample size (CI 95%)

I have two groups (treatment and placebo) and for each group I have the 95% CI mean calculated as per here. I want to do the same for calculating the 95% sample size as per table 11 of this paper: ...
1
vote
0answers
9 views

fitting non-normal multivariate distributions in R

I have many (n=317,823) observations on two variables. I want to fit a bivariate distribution to my observations, in order to identify descriptive features of the distribution (quantiles). However, my ...
-1
votes
0answers
22 views

How to get the Standard Deviation based on the sample [on hold]

The situation is I have 96 sets of paper, 4 questions on every paper and categorized into 5 categories.Every options consisting marks of 0,1,2,3,4. What's the formula to calculate the Standard ...
2
votes
1answer
22 views

A variance of how much is acceptable for a column before we can use it for modelling?

I am trying to create a classification model. While pre-processing the data. I look at the variance in each column. This is the amount of variance in each column. I am confused on which all columns ...
1
vote
0answers
3 views

interpreting rank difference trend

I am doing some research utilizing popularity ranks in Internet data. I've been able to model the rank data using Zipf Law effectively. I am interested in understanding how the rank changes, as well. ...
0
votes
0answers
11 views

State whether the statement are True or False for statistics modeling questions? And explain the reasoning [on hold]

For the first two statement, I think they are true, but I don't know how to explain them. For the last one I'm not sure whether it is true. The change deviance can be compared against chi-square to ...
0
votes
0answers
10 views

About using a neural network to train another neural network (on the same problem)

While experimenting with OpenAI gym to play a bit with reinforcement learning, I ended with using an approach that I can't exactly say what type of paradigm of training is, and I'd like to get your ...
0
votes
1answer
12 views

Long data format for Mixed linear modeling

I have a data set from an experiment with two conditions: a control condition, and a testing condition. It's an experiment performed in pairs. Each condition was undertaken by 20 pairs of subjects (...
0
votes
1answer
30 views

Uniform distribution of p-values

I have a question regarding the distribution of p-values. http://varianceexplained.org/statistics/interpreting-pvalue-histogram/ In this article it says that under the null hypothesis the ...
2
votes
2answers
45 views

T Test or Chi Square?

There are 2 variables, City and No Of Matches won. Variables: 1City Sydney Canberra 2No Of Matches Won? Sydney: 200 out of 300 Canberra: 210 out of 300 Question: Is team Canberra ...
0
votes
0answers
5 views

How to analyze EEG time-frequency data in a factorial design

I am trying to find the best strategy to analyze a set of EEG time-frequency data from 24 subjects, using 64 electrodes. This is an exploratory study as I have no a-priori hypothesis on where or when ...
0
votes
0answers
4 views

Feature engineering for verb\non-verb classification

Suppose we have data of X = words, and for each word we have a label indicating whether the word is a verb or a non-verb. So, y = labels. Assuming we can build all the unigrams and bigrams of each of ...
0
votes
0answers
5 views

Ordered statistics: Is the survivor function applied to a fixed point of the Mills ratio increases with the number of random variables

Let $X_1, ..., X_n$ be $n$ i.i.d. Let $F$ be their CDF and let $f$ be their PDF. Assume that $f$ is log-concave. The ordered statistic $\max_{i=1...n} X_i$ has for CDF $F_n\left(x\right) = \mathbb{P}\...
1
vote
0answers
21 views

Continuous time Fourier representation

I have learned that the Fourier transform of a continuous-time unit-periodic stochastic process is $x(t) = \sum\limits_{k=-\infty}^{\infty} a_k e^{i2\pi kt}$, where $a_k = \int\limits_{0}^{1} x(t)e^{-...
2
votes
0answers
9 views

Not testing for stationarity in a panel data set

Currently, I am analyzing a panel data set (different individuals over time) using panel data models with the lagged dependent variable as an explanatory variable. In this case the stationarity ...
0
votes
0answers
5 views

Finding significant explanatory values with unregularized logistic regression?

So i'm working currently with a colleague on a classification problem where both of us have different approaches. So i try to find the most important features through standard L1 regularization and he ...
0
votes
0answers
11 views

How to calculate bias and variance of Stacking model (a machine learning model)?

I'm doing bias-variance decomposition of Stacking model in regression case. The 1st-level learners include Random Forest, Extra-Trees and Gradient Boosting; then 2nd-level learner is Gradient Boosting....
0
votes
2answers
16 views

Are CNNs just an efficiency shortcut to Dense Layers?

I'm reading up on Convolutions in neural nets, and they seem like a neat and efficient way of finding "features" in the input. But am I right in thinking that a high enough number of layers and ...
0
votes
1answer
18 views

ARIMA(p,o,q) + GARCH(p,o,q) model

I found an article where they fit an ARIMA(p,o,q) model to a time series and then fit a GARCH(p,o,q) to the residual of the ARIMA (the parameters (p,o,q) are passed as the volatility model lags to ...
0
votes
0answers
9 views

Need some help/ advice with statistical methodology [on hold]

I was wondering if i could get some expert advice on how to analyse my data. I am trying to run the statistical analysis for a cross cultural study comprising of two country samples. I have 5 ...

15 30 50 per page