All Questions

0
votes
3answers
32 views

Guessing the gender of someone submitting a multiple choice questionnaire

If I have 100 people fill out a multiple choice questionnaire (containing 10 questions with 5 answer choices for each question) and the respondent writes their gender on their questionnaire, what is ...
0
votes
0answers
11 views

Ranking users by mean daily message count and least variation

I would like to rank users of a text and voice communication app by the number of messages sent, the amount of time spent on voice calls and number of interactions. This ranking should be updated in ...
0
votes
0answers
6 views

Data augmentation for animal measurements

I am looking for methods to do data augmentation for an animal study. What I want to do is generate a large number of data points by using a min and max measurements of the animal. So I have 2 ...
0
votes
0answers
7 views

Hyperparameter-free method for Moving Average/ Exponential smoothing?

I want to find hyperparameter-free method for Moving Average/ Exponential smoothing. Is there any related paper or python code? S(t)= alpha * F(t) + (1-alpha) * S(t-1) Any methods can avoid the ...
0
votes
0answers
12 views

Changing a conditional probability to a deterministic function [on hold]

Suppose that we have a conditional density function $p(y|x;\theta^*)$, where $\theta^*$ represents distribution parameters and are assumed to be deterministic. Is it possible that we write this ...
-2
votes
0answers
17 views

What Would Be The Best Scikit Learn Alogriths For Lottery Numbers? [on hold]

I want to start a machine learning model on lottery results using ScikitLearn. Below is the head of my database. If anyone has any suggestions I would appreciate it. Thanks. Head of Database: ...
0
votes
0answers
9 views

Estimating yearly diagnosis rate from multiple studies

I want to estimate yearly diagnosis rates from some 8 studies. Each of the 8 studies contains the following data: $n$ = total number of patients (e.g., 100) $t_{start}$ = start timestamp of the ...
2
votes
1answer
15 views

trend stationary with external regressors

Suppose I have two trend - stationary time series with strong correlation. In the case where there are no regressors, if a time series is trend-stationary, it becomes stationary by subtracting a ...
0
votes
0answers
8 views

Role of different exponential families in generalized linear models

I've gone through a variety of introductions to generalized linear models (GLM), and there's always a point in the discussion that confuses me. The story always begins saying that $P(y|x)$ belongs to ...
1
vote
1answer
8 views

a covariate versus a random effect [duplicate]

I have looked around on cross validated as well as other places but can't seem to find an answer. I'm running a generalized linear mixed-effects model. Y~initial abundance + Treatment + (1|Month) ...
0
votes
0answers
7 views

Comparing Gaussian and Poisson GLM when applied to count data; “Chi-Squared Error”

I have a fixed set of predictors ($[x_1,x_2,...,x_p]$), which I'm using to fit a GLM for univariate responses ($y_1$, $y_2$,...) of various types. E.g. I fit a GLM for $y_1 \sim [x_1,x_2,\dots,x_p]$, ...
0
votes
0answers
11 views

Meta Learning On Large Datasets

In a nutshell, the goal of meta-learning is to use a corpus of datasets to solve problems on small datasets (e.g. few-shot or zero-shot learning). Explicitly, we have a collection of datasets $\...
0
votes
0answers
9 views

Time complexity for locality sensitive hashing similar image search

I am trying to find most visually similar images for large image dataset. (N=1 million), using LSH (Locality Sensitive Hashing). Image feature vectors are 4096 dimensional VGG-16 features. Now, my ...
0
votes
0answers
8 views

Having trouble with counting problems

Im finding some permutations and combinations problems difficult to solve. Are there any tips for doing problems like these? I know the formulas but am always confused on how to use them
0
votes
0answers
10 views

When to use a certain sampling technique

I'm taking a class where we learn about different sampling techniques. For the most part it has come down to SRS, Ratio/Regression/Difference estimation, Stratified random sampling, Systematic ...
1
vote
1answer
20 views

Relationship between weak and covariance stationary

I have read that the definition of weak stationary is : $ Mean(t) = mean(t + \tau)\\ Cov(t_1,t_2) = cov(t_1-t_2,0)\\ E[|x(t)|^2] < \infty $ In this definition of weakly stationary, can the ...
1
vote
1answer
10 views

Model Size of Random Forest VS CatBoost

I trained models based on the same dataset, using random forest (sklearn) and CatBoost. I use n_estimators=1000 for random forest, and n_estimators(iterations)=1000 for CatBoost. The random forest ...
2
votes
2answers
30 views

What's the role of the data in Gibbs Sampling?

Trying to wrap my mind around Gibbs Sampling. Across many answers in this same forum, I constantly notice that the examples shown do not actually require an observed data set (First example (with R ...
0
votes
0answers
7 views

How to build an interval around a final Logistic Regression classification prediction?

I have a data set of 5000 hotel reviews. My boss wanted to know how many of the reviews were 'negative', so I set out to build a model. I began by tagging 200 of the reviews 'positive' or 'negative'. ...
0
votes
1answer
11 views

How to represent numerically the diversity of groups of datapoints with a single value?

I work for a performing arts organization, and am trying to create a numerical figure to represent the amount of genre-diversity in each of our seasons. So let's say that we have a Spring Season and a ...
0
votes
0answers
6 views

What could be a good quality index for an OCR scanner? [on hold]

I have an OCR (Optical Character Recognition) that, given the image of a word, tries to detect the letters giving, for each letter, the accuracy as a 0-1% probability. For example suppose that I have ...
1
vote
1answer
18 views

standard errors for computing means via OLS regression

Suppose I have a variable $y_{i,t}$ which is say a well-being index for individual of type $i$ in month $t$. There are 2 types of individual, $A$,$B$, which are mutually exclusive, and three non-...
2
votes
0answers
20 views

Don't understand why glmm random effect variance is zero. Have reviewed similar questions still dont get it

I study a colonially-nesting bird species. I am trying to perform an AICc evaluation of GLMMs for a nest site selection study. I collected data at nest sites and paired random sites. I want to ...
0
votes
0answers
11 views

Automate plotting individual side-by-side barplots of many data [on hold]

I have hundreds of tuples of means to plot on barcharts, along with unique error bars for each of the tuple of means. I want to know how to automate plotting them rather than plotting every tuple of ...
0
votes
0answers
15 views

In an RNN, if the gradients don't vanish for long/distant terms, won't the derivative of the error be either divergent to infinity or oscillatory?

In my question below, I'll refer to the following well known paper by Pascanu et. al: https://arxiv.org/pdf/1211.5063.pdf Assume we're working on somewhat simplified RNN, where, by their equation (...
1
vote
0answers
7 views

Fitting an ANCOVA model using aov or ANCOVA

I have data that I am trying to examine which looks something like: ...
0
votes
0answers
18 views

Smaller dataset or bigger but risking ovb?

I am testing whether financial access is correlated with the education of young individuals using a longitudinal panel. Among the controls I am using there is parents' education that is a determinant ...
0
votes
3answers
18 views

How to determine the values closest to the highest value in a data set?

I have a dataset which contains values calculated through a formula that processes a set of variables. The values are continuous, it's corresponding barplot graphic is: What I want to detect is the ...
0
votes
1answer
13 views

How does the sliding window work?

I am not sure how the "Sliding window" method work. Let's assume I have a dataset of number of logins by hour. a) A window of 24hours to predict the next 24h? b) A window of 24h to predict the next ...
0
votes
0answers
20 views

Variance of autocorrelation

At this link I have seen the following formula whereas $$ r_k = \frac {\sum_{t=k+1}^n a_t*a_{t-k}} {\sum_{t=1}^n a_t^2}$$ $$Var(r_k) = \frac {n-k}{n*(n+2)}$$ where $r_k$ is the autocorrelation at ...
0
votes
0answers
9 views

d to r conversion for extremely high versus extremely low groups

Context Suppose I am performing a meta-analysis on the relationship between two variables (say depression and life satisfaction). Unfortunately, many studies in the prior literature do not report ...
0
votes
0answers
11 views

How to find association between time spent and capital raised

I have a data set that I would like to analyse in order to determine the most efficient timeframe to fall within when meeting with clients (in terms of capital formed): ...
0
votes
0answers
14 views

Neural network model has way more features than samples but yields good test accuracy

I am recently doing a bioinformatic machine learning project. We have over 470,000 features and only 700 training samples and 300 test samples. We used a perceptron with 1 hidden layer to train. ...
1
vote
1answer
21 views

Definition of exponential family with dispersion parameter

I was recently reading a discussion of generalized linear models that considered the response to come from an exponential family with a dispersion parameter so $$ f(y|\theta,\phi) = \exp\left(\frac{y\...
0
votes
0answers
8 views

How do I set up my hyper-parameter space for optimizing a convolutional neural network (using packages Skopt and Tensorflow)

I just finished building a 1D CNN using TensorFlow, and I want to optimize a variety of hyper-parameters using Scikit-Optimize (skopt) (although, I would be willing to use whatever optimization ...
0
votes
0answers
8 views

Forecasting Process with Limited Historical Data and High Variance

I have a general inquiry regarding a project I am working on. I cannot reveal too much, but I would like to gauge the community here and hopefully be pointed towards the right direction in terms of ...
0
votes
0answers
12 views

onehotencoder issue [duplicate]

New to DNN. I have the following dataset My target value is "Purchase" column. I know I need to drop "product_id" column and create dummies for "Gender" and "City_category", but do i need to do ...
0
votes
0answers
10 views

Which robust covariance estimator to use if I want to 1) robust against outlier 2) stay positive definite

I am now working on a very noisy dataset $X$ , which is a m-by-n matrix, with m samples, each contains n features The n-by-n covariance matrix can be naively calculcated as $X^{T}X$ However, my ...
0
votes
0answers
7 views

Using groupwise averages of dependent variable as predictors

Let's say I have a binary classification problem with a number of categorical predictors. Is there an issue with creating new features by calculating the group wise average of one or more of the ...
1
vote
0answers
20 views

Repeated Measures in Mixed Model - Creating Graphics

In learning about mixed modeling and how to perform this in SAS I came across this article https://stats.idre.ucla.edu/sas/seminars/sas-repeatedmeasures/ However, I am not able to wrap my head ...
2
votes
1answer
36 views

Power calculation for two-sample Welch's t test

The R function power.t.test does power calculations (outputs power, sample size, effect size, or whichever parameter you leave out) for t-tests, but only has a ...
0
votes
2answers
32 views

What test do I use to check if two samples came from different population?

I'm aware that t-test checks if two sample data sets have the same difference in means with confidence. But testing difference in means is not equivalent to testing if the two distributions came from ...
0
votes
0answers
10 views

Conditional probability of n sequential time series points

I have hourly road traffic volume data. While I assume the volume data can be defined as normally distributed, there is some correlation. i.e. The traffic in the past hour may affect the traffic in ...
1
vote
0answers
7 views

Elastic net/LASSO with soft labels

Sometimes you do not have firm Y/N labels, but e.g. 80% probability of Y as a label. E.g. this happens, if you train a model on a small amount of labelled data, predict for a large amount of ...
0
votes
0answers
13 views

C5.0 - How to interpret a Tree of size 1?

I am using C5.0 to try to predict an output. The code is rather simple : ...
0
votes
0answers
10 views

What transformations can I use for time series with negative values?

Currently, I am working with different time series. Some of the series that I have can have all negative values or positive and negative values. When I have a time series with all positive values I ...
2
votes
0answers
8 views

How to get one composite score to represent a MIMIC factor (including both formative variables and reflective indicators)

I have a MIMIC model where the latent variable of interest has both formative causes and reflective indicators (as depicted in the figure). My question is: is there a way to get one composite score ...
1
vote
1answer
54 views

Is it true that Bahdanau's attention mechanism is not Global like Luong's?

I was reading the pytorch tutorial on a chatbot task and attention where it said: Luong et al. improved upon Bahdanau et al.’s groundwork by creating “Global attention”. The key difference is that ...
0
votes
1answer
8 views

Two datasets with same length give different number of extremes

I have two datasets of a given variable x that have the same length, let's say 14600 values in total each one. I need to extract the extreme observations within ...

15 30 50 per page