All Questions

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

randomForestOOB Error Rate Diverging with > nTrees

I am in a preliminary stage right now with classifying churned customers and have an interesting oob error rate chart that I would really like to get a better understanding of from someone more ...
1
vote
2answers
7 views

Dealing with rank deficiency when multiple regressors are inherently related / a non-binary ratio

I am running a linear mixed effects model in which three of the regressor are inherently related. For sake of conceptual example: let's say I would like to see how the relative time employees arrive ...
0
votes
0answers
6 views

Regression Prediction Interval with Population Data?

I have a set of data from a population I am studying. Note that the dataset is not a sample of the population, but consists of measurements from every single observation in the population on my ...
0
votes
0answers
8 views

Nemenyi Test Python

Lets say we have a 2D matrix defined as follows ...
0
votes
1answer
10 views

How to incorporate tolerance into the ANOVA test?

I am very new to the ANOVA test and I am curious to know how to improve the results since the P-value is much bigger than alpha. I have noticed that my readings are so close together and I am ...
2
votes
1answer
21 views

What do you call a statistical mean that is calculated from upper and lower extremes in any given dataset?

What do you call a statistical mean that is calculated from upper and lower extremes in any given dataset? For example, if you have a set: ...
0
votes
0answers
4 views

Should I use a Named Entity Recognition model or rather use simple heuristics?

I am relatively new in the field of NLP and I hope you will forgive me for my naive question, but why I don't understand whether I should use NER in my case. And if yes then why? Recently I have ...
0
votes
0answers
3 views

Parallelization in Caret with method = “timeslice”

Should parallelization work in the caret package when the method is set to timeslice? I use doMC to register the cores: ...
0
votes
0answers
8 views

Idea for a machine learning service or library to classify banking transactions

I'm looking for an online service or a library I can use to code a simple tool to classify my monthly banking transactions into a predefined set of categories (about 50). Every month, I retrieve my ...
0
votes
0answers
10 views

Asymptotic Mean Squared Error of Maximum Likelihood estimator

I want to show that $n$ times mean squared error for the maximum likelihood estimator converges to the inverse of Fisher information, where $n$ is the number of samples. But The standard proofs of ...
0
votes
1answer
13 views

Are there any statistical tests for which exact critical values are unknown?

I know that, for Jarque-Bera Test of Normality, critical values are simulated if the sample size is relatively small. Are there any other statistical tests for which critical values can only be ...
0
votes
0answers
4 views

Evaluating machine learning explainers?

I'm working on a project where multiple machine learning explainers (LIME and SHAP, potentially more coming) are applied to pre-trained models (neural networks) to help explain the predictions of ...
0
votes
0answers
7 views

Repeated Measure of 'robustness'

I have three small 20x5 matrices, with each one being collected on at a different time point. What I would like two is which columns show the smallest difference from each other across the 3 matrices,...
0
votes
0answers
6 views

Variation in shallow neural network

I have a binary classification problem for which I am using a shallow neural network. The input tensor shape is 7484x36 (7484 observations with 36 dimensions per observation). I have normalized ...
0
votes
0answers
19 views

Sampling from a mixture of product of Gaussians

I have the following density: $$p(y_*)=\int N(y_*\mid \mu(x_*),\Sigma(x_*) )\cdot N(x_*\mid Ax,Q)p(x,\theta)\ dx \ d\theta \ dx_*$$ with $\theta=(A,Q,\mu(\cdot),\Sigma(\cdot))$. How can I sample ...
2
votes
2answers
25 views

Test all models possibles is a good manner to choose the “best” model?

I have programmed a function in python to test all possibles linear regression models that I can do with 5 variables. I choose the "best" model in base its AIC and BIC. These models are ...
0
votes
0answers
12 views

Keras: validation loss decreases but accuracy does not increase

I am working on the development of a deep learning model for prediction of a disease from medical images. It is a binary classification algorithm. I am currently using a model built from scratch with ...
0
votes
0answers
8 views

Tuning the “strength” of updates to a posterior distribution for conjugates

I'm asking this question as a sanity check- I am not a statistician or research scientist, and just am doing a gut check on a model I am building. I want to quantify uncertainty of a specific metric ...
0
votes
0answers
8 views

Random Forest / Gradient Boosting with Random Effects

I want to run random forest regression (or the gradient boosting equivalent) on a dataset whose rows are not independent. Specifically, the rows are clustered, and you could consider the clustering ...
0
votes
0answers
10 views

Proof of Bandit Gradient Algorithm in Suttons Book

I am trying to understand a step in the proof of the "Bandit Gradient Algorithm As Stochastic Gradient Ascent" in Suttons Book (http://incompleteideas.net/book/RLbook2018.pdf) at page 39. Starting ...
0
votes
0answers
11 views

Fitting a regression model with spurious variables: do they vanish with large samples?

I am fitting the usual linear regression model $$y_j = x_j^T\beta + e_j,$$ where the errors $e_j$ are iid normal with unknown variance. If the vector of covariates $x_j$ contain spurious variables, ...
0
votes
0answers
17 views

Fitting distribution to log response ratios [on hold]

I am trying to analyze variables on repeated measures with a randomized block design, and used log response ratios to do so: $$ln\left(\frac{Y_{control}}{Y_{treatment}}\right)$$ I would like to model ...
2
votes
1answer
23 views

Low effective sample size but good R-hat is this a problem?

I am using Stan (Hamiltonian Monte-Carlo) to run a highly paramaterized model. One of the parameters in particular has a very low effective sample size (n_eff < .10*number of retained draws), but ...
0
votes
0answers
13 views

How can you interpret the ratio of the coefficients travel time to the coefficient of travel cost

Just the e) question This is the whole question, thank you all in advance.
-2
votes
0answers
9 views

how to count the number of peaks in this time series using python code [on hold]

import matplotlib.pyplot as plt import pandas as pd import numpy as np import tkinter as tk from tkinter import filedialog import time from datetime import datetime from scipy import signal import ...
1
vote
0answers
20 views

Is there a metric for data where only true positives are labeled (no true negatives)?

Let's say I have a dataset where each item is labeled with either (1) true positive or (2) unknown (could be true positive, could be true negative). It seems like if there are only true positives ...
1
vote
0answers
8 views

PCA that includes residuals scores AND the original regressed variable?

I have several variables that I included in a PCA: body length, mass, and development stage. We often calculate a variable we call 'condition'm which is the residuals of mass ~ (body length)^3. ...
1
vote
0answers
18 views

Confidence interval fit around GAMMA and BETA distribution

So currently I am using the new GAMLSS package and have plotted my model as shown in the photo. The blue line is my estimate (which I want to use at my point of estimate) when attempting to create a ...
1
vote
0answers
7 views

Power analysis for 3-level HLM model with lmer4 (longitudinal within person design)

I have been wracking my brain trying to figure out how to do a power analysis using simr with a two-level multilevel model. I have observations nested in people and am trying to do a post-hoc power ...
0
votes
1answer
13 views

Problem regarding visualizing high dimensional data using PCA

I have 10000 samples, each of which has 100 features. To visualize this high dimensional sample, I use 2 component PCA. Here is a the result: Here, results are colored based on a target value for ...
0
votes
1answer
14 views

Interpretation of simple regression results with a single categorical variable

I am slightly confused about my regression results. I have a SLR model with a single, dichotomous categorical variable. The results are: ...
0
votes
0answers
9 views

Quality of a quantile regression learner

Given a learning algorithm that selects and trains quantile models, how do we evaluate it? One idea is to - use the algorithm to train a model on a synthetic dataset with labels drawn from an ...
1
vote
1answer
28 views

Rolling d20, d12, d10, d8, d6 in a row, what's the probability of a 1 on at least one of the rolls?

So I roll a d20, d12, d10, d8, d6, one at a time, in a row. What are the chances of at least one roll to be a 1? And how do you calculate it?
0
votes
0answers
12 views

How can I calibrate my sigmas for Gaussian process regression?

I have a large-ish set of unevenly-spaced time-series data from instruments around the world, for which I'm using Gaussian process regression to do interpolation and short-term future prediction. My ...
0
votes
0answers
4 views
0
votes
0answers
16 views

Kronecker product/vectorisation proof

Apologies for the vague title... I was unsure what to name this question. I want to prove a lemma using the following notation but am unsure how. $vec(A)$ is the vectorisation of matrix $A$. I.e. it ...
0
votes
0answers
10 views

Applying Central Limit theorem to weighted binomial

I'm trying to apply the central limit theorem to a test statistic. The statistic is essentially a weighted Binomial given by $$T_{n} = \sum_{i=1}^{n}p_{i}w_{i} $$ The $p_{i}'s$ are 1 with probability $...
0
votes
1answer
11 views

Convolutional NN feature map mixing and filters dimension

I have two question about convolutional neural networks. The first: after we do a layer of convolution and get a set of feature maps are the next convolutional layer filters applied distinctly for ...
0
votes
0answers
12 views

How to minimise and expectation with respect to a parameter?

Suppose that $X$ is a random variable with distribution $G$. Let $H(X;\theta)$ be a parametric function with $\theta \in \Theta \subset {\mathbb R}^p$. I want to maximize the function $$\varphi(\...
0
votes
0answers
10 views

Error in model.frame.default(Terms, newdata, na.action = na.action, xlev = object$xlevels) : factor X has new levels Unknown [on hold]

I Have received a .rds file from my former colleague which is a trained file. When i am using it for predicting i am getting errorError in model.frame.default(Terms, newdata, na.action = na.action, ...
2
votes
1answer
18 views

Do I estimate factor loadings in a confirmatory factor anlysis (CFA) aimed at verifying an exploratory factor analysis (EFA)?

I decided to use a questionnaire published by another researcher (paper and supplementary here). In the article they perform an EFA, find two factors, and report the resulting factor loadings (...
0
votes
0answers
12 views

MCMC with small Log-Likelihood

I am using MCMC with metropolis hasting to determine the value of $\theta$, I have a function $f(x,\theta)$ which I have obtained points from to determine the Log-likelihood (using 10 inputs, $x$) $...
0
votes
0answers
4 views

Dealing with (# of events) = 0 in a bin when computing weight of evidence (WOE)

I want to compute the WOE of X (categorical) and Y (binary, 0 or 1). But one bin does not contain any Y = 0. As a result, I cannot compute the WOE of this bin. How should I proceed to compute the ...
0
votes
0answers
17 views

How to recover the original standard deviation from log and standardized data in the raw standard deviation scale

In order to preserve the anonymity of a client, we receive data in three forms: in one, the outcome variable is mean-centered, in another the outcome variable is first logged and then mean-centered, ...
1
vote
2answers
37 views

ANOVA or Linear Mixed Model?

I have been running several linear mixed effects models for some data of my current project, and now I'm moving on to different data I have. I say that because I'm in a mindset to use LME, and didn't ...
0
votes
0answers
27 views

Least squares with correlated noise

Assume I have the system \begin{equation} x(k+1) = \begin{bmatrix}A & B\end{bmatrix}\begin{bmatrix}x(k) \\ u(k)\end{bmatrix}+ w(k) \end{equation} where $x(0)\sim\mathcal{N}(0,\sigma_x^2)$ and $...
0
votes
0answers
9 views

How to compute derivatives and periods of change for interactions in GAMS?

I have been stuck in this problem for weeks, and I am not sure if I will be ever able to find a solution without your help. I am analyzing the relation between tweets and temperatures. I believe that ...
1
vote
1answer
9 views

How can I make XGBoost have an exponentially distributed output?

The input distribution is exponential, but XGBoost's predictions' distribution is always unimodal but not exponential. Is there a way to make it exponential?
0
votes
0answers
8 views

How to train a single discrete-time markov model?

I have a training set of sequences. I want to reach a discrete time Markov model (transition probability matrix). Is there a Bayesian way other than MLE to achieve this?
1
vote
0answers
5 views

Adjustment for multiple comparisons in complex cases

I use the emmeans package for post-hoc pairwise comparisons with different types of models. Standard contrast families are e.g. ...

15 30 50 per page