# All Questions

141,894 questions
Filter by
Sorted by
Tagged with
6 views

### randomForestOOB Error Rate Diverging with > nTrees

I am in a preliminary stage right now with classifying churned customers and have an interesting oob error rate chart that I would really like to get a better understanding of from someone more ...
7 views

### Dealing with rank deficiency when multiple regressors are inherently related / a non-binary ratio

I am running a linear mixed effects model in which three of the regressor are inherently related. For sake of conceptual example: let's say I would like to see how the relative time employees arrive ...
6 views

### Regression Prediction Interval with Population Data?

I have a set of data from a population I am studying. Note that the dataset is not a sample of the population, but consists of measurements from every single observation in the population on my ...
8 views

### Nemenyi Test Python

Lets say we have a 2D matrix defined as follows ...
10 views

### How to incorporate tolerance into the ANOVA test?

I am very new to the ANOVA test and I am curious to know how to improve the results since the P-value is much bigger than alpha. I have noticed that my readings are so close together and I am ...
21 views

### What do you call a statistical mean that is calculated from upper and lower extremes in any given dataset?

What do you call a statistical mean that is calculated from upper and lower extremes in any given dataset? For example, if you have a set: ...
4 views

### Should I use a Named Entity Recognition model or rather use simple heuristics?

I am relatively new in the field of NLP and I hope you will forgive me for my naive question, but why I don't understand whether I should use NER in my case. And if yes then why? Recently I have ...
3 views

### Parallelization in Caret with method = “timeslice”

Should parallelization work in the caret package when the method is set to timeslice? I use doMC to register the cores: ...
8 views

### Idea for a machine learning service or library to classify banking transactions

I'm looking for an online service or a library I can use to code a simple tool to classify my monthly banking transactions into a predefined set of categories (about 50). Every month, I retrieve my ...
10 views

### Asymptotic Mean Squared Error of Maximum Likelihood estimator

I want to show that $n$ times mean squared error for the maximum likelihood estimator converges to the inverse of Fisher information, where $n$ is the number of samples. But The standard proofs of ...
13 views

### Are there any statistical tests for which exact critical values are unknown?

I know that, for Jarque-Bera Test of Normality, critical values are simulated if the sample size is relatively small. Are there any other statistical tests for which critical values can only be ...
4 views

### Evaluating machine learning explainers?

I'm working on a project where multiple machine learning explainers (LIME and SHAP, potentially more coming) are applied to pre-trained models (neural networks) to help explain the predictions of ...
7 views

### Repeated Measure of 'robustness'

I have three small 20x5 matrices, with each one being collected on at a different time point. What I would like two is which columns show the smallest difference from each other across the 3 matrices,...
6 views

### Variation in shallow neural network

I have a binary classification problem for which I am using a shallow neural network. The input tensor shape is 7484x36 (7484 observations with 36 dimensions per observation). I have normalized ...
19 views

### Sampling from a mixture of product of Gaussians

I have the following density: $$p(y_*)=\int N(y_*\mid \mu(x_*),\Sigma(x_*) )\cdot N(x_*\mid Ax,Q)p(x,\theta)\ dx \ d\theta \ dx_*$$ with $\theta=(A,Q,\mu(\cdot),\Sigma(\cdot))$. How can I sample ...
25 views

### Test all models possibles is a good manner to choose the “best” model?

I have programmed a function in python to test all possibles linear regression models that I can do with 5 variables. I choose the "best" model in base its AIC and BIC. These models are ...
12 views

### Keras: validation loss decreases but accuracy does not increase

I am working on the development of a deep learning model for prediction of a disease from medical images. It is a binary classification algorithm. I am currently using a model built from scratch with ...
8 views

### Tuning the “strength” of updates to a posterior distribution for conjugates

I'm asking this question as a sanity check- I am not a statistician or research scientist, and just am doing a gut check on a model I am building. I want to quantify uncertainty of a specific metric ...
8 views

### Random Forest / Gradient Boosting with Random Effects

I want to run random forest regression (or the gradient boosting equivalent) on a dataset whose rows are not independent. Specifically, the rows are clustered, and you could consider the clustering ...
10 views

### Proof of Bandit Gradient Algorithm in Suttons Book

I am trying to understand a step in the proof of the "Bandit Gradient Algorithm As Stochastic Gradient Ascent" in Suttons Book (http://incompleteideas.net/book/RLbook2018.pdf) at page 39. Starting ...
11 views

### Fitting a regression model with spurious variables: do they vanish with large samples?

I am fitting the usual linear regression model $$y_j = x_j^T\beta + e_j,$$ where the errors $e_j$ are iid normal with unknown variance. If the vector of covariates $x_j$ contain spurious variables, ...
17 views

### Fitting distribution to log response ratios [on hold]

I am trying to analyze variables on repeated measures with a randomized block design, and used log response ratios to do so: $$ln\left(\frac{Y_{control}}{Y_{treatment}}\right)$$ I would like to model ...
23 views

### Low effective sample size but good R-hat is this a problem?

I am using Stan (Hamiltonian Monte-Carlo) to run a highly paramaterized model. One of the parameters in particular has a very low effective sample size (n_eff < .10*number of retained draws), but ...
13 views

### How can you interpret the ratio of the coefficients travel time to the coefficient of travel cost

Just the e) question This is the whole question, thank you all in advance.
9 views

### how to count the number of peaks in this time series using python code [on hold]

import matplotlib.pyplot as plt import pandas as pd import numpy as np import tkinter as tk from tkinter import filedialog import time from datetime import datetime from scipy import signal import ...
20 views

### Is there a metric for data where only true positives are labeled (no true negatives)?

Let's say I have a dataset where each item is labeled with either (1) true positive or (2) unknown (could be true positive, could be true negative). It seems like if there are only true positives ...
8 views

### PCA that includes residuals scores AND the original regressed variable?

I have several variables that I included in a PCA: body length, mass, and development stage. We often calculate a variable we call 'condition'm which is the residuals of mass ~ (body length)^3. ...
18 views

### Confidence interval fit around GAMMA and BETA distribution

So currently I am using the new GAMLSS package and have plotted my model as shown in the photo. The blue line is my estimate (which I want to use at my point of estimate) when attempting to create a ...
7 views

### Power analysis for 3-level HLM model with lmer4 (longitudinal within person design)

I have been wracking my brain trying to figure out how to do a power analysis using simr with a two-level multilevel model. I have observations nested in people and am trying to do a post-hoc power ...
13 views

### Problem regarding visualizing high dimensional data using PCA

I have 10000 samples, each of which has 100 features. To visualize this high dimensional sample, I use 2 component PCA. Here is a the result: Here, results are colored based on a target value for ...
14 views

### Interpretation of simple regression results with a single categorical variable

I am slightly confused about my regression results. I have a SLR model with a single, dichotomous categorical variable. The results are: ...
9 views

### Quality of a quantile regression learner

Given a learning algorithm that selects and trains quantile models, how do we evaluate it? One idea is to - use the algorithm to train a model on a synthetic dataset with labels drawn from an ...
28 views

### Rolling d20, d12, d10, d8, d6 in a row, what's the probability of a 1 on at least one of the rolls?

So I roll a d20, d12, d10, d8, d6, one at a time, in a row. What are the chances of at least one roll to be a 1? And how do you calculate it?
12 views

### How can I calibrate my sigmas for Gaussian process regression?

I have a large-ish set of unevenly-spaced time-series data from instruments around the world, for which I'm using Gaussian process regression to do interpolation and short-term future prediction. My ...
4 views

### my Durbin watson is coming as 1.068 indicating low auto correlation, what to do, multi-collinearity does not exist since VIF is undeer 10

Durbin watson is not lying in range 1.5-2.5 No multi-collinearity exists.shoould we proceed with analysis or not
16 views

### Kronecker product/vectorisation proof

Apologies for the vague title... I was unsure what to name this question. I want to prove a lemma using the following notation but am unsure how. $vec(A)$ is the vectorisation of matrix $A$. I.e. it ...
10 views

I'm trying to apply the central limit theorem to a test statistic. The statistic is essentially a weighted Binomial given by $$T_{n} = \sum_{i=1}^{n}p_{i}w_{i}$$ The $p_{i}'s$ are 1 with probability $... 1answer 11 views ### Convolutional NN feature map mixing and filters dimension I have two question about convolutional neural networks. The first: after we do a layer of convolution and get a set of feature maps are the next convolutional layer filters applied distinctly for ... 0answers 12 views ### How to minimise and expectation with respect to a parameter? Suppose that$X$is a random variable with distribution$G$. Let$H(X;\theta)$be a parametric function with$\theta \in \Theta \subset {\mathbb R}^p$. I want to maximize the function$$\varphi(\... 0answers 10 views ### Error in model.frame.default(Terms, newdata, na.action = na.action, xlev = object$xlevels) : factor X has new levels Unknown [on hold]

I Have received a .rds file from my former colleague which is a trained file. When i am using it for predicting i am getting errorError in model.frame.default(Terms, newdata, na.action = na.action, ...
18 views

### Do I estimate factor loadings in a confirmatory factor anlysis (CFA) aimed at verifying an exploratory factor analysis (EFA)?

I decided to use a questionnaire published by another researcher (paper and supplementary here). In the article they perform an EFA, find two factors, and report the resulting factor loadings (...
12 views

9 views

### How to compute derivatives and periods of change for interactions in GAMS?

I have been stuck in this problem for weeks, and I am not sure if I will be ever able to find a solution without your help. I am analyzing the relation between tweets and temperatures. I believe that ...
9 views

### How can I make XGBoost have an exponentially distributed output?

The input distribution is exponential, but XGBoost's predictions' distribution is always unimodal but not exponential. Is there a way to make it exponential?
8 views

### How to train a single discrete-time markov model?

I have a training set of sequences. I want to reach a discrete time Markov model (transition probability matrix). Is there a Bayesian way other than MLE to achieve this?
5 views

### Adjustment for multiple comparisons in complex cases

I use the emmeans package for post-hoc pairwise comparisons with different types of models. Standard contrast families are e.g. ...

15 30 50 per page