# All Questions

141,799 questions
Filter by
Sorted by
Tagged with
8 views

### Finding the weights of individual children items given the actual parent assembly/unit weights

I'm jumping head first into learning machine learning by trying to solve an actual problem at work. I'm trying to find individual weights of items/options that make up the total weights of assemblies/...
22 views

### Choosing which chi-square test to use when working with count data

I am analyzing data that compares responses to various questions across a variety of demographics. I want to see how the count of each response varies by each demographic group. The most basic test to ...
43 views

### Why estimate using GEEs inspite of their disadvantages over using ML?

I know sometimes I want to know the population-level estimates, but the problem with GEEs is I can't calculate the likelihood, and therefore all models I make with it aren't comparable, and I don't ...
20 views

### Non-linear Least squares fitting in R

I want to fit a curve to a set of 3 data points (1,65), (200,70), (800,75) in the form $y = \frac{ax}{b+x}$ Using the set of points I have this set of equations $a-65b = 65$ $200a-70b = 14000$ ...
6 views

### Estimating false accept rates from imposter scores below a threshold

I have a system that compares two items and produces a match score. Scores below a threshold are manually inspected to determine if they match or don't(imposter). Scores above the threshold are ...
9 views

### How to show or prove a dataset is not linearly separable

I am looking to be pointed in the right direction. I am learning about kernels and I have a homework assignment to use the dual perceptron algorithm to classify datapoints from a spiral dataset, with ...
40 views

### Expanding initial sample when the result isn't significant

This inspiring answer describes a variant of hypothesis, and I want to analysis its property further. Basically, it considers a two-sided test and interprets the $p$-value as a measure of how strong ...
27 views

### Machine learning and feature selection [on hold]

I developed my algorithm, Hybrid SVM algorithm, and correlation-based feature selection (for network intrusion detection), I have a suspicion that If I want to compare performance with other methods ...
7 views

### (How) Does different number of categories affect correlation?

I work with obesity in cats and many studies use a body condition score (BCS) to assess obesity. This is a somewhat subjective measure of how much fat covering an animal has. There are two commonly ...
8 views

### Only paralleling process in cluster Spark [on hold]

I have built a spark cluster using 6 computers, and a master node. I want a process that does not handle large databases and I want to repeat this process many times. Using the interactive shell ...
25 views

### having trouble interpreting confint function using R (logistic regression) [duplicate]

I'm trying to interpret these results of using R confint function , but I can not understand. This is a logistic regression about breast cancer. how to interpret confint function. What the 2.5% and 97....
7 views

### Classifier for detecting abnormal noise using FFT

As a simple machine learning application for quality evaluation, I want to detect abnormal sound in a product. A good product would make a signature humming sound, a bad product might make noises due ...
19 views

### How do I decompose significant main effects and interactions in a mixed model with multilevel factors? [on hold]

I have an ordinal mixed model for exploration and am wondering how to decompose significant main effects and interaction with multi-level predictors. In the 'main model' my predictors are centralised ...
10 views

### Setting Choosing alpha for generalized extreme Studentized deviate ESD

I'm working on a S-H-ESD implementation and I'm struggling to set the alpha for the ESD. The suggested alpha everywhere is 0.05. Is there a way to calculate an alpha based on the expected percent of ...
18 views

### Using a separate but related dataset for feature extraction (transfer learning)

I have two datasets of MRI images: a larger one of Altzheimer's paitents (AD), which is about 3 times the size of a smaller dataset of brain tumor paitents (BT). My aim is to make use of the AD data ...
13 views

### Temporal difference definition (Reinforcement Learning)

Reading for instance Szepesvari or this : i struggle to understand the rationale behind the temporal-difference definition $\delta_{t}=R_{t}+\gamma V_{k}(x_{t+1})-V_{k}(x_{t})$ with the notation ...
35 views

### Linear Regression : Can I use both levels and changes in the same model?

I have a linear model with 1 predictor variable in the form of: $Y = a + b_{1}*X$ Both $X$ and $Y$ are stationary variables and the fit of the model is good. I have also created 2 other models based ...
14 views

### a proper way of clustering 3D vectors(XYZ) [on hold]

I am new to clustering and I need to cluster a set of vectors in cartesian space (3d vectors - XYZ in space). It means that I have three points in space with a force vector on each of them. I wanna ...
35 views

### Modeling non-linear (short) time series and cross-validate them

beginner data scientist here. Time series analysis is a completly new area for me, so please correct me if i write something that makes no sense. I have many multivariante short time series, between ...
13 views

### Maximum entropy prior for r.v. supported on real line with no other constraints?

What would be a suitable maximum entropy prior for a random variable supported on the real line with no other constraints (i.e. unknown mean, unknown variance, unknown bounds)? All kinds of answers (...
14 views

### Increment columns of a data frame in a for loop in R [on hold]

For example, I have the closing prices for these companies Now I want to calculate the break points for each of these companies. I am going to use the function breakpoints from the package ...
23 views

### Number of Model Combinations

I have a model selection combinatorics problem. My covariate pool consists of six covariates: A1, A2, A3, B, C and D. However, covariates A2 and/or A3 can only be included if A1 is also included. ...
12 views

### Why is the SVM in canonical representation necessary?

For my exam preparation I have following question from an old exam: I know that the SVM in canonical representation is $$\min_{i=1,...,n} |\langle \omega,x_i \rangle + b | = 1$$ So why is it ...
8 views

### n for Hopkins statistics using get_clust_tendency

I started to use the Hopkins statistics to establish, if a dataset is 'clusterable'. I am using the following code - taken from here: ...
60 views

### Combining together principal components from PCA performed on different subsets of a large dataset

I'm trying to QA a process in which the data has over a million rows with approx 60,000 variables in a binary form. The aim of the process was to perform k-means clustering, but prior to this, the 60,...
21 views

### Ideal method for confounder adjustment

I have a fairly large dataset (15,000) in which I assessed the association of a three-level categorical risk factor to the outcome. The outcomes were significantly different for the three levels. ...
30 views

### What's the expected average time

We have the following situation: We are trying to find the path to a coffee shop. We have 3 streets when we exit our home and only one takes us to the coffee shop. If we take the 1st street, we make ...
39 views

### Dealing with Overdispersed Negative Binomial using glmmTMB

I'm new to the world of statistical modeling, but I was wondering if anyone had any input on how to handle overdispersed negative binomial data? I'm working on modeling bat activity as a response ...
10 views

### univariate regression on decile ranks versus multivariate dummy variable regression “equivalent”

Generally speaking, Which types of problems might it be appropriate to use a univariate OLS regression where the predictor takes 1 of 10 decile ranks and the response takes 1 of 10 decile ranks (i.e. ...
32 views

### Preprocessing: Why do we remove constant/invariant predictors?

For a recent research project we used machine-learning. In the preprocessing phase we removed 2 predictors because they contained mostly uninformative 0 values (x1 = 100%, x2 = 99%). Is it the right ...
22 views

### More output neurons than labels?

When we train a neural network model for a classification problem, we usually have a dense output layer of size equal to the number of labels we have. If the layer size was greater, the model can ...
10 views

### Placing constraints on free parameters across groups in lm or glm functions [on hold]

It's possible to place certain combinations of constraints on free parameters (intercepts, slopes, residual variances) across groups within the context of lm(), <...
18 views

### Hough Circle vs CNN object detection

Suppose I have a dataset that contains images like this: Each picture can have between 0 to 5 ellipses. The model has to count the number of ellipses in each test image. I would like to know what is ...
16 views

### Deep Learning applied to Multivariate Inputs Vs Univariate Input Theory Question

I'm currently working through a time series problem where I'm trying to develop a model that learns the historical relationships between several related financial markets. So, using the closing prices ...
15 views

### Is there a way to determine the number of the mixture components prior to run EM algorithm

I am working with mixture models. The common way to determine the number of the mixture components is fitting several mixture models with a different number of mixture components and then select the ...
28 views

### Hypothesis test between two coins

I am confused about how to proceed with solving 3rd and 4th question. The question is : Alice has two coins. The probability of Heads for the first coin is ${1\over4}$, and the probability of Heads ...
15 views

### Does an aberration imply overfitting?

I was reading-up on over-fitting for my project with a small dataset and it's clear that fluctuations in validation loss and accuracy imply overfitting, but does that include constant oscillation or ...
206 views

### How to determine the optimal threshold to achieve the highest accuracy

I have a list of probabilities outputted by a classifier on a balanced dataset. The metric I want to maximize is accuracy ($\frac{TP+TN}{P+N}$). Is there a way to calculate the best threshold (without ...
13 views

### Gradient of the variance of a Gaussian process

I'm trying to compute the spatial derivatives of the expected improvement acquisition function in Gaussian-process optimisation, and doing so requires the spatial derivatives of the predictive ...
16 views

### Using ML to optimize constants of a formula for minimizing error with a real measure

I have a formula of the form P = a.X0 + b.X1 + c.X2 + X3 and Ypred = d.(1-sqrt(1-e.P)) if P is positive and -rP if P is negative. X0,X1,X2,X3 are known attributes and a,b,c,d,e are constants that I ...
6 views

### How to compute the prediction performance/error of the L1 split in random survival analysis?

I'm trying to compute a prediction performance for the L1 split, which is a custom-built model by Hoora Moradian, Denis Larocque, and François Bellavance (L1 splitting rules in survival forests, 2015-...
18 views

### Non-linear dimensionality reduction for detecting coordinate systems

I am trying to find a way to automatically find the appropriate coordinate system for a physical problem. For example, in the case of a simple pendulum, polar coordinates are the most appropriate ...
27 views

### Is it possible to test overfitting with randomized data?

I have built machine learning models for a classification problem with four classes. They run at around 70% nested cross validation accuracy. I am looking to do further testing to check of ...
18 views

### Measure like relative frequency that also weights the total

Background I see a lot of publications mentioning relative frequencies to indicate the presence of certain genes in a specific area. For example: ...
14 views

### Fitting data to a distribution that appears lognormal without right-hump

I will like to fit a distribution and extract its moments in Python, given Length of Stay (LOS) data in hospitals. Asking out of curiosity, suppose the plot of the data looks like: The reason why I ...
14 views

### Is there a model to fit different data pattern obtained by identical method in R? [on hold]

I have been using a double exponential decay model to fit curves to my data. It has worked so far until last month when the data I obtained behaved differently (method for data collection was ...
8 views

### What method should I use for scoring usefulness of showing item in CD-CAT (Cognitive Diagnosis - Computerized Adaptive Testing)?

The goal of output of this method is to collect the largest possible amount of information when choosing greedy higher usefulness of item (question / task / activity). Method should score usefulness ...
15 views

### Learning Causal Graph from data

I am quite new to the theory of causal graphs, but from what I understand they are DAG, like Bayesian Networks. Since we have structure learning methods for Bayesian Networks like score based ...