All Questions

Filter by
Sorted by
Tagged with
-1
votes
0answers
8 views

Finding the weights of individual children items given the actual parent assembly/unit weights

I'm jumping head first into learning machine learning by trying to solve an actual problem at work. I'm trying to find individual weights of items/options that make up the total weights of assemblies/...
0
votes
0answers
22 views

Choosing which chi-square test to use when working with count data

I am analyzing data that compares responses to various questions across a variety of demographics. I want to see how the count of each response varies by each demographic group. The most basic test to ...
0
votes
2answers
43 views

Why estimate using GEEs inspite of their disadvantages over using ML?

I know sometimes I want to know the population-level estimates, but the problem with GEEs is I can't calculate the likelihood, and therefore all models I make with it aren't comparable, and I don't ...
0
votes
0answers
20 views

Non-linear Least squares fitting in R

I want to fit a curve to a set of 3 data points (1,65), (200,70), (800,75) in the form $y = \frac{ax}{b+x}$ Using the set of points I have this set of equations $a-65b = 65$ $200a-70b = 14000$ ...
0
votes
0answers
6 views

Estimating false accept rates from imposter scores below a threshold

I have a system that compares two items and produces a match score. Scores below a threshold are manually inspected to determine if they match or don't(imposter). Scores above the threshold are ...
0
votes
0answers
9 views

How to show or prove a dataset is not linearly separable

I am looking to be pointed in the right direction. I am learning about kernels and I have a homework assignment to use the dual perceptron algorithm to classify datapoints from a spiral dataset, with ...
4
votes
1answer
40 views

Expanding initial sample when the result isn't significant

This inspiring answer describes a variant of hypothesis, and I want to analysis its property further. Basically, it considers a two-sided test and interprets the $p$-value as a measure of how strong ...
1
vote
1answer
27 views

Machine learning and feature selection [on hold]

I developed my algorithm, Hybrid SVM algorithm, and correlation-based feature selection (for network intrusion detection), I have a suspicion that If I want to compare performance with other methods ...
0
votes
0answers
7 views

(How) Does different number of categories affect correlation?

I work with obesity in cats and many studies use a body condition score (BCS) to assess obesity. This is a somewhat subjective measure of how much fat covering an animal has. There are two commonly ...
0
votes
0answers
8 views

Only paralleling process in cluster Spark [on hold]

I have built a spark cluster using 6 computers, and a master node. I want a process that does not handle large databases and I want to repeat this process many times. Using the interactive shell ...
0
votes
0answers
25 views

having trouble interpreting confint function using R (logistic regression) [duplicate]

I'm trying to interpret these results of using R confint function , but I can not understand. This is a logistic regression about breast cancer. how to interpret confint function. What the 2.5% and 97....
0
votes
0answers
7 views

Classifier for detecting abnormal noise using FFT

As a simple machine learning application for quality evaluation, I want to detect abnormal sound in a product. A good product would make a signature humming sound, a bad product might make noises due ...
0
votes
0answers
19 views

How do I decompose significant main effects and interactions in a mixed model with multilevel factors? [on hold]

I have an ordinal mixed model for exploration and am wondering how to decompose significant main effects and interaction with multi-level predictors. In the 'main model' my predictors are centralised ...
0
votes
0answers
10 views

Setting Choosing alpha for generalized extreme Studentized deviate ESD

I'm working on a S-H-ESD implementation and I'm struggling to set the alpha for the ESD. The suggested alpha everywhere is 0.05. Is there a way to calculate an alpha based on the expected percent of ...
1
vote
1answer
18 views

Using a separate but related dataset for feature extraction (transfer learning)

I have two datasets of MRI images: a larger one of Altzheimer's paitents (AD), which is about 3 times the size of a smaller dataset of brain tumor paitents (BT). My aim is to make use of the AD data ...
0
votes
1answer
13 views

Temporal difference definition (Reinforcement Learning)

Reading for instance Szepesvari or this : i struggle to understand the rationale behind the temporal-difference definition $\delta_{t}=R_{t}+\gamma V_{k}(x_{t+1})-V_{k}(x_{t})$ with the notation ...
1
vote
1answer
35 views

Linear Regression : Can I use both levels and changes in the same model?

I have a linear model with 1 predictor variable in the form of: $Y = a + b_{1}*X$ Both $X$ and $Y$ are stationary variables and the fit of the model is good. I have also created 2 other models based ...
-1
votes
0answers
14 views

a proper way of clustering 3D vectors(XYZ) [on hold]

I am new to clustering and I need to cluster a set of vectors in cartesian space (3d vectors - XYZ in space). It means that I have three points in space with a force vector on each of them. I wanna ...
0
votes
1answer
35 views

Modeling non-linear (short) time series and cross-validate them

beginner data scientist here. Time series analysis is a completly new area for me, so please correct me if i write something that makes no sense. I have many multivariante short time series, between ...
1
vote
0answers
13 views

Maximum entropy prior for r.v. supported on real line with no other constraints?

What would be a suitable maximum entropy prior for a random variable supported on the real line with no other constraints (i.e. unknown mean, unknown variance, unknown bounds)? All kinds of answers (...
0
votes
0answers
14 views

Increment columns of a data frame in a for loop in R [on hold]

For example, I have the closing prices for these companies Now I want to calculate the break points for each of these companies. I am going to use the function breakpoints from the package ...
1
vote
1answer
23 views

Number of Model Combinations

I have a model selection combinatorics problem. My covariate pool consists of six covariates: A1, A2, A3, B, C and D. However, covariates A2 and/or A3 can only be included if A1 is also included. ...
0
votes
0answers
12 views

Why is the SVM in canonical representation necessary?

For my exam preparation I have following question from an old exam: I know that the SVM in canonical representation is $$\min_{i=1,...,n} |\langle \omega,x_i \rangle + b | = 1$$ So why is it ...
-1
votes
1answer
8 views

n for Hopkins statistics using get_clust_tendency

I started to use the Hopkins statistics to establish, if a dataset is 'clusterable'. I am using the following code - taken from here: ...
5
votes
1answer
60 views

Combining together principal components from PCA performed on different subsets of a large dataset

I'm trying to QA a process in which the data has over a million rows with approx 60,000 variables in a binary form. The aim of the process was to perform k-means clustering, but prior to this, the 60,...
0
votes
1answer
21 views

Ideal method for confounder adjustment

I have a fairly large dataset (15,000) in which I assessed the association of a three-level categorical risk factor to the outcome. The outcomes were significantly different for the three levels. ...
1
vote
1answer
30 views

What's the expected average time

We have the following situation: We are trying to find the path to a coffee shop. We have 3 streets when we exit our home and only one takes us to the coffee shop. If we take the 1st street, we make ...
0
votes
2answers
39 views

Dealing with Overdispersed Negative Binomial using glmmTMB

I'm new to the world of statistical modeling, but I was wondering if anyone had any input on how to handle overdispersed negative binomial data? I'm working on modeling bat activity as a response ...
0
votes
0answers
10 views

univariate regression on decile ranks versus multivariate dummy variable regression “equivalent”

Generally speaking, Which types of problems might it be appropriate to use a univariate OLS regression where the predictor takes 1 of 10 decile ranks and the response takes 1 of 10 decile ranks (i.e. ...
0
votes
1answer
32 views

Preprocessing: Why do we remove constant/invariant predictors?

For a recent research project we used machine-learning. In the preprocessing phase we removed 2 predictors because they contained mostly uninformative 0 values (x1 = 100%, x2 = 99%). Is it the right ...
0
votes
1answer
22 views

More output neurons than labels?

When we train a neural network model for a classification problem, we usually have a dense output layer of size equal to the number of labels we have. If the layer size was greater, the model can ...
0
votes
0answers
10 views

Placing constraints on free parameters across groups in lm or glm functions [on hold]

It's possible to place certain combinations of constraints on free parameters (intercepts, slopes, residual variances) across groups within the context of lm(), <...
0
votes
0answers
18 views

Hough Circle vs CNN object detection

Suppose I have a dataset that contains images like this: Each picture can have between 0 to 5 ellipses. The model has to count the number of ellipses in each test image. I would like to know what is ...
0
votes
0answers
16 views

Deep Learning applied to Multivariate Inputs Vs Univariate Input Theory Question

I'm currently working through a time series problem where I'm trying to develop a model that learns the historical relationships between several related financial markets. So, using the closing prices ...
0
votes
1answer
15 views

Is there a way to determine the number of the mixture components prior to run EM algorithm

I am working with mixture models. The common way to determine the number of the mixture components is fitting several mixture models with a different number of mixture components and then select the ...
0
votes
1answer
28 views

Hypothesis test between two coins

I am confused about how to proceed with solving 3rd and 4th question. The question is : Alice has two coins. The probability of Heads for the first coin is ${1\over4}$, and the probability of Heads ...
0
votes
0answers
15 views

Does an aberration imply overfitting?

I was reading-up on over-fitting for my project with a small dataset and it's clear that fluctuations in validation loss and accuracy imply overfitting, but does that include constant oscillation or ...
3
votes
2answers
206 views

How to determine the optimal threshold to achieve the highest accuracy

I have a list of probabilities outputted by a classifier on a balanced dataset. The metric I want to maximize is accuracy ($\frac{TP+TN}{P+N}$). Is there a way to calculate the best threshold (without ...
1
vote
0answers
13 views

Gradient of the variance of a Gaussian process

I'm trying to compute the spatial derivatives of the expected improvement acquisition function in Gaussian-process optimisation, and doing so requires the spatial derivatives of the predictive ...
0
votes
0answers
16 views

Using ML to optimize constants of a formula for minimizing error with a real measure

I have a formula of the form P = a.X0 + b.X1 + c.X2 + X3 and Ypred = d.(1-sqrt(1-e.P)) if P is positive and -rP if P is negative. X0,X1,X2,X3 are known attributes and a,b,c,d,e are constants that I ...
0
votes
0answers
6 views

How to compute the prediction performance/error of the L1 split in random survival analysis?

I'm trying to compute a prediction performance for the L1 split, which is a custom-built model by Hoora Moradian, Denis Larocque, and François Bellavance (L1 splitting rules in survival forests, 2015-...
0
votes
1answer
18 views

Non-linear dimensionality reduction for detecting coordinate systems

I am trying to find a way to automatically find the appropriate coordinate system for a physical problem. For example, in the case of a simple pendulum, polar coordinates are the most appropriate ...
1
vote
1answer
27 views

Is it possible to test overfitting with randomized data?

I have built machine learning models for a classification problem with four classes. They run at around 70% nested cross validation accuracy. I am looking to do further testing to check of ...
1
vote
1answer
18 views

Measure like relative frequency that also weights the total

Background I see a lot of publications mentioning relative frequencies to indicate the presence of certain genes in a specific area. For example: ...
0
votes
0answers
14 views

Fitting data to a distribution that appears lognormal without right-hump

I will like to fit a distribution and extract its moments in Python, given Length of Stay (LOS) data in hospitals. Asking out of curiosity, suppose the plot of the data looks like: The reason why I ...
0
votes
0answers
14 views

Is there a model to fit different data pattern obtained by identical method in R? [on hold]

I have been using a double exponential decay model to fit curves to my data. It has worked so far until last month when the data I obtained behaved differently (method for data collection was ...
0
votes
0answers
8 views

What method should I use for scoring usefulness of showing item in CD-CAT (Cognitive Diagnosis - Computerized Adaptive Testing)?

The goal of output of this method is to collect the largest possible amount of information when choosing greedy higher usefulness of item (question / task / activity). Method should score usefulness ...
0
votes
0answers
15 views

Learning Causal Graph from data

I am quite new to the theory of causal graphs, but from what I understand they are DAG, like Bayesian Networks. Since we have structure learning methods for Bayesian Networks like score based ...
1
vote
0answers
17 views

Understanding the general theory proposed by Neyman & Pearson

I'm reading Neyman & Pearson, 1933, i.e. Neyman and Pearson. On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. ...
0
votes
0answers
10 views

Use of Pearson Correlation when studying experimental data

My research focuses on creating and testing algorithms to optimise computer networks. Towards this end I perform experiments using a Monte-carlo simulator. I recently conducted a parameter study ...

15 30 50 per page