Match Maker EE
  • Member for 3 years, 9 months
  • Last seen more than a week ago
  • Netherlands
Handling missing data for a neural network
11 votes

The problem of missing data has in data analysis obtained considerable attention. In their reference book [1] Rubin and Little define three mechanisms behind data becoming missing (definitions from ...

View answer
Can't deep learning models now be said to be interpretable? Are nodes features?
8 votes

The subject of my Ph.D dissertation was to reveal the black-box properties of neural networks, specifically feed-forward neural networks, with one or two hidden layers. I will take up the challenge ...

View answer
Why does the Akaike Information Criterion (AIC) sometimes favor an overfitted model?
6 votes

The problem with AIC is that it does not take into account the stochastics of the parameter vector ${\boldsymbol { \beta}}$. Recall that in multiple regression, each estimate of the regression ...

View answer
Conditional Probability of Having Covid-19 Given Some Symptoms
6 votes

You are asking about conditional independence: $S_1 \; {\rm INDEP} \; S_2 \mid C19$. The way you write the joint probability, as a product over the probabilities of each feature - that model assumes ...

View answer
How are artificially balanced datasets corrected for?
Accepted answer
6 votes

I have practical experience with training classifiers from imbalanced training sets. There are problems with this. Basically, the variances of the parameters associated with the less frequent classes -...

View answer
Predicting if some variable is $\geq C$?
4 votes

Interesting question. First of all, classic linear regression was developed for applications where the scatter is normally distributed. If you plot the residual distribution it should have the classic ...

View answer
What is the formula to determine the sample size for machine learning models?
Accepted answer
3 votes

No such formula exists for the optimal sample size of machine learning models. The general answer is - it depends. There is the Vapnik–Chervonenkis dimension, which gives a very general expression of ...

View answer
Relationship between distribution and data generating process
Accepted answer
3 votes

The probability distribution is the actual mathematical function $P({\bf x}; \theta)$ that can assign a probability to each possible vector ${\bf x}$. It is given by the parameter vector $\theta$. The ...

View answer
When to preprocess data for neural network
Accepted answer
3 votes

Experience with many different datasets containing discrete or continuous input variables have shown me that normalization makes training much easier. The randomly selected set of initial weights is ...

View answer
Relationship between overfitting and robustness to outliers
3 votes

Interesting questions posed. I will address the two questions for the use case of statistical classifiers in order to demarcate the analysis to a model domain we can oversee. Before embarking onto an ...

View answer
Statistically compare neural network hyperparameter values
Accepted answer
3 votes

Training NNs is a practical task where you are only interested in well-performing neural networks. A learning session that ends up in a poor local minimum is quickly discarded as the solution is ...

View answer
What are supervised learning and unsupervised learning from a connectionist point of view
Accepted answer
3 votes

For clarity, I make a distinction between classification models and regression models. A classification problem is when a statistical model (the classifier) is being trained to predict the category ...

View answer
How to extract relative importance of features from a tensorflow DNNRegressor model?
3 votes

The importance of a feature-variable depends on the distributiuon of all the other feature-variables used, for the classification problem at hand. Generally, when changing a feature-value $x_i$ can ...

View answer
MultiClass Classification - Training OvO and OvA
2 votes

Multiclass problems can be partitioned into a set of 1-versus-many classification problems. Statistical classifiers handle such differently - let me make a distinction Probabilistic classifiers ...

View answer
Use different Naive Bayes classifiers to target different data
Accepted answer
2 votes

First, the term 'Naive Bayes' refers to the made assumption of conditional independence among feature variables, given the class outcome (that is, 'stroke' or 'no-stroke'). Taking the variables gender ...

View answer
How to estimate the maximum value from a set of data with errors?
Accepted answer
2 votes

In your case, I would turn to the nonparametric Komolgorov-Smirnov one-sample test. With this test you compare a set of measurements against a known cumulative distribution function. The expression to ...

View answer
Auto-encoders' learning process and overfitting
Accepted answer
2 votes

Relevant question. The most simple autoencoder has the NN-topology: $n-k-n$ with $n$ the number of input/output nodes and $k$ the number of hidden nodes. We talk of encoding/decoding when $k<n$. ...

View answer
Estimation of measurement reliability
2 votes

Your measurement process is somewhat complex. The following is the case $ \begin{split} a &= {\cal A} + \epsilon \\ b &= {\cal B} + \epsilon \\ \epsilon &\sim {\rm u}(-e,e) \\ \\ d &= (...

View answer
Machine-learning technique to predict children's wealth
Accepted answer
2 votes

You can try performing a nonlinear regression prediction using a neural network. Common practice is to let the output node (the $y$ your predict) not use a nonlinear activation function. You can use ...

View answer
Which distribution to use in the following scenario?
Accepted answer
2 votes

Sporty question. Why not use a binomial test for this purpose? You want to test whether your $H_1: P > 0.70$ as opposed to the opposite $H_0: P \leq 0.70$ The binomial test reasons on a fraction in ...

View answer
The Bayes' Theorem Components of the Probability Output of a Classifier
2 votes

I like the question. One point before the explanation. In statistics, we use a capital letter $P$ for probability, as your prior. For probability densities a small letter $p$ is used. The probability $...

View answer
Should I use highly skewed features in my model?
2 votes

Appropriate question. The added value of preprocessing depends on the type of classifier you will train. If you use nonparametric classifiers like C4.5 (ID3), CART, the multinomial classifier, the ...

View answer
Skills hard to find in machine learners?
2 votes

Having done scientific research in Machine learning / Statistical pattern recognition for 17 years - I can come up with a few skills that make a wanted-for data scientist stand out from others. ...

View answer
Interpretation of mutual information
2 votes

The mutual information measure $I(X;Y)$ is nonparametric measure of probabilistic dependence between the variables $X$ and $Y$. As follows from wikipedia: "Intuitively, mutual information ...

View answer
Past price data has predictive power, it's not 50/50
2 votes

Interesting deep question. The deeper answer lies in the underlying causality - the process being studied. Let me make a clear distinction. In classic physics we know that $F = m \cdot a \; $ and that ...

View answer
Sampling uncertainty of posterior probability distribution
2 votes

Interesting problem - which is most often overlooked in data science and machine learning. The output probabilities $\bf{y}$ are indeed estimates of the underlying (true) posterior probabilities (your ...

View answer
Expert Knowledge Acquisition and Machine learning
Accepted answer
2 votes

The short answer to your question is Bayesian modelling. Beta-distributed priors and Dirichlet priors - these are places to start with when you want to combine number statistics with export knowledge ...

View answer
semantic value of a convolution neural network
2 votes

A convolutional neural network with nonlinear activation functions performs nonlinear image processing. Let an $X \times Y$ 2D image be defined as $I(x,y)$ and a convolutional neural network as $NN(x,...

View answer
Neural network non binary output?
Accepted answer
2 votes

Neural networks can learn to solve $c$-class classification problems, where $c$ is the number of classes (categories) to be discriminated. The general goal is to categorize a set of patterns or ...

View answer
Overfitting with Linear Classifiers
2 votes

In the 70-ties, experiments with pattern recognition algorithms on large datasets revealed that adding extra features did in some cases increase the test-set error rates. This is counter intuitive ...

View answer