# Tag Info

4

MSE has several advantages over MAE, but also some disadvantages. Just list some of them, include but not limited to: Decomposition of MSE into Variance and Bias square is one of the most famous advantages. This property helps us to understand the logic behind error, especially MSE, while MAE has no such mathematical meaning. MAE with absolute value ...

4

There is a package called "darch" http://cran.um.ac.ir/web/packages/darch/index.html Quote from CRAN: darch: Package for deep architectures and Restricted-Bolzmann-Machines The darch package is build on the basis of the code from G. E. Hinton and R. R. Salakhutdinov (available under Matlab Code for deep belief nets : last visit: 01.08.2013). ...

3

Artificial neural network: computational power (Wikipedia): The multi-layer perceptron (MLP) is a universal function approximator, as proven by the Cybenko theorem. However, the proof is not constructive regarding the number of neurons required or the settings of the weights. Work by Hava Siegelmann and Eduardo D. Sontag has provided a proof ...

2

The testing set´s size is ranging from 10% to 30% of the training set, and validation set's size is ~10% of the training set. To prevent risk of overﬁtting, the size of the training set must be at least ﬁve times the number of weights. For a three-layers network it has be suggests that the hidden layer (neurons) should have approximately ...

2

Actually a three layer neural network can model arbitrary function with the linear and logistic functions, which was proved by Kolmogorov in 1957 (Kolmogorov, Andrei Nikolaevich. "On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition." Dokl. Akad. Nauk SSSR. Vol. 114. No. 5. ...

2

To get you started, the Elements of Statistical Learning have a nice discussion about regularization, and also sound discussions of different models To judge whether a particular regularization is a good idea, you need to take into account you data as well. E.g. for the LASSO, does it make sense for your data to assume that you have noise-only variates ...

1

It is possible that the additional units are over-fitting the data. Formal analysis of neural networks is limited to broad statements because they're exceedingly difficult to manipulate analytically. An experiment to test this for a particular dataset, is to perform nested k-fold cross-validation. Select a $n$ observations of the data to perform nested ...

1

This is the implemented function (extracted from the C-sources; filennet.c, lines 156-165): static double sigmoid(double sum) { if (sum < -15.0) return (0.0); else if (sum > 15.0) return (1.0); else return (1.0 / (1.0 + exp(-sum))); }

1

Not much to say about computational difficulties, since I haven't used a 3D map with real data. I don't think it will impose a great overhead. You just have to add one more dimension in the distance and neighborhood functions. Theoretically a 3D map will produce better clustering results because it will have a more flexible grid to adapt to the dataset but ...

1

From the linked paper (Paper #1) Each chromosome is defined as a floating-point vector, whose length corresponds to the number of variable in a certain problem. Each element in a vector is called as a gene and each chromosome consists of $N(N-1)$ genes, which are floating point numbers in the range $[-1, 1]$. When computing the FCM classifier's ...

1

Please refer to paper SVM Incremental Learning, Adaptation, and Optimization, which proposed an online SVM for binary classification. The code of above paper can be found here. In the code, two ways of online training are introduced: 1) train the SVM incrementally on one example at a time by calling svmtrain() and 2) perform batch training, incrementing all ...

1

Based on your update pseudo code it looks like you're using a sigmoid output. In this case given an input $x$ your output should be $$\sigma(x) = \frac{1}{1 + e^{-(w^T x + b)}}.$$ It is worth noting that if you assume a log-loss function (which is what you should use for classification) your setup is just binary logistic regression. In this case your ...

1

"I don't understand one thing. How to calculate output for vector input x?" If by single layer perceptron you mean the input layer plus the output layer: Then for each input to the output node, take the values applied to the inputs and multiply them by their cosponsoring weight values. Then sum these weighted inputs. This sum of weighted inputs is then ...

1

Regretfully, I am not familiar with WEKA. Still, here is some ideas that might help to you to look for that you need. There is no bound on the number of steps required for a network to converge. I would stress two points, Neural networks are not guaranteed to converge to a global optima, but to a local optima. They solve non-convex problems, which suffer ...

Only top voted, non community-wiki answers of a minimum length are eligible