Neural networks traditionally refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks (ANN), which are composed of artificial neurons or nodes - programming constructs that mimic the properties of biological neurons. ANN are ...
0
votes
0answers
23 views
How to calculate probability with sigmoid output in feedforward neural network?
first of all I'm sorry for my not very skilled English, but I will do my best to explain my problem.
I'm trying to create a feedforward neural network with one hidden layer (with probably arctan ...
2
votes
2answers
28 views
Is the restricted Boltzmann machine a type of graphical model (Bayesian network)?
In general, neural networks are not graphical models due to the choice of the cost function. But are restricted Boltzmann machines a special type of Bayesian networks?
0
votes
0answers
32 views
Appropriate method for supervised learning of small data set with few variables
What method exc. for regression can be used in order to get y=f(x1,x2) on a training set of 800 to 2000 samples? y is a whole number <0,15>, x1,x2 are real <0,40>?
I'm interested in prediction ...
1
vote
1answer
34 views
Checking autocorrelation in non-parametric methods
I would like to forecast short-term electric load by using Artificial Neural Network and Support Vector Regression. However, there's one question that sticks in my mind. In such forecasting with ...
2
votes
0answers
69 views
Universal Approximation Theorem — Neural Networks
I have posted this question elsewhere--MSE-Meta, MSE, TCS, MetaOptimize. Previously, no one had given a solution. But now, here is a really excellent and comprehensive answer.
Universal approximation ...
0
votes
0answers
40 views
Mathematically modeling neural networks as graphical models
I am struggling to make the mathematical connection between a neural network and a graphical model.
In graphical models the idea is simple: the probability distribution factorizes according to the ...
0
votes
1answer
29 views
Data normalization before giving to Neural Nets or Deep Learning algorithm?
What kind of normalization scheman is required for the best of NN algorithms? I saw some people just give the data to signum function before passing to NN and some of those process data by regular ...
0
votes
1answer
40 views
Loss Matrix Equivalent with Neural Networks and random Forest
I'm doing classification (0,1) on a dataset for which different types of errors should be weighted differently. IE, false positives would be weighted 10 x more than false negatives.
In decision ...
1
vote
0answers
31 views
Activation value at output neuron equals 1, and the network doesn't learn anything
I'm implementing a typical neural network with 1 hidden layer. The network does well with the logic XOR and other simple problems, but fails miserably when encountering a (16-input, 20~30 hidden, 3 ...
0
votes
1answer
67 views
matlab neural network strange simulation performance
I am not quite sure if this is the right place for a question like this, but asking anyway.
Having <14x10 double> input matrix (manually normalized) and <5x10 double> output matrix (manually ...
1
vote
1answer
93 views
Invariance in neural networks
I have tried to read about tangent propagation in neural networks (although I guess it could be applicable to other methods) which is a procedure to create models that are invariant to certain ...
5
votes
4answers
230 views
What does “degree of freedom” mean in neural networks?
In Bishop's book "Pattern Classification and Machine Learning", it describes a technique for regularization in the context of neural networks. However, I don't understand a paragraph describing that ...
1
vote
2answers
42 views
Neural network with skip-layer connections
I am interested in regression with neural networks.
Neural networks with zero hidden nodes + skip-layer connections are linear models.
What about the same neural nets but with hidden nodes ?
I am ...
0
votes
0answers
43 views
trouble in prediction in neural network classifier
I am training a 4-class neural network classifier.
The details of my data are:
featurelength = 280
...
1
vote
1answer
45 views
R - how can I use neural networks for a binary dependent variable in R?
I have a dataset from a bank with demographic data and one variable telling if the customer is a good customer or not (binary variable). I would like to do prediction on if the customer is good or not ...
1
vote
1answer
46 views
Averaging weights learned during backpropogation
Is it possible to train a neural network N times, using backpropogation and then average the weights learned to produce a more accurate classifier?
My tests are indicating no, but I'm unsure if it is ...
0
votes
3answers
108 views
Backpropagation vs Genetic Algorithm for Neural Network training
I've read a few papers discussing pros and cons of each method, some arguing that GA doesn't give any improvement in finding the optimal solution while others show that it is more effective. It seems ...
0
votes
2answers
113 views
Is ARIMA better in comparision with Neural Networks?
After working on Backpropagation Neural Network and ARIMA Time Series Model, I asked myself which one is better, but can't figure out the answer. They both use different approaches on the same problem ...
2
votes
2answers
80 views
ML with fastest classification speed
I have a data classification problem and I'm wondering what is the best machine learning approach to use for the particular constraints of my problem.
My constraints are as follows:
- the data ...
0
votes
0answers
25 views
Time delay estimation for non linear physical signals
I have a non linear physical system with 4 Inputs and 2 Outputs and I want to model it using a time series model such as NARX model. Because I'm new in system modeling and signal processing I do not ...
0
votes
0answers
54 views
lag in prediction outputs in one-step ahead neural network autoregressive model
I am working on an ARX forecasting problem mostly using feed-forward neural networks in MATLAB. The functional model is of the form
$y(t) = f(y(t-1),...,y(t-n),u(t))$. My data is at half hourly ...
0
votes
0answers
43 views
Weight Size in Neural Networks
I am using a Neuronal Network in combination with Reinforcement Learning. The network should learn the values of three actions in given states. The reward from the environment is scaled to [-0.9,0.9]. ...
0
votes
0answers
81 views
Modeling using neuralnet package in R - lots of issues
I am using neural net package in R. While I understand the basic neural network concepts, the details and back end is still a tough nut for me.
Currently all I can do is use brute force to change ...
2
votes
1answer
142 views
Predicting Football match winners based only on previous data of same match
I'm a huge football(soccer) fan and interested in Machine Learning too. As a project for my ML course I'm trying to build a model that would predict the chance of winning for the home team, given the ...
1
vote
4answers
152 views
Neural networks for simplistic image classification
I want to train a neural network to classify a few simple, cartoony images like the ones below (for the moment I only have the classes house, tree, and sword).
The images I am (currently) using ...
3
votes
1answer
221 views
What is the difference between a neural network and a deep belief network?
I am getting the impression that when people are referring to a 'deep belief' network that this is basically a neural network but very large. Is this correct or does a deep belief network also imply ...
2
votes
1answer
95 views
Must I normalize inputs into a perceptron that uses a sigmoid activation function?
I am building a neural network. Each perceptron in the network uses a sigmoid activation function.
Must I normalize my inputs (which currently range form 0 to 1200)? I ask this because the sigmoid ...
0
votes
0answers
61 views
Examples of Artificial Neural Networks performing Independent Component Analysis
Would someone point me to a reference/publication with a good example of an Artificial Neural Network that performs Independent Component Analysis?
Which types of architecture are suitable for this ...
0
votes
2answers
115 views
Data normalization and classification
I hope this clearly states the problem I have in hand. Here goes:
I've trained a neural network with one initial data set that was normalized in order to guarantee an equal participation of each ...
1
vote
0answers
42 views
Question about asymptotics of steepest descent method in the context of adaptive filtering
The model which will be used is defined as
$e(n) = d(n) - y(n)$
with
$y(n) = x(n)^Tw(n)$.
where $e(n)$ is the error term of the n-th observation, $x(n)$ the input vector of the n-th ...
0
votes
1answer
63 views
Cost function NN with weight derivation
In a single layered neural network with a sigmoid function (to make it easy to understand)
the cost function is
$E_j = \frac{1}{2} \sum_{k=1}^{K}(\text{target}_{jk} - \text{observed}_{jk})^2 + ...
3
votes
2answers
149 views
When would a neural network outperform an OLS estimate?
The optimization task is to find the operator $F(x_1, x_2, ...,x_n) \rightarrow y$.
Question: Under which conditions should NN provide better results than LS (in terms of mean square fit error)?
...
1
vote
2answers
165 views
What is the name of this perceptron-like classifier?
I wanted to find a variant of the perceptron which works for non-separable data, so I tried using $f(x)=\mathrm{\tanh}(x)$ instead of the hard threshold function and finding a $w$ that minimises the ...
0
votes
0answers
53 views
Why the prediction of neural network doesn't change?
I've created two neural networks for a prediction purposes,
the first is a network with one hidden layer and the second is two hidden layers,
I use the cross validation techniques,
the training error ...
1
vote
3answers
182 views
Using neural network trading in stock exchange, part 2
In several months, I have developed a framework for using neural networks (FANN library) in a chart trading software.
The framework allows me to combine any inputs for the NN, choose a learning rate, ...
-1
votes
1answer
171 views
Accuracy of neural network [closed]
How to calculate accuracy of a neural network?
is there any functions or formulas?
knowing that I use neural net work for prediction, with 10 neurons in the input layer, and 2 Hidden layers and 1 ...
1
vote
2answers
115 views
Interpreting the output of a neural network
I've implemented a neural network for prediction, and for the input data, I've used the following formula to normalize data:
...
0
votes
0answers
56 views
time series combination
I hope you applogize me if I have mistake in grammer or dictashion.I need an eurgent help about time series. in group A I have N time series ( these are value features extracted from EEG signal in ...
0
votes
0answers
34 views
Padding lags with 0 or NaN which is correct? Neural network application
How should I represent a lack of information in my lag locations when performing time series prediction with an explanatory set of multiple variables and a response set of 1 variable, with a 0 or NaN? ...
0
votes
1answer
72 views
What happens when many neurons have same weights?
Does it make any sense to have two (or more) neurons in a neural network with the same weights? (intuitively it makes no sense, since all the neurons would behave the same way).
Please consider both ...
3
votes
2answers
175 views
What are good initial weights in a neural network?
I have just heard, that it's a good idea to choose initial weights of a neural network from the range $(\frac{-1}{\sqrt d} , \frac{1}{\sqrt d})$, where $d$ is the number of inputs to a given neuron. ...
3
votes
2answers
198 views
Deep belief network performs worse than a simple MLP
I tried to train a deep belief network to recognize digits from the MNIST dataset. Everything works OK, I can train even quite a large network. The problem is that the best DBN is worse than a simple ...
0
votes
0answers
74 views
Modification of error function and weight update rule of backpropagation algorithm
I'm trying to understand the neural networks with a given example. On the graph, observed instances are shown. It states that the function is symmetric around x=3.5.
So the question is : How would ...
4
votes
3answers
174 views
What are good techniques for modeling small datasets?
I’m working on a classification problem. However, my training dataset is very small (just 800 items in training dataset) and each data item contains a small number of features (just 5 features). ...
0
votes
0answers
31 views
What is the correct way of implementing weight decay?
When applying weight decay, does one just use all available input features, an arbitrary large number of hidden layer nodes, and cross validate for the appropriate weight decay parameter? Or what is ...
1
vote
1answer
622 views
Example of time series prediction using neural networks in R
Anyone's got a quick short educational example how to use Neural Networks (nnet in R for example) for the purpose of prediction?
Here is an example, in R, of a time series
...
1
vote
1answer
199 views
Choosing cost function for a neural network continuous forecast
I´m using Octave to build a Neural Network regression (3 layers nn, using tanh as activation function, fmincg as optimizer and continuous output). The purpose of the model is to forecast demand for ...
2
votes
1answer
93 views
Examples of drastic improvements when using deep neural networks
We have already worked in predicting chemical compound activity using "classical" neural networks. Now there is all this hype about deep learning. I wonder if you know cases where predictive ability ...
2
votes
1answer
391 views
Cross-validation with neural networks yielding worse results than a standard neural network
Summary: when using a 10-fold cross-validation procedure where each training set is used to generate N bootstrap samples for processing with NNs. How do I provide my NN with correct sequence and ...
3
votes
1answer
127 views
How to improve neural network sensitivity with a lopsided binary outcome?
I'm working on predicting (not explaining) a 0/1 outcome that generally has only about 10% "1"s (I'm not at liberty to name the variables). N ~40,000.
Logistic regression proved unsatisfactory, both ...
