I am attempting to make a Radial Basis Function Network to see if a relationship exists between input/output data that I have been collecting. I have hit a bit of a brick wall with a few issues, and wondered if a subject matter expert could perhaps point me in the right direction?
All of my work is based in Excel 2007. I have 14940 datapoints. There are 32 input dimensions and one "target output" for each datapoint. All input data is normalised to be on a scale of -1 to 1 or 0 to 1 as appropriate. I have written a K-means clustering algorithm to cluster the data into 50 partitions. The centroid of each cluster is set as the centre of each of 50 Gaussian Radial Basis functions. The network architecture is normalised but does not use a regularisation parameter.
I have made a "random" sampling algorithm that picks 13000 training datapoints evenly across the clusters, and "saves" the rest as a validation set.
I have calculated the network weights using the standard matrices methods and then built the final network. I get an RMS error of 1.311, though obviously have no idea how good or bad this is. I thought maybe I could improve on this by adding RBF nodes, so I re-clustered the data with 80 clusters, and run the whole thing again. This time I got an error of 1.307, so barely any better.
I thought that my initial guess of the Basis Function widths (which was near the average distance between the network centroids) was probably not optimal, so I started re-creating the network for many different widths - spanning a whole order of magnitude. I found that the RMS error in the network barely varies at all with varying Basis Function width, which I found a little surprising. I was expecting there to be some variation, pointing to an "optimum value", as per the countless papers and tutorials I have read.
So this is my problem - I dont really know what to do, and wondered if someone with more experience in these things could suggest what I could do next. A few things I am thinking are:
1) Give up, and conclude that there is no pattern in the data and I am trying to get the RBF Network to look for a correlation that doesn't exist. Though this still doesn't explain why I don't see a better correlation when there are 80 centres versus 50 centres.
2) Use more basis functions. I feel like I have a large number of input dimensions and this is confusing or complicating the network. I have tried exhaustive searches on google to try to get estimates of an "initial guess" number of basis functions to try, but keep coming up with nothing. Does anyone have any thoughts or recommendations if my input dimensions are 32 and my training patterns are over 13000? I feel like I might be waaay off the required number of basis functions, and I may need several hundred?
3) Something else? I have checked and rechecked my formulas and macros for "silly" errors (Im an engineer, afterall!) and done a lot of validation by "manual calculation" checks, but cant seem to find anything thats causing it to trip up.
I apologise for the long question, but would greatly appreciate any insights that might help me continue with my model or improve my understanding.