|
I figured I may as well ask this to TL. OK, Neural Networks have interested me since I learned about them. But I always had 1 main hangup on them, and that is whether or not it was necessary to use thresholds, and if so what benefit they provided.
Now they basis of NN to my knowledge is a bottom layer of inputs, Followed by higher layers of nodes, that just weight and add the lower layers. Now generally it seems, that when talking about neural networks, most sources say to take the total of a Node and apply a threshold to it, if the number is lower than the threshold, the value of the node is 0, and if higher is 1(also potentially-1,1 or -1,0,1 ext..)
What I am curious about is what makes the threshold necessary. If not applied, ie just using the raw totals, the neural net should still function, you just get the raw total instead of 1 or 0. The threshold to me would just seem to reduce the possible outcomes/ subtleties the NN could provide. The only benefit to them I can immediately see, is for deeper, NN, to avoid having large totals, since potentially, large numbers could keep adding up as you get deeper in the layers.
TL:DR what is the benefit to putting a threshold on every node of NN?
|
While I have written neural network programs and studied them briefly, I am far from an expert, so I might be mistaken on what I say. I am doing research in the field of genetics-based machine learning. I personally find that there are several superior alternatives to neural networks. Of course, this depends on what type of problem you are solving.
Yes, I do think you could design it without using threshold values. But Artificial Neural Networks are modeled based on the nervous system, and neurons are modeled as function which will either fire or not fire, depending on the input. Thus, it is a question of modeling. To "remove" the threshold would be seen as defining it as 0, as the concept of threshold is a central part of neural networks. Yes, you can do that. There are a variety of ways to implement a neural network.
To "reduce the possible subtleties" does not necessarily have to be a bad thing, though. The neural network was probably trained on some kind of data, and it will not be able to provide limitless subtlety, due to bias or noise in the training data.
|
Out of curiosity what are the superior algorithms you are referring to?
|
In my REU this summer, SVM with some tweaking usually out performs it. Its very data dependent though and choice of classifier is only one part of the overall classification scheme so its hard to say X is better than Y
|
For classification, neural networks has the negative point of being black box. The user has no clear indication of why a value was classified as it was. On that part, utilizing fuzzy systems or a GBML (genetics-based machine learning)/LCS (learning classifier system) algorithm is strong, as they generally utilize rule sets as representation scheme. SVM is also a very strong approach, even if it also lacks in interpretability. It is not always that interpretability is an important factor, though.
For genetic fuzzy systems, you have a clear reason to provide the user of why a value was classified as it was. In my experience, it is also often able to achieve better test accuracy.
|
Interesting, wasn't aware of all of the alternate learning scheme's/algorithms. I guess I'll have to look into them.
On a side note I thought GA and NN, tended to be used for different sorts of problems? IE, NN finding shapes, and GA solving the 8 queens problem.
|
Yes, GA is often applied to solving such problems. GA can also be applied to machine learning and classification algorithms, which is referred to as GBML or LCS, depending on who you ask. With this, GA can also be utilized in finding shapes.
You can also design strong NN with the use of GA. There are almost limitless approaches to these kind of problems.
|
On July 24 2013 10:58 Tasaio wrote:
Yes, I do think you could design it without using threshold values. But Artificial Neural Networks are modeled based on the nervous system, and neurons are modeled as function which will either fire or not fire, depending on the input.
Incorrect. Neurons can also send different signals by firing at varying intervals, a bit like Morse code.
|
On July 24 2013 20:00 Dagobert wrote:Show nested quote +On July 24 2013 10:58 Tasaio wrote:
Yes, I do think you could design it without using threshold values. But Artificial Neural Networks are modeled based on the nervous system, and neurons are modeled as function which will either fire or not fire, depending on the input. Incorrect. Neurons can also send different signals by firing at varying intervals, a bit like Morse code.
Yes, that might be true. I really don't know much about the actual nervous system.
But I was talking about how neurons are modeled in artificial neural networks, using a threshold.
|
|
|
|
|
|
EPT
