View source for Talk:Oja learning rule

Reviewer B: This article is well written. This is one of best encyclopedia article that is written in a simple language such that an non-expert with little mathematical background can get the key points.  

Since there is still room, adding some more details maybe helpful. Probably, to keep the original easy readability is more preferred. If adding issues mathematically quite involved, a few sentence interpretation plus references maybe enough.

minor comments:

The   sentence before eq.(2) and the first sentence of the next paragraph maybe duplicated. The latter one can be removed.




Reviewer A:  The article is well written but oversimplified. For example, the mathematical analysis claims that equation following (5) (it has no number - let's call it (5a)) can be "averaged" to produce (6). How can you average (5a) when w is changing? The truth is a bit more complicated. There must be a reference to stochastic approximation theory (see Kushner & Clark) where it is shown that the difference equation asymptotically approximates the differential equation with the averaged values provided that the learning rate "alpha" is small and the sequence of alpha's sums up to infinity. By the way, please give a number to equation between (5) and (6).
Moreover, it is claimed that "(ICA) is a technique that is related to PCA, but is potentially much more powerful". In reality, ICA is DIFFERENT from PCA but not more POWERFUL. For example, both ICA and PCA can perform blind source separation under different assumptions.
Finally, the article often refers to a Figure which I cannot find anywhere.

[1] H.J. Kushner and D.S. Clark, ''Stochastic Approximation Methods for Constrained and Unconstrained Systems'',Springer-Verlag, 1978.


AUTHOR'S REPLY:  I thank both the reviewers. Perhaps the article is oversimplified, I tried to make it as readable as 
possible and then had to leave out all math details. I have now added some notes on the connection to stochastic appproximation (the Kushner-Clark book is of course the central reference in the original analysis of the Oja rule in the book Oja, 1983).