Neural Network FAQ (opens in new tab)

Sorry about the Google cache; HN won't let you submit FTP links.

This was written by Warren S. Sarle from 1997-2002. I ran into it while trying to look for papers on benchmarking neural networks. I highly recommend it for its frank and accessible style.

Part 2 of that FAQ (which regrettably seems not to be in the Google cache) links to another piece by Warren Sarle, which I consider to be the best description anywhere of a wide variety of optimization methods (some but not all specific to neural networks): http://www.creative.net.au/mirrors/neural/kangaroos .

These are a great read... The Usenet FAQ archive has all of them available:

http://www.faqs.org/faqs/ai-faq/neural-nets/part1/

http://www.faqs.org/faqs/ai-faq/neural-nets/part2/

http://www.faqs.org/faqs/ai-faq/neural-nets/part3/

http://www.faqs.org/faqs/ai-faq/neural-nets/part4/

http://www.faqs.org/faqs/ai-faq/neural-nets/part5/

http://www.faqs.org/faqs/ai-faq/neural-nets/part6/

http://www.faqs.org/faqs/ai-faq/neural-nets/part7/