Very well delivered online course from Caltech: Introductory Machine Learning online course (MOOC). I’ve only been listening via iTunes University, however it’s delivered so well that even without the visuals the high level details are clear. (I will need to revisit everything after lecture 4 a second time…)

Currently up to lecture 11. Combining some of the ideas from lectures 10 and 11 on nature inspired systems and over fitting, It’s really tempting to apply theses principles to human learning, and he has provided some good analogies. Over fitting as hallucinations, or the noise that will be introduced if children attempt to understand concepts well beyond their education level. It seems there may be people in this world that are completely satisfied with minimizing in-sample errors, such that they actively reject new sample data so that their perfectly formed hypothesis is not hurt when it’s exposed to out-of-sample errors. I suspect this analysis is also falling victim to the same process.

Other Random Thoughts Regarding Over Fitting. Noise is broken into two categories stochastic and deterministic.

• The stochastic noise seems to be evenly spread over the ‘spectrum’ of the solution space. Filtering it would not help.
• The deterministic noise seems to have a distribution over the ‘spectrum’ of the solution space, is the solution of regularization analogous to using filtering to deal with noise in signal processing?