The convergence proof for the Perceptron algorithm is one of the most elegant pieces of math I’ve seen in ML. Most useful: Boosting , especially boosted decision trees.