手工实现机器学习的基础算法

1. 广义线性模型

• LinearRegression (closed-form solution)

• LogisticRegression (closed-form solution, gradient descent, Newton)

2. 集成学习 (Ensembling)

• Boosting
• Bagging
• RandomForest

3. 聚类算法 (Clustering)

• k-means
• DBSCAN

4. 降维 (DimensionReduction)

• PCA
• LDA
• MDS
• Isomap

5. 其他常见算法

• DecisionTree (ID3)
• SVM (QP with hard/soft margin, SMO)
• NaiveBayesClassifier (with Laplacion correction)
• NeuralNetwork (structure 2-2-1)
• Perceptron
• KNN