Final project of MDS6106 Optimization

• Solving variant-SVM and variant-Logistic Regression by implementing GM, AGM, BFGS, L-BFGS, SGD
• 4 self-generated data sets & some large-scale datasets from LIBSVM

• Create to do list

• Generating Data

• Create Mini-Batch

• Solve Logistic Regression Problem from Scratch (In order to get some sense)

  • Initializing Model Parameters
  • Define the Model
  • Define the Loss Function
  • Define the Optimization Algorithm (SGD, first try simplest version)
  • Define Classification Accuracy and eval_accuracy for given model on specific dataset
  • Training
  • Plot the Result (With seperating line)
  • Choose $\lambda$ by using Cross-Validation
  • Rewrite the optimization algorithm (SGD, the average version)

• Optimization Part

  • Implement basic Gradient Method
  • Implement Accelerated Gradient Method
  • Implement BFGS
  • Implement L-BFGS

• SVM Part