Section 2.1.4
plot objection function value v.s. iteration with different step size
summary:
1. When step size becomes large, the speed of converge is increasing. The optimized step size is about
0.4 to 0.6.
2. When step size is 0.672, the objection value oscillates. But there still exists a stable condition.
3. When step size is 0.814, the objection value oscillates. But there still doesn't exists a stable condition.
4. Although the speed of converge is increasing by increasing step size, this method may cause the search will
jump outside the velly, which means it can't get the minimum point.
'''
mysvm = SVM.MySVM(alpha, beta, C, h, max_iter, step_size, rnd_number, train_XX,
train_yy, validation_XX, validation_yy, gradient_error,
improve_ratio)
obj_list = list()
step_list = list()
mysvm.max_iter = 20
for k in [2 * i for i in range(12)]:
mysvm.step_size = 0.1 * math.pow(1.1, k)
mysvm.my_gradient_decent(1, True)
step_list.append(mysvm.step_size)
obj_list.append(mysvm.obj_record)
# end for
show_obj_versus_iteration(step_list, obj_list)
'''
Section 2.1.4
