The optimizing algorithm I am using is once again the same old gradient descent.def gradientDescentnn(X,y,initial_nn_params,alpha,num_iters,Lambda,input_layer_size, hidden_layer_size, num_labels): """ Take in numpy array X, y and theta and update theta by taking num_iters gradient steps with learning rate of alpha return theta and the list of the cost of theta during each iteration """ Theta1 = initial_nn_params[:((input_layer_size+1) * hidden_layer_size)].reshape(hidden_layer_size,input_layer_size+1) Theta2 = initial_nn_params[((input_layer_size +1)* hidden_layer_size ):].reshape(num_labels,hidden_layer_size+1) m=len(y) J_history =[] for i in range(num_iters): nn_params = np.append(Theta1.flatten(),Theta2.flatten()) cost, grad1, grad2 = nnCostFunction(nn_params,input_layer_size, hidden_layer_size, num_labels,X, y,Lambda)[3:] Theta1 = Theta1 – (alpha * grad1) Theta2 = Theta2 – (alpha * grad2) J_history.append(cost) nn_paramsFinal = np.append(Theta1.flatten(),Theta2.flatten()) return nn_paramsFinal , J_historynnTheta, nnJ_history = gradientDescentnn(X,y,initial_nn_params,0.8,800,1,input_layer_size, hidden_layer_size, num_labels)Theta1 = nnTheta[:((input_layer_size+1) * hidden_layer_size)].reshape(hidden_layer_size,input_layer_size+1)Theta2 = nnTheta[((input_layer_size +1)* hidden_layer_size ):].reshape(num_labels,hidden_layer_size+1)A warning to those executing the code.It will take a fair bit of time to compute depending on your computing power, and even longer if you are optimizing your alpha and num_iters values.I use 0.8 for alpha and 800 for num_iters but I believe it can get better accuracy with more tunning.pred3 = predict(Theta1, Theta2, X)print("Training Set Accuracy:",sum(pred3[:,np.newaxis]==y)[0]/5000*100,"%")With this, I am halfway through the series.The jupyter notebook will be uploaded to my GitHub at (https://github.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python).For other python implementation in the series,Linear RegressionLogistic RegressionRegularized Logistic RegressionThank you for reading.. More details