機器學習之邏輯迴歸(logistics regression)程式碼(牛頓法實現)
阿新 • • 發佈:2019-01-27
先貼一張圖解釋牛頓法原理:
然後以一道問題為例:
這個問題是《機器學習》周志華版本上的題目,給了西瓜的密度和含糖率資料,判斷西瓜的好壞。資料在程式碼裡。
下面貼一下程式碼:
<span style="font-size:14px;">clear all; close all; clc; density = [0.697, 0.774, 0.634, 0.608, 0.556, 0.403, 0.481, 0.437, 0.666, 0.243, 0.245, 0.343, 0.639, 0.657, 0.360, 0.593, 0.719]; density = density'; sugar = [0.460, 0.376, 0.264, 0.318, 0.215, 0.237, 0.149, 0.211, 0.091, 0.267, 0.057, 0.099, 0.161, 0.198, 0.370, 0.042, 0.103]; sugar = sugar'; y = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]; y = y'; x = [ones(size(y,1),1) density sugar]; [m,n] = size(x); figure, pos = find(y); neg = find(y==0); plot(x(pos,2),x(pos,3),'o'); hold on plot(x(neg,2),x(neg,3),'*'); xlabel('density'),ylabel('sugar'); theta = zeros(n,1); MaxIter = 10; J=zeros(MaxIter,1); for i=1:MaxIter z = x*theta; h = sigmoid(z); grad = 1./m*x'*(h-y); H = (1/m).*x' * diag(h) * diag(1-h) * x; theta = theta-H\grad; J(i) = 1/m*sum(-y.*log(h)-(1-y).*(log(1-h))); end hold on plot_x = [min(x(:,2)),max(x(:,2))]; plot_y = (-1/theta(3))*(theta(1)+theta(2)*plot_x); plot(plot_x,plot_y); figure, plot(0:MaxIter-1,J,'o--', 'MarkerFaceColor', 'r', 'MarkerSize', 8) xlabel('Iteration'); ylabel('J')</span>
最後,貼一下結果。
對訓練資料的分類結果:
損失函式隨迭代次數的變化,可以看出,下降非常快!