習題3.3 與 習題 3.5 程式設計實現對率迴歸、線性判別分析

原始資料：

其中‘+’表示好瓜，‘o'表示壞瓜

對率迴歸即線性判別分析原理見書。

以下為程式碼及結果，Matlab實現：

1、對率迴歸（邏輯斯蒂迴歸）

clear all; clc;

x = importdata('ex4x.txt');
y = importdata('ex4y.txt');

[m,n] = size(x);
sample_num = m;
x = [x,ones(m,1)];

figure;   %畫初始分佈圖
pos = find(y == 1);neg = find(y == 0);
plot(x(pos,1),x(pos,2),'+');
hold on;
plot(x(neg,1),x(neg,2),'o');
xlabel('density');
ylabel('sweet');

g = @(z)(exp(z) ./ (1.0 + exp(z))); 
beta = zeros(n+1,1);
times = 500;   %100次迭代
for i = 1:times
    delta1 = 0;
    delta2 = 0;
    for j=1:m
        tmp_x = x(j,:)';
        p = g(beta' * tmp_x);
        delta1 = delta1 - tmp_x * (y(j,:) - p);
        delta2 = delta2 + tmp_x * tmp_x' * p * (1-p);
    end
    delta = delta2\delta1;
    beta = beta - delta;
end 

figure;
plot_x = [0, 1];
plot_y = (1 / beta(2)) *(- beta(1) * plot_x - beta(3));
plot(x(pos,1),x(pos,2),'+');
hold on;
plot(x(neg,1),x(neg,2),'o');
plot(plot_x, plot_y);
xlabel('density');
ylabel('sweet');
title('對率迴歸');

 

2、線性判別分析

程式碼：

clear all;
clc;

x = importdata('ex4x.txt');
y = importdata('ex4y.txt');

[m,n] = size(x);
u = zeros(2,n);
num = zeros(2);

figure;   %畫初始分佈圖
pos = find(y == 1);neg = find(y == 0);
plot(x(pos,1),x(pos,2),'+');
hold on;
plot(x(neg,1),x(neg,2),'o');
xlabel('density');
ylabel('sweet');

%計算均值向量
for i=1:m
    u(y(i)+1,:) = u(y(i)+1,:) + x(i,:);
    num(y(i)+1) = num(y(i)+1)+1;
end
u(1,:) = u(1,:)/num(1);
u(2,:) = u(2,:)/num(2);

Sb = (u(1,:)-u(2,:)) * (u(1,:)-u(2,:))';
Sw = 0;
for i=1:m
    tmp = x(i,:) - u(y(i)+1,:);
    Sw = Sw + tmp * tmp';
end

w = Sw\(u(1,:)-u(2,:));

figure;
pos = find(y == 1);neg = find(y == 0);
plot(x(pos,1),x(pos,2),'+');
hold on;
plot(x(neg,1),x(neg,2),'o');
plot_x = [0.1,0.9];
plot_y = -w(1) * plot_x / w(2);
plot(plot_x, plot_y);

xlabel('density');
ylabel('sweet');
title('線性判別分析');

 

結果分析：由於資料給的並不是那麼友善。。。。。邏輯斯蒂迴歸的結果只算差強人意，而線性判別分析的結果完全不理想。