機器學習訓練營——機器學習愛好者的自由交流空間（qq 群號：696721295）

這個例子說明在判別分析裡使用縮水(shrinkage)的方法，可以提高分類的準確率。所謂“縮水”，是指減少預測的特徵。我們使用的資料集是模擬資料，你也可以在真實資料集上驗證縮水判別分析的分類效果。

例項詳解

首先，匯入必需的庫。

from __future__ import division

import numpy as np
import matplotlib.pyplot as plt

from sklearn.datasets import make_blobs
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
 

準備工作

n_train = 20  # samples for training
n_test = 200  # samples for testing
n_averages = 50  # how often to repeat classification
n_features_max = 75  # maximum number of features
step = 4  # step size for the calculation

函式generate_data()

函式generate_data()用來生成模擬資料集。它有兩個引數n_samples, n_features

def generate_data(n_samples, n_features):
    """Generate random blob-ish data with noisy features.

    This returns an array of input data with shape `(n_samples, n_features)`
    and an array of `n_samples` target labels.

    Only one feature contains discriminative information, the other features
    contain only noise.
    """
    X, y = make_blobs(n_samples=n_samples, n_features=1, centers=[[-2], [2]])

    # add non-discriminative features
    if n_features > 1:
        X = np.hstack([X, np.random.randn(n_samples, n_features - 1)])
    return X, y

 

普通判別分析 v.s. 縮水判別分析

scikit-learn的線性判別分析函式LinearDiscriminantAnalysis使用一個線性邊界的分類器，它使用貝葉斯規則的類條件密度擬合數據。模型假設所有的類有相同的協方差陣，對每一個類擬合一個高斯密度。LinearDiscriminantAnalysis的主要引數solver是一個字串，指定使用的解形式。在這裡，它取lsqr, 表示最小二乘解，可以與縮水法結合。引數shrinkage預設取值None, 表示沒有縮水。在這裡，它取auto, 表示使用Ledoit-Wolf(估計縮水的協方差陣)自動縮水。

acc_clf1, acc_clf2 = [], []
n_features_range = range(1, n_features_max + 1, step)
for n_features in n_features_range:
    score_clf1, score_clf2 = 0, 0
    for _ in range(n_averages):
        X, y = generate_data(n_train, n_features)

        clf1 = LinearDiscriminantAnalysis(solver='lsqr', shrinkage='auto').fit(X, y)
        clf2 = LinearDiscriminantAnalysis(solver='lsqr', shrinkage=None).fit(X, y)

        X, y = generate_data(n_test, n_features)
        score_clf1 += clf1.score(X, y)
        score_clf2 += clf2.score(X, y)

    acc_clf1.append(score_clf1 / n_averages)
    acc_clf2.append(score_clf2 / n_averages)

features_samples_ratio = np.array(n_features_range) / n_train

視覺化分類效果

plt.plot(features_samples_ratio, acc_clf1, linewidth=2,
         label="Linear Discriminant Analysis with shrinkage", color='navy')
plt.plot(features_samples_ratio, acc_clf2, linewidth=2,
         label="Linear Discriminant Analysis", color='gold')

plt.xlabel('n_features / n_samples')
plt.ylabel('Classification accuracy')

plt.legend(loc=1, prop={'size': 12})
plt.suptitle('Linear Discriminant Analysis vs. \
shrinkage Linear Discriminant Analysis (1 discriminative feature)')
plt.show()

閱讀更多精彩內容，請關注微信公眾號：統計學習與大資料