R語言（遞迴分割樹[傳統決策樹]）分類模型（一）
阿新 • • 發佈：2018-12-31
簡介
分類演算法是基於類標號已知的訓練資料集建立分類模型並使用其對新觀測值（測試資料集）進行分類的演算法，因而也和迴歸一樣屬於監督學習演算法，都是使用訓練集的已知結論（類標號）預測測試資料集的分類結果，分類與迴歸的最大區別是後者對連續值進行處理。
我們先使用churn資料集分別建立訓練資料集和測試資料集，然後使用不用的分類模型對其分類。緊接著使用傳統分類樹與條件推理樹來介紹基於樹的分類方法，還將介紹基於延遲與基於概率的演算法。這些演算法都會基於訓練資料建立分類資料集，然後利用分類模型建立預測測試資料的分類結果，我們還將構建一個混淆矩陣來評測這些模型的預測效能。
準備訓練與測試資料集
使用C50包中的客戶流失資料(churn資料集)，有3333個樣例，資料維度為20，我們建立一個分類模型判斷客戶是否會流失，因為爭取一個新客戶的成本要小於一維護一個老客戶，因此預測的結果就比較重要。
在建立模型時，首先對資料進行預處理，通過觀測state、area_code、account_length對建模分類沒有作用，因此先去掉這三個屬性。
完成預處後將資料集分為訓練和測試兩個集合，我們使用一個樣本隨機函式生成一個序列，序列大小等於70%的樣例大小，再生成一個大小等於30%的樣例序列。
library(grid)
library(partykit)
library(C50)
data(churn)
str(churnTrain)
'data.frame': 3333 obs. of 20 variables:
$ state : Factor w/ 51 levels "AK","AL","AR",..: 17 36 32 36 37 2 20 25 19 50 ...
$ account_length : int 128 107 137 84 75 118 121 147 117 141 ... 使用遞迴分割樹建立分類模型
分類樹對分類結果的預測是基於一個或者多個輸入變數並結合劃分條件完成的。分裂過程從分類樹樹根結點開始:在每一個節點，演算法將根據劃分條件檢查輸入變數是否需要斷續向左子葉樹與右子葉樹遞迴進行劃分，當達到任意分類樹的子節點（終點）時，停止分裂。
churn.rp = rpart(churn ~ .,data = trainset)
churn.rp
n= 2315
node), split, n, loss, yval, (yprob)
* denotes terminal node
1) root 2315 342 no (0.14773218 0.85226782)
2) total_day_minutes>=265.45 144 59 yes (0.59027778 0.40972222)
4) voice_mail_plan=no 110 29 yes (0.73636364 0.26363636)
8) total_eve_minutes>=188.5 67 3 yes (0.95522388 0.04477612) *
9) total_eve_minutes< 188.5 43 17 no (0.39534884 0.60465116)
18) total_day_minutes>=282.7 19 6 yes (0.68421053 0.31578947) *
19) total_day_minutes< 282.7 24 4 no (0.16666667 0.83333333) *
5) voice_mail_plan=yes 34 4 no (0.11764706 0.88235294) *
3) total_day_minutes< 265.45 2171 257 no (0.11837863 0.88162137)
6) number_customer_service_calls>=3.5 168 82 yes (0.51190476 0.48809524)
12) total_day_minutes< 160.2 71 10 yes (0.85915493 0.14084507) *
13) total_day_minutes>=160.2 97 25 no (0.25773196 0.74226804)
26) total_eve_minutes< 155.5 20 7 yes (0.65000000 0.35000000) *
27) total_eve_minutes>=155.5 77 12 no (0.15584416 0.84415584) *
7) number_customer_service_calls< 3.5 2003 171 no (0.08537194 0.91462806)
14) international_plan=yes 188 76 no (0.40425532 0.59574468)
28) total_intl_calls< 2.5 38 0 yes (1.00000000 0.00000000) *
29) total_intl_calls>=2.5 150 38 no (0.25333333 0.74666667)
58) total_intl_minutes>=13.1 32 0 yes (1.00000000 0.00000000) *
59) total_intl_minutes< 13.1 118 6 no (0.05084746 0.94915254) *
15) international_plan=no 1815 95 no (0.05234160 0.94765840)
30) total_day_minutes>=224.15 251 50 no (0.19920319 0.80079681)
60) total_eve_minutes>=259.8 36 10 yes (0.72222222 0.27777778) *
61) total_eve_minutes< 259.8 215 24 no (0.11162791 0.88837209) *
31) total_day_minutes< 224.15 1564 45 no (0.02877238 0.97122762) *接下來，呼叫printcp來檢查複雜性引數
printcp(churn.rp)
Classification tree:
rpart(formula = churn ~ ., data = trainset)
Variables actually used in tree construction:
[1] international_plan number_customer_service_calls
[3] total_day_minutes total_eve_minutes
[5] total_intl_calls total_intl_minutes
[7] voice_mail_plan
Root node error: 342/2315 = 0.14773
n= 2315
CP nsplit rel error xerror xstd
1 0.076023 0 1.00000 1.00000 0.049920
2 0.074561 2 0.84795 0.99708 0.049860
3 0.055556 4 0.69883 0.76023 0.044421
4 0.026316 7 0.49415 0.52632 0.037673
5 0.023392 8 0.46784 0.52047 0.037481
6 0.020468 10 0.42105 0.50877 0.037092
7 0.017544 11 0.40058 0.47076 0.035788
8 0.010000 12 0.38304 0.47661 0.035993plotcp(churn.rp)
成本複雜性函式
最後使用summary檢查已經建立的模型
summary(churn.rp)
Call:
rpart(formula = churn ~ ., data = trainset)
n= 2315
CP nsplit rel error xerror xstd
1 0.07602339 0 1.0000000 1.0000000 0.04992005
2 0.07456140 2 0.8479532 0.9590643 0.04906076
3 0.05555556 4 0.6988304 0.7953216 0.04530196
4 0.02631579 7 0.4941520 0.5233918 0.03757730
5 0.02339181 8 0.4678363 0.5263158 0.03767329
6 0.02046784 10 0.4210526 0.5175439 0.03738427
7 0.01754386 11 0.4005848 0.5058480 0.03699399
8 0.01000000 12 0.3830409 0.4970760 0.03669750
Variable importance
total_day_minutes total_day_charge number_customer_service_calls
18 18 10
total_intl_minutes total_intl_charge total_eve_charge
8 8 8
total_eve_minutes international_plan total_intl_calls
8 7 6
number_vmail_messages voice_mail_plan total_night_calls
3 3 1
total_eve_calls
1
Node number 1: 2315 observations, complexity param=0.07602339
predicted class=no expected loss=0.1477322 P(node) =1
class counts: 342 1973
probabilities: 0.148 0.852
left son=2 (144 obs) right son=3 (2171 obs)
Primary splits:
total_day_minutes < 265.45 to the right, improve=60.145020, (0 missing)
total_day_charge < 45.125 to the right, improve=60.145020, (0 missing)
number_customer_service_calls < 3.5 to the right, improve=53.641430, (0 missing)
international_plan splits as RL, improve=43.729370, (0 missing)
voice_mail_plan splits as LR, improve= 6.089388, (0 missing)
Surrogate splits:
total_day_charge < 45.125 to the right, agree=1, adj=1, (0 split)
Node number 2: 144 observations, complexity param=0.07602339
predicted class=yes expected loss=0.4097222 P(node) =0.06220302
class counts: 85 59
probabilities: 0.590 0.410
left son=4 (110 obs) right son=5 (34 obs)
Primary splits:
voice_mail_plan splits as LR, improve=19.884860, (0 missing)
number_vmail_messages < 9.5 to the left, improve=19.884860, (0 missing)
total_eve_minutes < 167.05 to the right, improve=14.540020, (0 missing)
total_eve_charge < 14.2 to the right, improve=14.540020, (0 missing)
total_day_minutes < 283.9 to the right, improve= 6.339827, (0 missing)
Surrogate splits:
number_vmail_messages < 9.5 to the left, agree=1.000, adj=1.000, (0 split)
total_night_minutes < 110.3 to the right, agree=0.785, adj=0.088, (0 split)
total_night_charge < 4.965 to the right, agree=0.785, adj=0.088, (0 split)
total_night_calls < 50 to the right, agree=0.778, adj=0.059, (0 split)
total_intl_minutes < 15.3 to the left, agree=0.771, adj=0.029, (0 split)
Node number 3: 2171 observations, complexity param=0.0745614
predicted class=no expected loss=0.1183786 P(node) =0.937797
class counts: 257 1914
probabilities: 0.118 0.882
left son=6 (168 obs) right son=7 (2003 obs)
Primary splits:
number_customer_service_calls < 3.5 to the right, improve=56.398210, (0 missing)
international_plan splits as RL, improve=43.059160, (0 missing)
total_day_minutes < 224.15 to the right, improve=10.847440, (0 missing)
total_day_charge < 38.105 to the right, improve=10.847440, (0 missing)
total_intl_minutes < 13.15 to the right, improve= 6.347319, (0 missing)
Node number 4: 110 observations, complexity param=0.02631579
predicted class=yes expected loss=0.2636364 P(node) =0.0475162
class counts: 81 29
probabilities: 0.736 0.264
left son=8 (67 obs) right son=9 (43 obs)
Primary splits:
total_eve_minutes < 188.5 to the right, improve=16.419610, (0 missing)
total_eve_charge < 16.025 to the right, improve=16.419610, (0 missing)
total_night_minutes < 206.85 to the right, improve= 5.350500, (0 missing)
total_night_charge < 9.305 to the right, improve= 5.350500, (0 missing)
total_day_minutes < 281.15 to the right, improve= 5.254545, (0 missing)
Surrogate splits:
total_eve_charge < 16.025 to the right, agree=1.000, adj=1.000, (0 split)
total_night_calls < 82 to the right, agree=0.655, adj=0.116, (0 split)
total_intl_minutes < 3.35 to the right, agree=0.636, adj=0.070, (0 split)
total_intl_charge < 0.905 to the right, agree=0.636, adj=0.070, (0 split)
total_day_minutes < 268.55 to the right, agree=0.627, adj=0.047, (0 split)
Node number 5: 34 observations
predicted class=no expected loss=0.1176471 P(node) =0.01468683
class counts: 4 30
probabilities: 0.118 0.882
Node number 6: 168 observations, complexity param=0.0745614
predicted class=yes expected loss=0.4880952 P(node) =0.07257019
class counts: 86 82
probabilities: 0.512 0.488
left son=12 (71 obs) right son=13 (97 obs)
Primary splits:
total_day_minutes < 160.2 to the left, improve=29.655880, (0 missing)
total_day_charge < 27.235 to the left, improve=29.655880, (0 missing)
total_eve_minutes < 180.65 to the left, improve= 8.556953, (0 missing)
total_eve_charge < 15.355 to the left, improve= 8.556953, (0 missing)
number_customer_service_calls < 4.5 to the right, improve= 5.975362, (0 missing)
Surrogate splits:
total_day_charge < 27.235 to the left, agree=1.000, adj=1.000, (0 split)
total_night_calls < 79 to the left, agree=0.625, adj=0.113, (0 split)
total_intl_calls < 2.5 to the left, agree=0.619, adj=0.099, (0 split)
number_customer_service_calls < 4.5 to the right, agree=0.607, adj=0.070, (0 split)
total_eve_calls < 89.5 to the left, agree=0.601, adj=0.056, (0 split)
Node number 7: 2003 observations, complexity param=0.05555556
predicted class=no expected loss=0.08537194 P(node) =0.8652268
class counts: 171 1832
probabilities: 0.085 0.915
left son=14 (188 obs) right son=15 (1815 obs)
Primary splits:
international_plan splits as RL, improve=42.194510, (0 missing)
total_day_minutes < 224.15 to the right, improve=16.838410, (0 missing)
total_day_charge < 38.105 to the right, improve=16.838410, (0 missing)
total_intl_minutes < 13.15 to the right, improve= 6.210678, (0 missing)
total_intl_charge < 3.55 to the right, improve= 6.210678, (0 missing)
Node number 8: 67 observations
predicted class=yes expected loss=0.04477612 P(node) =0.02894168
class counts: 64 3
probabilities: 0.955 0.045
Node number 9: 43 observations, complexity param=0.02046784
predicted class=no expected loss=0.3953488 P(node) =0.01857451
class counts: 17 26
probabilities: 0.395 0.605
left son=18 (19 obs) right son=19 (24 obs)
Primary splits:
total_day_minutes < 282.7 to the right, improve=5.680947, (0 missing)
total_day_charge < 48.06 to the right, improve=5.680947, (0 missing)
total_night_minutes < 212.65 to the right, improve=4.558140, (0 missing)
total_night_charge < 9.57 to the right, improve=4.558140, (0 missing)
total_eve_minutes < 145.4 to the right, improve=4.356169, (0 missing)
Surrogate splits:
total_day_charge < 48.06 to the right, agree=1.000, adj=1.000, (0 split)
total_day_calls < 103 to the left, agree=0.674, adj=0.263, (0 split)
total_eve_calls < 104.5 to the left, agree=0.674, adj=0.263, (0 split)
total_intl_minutes < 11.55 to the left, agree=0.651, adj=0.211, (0 split)
total_intl_charge < 3.12 to the left, agree=0.651, adj=0.211, (0 split)
Node number 12: 71 observations
predicted class=yes expected loss=0.1408451 P(node) =0.03066955
class counts: 61 10
probabilities: 0.859 0.141
Node number 13: 97 observations, complexity param=0.01754386
predicted class=no expected loss=0.257732 P(node) =0.04190065
class counts: 25 72
probabilities: 0.258 0.742
left son=26 (20 obs) right son=27 (77 obs)
Primary splits:
total_eve_minutes < 155.5 to the left, improve=7.753662, (0 missing)
total_eve_charge < 13.22 to the left, improve=7.753662, (0 missing)
total_intl_minutes < 13.55 to the right, improve=2.366149, (0 missing)
total_intl_charge < 3.66 to the right, improve=2.366149, (0 missing)
number_customer_service_calls < 4.5 to the right, improve=2.297667, (0 missing)
Surrogate splits:
total_eve_charge < 13.22 to the left, agree=1.000, adj=1.00, (0 split)
total_night_calls < 143.5 to the right, agree=0.814, adj=0.10, (0 split)
total_eve_calls < 62 to the left, agree=0.804, adj=0.05, (0 split)
Node number 14: 188 observations, complexity param=0.05555556
predicted class=no expected loss=0.4042553 P(node) =0.0812095
class counts: 76 112
probabilities: 0.404 0.596
left son=28 (38 obs) right son=29 (150 obs)
Primary splits:
total_intl_calls < 2.5 to the left, improve=33.806520, (0 missing)
total_intl_minutes < 13.1 to the right, improve=30.527050, (0 missing)
total_intl_charge < 3.535 to the right, improve=30.527050, (0 missing)
total_day_minutes < 221.95 to the right, improve= 3.386095, (0 missing)
total_day_charge < 37.735 to the right, improve= 3.386095, (0 missing)
Node number 15: 1815 observations, complexity param=0.02339181
predicted class=no expected loss=0.0523416 P(node) =0.7840173
class counts: 95 1720
probabilities: 0.052 0.948
left son=30 (251 obs) right son=31 (1564 obs)
Primary splits:
total_day_minutes < 224.15 to the right, improve=12.5649300, (0 missing)
total_day_charge < 38.105 to the right, improve=12.5649300, (0 missing)
total_eve_minutes < 244.95 to the right, improve= 4.7875890, (0 missing)
total_eve_charge < 20.825 to the right, improve= 4.7875890, (0 missing)
total_night_minutes < 163.85 to the right, improve= 0.9074391, (0 missing)
Surrogate splits:
total_day_charge < 38.105 to the right, agree=1, adj=1, (0 split)
Node number 18: 19 observations
predicted class=yes expected loss=0.3157895 P(node) =0.008207343
class counts: 13 6
probabilities: 0.684 0.316
Node number 19: 24 observations
predicted class=no expected loss=0.1666667 P(node) =0.01036717
class counts: 4 20
probabilities: 0.167 0.833
Node number 26: 20 observations
predicted class=yes expected loss=0.35 P(node) =0.008639309
class counts: 13 7
probabilities: 0.650 0.350
Node number 27: 77 observations
predicted class=no expected loss=0.1558442 P(node) =0.03326134
class counts: 12 65
probabilities: 0.156 0.844
Node number 28: 38 observations
predicted class=yes expected loss=0 P(node) =0.01641469
class counts: 38 0
probabilities: 1.000 0.000
Node number 29: 150 observations, complexity param=0.05555556
predicted class=no expected loss=0.2533333 P(node) =0.06479482
class counts: 38 112
probabilities: 0.253 0.747
left son=58 (32 obs) right son=59 (118 obs)
Primary splits:
total_intl_minutes < 13.1 to the right, improve=45.356840, (0 missing)
total_intl_charge < 3.535 to the right, improve=45.356840, (0 missing)
total_day_calls < 95.5 to the left, improve= 4.036407, (0 missing)
total_day_minutes < 237.75 to the right, improve= 1.879020, (0 missing)
total_day_charge < 40.42 to the right, improve= 1.879020, (0 missing)
Surrogate splits:
total_intl_charge < 3.535 to the right, agree=1.0, adj=1.000, (0 split)
total_day_minutes < 52.45 to the left, agree=0.8, adj=0.063, (0 split)
total_day_charge < 8.92 to the left, agree=0.8, adj=0.063, (0 split)
Node number 30: 251 observations, complexity param=0.02339181
predicted class=no expected loss=0.1992032 P(node) =0.1084233
class counts: 50 201
probabilities: 0.199 0.801
left son=60 (36 obs) right son=61 (215 obs)
Primary splits:
total_eve_minutes < 259.8 to the right, improve=22.993380, (0 missing)
total_eve_charge < 22.08 to the right, improve=22.993380, (0 missing)
voice_mail_plan splits as LR, improve= 4.745664, (0 missing)
number_vmail_messages < 7.5