機器學習1/100天-資料預處理
阿新 • • 發佈:2019-01-01
Day1 Data PreProcessing
github: 100-Days-Of-ML-Code
1.匯入兩個常用的python庫,numpy, pandas
import numpy as np
import pandas as pd
2.讀取資料檔案
dataset = pd.read_csv("Data.csv")
X = dataset.iloc[:,:-1].values
Y = dataset.iloc[:,3].values
pd函式read_csv讀取資料檔案
而後dataframe.iloc按照位置選取資料,劃分成X和Y
3.預設值處理
使用sklearn.preprocessing.Imputer處理預設值,以均值代替NaN
from sklearn.preprocessing import Imputer
imputer = Imputer(missing_values = "NaN", strategy = "mean", axis = 0)
imputer = imputer.fit(X[:,1:3])
X[:,1:3] = imputer.transform(X[:,1:3])
4.將文字資料編碼
使用sklearn.preprocessing.LabelEncoder和OneHotEncoder編碼資料。
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
labelencoder_X = LabelEncoder()
X[:,0] = labelencoder_X.fit_transform(X[:,0])
onehotencoder = OneHotEncoder(categorical_features = [0])
X = onehotencoder.fit_transform(X).toarray()
labelEncoder_Y = LabelEncoder()
Y = labelEncoder_Y.fit_transform(Y )
LabelEncoder文字變數值,OneHotEncoder數值變OneHot編碼
5.劃分訓練集和測試集
在新版本中train_test_split函式位於model_select module
from sklearn.cross_validation import train_test_split
X_train, X_test, Y_train, Y_test = train_test_split(X,Y,test_size=0.2,random_state=0)
6.資料標準化
from sklearn.preprocessing import StandardScaler
sc_X = StandardScaler()
X_train = sc_X.fit_transform(X_train)
X_test = sc_X.fit_transform(X_test)