Using Neural Networks:

Define the model :

# Build a neural network :NN_model = Sequential()

NN_model.add(Dense(128, input_dim = 68, activation='relu'))NN_model.add(Dense(256, activation='relu'))NN_model.add(Dense(256, activation='relu'))NN_model.add(Dense(256, activation='relu'))NN_model.add(Dense(1, activation='sigmoid'))NN_model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

Define a checkpoint callback :

checkpoint_name = 'Weights-{epoch:03d}-{val_acc:.5f}.hdf5' checkpoint = ModelCheckpoint(checkpoint_name, monitor='val_acc', verbose = 1, save_best_only = True, mode ='max')callbacks_list = [checkpoint]

Train the model :

NN_model.fit(train_X, train_y, epochs=150, batch_size=64, validation_split = 0.2, callbacks=callbacks_list)

The training process :

Epoch 00044: val_acc did not improve from 0.88060 Epoch 45/150 534/534 [==============================] - 0s 149us/step - loss: 0.3196 - acc: 0.8652 - val_loss: 0.4231 - val_acc: 0.8433  Epoch 00045: val_acc did not improve from 0.88060 Epoch 46/150 534/534 [==============================] - 0s 134us/step - loss: 0.3156 - acc: 0.8670 - val_loss: 0.4175 - val_acc: 0.8358  Epoch 00046: val_acc did not improve from 0.88060 Epoch 47/150 534/534 [==============================] - 0s 144us/step - loss: 0.3031 - acc: 0.8689 - val_loss: 0.4214 - val_acc: 0.8433  Epoch 00047: val_acc did not improve from 0.88060 Epoch 48/150 534/534 [==============================] - 0s 131us/step - loss: 0.3117 - acc: 0.8689 - val_loss: 0.4095 - val_acc: 0.8582..Epoch 00148: val_acc did not improve from 0.88060Epoch 149/150534/534 [==============================] - 0s 146us/step - loss: 0.1599 - acc: 0.9382 - val_loss: 1.0482 - val_acc: 0.7761Epoch 00149: val_acc did not improve from 0.88060Epoch 150/150534/534 [==============================] - 0s 133us/step - loss: 0.1612 - acc: 0.9307 - val_loss: 1.1589 - val_acc: 0.7836Epoch 00150: val_acc did not improve from 0.88060<keras.callbacks.History at 0x7f47cb549320>

Load wights file of the best model :

wights_file = './Weights-016-0.88060.hdf5' # choose the best checkpoint NN_model.load_weights(wights_file) # load itNN_model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

Test the trained model on test data :

predictions = NN_model.predict(test_X)# round predictionsrounded = [round(x[0]) for x in predictions]predictions = roundedscore = accuracy_score(test_y ,predictions)print(score)

0.81165

The accuracy of this neural network model is 81%, we notice that using random forest gives us a higher accuracy.