Understanding Confusion Matrix | Model Evaluation Metrics

Example of confusion matrix used to evaluate the quality of the output of a classifier on our data set. The diagonal elements represent the number of points for which the predicted label is equal to the true label, while off-diagonal elements are those that are mislabeled by the classifier. The higher the diagonal values of the confusion matrix the better, indicating many correct predictions.

The figures show the confusion matrix with and without normalization by class support size (number of elements in each class). This kind of normalization can be interesting in the case of class imbalance to have a more visual interpretation of which class is being misclassified.

#Plotting confusion matrix
def plot_confusion_matrix(y_test, y_pred, classes,
normalize=False,
title=None,
cmap=plt.cm.Blues):
"""
This function prints and plots the confusion matrix.
Normalization can be applied by setting `normalize=True`.
"""
if not title:
if normalize:
title = 'Normalized confusion matrix'
else:
title = 'Confusion matrix, without normalization'

# Compute confusion matrix
cm = confusion_matrix(y_test, y_pred)
# Only use the labels that appear in the data
# For unique lebel read this:
# https://scikit-learn.org/stable/modules/generated/sklearn.utils.multiclass.unique_labels.html#sklearn.utils.multiclass.unique_labels

classes = unique_labels(y_test, y_pred)
if normalize:
cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
print("Normalized confusion matrix")
else:
print('Confusion matrix, without normalization')

print(cm)

fig, ax = plt.subplots()
im = ax.imshow(cm, interpolation='nearest', cmap=cmap)
ax.figure.colorbar(im, ax=ax)
# We want to show all ticks...
ax.set(xticks=np.arange(cm.shape[1]),
yticks=np.arange(cm.shape[0]),
# ... and label them with the respective list entries
xticklabels=classes, yticklabels=classes,
title=title,
ylabel='True label',
xlabel='Predicted label')

# Rotate the tick labels and set their alignment.
plt.setp(ax.get_xticklabels(), rotation=45, ha="right",
rotation_mode="anchor")

# Loop over data dimensions and create text annotations.
fmt = '.2f' if normalize else 'd'
thresh = cm.max() / 2.
for i in range(cm.shape[0]):
for j in range(cm.shape[1]):
ax.text(j, i, format(cm[i, j], fmt),
ha="center", va="center",
color="white" if cm[i, j] > thresh else "black")
fig.tight_layout()
return ax

1. Plot non-normalized confusion matrix

from sklearn.utils.multiclass import unique_labels

# Plot non-normalized confusion matrix
plot_confusion_matrix(y_test, y_pred, classes=[0,1],
title='Confusion matrix, without normalization')


2. Plot normalized confusion matrix

# Plot normalized confusion matrix
plot_confusion_matrix(y_test, y_pred, classes=[0,1], normalize=True,
title='Normalized confusion matrix')

Here the results are not as good as they could be as our choice for the regularization parameter C was not the best. In real life applications, this parameter is usually chosen using Tuning the hyper-parameters of an estimator.