Evaluating TensorFlow Models: From Data to Insight

Lesson Overview

Welcome to today's lesson on Evaluating a Model with Tensorflow . In this lesson, we're going to explore how to evaluate the performance of a model that we previously trained using TensorFlow . Specifically, we will be using the evaluate() function provided by TensorFlow to assess how well our model performs on unseen data. Model evaluation is an essential step in the machine learning pipeline as it helps us gauge the effectiveness of our model and its ability to generalize to new data. We will also discuss the importance of splitting our data into training and testing sets for robust model evaluation. After this lesson, you should have a good understanding of how to perform model evaluation and interpret the results to fine-tune your model.

Understanding the Dataset

Before we dive into model evaluation, imagine we have a dataset containing the study habits of a group of students. More specifically, we have data on the number of hours each student studied and the amount of sleep they got. Python import numpy as np # Example data: hours studied, hours slept X = np.array([[4, 6], [5, 7], [2, 8], [1, 3], [3, 4], [0, 5], [1, 1], [2, 4], [3, 5], [5, 5], [0, 4], [4, 4],]) import numpy as np # Example data: hours studied, hours slept X = np.array([[4, 6], [5, 7], [2, 8], [1, 3], [3, 4], [0, 5], [1, 1], [2, 4], [3, 5], [5, 5], [0, 4], [4, 4],]) The dataset has 12 observations and each observation has two features: hours studied and hours slept. We are using this data to predict whether a student passes (denoted as 1) or fails (denoted as 0) their exam. For the sake of simplicity, we've already labeled our data. Python # Labels: 1 if passed, 0 if failed y = np.array([[1], [1], [1], [0], [0], [0], [0], [0], [1], [1], [0], [1]]) # Labels: 1 if passed, 0 if failed y = np.array([[1], [1], [1], [0], [0], [0], [0], [0], [1], [1], [0], [1]]) We would like to build a model that takes in these two features and outputs a prediction of whether a student is likely to pass or fail.

Data Splitting - Training and Test Datasets

In machine learning, it's crucial that we have two sets of data: a training set and a testing set. Our model learns from the training set and we evaluate our model's performance using the testing set. We can use the train_test_split function from sklearn's model_selection module to divide our data. The test_size parameter specifies the proportion of the dataset to include in the test split, and the random_state parameter is used to shuffle and partition the data randomly. Python from sklearn.model_selection import train_test_split # Split the dataset into 80% training and 20% testing X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) from sklearn.model_selection import train_test_split # Split the dataset into 80% training and 20% testing X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) We chose a split of 80% training and 20% testing, which is a common choice in machine learning projects. Now that our data is ready, let's move back to our model.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Revisiting the Model

Our TensorFlow model is a simple neural network consisting of a Sequential model with two Dense layers. The first Dense layer has 5 neurons and uses the 'relu' activation function, while the second (output) layer has one neuron and uses the 'sigmoid' activation function.

The 'relu' activation function is one of the most commonly used activation functions due to its simplicity and efficiency, while the 'sigmoid' function is typically used in the output layer for binary classification problems. After building our model, we need to compile and train it, so it can be ready for evaluation

Python

import tensorflow as tf

# Initializing the model
model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(2,)), 
    tf.keras.layers.Dense(5, activation='relu'), 
    tf.keras.layers.Dense(1, activation='sigmoid') 
])

# Compiling the model with 'accuracy' as chosen metric
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Training the model with the training data
model.fit(X_train, y_train, epochs=10, verbose=0)

import tensorflow as tf

# Initializing the model
model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(2,)), 
    tf.keras.layers.Dense(5, activation='relu'), 
    tf.keras.layers.Dense(1, activation='sigmoid') 
])

# Compiling the model with 'accuracy' as chosen metric
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Training the model with the training data
model.fit(X_train, y_train, epochs=10, verbose=0)

In the compilation step, we chose accuracy as the metric to evaluate our model's performance. This choice directly influences our evaluation because the model.evaluate() function will return the metrics chosen during compilation. Other metrics could have been used to provide more detailed insights into model performance, such as precision, which measures how many of the positive predictions made by the model are actually correct.

In the training step, we set verbose=0 to ensure that no output is generated during the training process. This can be useful when running multiple experiments or when you want to keep the output clean and uncluttered. In this lesson, we are focusing on other outputs for the evaluation.

Lesson Overview

Understanding the Dataset

Data Splitting - Training and Test Datasets

Join the 1M+ learners on CodeSignal

Revisiting the Model

Evaluating the Model

Lesson Summary