Training and Evaluating Neural Networks

Topic Overview

Today, we dive into an integral piece of the deep learning puzzle: training your neural network . In this lesson, we will demystify what training entails and learn how to implement it using TensorFlow . By training the model, the neural network learns from the input data, gradually adjusting its parameters (weights and biases) to minimize the error in its predictions.

Importance of Training a Neural Network

Training a neural network is akin to teaching a child to recognize shapes. The child learns from repeated exposure and feedback, just as a neural network learns from training datasets. In practice, the training process involves several rounds of forwarding input data through the network, calculating the error (the difference between the network's output and the actual, desired output), and adjusting the weights and biases to minimize this error. This process is much like a child adjusting their understanding of shapes based on feedback! This iterative method allows the neural network to learn independently from the data and can eventually lead to accurate predictions or classifications, thereby enabling us to create powerful and predictive models.

Understanding the `model.fit()` Method

The model.fit() method in TensorFlow is our main tool for training a neural network. This method takes in inputs and their corresponding target values, fitting the model to this data over a certain number of iterations known as epochs. Here are the key parameters we need to understand: X: Input data. This is the data from which your model will learn. y: Target data. These are the answers or results that your model should learn to predict. epochs: One epoch is one complete pass through the entire training dataset. batch_size: This is the number of samples per gradient update. It's akin to breaking our dataset into smaller chunks, updating our model's learning parameters after each chunk. validation_split: This value (between 0 and 1) determines the fraction of your training data that should be set aside for validation. Validation data guides the training process by providing a measure of model performance on unseen data. Let's see this method in action with some code: Pythonfrom sklearn.datasets import load_digits from sklearn.model_selection import train_test_split from keras.utils import to_categorical from keras.models import Sequential from keras.layers import Dense # Load data digits = load_digits() X = digits.data y = digits.target # Convert to one-hot encoding y = to_categorical(y) # Split the data into training and test sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2) # Create model model = Sequential() model.add(Dense(64, input_dim=len(X[0]), activation='relu')) model.add(Dense(len(y[0]), activation='softmax')) # Compile model model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) # Train the model history = model.fit(X_train, y_train, epochs=5, batch_size=32, validation_split=0.2)

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Visualizing Training History

Understanding how well the training is progressing is key when training a model. This is where the 'history' object from model.fit() comes into play. This object contains information about the training and validation accuracy and loss for each epoch, which we can use to track the learning progress of our network.

We can extract this data and visualize them using plots. The following code creates a simple line plot for the training and validation accuracy for each epoch, which gives us insight into how effectively our model is learning.

Python

import matplotlib.pyplot as plt

# Plot the training history
plt.plot(history.history['accuracy'], label='accuracy') # Plotting training accuracy
plt.plot(history.history['val_accuracy'], label = 'val_accuracy') # Plotting validation accuracy
plt.xlabel('Epoch') # Label for x-axis
plt.ylabel('Accuracy') # Label for y-axis
plt.ylim([0, 1]) # Setting limit for y-axis
plt.legend(loc='lower right') # Positioning legend
plt.show()  # Displaying the plot

import matplotlib.pyplot as plt

# Plot the training history
plt.plot(history.history['accuracy'], label='accuracy') # Plotting training accuracy
plt.plot(history.history['val_accuracy'], label = 'val_accuracy') # Plotting validation accuracy
plt.xlabel('Epoch') # Label for x-axis
plt.ylabel('Accuracy') # Label for y-axis
plt.ylim([0, 1]) # Setting limit for y-axis
plt.legend(loc='lower right') # Positioning legend
plt.show()  # Displaying the plot

ouput:

 1/36 [..............................] - ETA: 0s - loss: 0.4314 - accuracy: 0.8125
13/36 [=========>....................] - ETA: 0s - loss: 0.3962 - accuracy: 0.8774
25/36 [===================>..........] - ETA: 0s - loss: 0.3419 - accuracy: 0.8875
36/36 [==============================] - 0s 7ms/step - loss: 0.3615 - accuracy: 0.8842 - val_loss: 0.3589 - val_accuracy: 0.8785

 1/36 [..............................] - ETA: 0s - loss: 0.4314 - accuracy: 0.8125
13/36 [=========>....................] - ETA: 0s - loss: 0.3962 - accuracy: 0.8774
25/36 [===================>..........] - ETA: 0s - loss: 0.3419 - accuracy: 0.8875
36/36 [==============================] - 0s 7ms/step - loss: 0.3615 - accuracy: 0.8842 - val_loss: 0.3589 - val_accuracy: 0.8785

In the graph, you can see the accuracy (both 'training' and 'validation') plotted against the number of epochs. An epoch is one round of passing all samples through the model. This visualization provides a view into the progression of learning and can guide the adjustments needed in the learning parameters if necessary.

You can notice that with each passing epoch our accuracy measure (metric against the training set) and our val_accuracy (metric against the validation set, i.e. unseen data) gets better but we are still below 0.9 on both. How can we increase it? Given the trend we are seeing, increasing the number of epochs should be the right approach.

Improving our model

When we double the number of epochs we get the following code: