Preprocessing the Data

Welcome back! Now that you've learned how to load and understand the dataset for drawing recognition, it's time to move on to the next crucial step: preprocessing the data. Preprocessing is essential because it prepares your data for the machine learning model, ensuring that it can learn effectively. By the end of this lesson, you'll be equipped with the skills to clean and normalize your dataset, setting the stage for successful model training.

In this lesson, you will learn how to preprocess the dataset to make it suitable for training a drawing recognition model. We'll cover three main tasks: cleaning and normalizing the data. Here's a glimpse of the code you'll be working with:

import numpy as np
from sklearn.model_selection import train_test_split

# Load and prepare data
data = []
labels = []

for idx, cat in enumerate(categories):
    filepath = f'quickdraw_data/{cat}.npy'
    imgs = np.load(filepath)[:15000]
    if imgs.dtype != np.uint8:
        imgs = imgs.astype(np.uint8)
    data.append(imgs)
    labels.append(np.full(imgs.shape[0], idx))

data = np.concatenate(data, axis=0)
labels = np.concatenate(labels, axis=0)

# Shuffle data
indices = np.arange(len(data))
np.random.shuffle(indices)
data, labels = data[indices], labels[indices]

# Reshape and normalize
data = data.reshape(-1, 28, 28, 1).astype('float32') / 255.0

# Split data into training and testing sets
x_train, x_test, y_train, y_test = train_test_split(data, labels, test_size=0.2, random_state=42)

This code snippet demonstrates how to load, clean, and normalize the data, as well as how to split it into training and testing sets. You'll learn how to ensure your data is in the right format and ready for model training.

Preprocessing is a vital step in any machine learning project because it directly impacts the model's performance. By cleaning the data, you remove any inconsistencies or errors that could skew the results. Normalizing the data ensures that all input features are on a similar scale, which helps the model learn more effectively.

Are you excited to dive in? Let's start the practice section and apply these preprocessing techniques to your dataset!