Preprocessing Text Data: Train-Test Split and Stratified Cross-Validation | CodeSignal Learn

Preprocessing Text Data: Train-Test Split and Stratified Cross-Validation | CodeSignal Learn

Topic Overview and Actualization

Greetings in this segment of Introduction to Modeling Techniques for Text Classification! This part focuses on the heart of preprocessing techniques in modeling — Train-Test Split and Stratified Cross-Validation. Rails of any machine learning model are laid by creating an effective split in the dataset and ensuring class balance. You'll not just learn about these core concepts but also implement them using Python's powerful library, scikit-learn . Using these techniques, you'll split the SMS Spam Collection dataset for effective text classification later in the course.

Understanding the Dataset

In real life, as you browse your inbox, you come across various legitimate (ham) and promotional or unsolicited (spam) messages. Machine Learning models help distinguish between these, by labeling an incoming message as spam or ham . A good model is crucial for avoiding a cluttered inbox. Let's start by loading the dataset. The datasets library can pull the data directly, and we'll convert it into a pandas DataFrame for easier data manipulation. Python # Import necessary libraries import datasets import pandas as pd # Load the dataset spam_dataset = datasets.load_dataset('codesignal/sms-spam-collection', split='train') spam_dataset = pd.DataFrame(spam_dataset) # Display the first few rows of the dataset print(spam_dataset.head(3)) The output will be: text label message 0 ham Go until jurong point, crazy.. Available only ... 1 ham Ok lar... Joking wif u oni... 2 spam Free entry in 2 a wkly comp to win FA Cup fina... This output displays the first three rows of the dataset, showcasing two ham messages and one spam message. From the dataset, you can see that each message is labeled as either ham or spam under the 'label' column, giving an indicator of the class of each message. By running the above code blocks, you have loaded the SMS Spam Collection dataset - a collection of 5572 text messages, each classified as either ham or spam, into a pandas DataFrame, a data structure ideal for data manipulation tasks. It's crucial to familiarize yourself with the dataset before further processing to provide a foundation for the preprocessing tasks.

Diving into Train-Test Split

Before we start our journey of text classification, let's understand Train-Test Split. It is a method used to separate our dataset into two parts — a training set and a test set. The training set is what our machine learning model trains on, while the test set is used to evaluate the performance of our trained model. But why do we split our dataset? It prevents our model from overlearning the training data and ensures that it predicts unseen data robustly, improving model generalizability. Let's implement train-test split on the data: Python from sklearn.model_selection import train_test_split # Define X (input features) and Y (output labels) X = spam_dataset["message"] Y = spam_dataset["label"] # Perform the train test split X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=42) By specifying test_size as 0.2, we're splitting our data such that 80% of it goes to training, and the remaining 20% will be used for testing.

Sign up

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Stratified Cross-Validation

While any splitting would have worked, why did we choose "Stratified Cross-Validation"? It's because we want to ensure that both our training and testing datasets contain an equal representation of both spam and ham classes. This strategy is especially helpful when we have an imbalanced dataset, where one class heavily outnumbers the other. Let's revise our train-test split and apply stratified cross-validation: # Perform the train test split using stratified cross-validation X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=42, stratify=Y) Now that our data is prepared, let's validate our split: # Display the number of samples in training and test datasets print(f"Training dataset: {len(X_train)} samples") print(f"Test dataset: {len(X_test)} samples") The output will be: Training dataset: 4459 samples Test dataset: 1115 samples This output confirms the successful split of our dataset into training and test datasets, with 4459 samples allocated for training and 1115 samples designated for testing, ensuring a balanced representation of classes in both sets.