Training a Baseline Model

Welcome to this lesson on training a baseline model! Establishing a baseline model is crucial in any machine learning project. It serves as a reference point, allowing you to measure the performance of more complex models you might develop later. Our goal today is to understand how to train a simple baseline model using logistic regression and evaluate its performance. This will provide you with a solid foundation for tackling more advanced models in the future.

Before model training, it's essential to preprocess our data. Preprocessing steps from the previous units will be stored as a function in the preprocessing.py file.

Once our data is preprocessed, we separate it into features and target variables, which is crucial for model training.

• Separating Features and Target: We drop the label column to create our feature set X_train and use the label column as our target y_train.

  Separating the label column is necessary because the model needs to learn the mapping between input features (X_train) and target outcomes (y_train). Including the label in the feature set would result in data leakage, causing the model to unfairly learn from the output it is supposed to predict.

  Python

  X_train = train_data.drop('label', axis=1)
  y_train = train_data['label']

• Feature Scaling: Scaling ensures all numerical features contribute equally to the model's performance. We use StandardScaler from scikit-learn to standardize our numerical features.

  Python

  from sklearn.preprocessing import StandardScaler
  scaler = StandardScaler()
  numeric_columns = ['age', 'income', 'credit_score', 'num_purchases', 'time_on_site', 'num_visits']
  X_train_scaled = X_train.copy()
  X_train_scaled[numeric_columns] = scaler.fit_transform(X_train[numeric_columns])

By scaling the numerical features, we ensure our model is not biased towards features with larger magnitudes.

Now that our data is ready, we can train a baseline model. Logistic regression is a simple yet effective classification algorithm, making it an excellent choice for our baseline.

• Training the Model: We initialize a logistic regression model and fit it to our scaled training data.

  Python

  from sklearn.linear_model import LogisticRegression
  model = LogisticRegression(max_iter=1000)
  model.fit(X_train_scaled, y_train)

The max_iter parameter specifies the maximum number of iterations for the solver to converge, ensuring the model finds an optimal solution.

Evaluating our model's performance is crucial to understanding its effectiveness. We use the test data to assess how well our model generalizes to unseen data.

• Preprocessing Test Data: We preprocess the test data using the same steps as the training data.

  Python

  test_data = preprocess_data('data/test.csv')
  X_test = test_data.drop('label', axis=1)
  y_test = test_data['label']
  X_test_scaled = X_test.copy()
  X_test_scaled[numeric_columns] = scaler.transform(X_test[numeric_columns])

• Making Predictions and Calculating Accuracy: We use our trained model to make predictions on the test data and calculate the accuracy score.

  Python

  from sklearn.metrics import accuracy_score
  y_pred = model.predict(X_test_scaled)
  accuracy = accuracy_score(y_test, y_pred)
  print(f"Accuracy on the testing set: {accuracy:.2f}")  # You will see an output once you train your model during practice

The accuracy score provides a straightforward measure of how well our model performs, indicating the proportion of correctly classified instances.

In this lesson, we've covered the essential steps for training a baseline model: data preprocessing, feature scaling, model training, and evaluation. Establishing a baseline is a critical step in any machine learning project, providing a reference point for future model improvements. With this foundation, you're now ready to explore more complex models and techniques.

Now it's time to put your knowledge into practice! You'll have the opportunity to apply these concepts to our dataset, reinforcing your understanding and gaining hands-on experience. Let's get started!