Heatmap of Correlation Matrix

Topic Overview

Hello and welcome! In this lesson, you'll learn how to compute and visualize a correlation matrix using the diamonds dataset . The goal is to understand how different features in the dataset relate to each other through correlations and visualize these relationships using a heatmap .

Convert Categorical Variables

The diamonds dataset contains categorical features such as cut, color, and clarity . Correlation matrices require numerical data, so we need to convert these categorical variables to numerical codes. First, let's identify the categorical columns that need conversion, then we'll convert them using astype('category').cat.codes . astype('category') makes sure the feature is a categorical type, after which .cat.codes can be applied to convert it to a unique integer code ranging from 0 to number_of_categories - 1 . Python import seaborn as sns import pandas as pd # Load the diamonds dataset diamonds = sns.load_dataset('diamonds') # Convert categorical variables for correlation calculation category_columns = ['cut', 'color', 'clarity'] for col in category_columns: diamonds[col] = diamonds[col].astype('category').cat.codes print(diamonds.head()) import seaborn as sns import pandas as pd # Load the diamonds dataset diamonds = sns.load_dataset('diamonds') # Convert categorical variables for correlation calculation category_columns = ['cut', 'color', 'clarity'] for col in category_columns: diamonds[col] = diamonds[col].astype('category').cat.codes print(diamonds.head()) By converting these columns, you enable the dataset to be used in correlation computations where all features need to be numerical: plaintext carat cut color clarity depth table price x y z 0 0.23 0 1 6 61.5 55.0 326 3.95 3.98 2.43 1 0.21 1 1 5 59.8 61.0 326 3.89 3.84 2.31 2 0.23 3 1 3 56.9 65.0 327 4.05 4.07 2.31 3 0.29 1 5 4 62.4 58.0 334 4.20 4.23 2.63 4 0.31 3 6 6 63.3 58.0 335 4.34 4.35 2.75 carat cut color clarity depth table price x y z 0 0.23 0 1 6 61.5 55.0 326 3.95 3.98 2.43 1 0.21 1 1 5 59.8 61.0 326 3.89 3.84 2.31 2 0.23 3 1 3 56.9 65.0 327 4.05 4.07 2.31 3 0.29 1 5 4 62.4 58.0 334 4.20 4.23 2.63 4 0.31 3 6 6 63.3 58.0 335 4.34 4.35 2.75

Compute the Correlation Matrix

Next, we'll compute the correlation matrix. A correlation matrix is a table that shows correlation coefficients between variables. Each cell in the table shows the correlation between two variables. We'll use the corr() method from pandas for this: Pythonimport seaborn as sns import pandas as pd # Load the diamonds dataset diamonds = sns.load_dataset('diamonds') # Convert categorical variables for correlation calculation category_columns = ['cut', 'color', 'clarity'] for col in category_columns: diamonds[col] = diamonds[col].astype('category').cat.codes # Compute the correlation matrix correlation_matrix = diamonds.corr() print(correlation_matrix)import seaborn as sns import pandas as pd # Load the diamonds dataset diamonds = sns.load_dataset('diamonds') # Convert categorical variables for correlation calculation category_columns = ['cut', 'color', 'clarity'] for col in category_columns: diamonds[col] = diamonds[col].astype('category').cat.codes # Compute the correlation matrix correlation_matrix = diamonds.corr() print(correlation_matrix) The correlation matrix will give us an understanding of how each feature relates to every other feature in the dataset. The values will range from -1 to 1, where: 1 indicates a perfect positive correlation -1 indicates a perfect negative correlation 0 indicates no correlation Understanding these values helps us see the strength and direction of the relationships between different features in the dataset. text carat cut color clarity depth table price \ carat 1.000000 0.134967 0.291437 0.352841 0.028224 0.181618 0.921591 cut 0.134967 1.000000 0.020519 0.189175 0.218055 0.433405 0.053491 color 0.291437 0.020519 1.000000 -0.025631 0.047279 0.026465 0.172511 clarity 0.352841 0.189175 -0.025631 1.000000 0.067384 0.160327 0.146800 depth 0.028224 0.218055 0.047279 0.067384 1.000000 -0.295779 -0.010647 table 0.181618 0.433405 0.026465 0.160327 -0.295779 1.000000 0.127134 price 0.921591 0.053491 0.172511 0.146800 -0.010647 0.127134 1.000000 x 0.975094 0.125565 0.270287 0.371999 -0.025289 0.195344 0.884435 y 0.951722 0.121462 0.263584 0.358420 -0.029341 0.183760 0.865421 z 0.953387 0.149323 0.268227 0.366952 0.094924 0.150929 0.861249 x y z carat 0.975094 0.951722 0.953387 cut 0.125565 0.121462 0.149323 color 0.270287 0.263584 0.268227 clarity 0.371999 0.358420 0.366952 depth -0.025289 -0.029341 0.094924 table 0.195344 0.183760 0.150929 price 0.884435 0.865421 0.861249 x 1.000000 0.974701 0.970772 y 0.974701 1.000000 0.952006 z 0.970772 0.952006 1.000000 carat cut color clarity depth table price \ carat 1.000000 0.134967 0.291437 0.352841 0.028224 0.181618 0.921591 cut 0.134967 1.000000 0.020519 0.189175 0.218055 0.433405 0.053491 color 0.291437 0.020519 1.000000 -0.025631 0.047279 0.026465 0.172511 clarity 0.352841 0.189175 -0.025631 1.000000 0.067384 0.160327 0.146800 depth 0.028224 0.218055 0.047279 0.067384 1.000000 -0.295779 -0.010647 table 0.181618 0.433405 0.026465 0.160327 -0.295779 1.000000 0.127134 price 0.921591 0.053491 0.172511 0.146800 -0.010647 0.127134 1.000000 x 0.975094 0.125565 0.270287 0.371999 -0.025289 0.195344 0.884435 y 0.951722 0.121462 0.263584 0.358420 -0.029341 0.183760 0.865421 z 0.953387 0.149323 0.268227 0.366952 0.094924 0.150929 0.861249 x y z carat 0.975094 0.951722 0.953387 cut 0.125565 0.121462 0.149323 color 0.270287 0.263584 0.268227 clarity 0.371999 0.358420 0.366952 depth -0.025289 -0.029341 0.094924 table 0.195344 0.183760 0.150929 price 0.884435 0.865421 0.861249 x 1.000000 0.974701 0.970772 y 0.974701 1.000000 0.952006 z 0.970772 0.952006 1.000000

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal