Highlighting Correlation Values Within a Range

Introduction to Correlation and Its Importance

Hello! In today's lesson, we will dive into the concept of correlation and focus specifically on highlighting certain correlation values within the diamonds dataset. Correlation is a statistical measure that describes the extent to which two variables change together. Understanding correlations is crucial in data analysis as it helps us identify relationships between different variables. For example: Positive Correlation: As one variable increases, the other also increases (e.g., height and weight). Negative Correlation: As one variable increases, the other decreases (e.g., speed and travel time). By the end of this lesson, you will be able to compute, mask, and visually represent these correlations to get a clearer picture of the underlying data relationships.

Computing the Correlation Matrix

Let's compute the correlation matrix for our prepared diamonds dataset. As mentioned before, the correlation matrix is a table showing correlation coefficients between many variables. Each cell in the table shows the correlation between two variables. You might be familiar with the process by now, but here's how to compute and display the correlation matrix using pandas: Pythonimport seaborn as sns import pandas as pd # Load the diamonds dataset diamonds = sns.load_dataset('diamonds') diamonds['cut'] = diamonds['cut'].astype('category').cat.codes diamonds['color'] = diamonds['color'].astype('category').cat.codes diamonds['clarity'] = diamonds['clarity'].astype('category').cat.codes # Calculate the correlation matrix correlation_matrix = diamonds.corr() # Display the correlation matrix print(correlation_matrix) Output: text carat cut color clarity depth table price \ carat 1.000000 0.134967 0.291437 0.352841 0.028224 0.181618 0.921591 cut 0.134967 1.000000 0.020519 0.189175 0.218055 0.433405 0.053491 color 0.291437 0.020519 1.000000 -0.025631 0.047279 0.026465 0.172511 clarity 0.352841 0.189175 -0.025631 1.000000 0.067384 0.160327 0.146800 depth 0.028224 0.218055 0.047279 0.067384 1.000000 -0.295779 -0.010647 table 0.181618 0.433405 0.026465 0.160327 -0.295779 1.000000 0.127134 price 0.921591 0.053491 0.172511 0.146800 -0.010647 0.127134 1.000000 x 0.975094 0.125565 0.270287 0.371999 -0.025289 0.195344 0.884435 y 0.951722 0.121462 0.263584 0.358420 -0.029341 0.183760 0.865421 z 0.953387 0.149323 0.268227 0.366952 0.094924 0.150929 0.861249 x y z carat 0.975094 0.951722 0.953387 cut 0.125565 0.121462 0.149323 color 0.270287 0.263584 0.268227 clarity 0.371999 0.358420 0.366952 depth -0.025289 -0.029341 0.094924 table 0.195344 0.183760 0.150929 price 0.884435 0.865421 0.861249 x 1.000000 0.974701 0.970772 y 0.974701 1.000000 0.952006 z 0.970772 0.952006 1.000000 In the matrix, correlation coefficients range from -1 to 1. Values close to 1 imply a strong positive correlation, while values close to -1 imply a strong negative correlation. Values near 0 imply little to no correlation.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Masking Values in the Correlation Matrix

To enhance visibility, we'll mask correlation values within a specified range (e.g., -0.3 to 0.3). Masking helps us focus on more significant relationships.

We'll use the map function in pandas to mask these values:

Python

# Masking the correlation matrix for values within the range -0.3 to 0.3
mask = correlation_matrix.map(lambda x: -0.3 < x < 0.3)
corr_to_plot = correlation_matrix.mask(mask)

# Display the masked correlation matrix
print(corr_to_plot)

# Masking the correlation matrix for values within the range -0.3 to 0.3
mask = correlation_matrix.map(lambda x: -0.3 < x < 0.3)
corr_to_plot = correlation_matrix.mask(mask)

# Display the masked correlation matrix
print(corr_to_plot)

Output:

text

            carat       cut  color  ...         x         y         z
carat    1.000000       NaN    NaN  ...  0.975094  0.951722  0.953387
cut           NaN  1.000000    NaN  ...       NaN       NaN       NaN
color         NaN       NaN    1.0  ...       NaN       NaN       NaN
clarity  0.352841       NaN    NaN  ...  0.371999  0.358420  0.366952
depth         NaN       NaN    NaN  ...       NaN       NaN       NaN
table         NaN  0.433405    NaN  ...       NaN       NaN       NaN
price    0.921591       NaN    NaN  ...  0.884435  0.865421  0.861249
x        0.975094       NaN    NaN  ...  1.000000  0.974701  0.970772
y        0.951722       NaN    NaN  ...  0.974701  1.000000  0.952006
z        0.953387       NaN    NaN  ...  0.970772  0.952006  1.000000

            carat       cut  color  ...         x         y         z
carat    1.000000       NaN    NaN  ...  0.975094  0.951722  0.953387
cut           NaN  1.000000    NaN  ...       NaN       NaN       NaN
color         NaN       NaN    1.0  ...       NaN       NaN       NaN
clarity  0.352841       NaN    NaN  ...  0.371999  0.358420  0.366952
depth         NaN       NaN    NaN  ...       NaN       NaN       NaN
table         NaN  0.433405    NaN  ...       NaN       NaN       NaN
price    0.921591       NaN    NaN  ...  0.884435  0.865421  0.861249
x        0.975094       NaN    NaN  ...  1.000000  0.974701  0.970772
y        0.951722       NaN    NaN  ...  0.974701  1.000000  0.952006
z        0.953387       NaN    NaN  ...  0.970772  0.952006  1.000000

With the mask applied, we'll only see the correlations with absolute values greater than 0.3.

Introduction to Correlation and Its Importance

Computing the Correlation Matrix

Join the 1M+ learners on CodeSignal

Masking Values in the Correlation Matrix

Visualizing the Correlation Matrix with a Heatmap

Lesson Summary and Practice