Introduction to Correlation and Its Importance

Hello! In today's lesson, we will dive into the concept of correlation and focus specifically on highlighting certain correlation values within the diamonds dataset.

Correlation is a statistical measure that describes the extent to which two variables change together. Understanding correlations is crucial in data analysis as it helps us identify relationships between different variables.

For example:

  • Positive Correlation: As one variable increases, the other also increases (e.g., height and weight).
  • Negative Correlation: As one variable increases, the other decreases (e.g., speed and travel time).

By the end of this lesson, you will be able to compute, mask, and visually represent these correlations to get a clearer picture of the underlying data relationships.

Computing the Correlation Matrix

Let's compute the correlation matrix for our prepared diamonds dataset. As mentioned before, the correlation matrix is a table showing correlation coefficients between many variables. Each cell in the table shows the correlation between two variables.

You might be familiar with the process by now, but here's how to compute and display the correlation matrix using pandas:

Python
import seaborn as sns
import pandas as pd

# Load the diamonds dataset
diamonds = sns.load_dataset('diamonds')

diamonds['cut'] = diamonds['cut'].astype('category').cat.codes
diamonds['color'] = diamonds['color'].astype('category').cat.codes
diamonds['clarity'] = diamonds['clarity'].astype('category').cat.codes

# Calculate the correlation matrix
correlation_matrix = diamonds.corr()

# Display the correlation matrix
print(correlation_matrix)

Output:

            carat       cut     color   clarity     depth     table     price  \
carat    1.000000  0.134967  0.291437  0.352841  0.028224  0.181618  0.921591   
cut      0.134967  1.000000  0.020519  0.189175  0.218055  0.433405  0.053491   
color    0.291437  0.020519  1.000000 -0.025631  0.047279  0.026465  0.172511   
clarity  0.352841  0.189175 -0.025631  1.000000  0.067384  0.160327  0.146800   
depth    0.028224  0.218055  0.047279  0.067384  1.000000 -0.295779 -0.010647   
table    0.181618  0.433405  0.026465  0.160327 -0.295779  1.000000  0.127134   
price    0.921591  0.053491  0.172511  0.146800 -0.010647  0.127134  1.000000   
x        0.975094  0.125565  0.270287  0.371999 -0.025289  0.195344  0.884435   
y        0.951722  0.121462  0.263584  0.358420 -0.029341  0.183760  0.865421   
z        0.953387  0.149323  0.268227  0.366952  0.094924  0.150929  0.861249   

                x         y         z  
carat    0.975094  0.951722  0.953387  
cut      0.125565  0.121462  0.149323  
color    0.270287  0.263584  0.268227  
clarity  0.371999  0.358420  0.366952  
depth   -0.025289 -0.029341  0.094924  
table    0.195344  0.183760  0.150929  
price    0.884435  0.865421  0.861249  
x        1.000000  0.974701  0.970772  
y        0.974701  1.000000  0.952006  
z        0.970772  0.952006  1.000000  

In the matrix, correlation coefficients range from -1 to 1. Values close to 1 imply a strong positive correlation, while values close to -1 imply a strong negative correlation. Values near 0 imply little to no correlation.

Sign up
Join the 1M+ learners on CodeSignal
Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal