Welcome to the first lesson of the course, where we will lay the foundation for working with PredictHealth's insurance dataset. In this lesson, you will learn how to load, explore, and perform basic manipulations on a real-world dataset using Python. These are essential first steps in any data analysis project, as they help you understand the data you are working with and prepare it for deeper analysis or modeling.

By the end of this lesson, you will be able to confidently load a dataset, inspect its structure, check for data quality issues, and perform simple filtering operations. These skills are crucial for anyone interested in data science, analytics, or working with health insurance data.

To begin, we need to use some popular Python libraries that make data analysis easier and more efficient. The main libraries we will use for this course are pandas for data manipulation, numpy for numerical operations, and matplotlib.pyplot and seaborn for data visualization. In most environments, you would need to install these libraries using commands like pip install pandas, but on CodeSignal, these libraries are already installed and ready to use. This allows you to focus on learning and practicing without worrying about setup.

Here is how you import these libraries and load the PredictHealth insurance dataset:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

insurance_data = pd.read_csv('datasets/insurance.csv')

This code imports the necessary libraries and loads the dataset from a CSV file named insurance.csv into a pandas DataFrame called insurance_data. The DataFrame is a powerful data structure that allows you to easily explore and manipulate tabular data.

Once the dataset is loaded, it is important to take a first look at its contents and structure. This helps you get familiar with the data and spot any immediate issues or interesting patterns. You can use the .head() method to display the first few rows of the dataset, which gives you a quick overview of what the data looks like. By default, .head() shows 5 rows, but you can specify a different number by passing it as an argument (e.g., .head(10) to view the first 10 rows).

print("First 5 rows of the dataset:")
print(insurance_data.head())

# To view more rows if needed
print("\nFirst 10 rows of the dataset:")
print(insurance_data.head(10))

The output for the first 5 rows might look like this:

First 5 rows of the dataset:
   age     sex     bmi  children smoker     region      charges
0   19  female  27.900         0    yes  southwest  16884.9240
1   18    male  33.770         1     no  southeast   1725.5523
2   28    male  33.000         3     no  southeast   4449.4620
3   33    male  22.705         0     no  northwest  21984.4706
4   32    male  28.880         0     no  northwest   3866.8552

To get more detailed information about the dataset, such as the number of rows and columns, column names, and data types, you can use the .info() method:

print("\nDataset Information:")
print(insurance_data.info())

This will output something like:

Dataset Information:
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1338 entries, 0 to 1337
Data columns (total 7 columns):
 #   Column    Non-Null Count  Dtype  
---  ------    --------------  -----  
 0   age       1338 non-null   int64  
 1   sex       1338 non-null   object 
 2   bmi       1338 non-null   float64
 3   children  1338 non-null   int64  
 4   smoker    1338 non-null   object 
 5   region    1338 non-null   object 
 6   charges   1338 non-null   float64
dtypes: float64(2), int64(2), object(3)
memory usage: 73.3+ KB
None

This information helps you understand the size and structure of your data, which is important before moving on to analysis.

After understanding the structure, it is helpful to look at a statistical summary of the numerical columns in the dataset. The .describe() method provides useful statistics such as mean, standard deviation, minimum, and maximum values for each numerical column.

print("\nStatistical Summary:")
print(insurance_data.describe())

The output will look like this:

Statistical Summary:
       age          bmi     children      charges
count  1338.000000  1338.000000  1338.000000   1338.000000
mean     39.207025    30.663397     1.094918  13270.422265
std      14.049960     6.098187     1.205493  12110.011237
min      18.000000    15.960000     0.000000   1121.873900
25%      27.000000    26.296250     0.000000   4740.287150
50%      39.000000    30.400000     1.000000   9382.033000
75%      51.000000    34.693750     2.000000  16639.912515
max      64.000000    53.130000     5.000000  63770.428010

This summary gives you a sense of the distribution and range of values in your data.

It is also important to check for missing values, as they can affect your analysis. You can do this using the .isnull().sum() method:

print("\nMissing Values:")
print(insurance_data.isnull().sum())

If there are no missing values, the output will be:

Missing Values:
age         0
sex         0
bmi         0
children    0
smoker      0
region      0
charges     0
dtype: int64

Finally, to confirm the data types of each column, you can use the .dtypes attribute:

print("\nData Types:")
print(insurance_data.dtypes)

This will show you which columns are numerical and which are categorical, which is important for later analysis.

Data Types:
age           int64
sex          object
bmi         float64
children      int64
smoker       object
region       object
charges     float64
dtype: object

Now that you have explored the dataset, let's perform a simple data manipulation task. Suppose you want to analyze only the records of people who smoke. You can filter the dataset using a condition on the smoker column. This is a common step in data analysis when you want to focus on a specific group.

Here is how you can filter the dataset to include only smokers:

smokers = insurance_data[insurance_data['smoker'] == 'yes']
print("\nFirst 5 rows of smokers data:")
print(smokers.head())

The output will look like this:

First 5 rows of smokers data:
   age     sex     bmi  children smoker     region      charges
0   19  female  27.900         0    yes  southwest  16884.9240
11  19    male  27.900         0    yes  southwest  1837.2370
23  22    male  30.400         0    yes  southeast  35585.5760
30  31    male  25.740         0    yes  southeast  38711.0000
34  34    male  31.920         1    yes  northeast  37701.8768

Filtering data like this is critical for targeted analysis. For example, you might want to compare the insurance charges of smokers versus non-smokers or study the health risks associated with smoking. Being able to select specific groups from your data is a key skill in data analysis.

In this lesson, you learned how to import essential Python libraries, load the PredictHealth insurance dataset, and explore its structure and quality. You also learned how to generate statistical summaries, check for missing values, and filter the data to focus on specific groups, such as smokers. These are the foundational steps in any data analysis project and will help you build confidence as you move forward.

Next, you will have the opportunity to practice these skills with hands-on exercises. These exercises will reinforce what you have learned and prepare you for more advanced topics, such as data visualization and regression analysis. Take your time to review the code examples and outputs from this lesson, and get ready to apply these techniques on your own. If you have any questions or need to revisit a concept, feel free to refer back to this lesson as you practice.