Welcome to our next course, Deep Dive into Numpy and Pandas with Housing Data. In this course, you will unlock the secrets of efficient data manipulation and analysis with Numpy and Pandas. We will build your skills from a foundational to an advanced level, strengthening your grasp of Python and preparing you for the world of Data Science.

In this first lesson, we will study the California Housing dataset. This important dataset is often used as a benchmark in machine learning and data analysis. It contains detailed information about housing values in California suburbs. The California housing market, due to its high prices and shortages, has been the subject of study for many years. This makes the dataset particularly relevant today. In this lesson, our main objective is to explore the fundamental attributes of this dataset. We aim to understand various attributes such as median income, population, average number of rooms per household, and their influence on house prices. Let's get started!

To load the California Housing dataset, we can use the sklearn library, which is powerful, easy to use, and contains many ready-to-use machine learning algorithms. It also comes with a few pre-loaded datasets, including the California Housing dataset. We can load the dataset by simply importing the appropriate sklearn module and calling a function.

from sklearn.datasets import fetch_california_housing

dataset = fetch_california_housing()

After loading the California Housing dataset, we receive the data in a Bunch object; it's similar to a dictionary but with added functionalities. It has keys like data, target, feature_names, each leading to a different part of the dataset. The data key contains all the input features, the target key has the output values we might want to predict (median house values), and feature_names holds the names of the features.

A pandas DataFrame is more familiar and offers more functionality, making it easier to work with. Let's convert our dataset into a pandas DataFrame.

import pandas as pd

df = pd.DataFrame(data=dataset.data, columns=dataset.feature_names)
df["MedHouseValue"] = dataset.target
print(df.head())
"""
   MedInc  HouseAge  AveRooms  ...  Latitude  Longitude  MedHouseValue
0  8.3252      41.0  6.984127  ...     37.88    -122.23          4.526
1  8.3014      21.0  6.238137  ...     37.86    -122.22          3.585
2  7.2574      52.0  8.288136  ...     37.85    -122.24          3.521
3  5.6431      52.0  5.817352  ...     37.85    -122.25          3.413
4  3.8462      52.0  6.281853  ...     37.85    -122.25          3.422

[5 rows x 9 columns]
"""

This code converts the different parts of the Bunch object into a Pandas DataFrame. We also add the target values to the dataframe as a new column, MedHouseValue. Now, we have successfully imported our dataset and already explored the first few rows using the head() function!