Data Aggregation

In this lesson, we'll explore data aggregation, a powerful tool in data analysis. Aggregation helps you summarize and simplify large sets of data to gain insights quickly. By the end, you'll know how to use data aggregation techniques to find specific information about groups in your dataset.

Data aggregation involves combining, summarizing, or consolidating data points into a single representation. Imagine you have a large set of student test scores. Instead of looking at every individual score, you might want to know the average score for each class. This simplifies your data and helps you see the bigger picture.

Common functions used in data aggregation include:

• Maximum (max): Finds the highest value in a group.
• Mean (mean): Calculates the average value of a group.
• Sum (sum): Calculates the total sum of a group.
• Standard Deviation (std): Calculates the dispersion or spread of a group.

Let's start with a simple example:

scores = [89, 76, 92, 54, 88]
print("Maximum score:", max(scores))  # Maximum score: 92
print("Average score:", sum(scores) / len(scores))  # Average score: 79.8

Here, max gives us the highest score, and calculating the average (mean) summarizes the test scores.

Pandas offers an easy way to perform data aggregation using groupby and agg methods. Here’s how you can use them:

• groupby: Splits the data into groups based on some criteria.
• agg: Applies one or more aggregation functions to these groups.

Let’s see this in action.

We'll use an example dataset containing information about different products sold in various stores. Here's a small sample:

import pandas as pd

data = {
    'store': ['Store A', 'Store A', 'Store B', 'Store B', 'Store C'],
    'product': ['Apples', 'Bananas', 'Apples', 'Bananas', 'Apples'],
    'units_sold': [30, 50, 40, 35, 90],
    'price': [1.20, 0.50, 1.00, 0.50, 1.30]
}

df = pd.DataFrame(data)
print(df)

     store  product  units_sold  price
0  Store A   Apples          30   1.20
1  Store A  Bananas          50   0.50
2  Store B   Apples          40   1.00
3  Store B  Bananas          35   0.50
4  Store C   Apples          90   1.30