Deep Dive into Conditional Selection

Hello! Today we're delving deeper into Conditional Selection. As you might remember, it's a technique for selecting data in a data frame that meets given conditions. It's a key tool for data analysis as it allows us to focus on the most pertinent information.

In today's lesson, we're going to look at more complex conditional selection scenarios and learn about an important method, where(). Our journey will start with a short refresher on conditional selection, move on to more sophisticated compound conditions, and finally, we'll dive into the where() method. Let's get started!

Before we venture into uncharted territory, let's refresh our memory on conditional selection. Essentially, with conditional selection, we're requesting Python to sift through our data and return elements that meet our stipulations. We do this by comparing columns or rows of our data frame against certain conditions.

For instance, we have a pandas data frame scores_df consisting of a list of students' names and their test scores.

import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie', 'Dave'], 'Score': [88, 92, 95, 80]}
scores_df = pd.DataFrame(data)

print(scores_df)
#      Name  Score
# 0   Alice     88
# 1     Bob     92
# 2 Charlie     95
# 3    Dave     80

Let's find out who scored more than 90:

print(scores_df[scores_df['Score'] > 90])
#      Name  Score
# 1     Bob     92
# 2 Charlie     95

By using 'Score' > 90, we've created a mask and used it to filter rows that resolve to True. Pretty cool, right?

In real-world scenarios, it might be necessary to select data based on more than one condition. In these cases, we would need to deploy compound conditions.

Here we introduce two operators — & (and) and | (or). & insists that all conditions must be true, and | requires any condition to be true.

Interestingly, we can negate a condition using ~ (not).

Make sure to place your conditions in parentheses when using & (and) or | (or). This is required in Python to ensure that the conditions are evaluated before the conjunction is done.

Consider this example:

print(scores_df[(scores_df['Score'] > 85) & (scores_df['Name'].str.startswith('A'))])
#     Name  Score
# 0  Alice     88

And there's Alice! She scored more than 85 and her name starts with an 'A'.

Now, what if we want all students except for 'Bob'. Simply employ ~:

print(scores_df[~(scores_df['Name'] == 'Bob')])
#      Name  Score
# 0   Alice     88
# 2 Charlie     95
# 3    Dave     80

Adios, Bob!