Descriptive Statistics and Probability Distributions with SciPy

Welcome to the first lesson in our "Statistics with SciPy" course. In this lesson, you will gain a foundational understanding of descriptive statistics and probability distributions and learn how to use SciPy, a powerful Python library, to work with these concepts.

Descriptive statistics summarize and describe the main features of a dataset, helping to simplify and present data in an informative way. Probability distributions, on the other hand, describe how the values of a random variable are distributed. Together, these concepts are pivotal for data analysis and decision-making processes. SciPy enables you to perform these analyses with ease, offering tools to perform various statistical operations.

Let's delve into the core descriptive statistics: mean, median, mode, and standard deviation. We'll use a sample dataset to illustrate these concepts.

First, we need to understand what these terms mean:

• Mean is the average of a dataset.
• Median is the middle value when the data is sorted.
• Mode is the most frequently occurring value.
• Standard Deviation measures the amount of variation or dispersion in a set of values.

To calculate these, we'll use SciPy. Let's start with defining our sample dataset.

from scipy import stats
import numpy as np

# Sample data
data = np.array([1, 2, 2, 3, 4, 4, 4, 5, 6, 6])

In this code snippet, we're using NumPy to create an array named data that holds our sample dataset, which we will analyze using SciPy.

Now, let's calculate the mean, median, standard deviation, and mode using SciPy functions:

# Calculate mean, median, and standard deviation using SciPy
mean_value = stats.tmean(data)
median_value = np.percentile(data, 50)  # Note: For finite datasets, the median is the 50th percentile.
std_deviation = stats.tstd(data)

print(f"Mean: {mean_value}, Median: {median_value}, Standard Deviation: {std_deviation}")

• stats.tmean(data) calculates the average of the dataset.
• np.percentile(data, 50) finds the 50th percentile, which is equivalent to the median for finite datasets.
• stats.tstd(data) computes how far elements of the dataset are spread from the average.

Output:

Mean: 3.7, Median: 4.0, Standard Deviation: 1.699673171197595

Next, let's find the mode using SciPy:

# Calculate mode
mode_value = stats.mode(data).mode

print(f"Mode: {mode_value}")  # 4

stats.mode(data) returns an object containing two arrays: the mode values and their respective counts. To extract the most frequently occurring value, we use .mode on the result. If two or more items have the same maximal frequency, the smallest of those is returned. In this case, the mode of the dataset is 4, which appears the most frequently.