Welcome to our lesson on basic derivatives! In machine learning, understanding how things change is crucial. Derivatives help measure these changes, which is key for optimizing models. By the end, you'll understand derivatives, know how to calculate them, and see a practical example using Python.
What is a Derivative?
Let's understand the concept of a derivative. Imagine you're driving a car. Your speed indicates how your position changes over time. Similarly, a derivative measures how a function's output changes as its input changes. It's like the speed for functions. The derivative is calculated for every input of the original function. So, the derivative is also a function of the same input.
Calculating an Approximation for a Derivative: part 1
Calculating an Approximation for a Derivative: part 2
Calculating an Approximation for a Derivative: part 3
Let's demonstrate this with code:
# Define the function f(x) = x^2def f(x): return x**2delta_x_values = [1, 0.5, 0.1, 0.01, 0.001]x = 2for delta_x in delta_x_values: approx_derivative = (f(x + delta_x) - f(x)) / delta_x print(f'Approx. derivative with delta_x = {delta_x}: {approx_derivative}')
In this code, we make delta_x smaller and smaller and see how the approximation for a derivative changes.. For each delta_x, the approximate derivative is calculated using the difference quotient formula. The result is printed with an f-string, including the current delta_x, and a calculated approximate derivative for it. Here is the output:
Approx. derivative with delta_x = 1: 5.0Approx. derivative with delta_x = 0.5: 4.5Approx. derivative with delta_x = 0.1: 4.100000000000001Approx. derivative with delta_x = 0.01: 4.009999999999891Approx. derivative with delta_x = 0.001: 4.000999999999699
As you can see, the value gets closer and closer to 4.
Limiting to 0
Implementing a Derivative in Python
Let's see how to calculate it using Python. We'll use the forward difference method to estimate the derivative numerically.
Here’s the function:
# Numerical derivative using forward differencedef derivative(f, x, h=1e-5): return (f(x + h) - f(x)) / h# Sample function: f(x) = x^2f = lambda x: x**2# Compute derivative of f at x=3print("Derivative of f(x) at x=3:", derivative(f, 3)) # Derivative of f(x) at x=3: 6.000009999951316
In this code:
derivative takes a function f, a point x, and a small step h.
A small h (1e-5) makes the approximation accurate.
We define f(x) = x^2 using a lambda function.
We compute and print the derivative of f at x = 3.
Lesson Summary
To summarize, today we:
Learned what derivatives are and how they measure change.
Used pictures and Python code to visualize and approximate derivatives.
Calculated derivatives using numerical methods.
Next, you'll move to practice, where you'll calculate derivatives for different functions using Python. Happy coding!
Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal
Let's try to approximate the derivative of the function f(x)=x2 at x=2. In other words, we want to see the speed of the function's growth (or decay) at this point. Using a small change Δx, we will find how much function changed when the x changed:
Δxf(x+Δx)−f(x)
Here’s a graph to illustrate this:
In the picture, the green dashed line shows our slope approximation using Δx=1. We see, that the function changed from 4 to 9, while x changed from 2 to 3. We can make a conclusion that the speed of the function at x=2 is roughly 19−4, which equal to 5.
We care about the derivative at point x=2, but what we really calculate is the speed of the function on the [2,3] interval. Let's improve the accuracy of our approximation by choosing a smaller Δx. Here is a new graph:
Here, the Δx=0.5. The derivative approximation is 0.5f(2.5)−f(2), which is equal to 4.5. This time, we calculated the function's growth speed on the [2,2.5] interval, which gives us a better approximation quality. By shrinking this interval (in other words, by making Δx lower), we can improve the approximation quality.
For example, making the same calculations on the [2,2.00001] interval will give you a value, which is very close to the actual derivative at point x=2. However, it won't be exact.
Python
# Define the function f(x) = x^2def f(x): return x**2delta_x_values = [1, 0.5, 0.1, 0.01, 0.001]x = 2for delta_x in delta_x_values: approx_derivative = (f(x + delta_x) - f(x)) / delta_x print(f'Approx. derivative with delta_x = {delta_x}: {approx_derivative}')
Approx. derivative with delta_x = 1: 5.0Approx. derivative with delta_x = 0.5: 4.5Approx. derivative with delta_x = 0.1: 4.100000000000001Approx. derivative with delta_x = 0.01: 4.009999999999891Approx. derivative with delta_x = 0.001: 4.000999999999699
To get an exact derivative, make Δx as small as possible. When Δx is closer to 0, the approximation becomes more accurate. So, to find the exact value of the derivative, find a limit of our expression when Δx approaches 0.
h→0limhf(x+h)−f(x)
Evaluating this will give us an exact derivative function, which can be used to calculate the derivative at any given input.
However, this course aims to give you an understanding of the idea behind derivative, not to teach how to calculate it. So, to calculate a derivative, we will use numerical methods, which give us a good approximation and doesn't require heavy math.
Python
# Numerical derivative using forward differencedef derivative(f, x, h=1e-5): return (f(x + h) - f(x)) / h# Sample function: f(x) = x^2f = lambda x: x**2# Compute derivative of f at x=3print("Derivative of f(x) at x=3:", derivative(f, 3)) # Derivative of f(x) at x=3: 6.000009999951316
