Basic Derivatives

Basic Derivatives | CodeSignal Learn

Lesson Introduction

Welcome to our lesson on basic derivatives! In machine learning, understanding how things change is crucial. Derivatives help measure these changes, which is key for optimizing models. By the end, you'll understand derivatives, know how to calculate them, and see a practical example using Python.

What is a Derivative?

Let's understand the concept of a derivative. Imagine you're driving a car. Your speed indicates how your position changes over time. Similarly, a derivative measures how a function's output changes as its input changes. It's like the speed for functions. The derivative is calculated for every input of the original function. So, the derivative is also a function of the same input.

Calculating an Approximation for a Derivative: part 1

f(x) = x^2

Calculating an Approximation for a Derivative: part 2

x=2

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Let's try to approximate the derivative of the function $f(x) = x^2$ at $x=2$ . In other words, we want to see the speed of the function's growth (or decay) at this point. Using a small change $\Delta x$ , we will find how much function changed when the x changed:

\frac{{f(x + \Delta x) - f(x)}}{\Delta x}

Here’s a graph to illustrate this:

In the picture, the green dashed line shows our slope approximation using $\Delta x = 1$ . We see, that the function changed from 4 to 9, while x changed from 2 to 3. We can make a conclusion that the speed of the function at $x=2$ is roughly $\frac{9-4}{1}$ , which equal to 5.

We care about the derivative at point $x=2$ , but what we really calculate is the speed of the function on the $[2, 3]$ interval. Let's improve the accuracy of our approximation by choosing a smaller $\Delta x$ . Here is a new graph:

Here, the $\Delta x=0.5$ . The derivative approximation is $\frac{f(2.5) - f(2)}{0.5}$ , which is equal to 4.5. This time, we calculated the function's growth speed on the $[2, 2.5]$ interval, which gives us a better approximation quality. By shrinking this interval (in other words, by making $\Delta x$ lower), we can improve the approximation quality.

For example, making the same calculations on the $[2, 2.00001]$ interval will give you a value, which is very close to the actual derivative at point $x=2$ . However, it won't be exact.

Calculating an Approximation for a Derivative: part 3

Let's demonstrate this with code: Python# Define the function f(x) = x^2 def f(x): return x**2 delta_x_values = [1, 0.5, 0.1, 0.01, 0.001] x = 2 for delta_x in delta_x_values: approx_derivative = (f(x + delta_x) - f(x)) / delta_x print(f'Approx. derivative with delta_x = {delta_x}: {approx_derivative}')# Define the function f(x) = x^2 def f(x): return x**2 delta_x_values = [1, 0.5, 0.1, 0.01, 0.001] x = 2 for delta_x in delta_x_values: approx_derivative = (f(x + delta_x) - f(x)) / delta_x print(f'Approx. derivative with delta_x = {delta_x}: {approx_derivative}') In this code, we make delta_x smaller and smaller and see how the approximation for a derivative changes.. For each delta_x, the approximate derivative is calculated using the difference quotient formula. The result is printed with an f-string, including the current delta_x, and a calculated approximate derivative for it. Here is the output: Approx. derivative with delta_x = 1: 5.0 Approx. derivative with delta_x = 0.5: 4.5 Approx. derivative with delta_x = 0.1: 4.100000000000001 Approx. derivative with delta_x = 0.01: 4.009999999999891 Approx. derivative with delta_x = 0.001: 4.000999999999699Approx. derivative with delta_x = 1: 5.0 Approx. derivative with delta_x = 0.5: 4.5 Approx. derivative with delta_x = 0.1: 4.100000000000001 Approx. derivative with delta_x = 0.01: 4.009999999999891 Approx. derivative with delta_x = 0.001: 4.000999999999699 As you can see, the value gets closer and closer to 4.

Implementing a Derivative in Python

Let's see how to calculate it using Python. We'll use the forward difference method to estimate the derivative numerically. Here’s the function: Python # Numerical derivative using forward difference def derivative(f, x, h=1e-5): return (f(x + h) - f(x)) / h # Sample function: f(x) = x^2 f = lambda x: x**2 # Compute derivative of f at x=3 print("Derivative of f(x) at x=3:", derivative(f, 3)) # Derivative of f(x) at x=3: 6.000009999951316 # Numerical derivative using forward difference def derivative(f, x, h=1e-5): return (f(x + h) - f(x)) / h # Sample function: f(x) = x^2 f = lambda x: x**2 # Compute derivative of f at x=3 print("Derivative of f(x) at x=3:", derivative(f, 3)) # Derivative of f(x) at x=3: 6.000009999951316 In this code: derivative takes a function f, a point x, and a small step h . A small h (1e-5) makes the approximation accurate. We define f(x) = x^2 using a lambda function. We compute and print the derivative of f at x = 3 .

To get an exact derivative, make $\Delta x$ as small as possible. When $\Delta x$ is closer to 0, the approximation becomes more accurate. So, to find the exact value of the derivative, find a limit of our expression when $\Delta x$ approaches 0.

\lim_{{h \to 0}} \frac{{f(x + h) - f(x)}}{h}

Evaluating this will give us an exact derivative function, which can be used to calculate the derivative at any given input.

However, this course aims to give you an understanding of the idea behind derivative, not to teach how to calculate it. So, to calculate a derivative, we will use numerical methods, which give us a good approximation and doesn't require heavy math.