Welcome to our lesson on basic derivatives! In machine learning, understanding how things change is crucial. Derivatives help measure these changes, which is key for optimizing models. By the end, you'll understand derivatives, know how to calculate them, and see a practical example using Python.

Let's understand the concept of a derivative. Imagine you're driving a car. Your speed indicates how your position changes over time. Similarly, a derivative measures how a function's output changes as its input changes. It's like the speed for functions. The derivative is calculated for every input of the original function. So, the derivative is also a function of the same input.

Let's try to approximate the derivative of the function $f(x) = x^2$ at $x=2$. In other words, we want to see the speed of the function's growth (or decay) at this point. Using a small change $\Delta x$, we will find how much function changed when the changed:

We care about the derivative at point $x=2$, but what we really calculate is the speed of the function on the $[2, 3]$ interval. Let's improve the accuracy of our approximation by choosing a smaller $\Delta x$. Here is a new graph:

Here, the $\Delta x=0.5$. The derivative approximation is , which is equal to . This time, we calculated the function's growth speed on the interval, which gives us a better approximation quality. By shrinking this interval (in other words, by making lower), we can improve the approximation quality.

Let's demonstrate this with code:

In this code, we make delta_x smaller and smaller and see how the approximation for a derivative changes.. For each delta_x, the approximate derivative is calculated using the difference quotient formula. The result is printed with an f-string, including the current delta_x, and a calculated approximate derivative for it. Here is the output:

As you can see, the value gets closer and closer to 4.

To get an exact derivative, make $\Delta x$ as small as possible. When $\Delta x$ is closer to 0, the approximation becomes more accurate. So, to find the exact value of the derivative, find a limit of our expression when $\Delta x$ approaches 0.

Let's see how to calculate it using Python. We'll use the forward difference method to estimate the derivative numerically.

Here’s the function:

In this code:

• derivative takes a function f, a point x, and a small step h.
• A small h (1e-5) makes the approximation accurate.
• We define f(x) = x^2 using a lambda function.
• We compute and print the derivative of f at x = 3.

To summarize, today we:

• Learned what derivatives are and how they measure change.
• Used pictures and Python code to visualize and approximate derivatives.
• Calculated derivatives using numerical methods.

Next, you'll move to practice, where you'll calculate derivatives for different functions using Python. Happy coding!