Optimization for Multivariable Functions in SciPy

Welcome back to our course on optimization with SciPy. In previous lessons, you learned about defining functions in Python and explored the basics of optimization using SciPy. Today, we’ll dive into optimization for multivariable functions, an essential skill in many fields such as machine learning, engineering, and economics.

Understanding optimization and its real-world applications will help you solve complex problems more efficiently. Multivariable optimization deals with functions that have more than one input variable. We'll use practical examples to guide you through this concept.

An objective function is central to optimization. It's the function we aim to minimize or maximize. In multivariable optimization, the objective function may depend on several input variables.

Let's create a simple objective function. We'll use the function $f(x, y) = x^2 + y^2 + 4\sin(x) + 4\cos(y)$. This function takes two input variables, $x$ and $y$, and involves quadratic and trigonometric terms.

import numpy as np

def objective_function(x):
    return x[0]**2 + x[1]**2 + 4*np.sin(x[0]) + 4*np.cos(x[1])

# Example input for x
example_x = [0, 0]
print("Objective function value for example x:", objective_function(example_x))  # 4.0

• objective_function is defined to take a list x containing the two variables, $x_0$ and $x_1$.
• $x_0$ represents $x$, and $x_1$ represents $y$.
• It calculates the sum of squares, sine, and cosine terms accordingly.

The initial guess is crucial in optimization as it influences both the convergence speed and the solution found. Let's set an initial guess for our problem.

initial_guess = [1, 1]

• Here, [1, 1] is our starting point for the variables $x$ and $y$.
• An initial guess can be based on prior knowledge or can be a simple starting point like the vector of ones we used here.

Now let's move on to using the minimize function from SciPy to find the minimum of our objective function.

from scipy.optimize import minimize
import numpy as np

def objective_function(x):
    return x[0]**2 + x[1]**2 + 4*np.sin(x[0]) + 4*np.cos(x[1])

initial_guess = [1, 1]
result = minimize(objective_function, initial_guess)

print("Optimal [x, y]:", result.x)
print("Function value at minimum:", result.fun)
print("Number of iterations:", result.nit)
print("Successful:", result.success)

Output:

Optimal [x, y]: [-1.02986652  1.89549428]
Function value at minimum: -0.051487585323559903
Number of iterations: 6
Successful: True

Explanation:

• minimize is a versatile function for optimization tasks.
• It takes the objective_function and initial_guess as inputs to start the optimization process.
• The printed results give the optimal values for $x$ and $y$, the minimum value of the objective function at the optimal solution, the number of iterations it took to reach the solution, and whether the optimization was successful.