Optimization with Constraints Using SciPy

Welcome to the final lesson of the course: Optimization with Constraints. Building on your knowledge of function optimization using SciPy, we're now going to tackle optimization challenges that involve constraints, which are crucial for many real-world applications. Consider scenarios where resource limitations must be honored — this is where constrained optimization shines. By the end of this lesson, you'll be equipped to solve such problems using SciPy, enhancing your ability to model and solve complex problems.

Before we dive into the code, let's revisit the concept of constraints. Constraints in optimization are conditions that the solution must satisfy. These can be of two types: inequality constraints, which specify that a certain expression should be greater than or equal to zero, and equality constraints, which specify that a certain expression should be exactly zero.

Let's use the following constraint for an example:

$x + y <= 1$

This constraint means that we are limited to the region under the $y = 1 - x$ line:

We will search for a solution only in the grey region, including the black line itself.

In SciPy, we define constraints using a dictionary format. Here's how you define an inequality constraint where the sum of variables x[0] and x[1] is less than or equal to 1:

# Inequality constraint: x[0] + x[1] <= 1
constraint = {
    'type': 'ineq',  # 'ineq' for inequality
    'fun': lambda x: 1 - (x[0] + x[1])
}

In this constraint dictionary, 'type' signifies the kind of constraint ('ineq' for inequality), while 'fun' provides the function definition for the constraint. The lambda function returns a positive value when x[0] + x[1] is less than or equal to 1, which ensures the constraint is met.

Now that we understand constraints, let's formulate and solve a constrained optimization problem using SciPy. Below is a complete code snippet that includes defining the objective function, setting an initial guess, defining constraints, and solving the problem using SciPy's minimize function.

import numpy as np
from scipy.optimize import minimize

# 1. Objective Function
def f(x):
    return 2*x[0]**2 - 5*x[0] + 3*x[1]**2

# 2. Initial Guess
initial_guess = [1, 1]

# 3. Define Constraints
constraint = {
    'type': 'ineq',
    'fun': lambda x: 1 - (x[0] + x[1])
}

# 4. Solving with SciPy's `minimize`
result_constraints = minimize(f, initial_guess, constraints=[constraint])

# Output the result
print("Optimal solution:", result_constraints.x)  # [ 1.1 -0.1]
print("Objective function value at optimal solution:", result_constraints.fun)  # -3.05

Explanation:

1. Objective Function: The function f(x) is defined to be minimized. It is a quadratic function of two variables.

2. Initial Guess: We start the optimization process with an initial guess of [1, 1].

3. Define Constraints: The constraint x[0] + x[1] <= 1 is defined using a dictionary format, specifying it as an inequality constraint.

4. Solving with SciPy's minimize: The minimize function is used to find the values of x that minimize the objective function while satisfying the constraints. The result is stored in result_constraints, and the optimal solution and its corresponding function value are printed.