Building Multi Step Workflows

Welcome back! In the last lesson, you learned how to build and deploy simple serverless APIs using Google Cloud Functions and API Gateway. You saw how a single Cloud Function can handle a request and return a response.

However, many real-world applications require more than just a single step. For example, when processing an online order, you might need to:

• Validate the order details
• Charge the customer
• Record the order in a database

Each of these steps could be handled by a separate Cloud Function. But how do you make sure they run in the right order and that data flows smoothly from one step to the next? This is where Google Cloud Workflows come in. Workflows let you organize and manage multi-step processes in your serverless applications.

Before we dive into Workflows, let's quickly remind ourselves how Cloud Functions fit into serverless applications.

A Cloud Function is a small piece of code that runs in response to an event, such as an API call or a message. In the previous lesson, you learned how to:

• Write a Cloud Function that processes input and returns output
• Connect a Cloud Function to an API Gateway endpoint

In this lesson, we will see how to connect several Cloud Functions together to handle a more complex process.

Google Cloud Workflows help you coordinate multiple Cloud Functions (or other Google Cloud services) into a single workflow.

A workflow is like a flowchart: it defines a series of steps and the order in which they run. Each step can perform a task, make a choice, or wait for something to happen.

Let's use a real-world example: processing an online order. Imagine you have three steps:

1. Validate the order (check if the amount is valid)
2. Charge the customer (simulate payment)
3. Record the order (save it to a database)

With Workflows, you can define a process that runs these steps one after another, passing the result from one step to the next.

Let's build up the order processing workflow step by step. We will use three Cloud Functions: one for each step in the process.

First, we need a Cloud Function to check if the order amount is valid.

# main.py (validate_order function)
def validate_order(request):
    request_json = request.get_json()
    order = request_json["order"]
    if float(order["amount"]) <= 0:
        return {"error": "Invalid amount"}, 400
    order["validated"] = True
    return {"order": order}

• This function receives a request with an order object.
• It checks if the amount is greater than 0. If not, it returns an error.
• If the amount is valid, it adds a "validated": True field to the order and returns it.

Example input:

{"order": {"order_id": "test-123", "amount": "99.99"}}

Example output:

{"order": {"order_id": "test-123", "amount": "99.99", "validated": true}}

Next, we need a Cloud Function to simulate charging the customer.

# main.py (charge_customer function)
def charge_customer(request):
    request_json = request.get_json()
    order = request_json["order"]
    request_json["payment_id"] = f"pay_{order['order_id']}"
    request_json["charged"] = True
    return request_json

• This function takes the output from the previous step.
• It creates a payment_id based on the order ID and adds a charged flag.
• It returns the updated event.

Example input:

{"order": {"order_id": "test-123", "amount": "99.99", "validated": true}}

Example output:

{"order": {"order_id": "test-123", "amount": "99.99", "validated": true}, "payment_id": "pay_test-123", "charged": true}

Finally, we need a Cloud Function to save the order to a Firestore collection.

# main.py (record_order function)
import time
from google.cloud import firestore

def record_order(request):
    db = firestore.Client()
    request_json = request.get_json()
    order = request_json["order"]
    item = {
        "order_id": order["order_id"],
        "amount": float(order["amount"]),
        "payment_id": request_json["payment_id"],
        "ts": int(time.time())
    }
    db.collection("orders").document(order["order_id"]).set(item)
    return {"status": "ok", "order": item}

• This function connects to a Firestore collection called orders.
• It creates a document with the order ID, amount, payment ID, and a timestamp.
• It saves the document to the collection and returns a status.

Example input:

{"order": {"order_id": "test-123", "amount": "99.99", "validated": true}, "payment_id": "pay_test-123", "charged": true}

Example output:

{"status": "ok", "order": {"order_id": "test-123", "amount": 99.99, "payment_id": "pay_test-123", "ts": 1710000000}}

Now, let's see how these functions are connected using Google Cloud Workflows in a YAML definition file.

Here is the relevant part of the workflow definition:

# order_workflow.yaml
main:
  params: [input]
  steps:
    - validate:
        call: http.post
        args:
          url: https://REGION-PROJECT_ID.cloudfunctions.net/validate_order
          body: ${input}
        result: validated
    - charge:
        call: http.post
        args:
          url: https://REGION-PROJECT_ID.cloudfunctions.net/charge_customer
          body: ${validated.body}
        result: charged
    - record:
        call: http.post
        args:
          url: https://REGION-PROJECT_ID.cloudfunctions.net/record_order
          body: ${charged.body}
        result: recorded
    - return_output:
        return: ${recorded.body}

• The workflow starts by calling the validate_order function.
• Each step calls the next Cloud Function, passing along the output from the previous step.
• The workflow ends by returning the final output from the record_order function.

This setup ensures that the order is validated, charged, and recorded in sequence.

To deploy and test this workflow, you can use the gcloud CLI and the Google Cloud Console. Here’s a high-level overview of the process:

1. Deploy the Cloud Functions:

   Shell

   gcloud functions deploy validate_order --runtime python310 --trigger-http --allow-unauthenticated
   gcloud functions deploy charge_customer --runtime python310 --trigger-http --allow-unauthenticated
   gcloud functions deploy record_order --runtime python310 --trigger-http --allow-unauthenticated

2. Deploy the Workflow:

   Shell

   gcloud workflows deploy order-workflow --source=order_workflow.yaml --location=REGION

3. Execute the Workflow: You can trigger the workflow with a sample order:

   Shell

   gcloud workflows execute order-workflow --data='{"order": {"order_id": "test-123", "amount": "99.99"}}' --location=REGION

Expected output:
If everything works, the workflow will process the order through all three steps. The final output will look like this:

{"status": "ok", "order": {"order_id": "test-123", "amount": 99.99, "payment_id": "pay_test-123", "ts": 1710000000}}

This shows that the order was validated, charged, and recorded successfully.

In this lesson, you learned how Google Cloud Workflows help you organize and run multi-step workflows in your serverless applications. You saw how to:

• Break a process into separate Cloud Functions
• Connect those functions in a workflow using Workflows
• Deploy and test the workflow using Google Cloud tools

Next, you'll get hands-on practice building and running your own Workflows. This will help you reinforce what you've learned and prepare you for more advanced serverless patterns. Good luck!