Getting Started with Compute Engine

Welcome to your first lesson in GCP Core Compute and Storage Services! We're starting with Google Compute Engine (GCE), which is the foundation of Google Cloud Platform's compute offerings and is likely the service you'll use most frequently as you build applications in the cloud.

Think of Compute Engine as a way to rent virtual machines (VMs) in Google's data centers. Instead of buying physical servers, waiting for delivery, and setting up hardware in your office, you can launch a virtual machine in minutes and pay only for what you use. These virtual machines, called instances, can run the same operating systems and applications as physical computers, but with the flexibility to resize, relocate, or replace them instantly.

Compute Engine is fundamental because it provides general-purpose virtual machines that give you the flexibility to run virtually any workload. Many applications you build will either run directly on Compute Engine or use it alongside other GCP services. Whether you're hosting a simple website, running complex data processing jobs, or building microservices, Compute Engine provides the computational power you need.

By the end of this lesson, you'll have launched your very own Compute Engine instance, created secure access credentials, and connected to your server via SSH. You'll understand how to find the right boot disk image, choose appropriate hardware specifications, and manage your instance's lifecycle. This hands-on experience will give you the confidence to work with Compute Engine in real-world scenarios.

A boot disk image in GCP is essentially a template that contains everything needed to launch a Compute Engine instance. Think of it as a snapshot of a disk containing an operating system, pre-installed software, and configuration settings. When you launch an instance, GCP uses the selected image as a blueprint to create your virtual machine's boot disk.

GCP provides a wide variety of public images for popular operating systems like Debian, Ubuntu, CentOS, and Windows Server. These images are maintained by Google and receive regular security updates. You can also create your own custom images with your specific software and configurations pre-installed. Custom images capture the contents of a single disk and are ideal for standardizing deployments across multiple instances.

The key to working with images effectively is understanding how to find and select the right one for your needs. GCP organizes images into families (such as debian-12 or ubuntu-2204-lts), and you can always select the latest image in a family.

Let's look at how to programmatically find the latest Debian image using the Google Cloud Python SDK (google-cloud-compute):

from google.cloud import compute_v1

def get_latest_debian_image(project="debian-cloud", family="debian-12"):
    image_client = compute_v1.ImagesClient()
    image = image_client.get_from_family(project=project, family=family)
    return image.self_link  # or image.name if you need just the name

# Example usage:
image_link = get_latest_debian_image()
print("Latest Debian image link:", image_link)

This function uses the get_from_family method to retrieve the latest image from the specified family. The project parameter specifies the image project (for official Debian images, this is debian-cloud). The returned self_link can be used when launching your instance.

Security is paramount when working with cloud servers, and SSH key pairs provide a secure way to access your instances. When you connect to a Compute Engine instance, your SSH client uses your private key to prove your identity. The public key is stored in the instance's metadata, and the private key remains on your computer.

GCP offers multiple approaches to SSH key management:

1. Metadata-based SSH keys: You can add your own public SSH key to an instance's metadata, which is the approach we'll use in this lesson. This gives you direct control over which keys can access your instances.

2. OS Login: GCP also supports OS Login, which centralizes SSH key management by linking SSH keys to your Google Cloud user account. This approach enhances security by providing centralized access control, two-factor authentication support, and automatic key rotation. OS Login is particularly useful in enterprise environments where you need to manage access for multiple users across many instances.

For learning purposes, we'll focus on the metadata-based approach, which provides a clear understanding of how SSH keys work. You can add your SSH key to an instance's metadata programmatically. Here's how to generate a key pair and add the public key to your instance using Python:

import os
import subprocess
from google.cloud import compute_v1

def generate_ssh_key_pair(username, key_path):
    # Note: For production, prefer ed25519 keys and a passphrase:
    # ssh-keygen -t ed25519 -f <key_path> -N "<strong-passphrase>" -C <username>
    # The RSA/no-passphrase example below is for demonstration only.
    # Generate SSH key pair if it doesn't exist
    if not os.path.exists(key_path):
        subprocess.run([
            "ssh-keygen", "-t", "rsa", "-f", key_path, "-N", "", "-C", username
        ], check=True)
    with open(key_path + ".pub", "r") as pubkey_file:
        public_key = pubkey_file.read().strip()
    return public_key

def set_instance_ssh_key(project, zone, instance_name, username, public_key):
    instance_client = compute_v1.InstancesClient()
    instance = instance_client.get(project=project, zone=zone, instance=instance_name)
    # Prepare SSH key in the format: "username:ssh-rsa ... username"
    ssh_key_entry = f"{username}:{public_key}"
    # Add to instance metadata
    items = list(instance.metadata.items)
    # Remove any existing ssh-keys entry
    items = [item for item in items if item.key != "ssh-keys"]
    items.append(compute_v1.types.Items(key="ssh-keys", value=ssh_key_entry))
    metadata = compute_v1.types.Metadata(items=items)
    operation = instance_client.set_metadata(
        project=project, zone=zone, instance=instance_name, metadata=metadata
    )
    return operation

# Example usage:
username = "gcpuser"
key_path = "gcpuser_key"
public_key = generate_ssh_key_pair(username, key_path)
# set_instance_ssh_key(project, zone, instance_name, username, public_key)

This approach ensures that only someone with the private key can access the instance. Remember to keep your private key secure and set its permissions to 600 (chmod 600 gcpuser_key).

For production environments, prefer OS Login over project or instance metadata keys:

• Centralized access via IAM roles with optional two-factor authentication
• Automatic key association with users, simpler revocation and auditing
• Works well with IAP TCP forwarding to avoid exposing SSH to the public internet

References:

• OS Login: https://cloud.google.com/compute/docs/oslogin
• IAP TCP forwarding: https://cloud.google.com/iap/docs/tcp-forwarding-overview

Compute Engine offers a wide range of machine types, each optimized for different use cases. Understanding the naming convention helps you choose the right one for your needs. Machine types follow the pattern of family, series, and size, such as e2-micro, n1-standard-1, or n2-highmem-4.

• The family (e.g., e2, n1, n2) indicates the generation and capabilities of the machine type.
• The type (e.g., micro, standard, highmem, highcpu) describes the balance of CPU and memory.
• The number at the end (e.g., 1, 2, 4) indicates the number of vCPUs.

For learning and development, e2-micro is an excellent starting point and is included in the GCP Free Tier. It provides 2 vCPUs (shared) and 1 GB of memory, which is sufficient for basic web servers, development environments, and learning exercises.

Cost considerations are important when choosing machine types. Larger machine types cost more per hour, and costs can add up quickly if you forget to stop or delete instances when you're not using them. Always start with the smallest machine type that meets your needs, and scale up only when necessary.

Now that we understand images, SSH keys, and machine types, let's put it all together to launch a Compute Engine instance. The process involves several steps: creating a firewall rule to allow SSH access, specifying the image, choosing a machine type, configuring security settings, and waiting for the instance to become available.

First, let's create a firewall rule to allow SSH (port 22) access:

from google.cloud import compute_v1

def create_firewall_rule(project, network="global/networks/default"):
    firewall_client = compute_v1.FirewallsClient()
    rule_name = "allow-ssh"
    firewall_rule = compute_v1.Firewall(
        name=rule_name,
        network=network,
        allowed=[compute_v1.Allowed(
            IPProtocol="tcp",
            ports=["22"]
        )],
        direction="INGRESS",
        # Demo-only: opens SSH to the internet. For production, restrict to your IP (/32)
        # or use OS Login with IAP TCP forwarding.
        source_ranges=["0.0.0.0/0"],
        target_tags=["allow-ssh"]
    )
    try:
        firewall_client.insert(project=project, firewall_resource=firewall_rule)
    except Exception as e:
        print("Firewall rule may already exist:", e)
    return rule_name

Security note: The example above allows SSH from anywhere (0.0.0.0/0) for demonstration purposes. In production, restrict SSH to your IP address using a /32 CIDR (for example, ["203.0.113.10/32"]), enable OS Login, and consider IAP TCP forwarding to avoid exposing SSH publicly.

Next, let's launch the instance:

from google.cloud import compute_v1

def launch_instance(project, zone, instance_name, image_link, machine_type="e2-micro", username="gcpuser", public_key=None):
    instance_client = compute_v1.InstancesClient()
    machine_type_url = f"zones/{zone}/machineTypes/{machine_type}"
    network_interface = compute_v1.NetworkInterface(
        network="global/networks/default",
        access_configs=[compute_v1.AccessConfig(name="External NAT", type_="ONE_TO_ONE_NAT")]
    )
    # Add SSH key to instance metadata
    metadata_items = []
    if public_key:
        metadata_items.append(compute_v1.types.Items(
            key="ssh-keys",
            value=f"{username}:{public_key}"
        ))
    instance = compute_v1.Instance(
        name=instance_name,
        machine_type=machine_type_url,
        network_interfaces=[network_interface],
        disks=[compute_v1.AttachedDisk(
            boot=True,
            auto_delete=True,
            initialize_params=compute_v1.AttachedDiskInitializeParams(
                source_image=image_link,
                disk_size_gb=10
            )
        )],
        metadata=compute_v1.Metadata(items=metadata_items),
        tags=compute_v1.Tags(items=["allow-ssh"])
    )
    operation = instance_client.insert(
        project=project,
        zone=zone,
        instance_resource=instance
    )
    return operation

# Example usage:
# launch_instance(project, zone, "my-instance", image_link, "e2-micro", username, public_key)

After launching, you can retrieve the external IP address:

def get_instance_external_ip(project, zone, instance_name):
    instance_client = compute_v1.InstancesClient()
    instance = instance_client.get(project=project, zone=zone, instance=instance_name)
    for iface in instance.network_interfaces:
        for access_config in iface.access_configs:
            if access_config.nat_i_p:
                return access_config.nat_i_p
    return None

Understanding Compute Engine instance states is crucial for managing costs and resources effectively. An instance progresses through several states during its lifecycle: PROVISIONING (being created), STAGING (preparing to start), RUNNING (available for use), STOPPING (shutting down), TERMINATED (stopped), and DELETING (being removed). Note that Compute Engine supports additional states such as SUSPENDING, SUSPENDED, and REPAIRING, though we're focusing on the most commonly encountered states for this introductory lesson.

• Stopping an instance shuts it down but preserves the boot disk and any attached storage. You're not charged for compute time while the instance is stopped, but you do pay for any persistent disk storage.
• Deleting an instance permanently removes it and, by default, deletes the boot disk. This action cannot be undone.

Here are functions to manage the instance lifecycle:

def stop_instance(project, zone, instance_name):
    instance_client = compute_v1.InstancesClient()
    operation = instance_client.stop(project=project, zone=zone, instance=instance_name)
    return operation

def delete_instance(project, zone, instance_name):
    instance_client = compute_v1.InstancesClient()
    operation = instance_client.delete(project=project, zone=zone, instance=instance_name)
    return operation

From a cost management perspective, always stop instances when you're not using them, especially during development and testing. For temporary instances or completed experiments, deletion is often the best choice to avoid any ongoing storage costs.

Once your instance is running and you have its external IP address, you can connect to it using SSH. GCP provides multiple ways to establish SSH connections:

Command-Line SSH:

The traditional approach uses an SSH client with your private key file to authenticate to the server. The general SSH command format is ssh -i [key-file] [username]@[external-ip]. For most public images, the default username is the one you specified when generating your SSH key. For Debian and Ubuntu images, you can use your chosen username.

For example, if you created a key for gcpuser and your instance's external IP is 34.123.45.67, the command would be:

ssh -i gcpuser_key gcpuser@34.123.45.67

The first time you connect, SSH will ask you to verify the server's fingerprint. This is a security feature to prevent man-in-the-middle attacks. Type "yes" to accept the fingerprint and establish the connection.

Browser-Based SSH:

GCP also offers a browser-based SSH terminal accessible directly from the Google Cloud Console. This feature allows you to connect to your instances without needing to manage SSH keys locally or use a terminal application. Simply navigate to your instance in the Console and click the "SSH" button. This is particularly convenient for quick administrative tasks or when you're working from a computer where you don't have your SSH keys available.

Common connection issues include incorrect file permissions on the key file, network connectivity problems, firewall rules that block SSH traffic, or missing external IP addresses. If you encounter connection problems, first verify that:

• Your key file has 600 permissions
• You're using the correct username for your image type
• Your firewall rule allows inbound SSH traffic on port 22
• Your instance has an external IP address assigned

Once connected, you'll have a command-line interface to your Compute Engine instance. You can install software, configure services, transfer files, and perform any tasks you would on a physical Linux server. Remember that this is a real computer running in Google's data center, and you have full administrative access to it.

In this lesson, we've covered the fundamental concepts needed to launch and manage Compute Engine instances. You've learned how boot disk images serve as blueprints for your virtual servers, how SSH key pairs provide secure access, and how to choose appropriate machine types for your needs. We've walked through the complete process of launching an instance programmatically and understanding its lifecycle states.

The workflow we've established follows a logical progression: find the right image for your operating system needs, create secure access credentials through key pairs, launch an appropriately sized instance, and connect to it via SSH. This pattern will serve you well whether you're deploying a simple web application or building complex distributed systems.

Looking ahead, you'll have the opportunity to implement this code yourself in the upcoming practice exercises. You'll work with the CodeSignal environment, where the google-cloud-compute library and other necessary tools are pre-installed, allowing you to focus on learning the concepts rather than environment setup.

As you work through the exercises, remember the importance of cost management. Always clean up resources when you're finished with them, either by stopping instances for temporary pauses or deleting them when you're completely done. The GCP Free Tier provides generous limits for learning, but developing good cleanup habits now will serve you well when working with production systems.

The skills you've learned in this lesson form the foundation for everything else we'll cover in this course. Whether we're discussing storage services, load balancers, or auto-scaling groups, they all build upon the basic Compute Engine concepts you now understand. You're ready to start launching and managing your own virtual servers in the Google Cloud Platform.