Working with Cloud Datastore

Welcome to the next chapter in your cloud database journey! Having mastered Cloud Firestore in the previous lesson—Google's cutting-edge NoSQL document database—you're now ready to explore its predecessor. Meet Cloud Datastore, Google's original NoSQL database service that remains a powerful and widely-adopted solution in the GCP ecosystem.

While Firestore might be the newer star, Datastore continues to serve as the backbone for countless production applications, offering unique strengths that make it the preferred choice for specific use cases. In this lesson, we'll dive deep into Cloud Datastore and discover why it's still such a vital part of Google Cloud's database offerings.

In this lesson, you will learn how to:

• Create and store an entity in Cloud Datastore
• Retrieve a specific entity from Cloud Datastore
• List entities of a particular kind
• Update and delete entities in Cloud Datastore

By the end of this lesson, you will understand both major NoSQL options in GCP and can choose the right one for your applications.

While both Firestore and Datastore are NoSQL databases, they have different data models and use cases:

• Cloud Firestore organizes data into collections and documents with real-time synchronization and strong consistency for document reads and most queries
• Cloud Datastore organizes data into kinds and entities and supports ACID transactions primarily within entity groups; it provides strong consistency for key lookups and ancestor queries, and eventual consistency for non-ancestor (global) queries

Cloud Datastore is commonly used for applications that benefit from entity-group transactions, predictable performance with hierarchical data models, and integration with Google App Engine. It supports rich queries, but you should design with entity groups when you need strongly consistent, transactional operations; non-ancestor queries are eventually consistent.

Just like with Cloud Firestore in the previous lesson, cloud compute instances can securely access Datastore using service accounts and the metadata server. This means you do not need to manually provide credentials in your code when running on a cloud instance.

To interact with Cloud Datastore, you will use a different client library than Firestore. The client library automatically uses the credentials provided by the environment, so you do not need to configure anything extra.

Let's start by setting up our imports and client connection:

from google.cloud import datastore

datastore_client = datastore.Client(project="datastore-demo")

Here, datastore.Client() creates a client object that lets you interact with Cloud Datastore. The project parameter specifies which GCP project to connect to - in our case, we're using datastore-demo which is the project configured for our local Datastore emulator. You will use this datastore_client object for all your Cloud Datastore operations in this lesson.

Unlike Firestore's collections and documents, Datastore uses kinds and entities. A kind is like a category or table that groups similar data together (similar to a Firestore collection). An entity is an individual record within that kind (similar to a Firestore document). Just like with Firestore, you don't need to create kinds beforehand - they are created automatically when you store your first entity.

Let's define the kind we'll work with in our examples:

import uuid
from datetime import datetime

KIND = "SampleData"
ENTITY_NAME = f"sample-{str(uuid.uuid4())[:8]}"

• KIND is the name of our data category (like "SampleData")
• ENTITY_NAME creates a unique identifier for our entity using a random string

Now let's create an entity and store it in Datastore. An entity contains key-value pairs of data, similar to how a Firestore document contains fields.

# Create an entity with a specific key
key = datastore_client.key(KIND, ENTITY_NAME)
entity = datastore.Entity(key=key)
entity.update({
    'message': 'Hello from Cloud Datastore!',
    'created_at': datetime.utcnow(),
    'version': 1,
    'active': True
})
datastore_client.put(entity)
print("Created entity:", ENTITY_NAME)

Explanation:

• datastore_client.key(KIND, ENTITY_NAME) creates a unique key for our entity
• datastore.Entity(key=key) creates a new entity with that key
• entity.update({...}) adds data properties to the entity
• datastore_client.put(entity) saves the entity to Datastore

Example output:

Created entity: sample_1a2b3c4d

After storing an entity, you can retrieve it using its key. Let's fetch the entity we just created.

# Read the entity back
retrieved_entity = datastore_client.get(key)
print("Read:", {
    'message': retrieved_entity.get('message'),
    'version': retrieved_entity.get('version'),
    'active': retrieved_entity.get('active')
})

Explanation:

• datastore_client.get(key) retrieves the entity using its key
• entity.get('property_name') safely gets the value of a property
• If the entity doesn't exist, get() returns None

Example output:

Read: {'message': 'Hello from Cloud Datastore!', 'version': 1, 'active': True}

You can query for multiple entities of the same kind. Let's list all entities of our sample kind.

# List entities of this kind
query = datastore_client.query(kind=KIND)
query.order = ['created_at']
entities = list(query.fetch(limit=5))
print(f"Found {len(entities)} entities of kind '{KIND}'")

Explanation:

• datastore_client.query(kind=KIND) creates a query for entities of the specified kind
• query.order = ['created_at'] sorts results by the created_at property
• query.fetch(limit=5) executes the query and limits results to 5 entities
• We convert the results to a list to get the count

Example output:

Found 1 entities of kind 'SampleData'

Finally, let's see how to update an existing entity and then delete it.

# Update the entity
retrieved_entity['message'] = 'Updated message from Cloud Datastore!'
retrieved_entity['version'] = retrieved_entity.get('version', 0) + 1
retrieved_entity['updated_at'] = datetime.utcnow()
datastore_client.put(retrieved_entity)
print("Updated entity:", ENTITY_NAME)

# Read updated entity
updated_entity = datastore_client.get(key)
print("Updated read:", {
    'message': updated_entity.get('message'),
    'version': updated_entity.get('version')
})

# Cleanup: delete the entity
datastore_client.delete(key)
print("Deleted entity:", ENTITY_NAME)

Explanation:

• To update: modify the entity's properties directly and save it back with put()
• entity['property'] = value updates or adds a property
• datastore_client.delete(key) removes the entity from Datastore using its key

Example output:

Updated entity: sample_1a2b3c4d
Updated read: {'message': 'Updated message from Cloud Datastore!', 'version': 2}
Deleted entity: sample_1a2b3c4d

While the code patterns are similar, Datastore and Firestore have important behavioral differences:

Data Model:

• Firestore: Collections → Documents (flexible, nested data)
• Datastore: Kinds → Entities (flatter, more structured)

Real-time Features:

• Firestore: Built-in real-time listeners for instant updates
• Datastore: No real-time sync (must poll for changes)

Transactions:

• Firestore: ACID transactions across multiple documents and collections without constraints.
• Datastore: ACID transactions work best within entity groups; cross-group transactions have limitations.

Best Use Cases:

• Choose Firestore for: Mobile apps, real-time dashboards, and applications needing broad strong consistency
• Choose Datastore for: App Engine backends and systems requiring entity-group transactions

Both services integrate seamlessly with other GCP services and use similar authentication patterns, making it easy to switch between them based on your application's specific needs.

In this lesson, you learned how to:

• Create and store an entity in Cloud Datastore
• Retrieve a specific entity from Cloud Datastore
• List entities of a particular kind
• Update and delete entities in Cloud Datastore

You now have experience with both of Google Cloud's NoSQL database services: Firestore (with collections and documents) and Datastore (with kinds and entities). Each has its strengths - Firestore excels at real-time applications, while Datastore provides strong consistency and robust querying capabilities.

Up next, you'll get to practice these steps yourself in hands-on exercises. This will help you get comfortable with Datastore operations and prepare you for more advanced cloud development tasks. Good luck!