Schema Validation in MongoDB

Welcome back to our journey through MongoDB. In our last few lessons, we've explored various data types supported by MongoDB and tackled different types of relations: one-to-one, one-to-many, and many-to-many. Now, we'll be adding another tool to our toolkit: Schema Validation. Validation is a way for us to enforce certain rules and standards on our data, ensuring that all entries in our database meet certain requirements.

As mentioned earlier, MongoDB is schemaless, which means that a single collection can store documents with entirely different structures. For example:

[
    { "title": "To Kill a Mockingbird", "published_year": NumberInt(1949) },
    { "name": "To Kill a Mockingbird", "published_at": NumberInt(1949) },
    { "book_title": "To Kill a Mockingbird", "year": NumberInt(1949) }
]

Although this is technically possible, developers typically ensure that all documents in a collection share a similar structure or at least have consistent core fields to avoid complications in querying and data integrity. For instance:

[
    { "title": "To Kill a Mockingbird", "author": "Harper Lee", "published_year": NumberInt(1960), "genre": "Fiction", "copies_sold": NumberInt(40000000) },
    { "title": "1984", "author": "George Orwell", "published_year": NumberInt(1949), "cover_type": "Hardcover" },
    { "title": "Brave New World", "author": "Aldous Huxley", "published_year": NumberInt(1932), "language": "English", "pages": NumberInt(288) }
]

In this example, while title, author, and published_year are the core fields required for each document, several optional fields like genre, copies_sold, cover_type, language, and pages are also included. These optional fields provide additional information but aren't mandatory for each document. To enforce the required fields and maintain consistency, we can use Schema Validation.

Schema validation allows us to define specific rules that all documents in a collection must follow. These rules can range from simple—such as requiring a field to be a string—to complex, like ensuring a numerical field falls within a certain range. By implementing validation, we ensure data integrity and consistency across our collections.

MongoDB provides two important options for fine-tuning schema validation: validationLevel and validationAction.

The validationLevel option specifies which documents should undergo validation:

• strict: This is the default setting. All documents must comply with the validation rules, including those already in the collection and any newly inserted or updated documents.
• moderate: This setting enforces validation only during updates or insertions. Existing documents that do not conform to the rules will not cause errors unless they are modified.

The validationAction option specifies what should happen when validation fails:

• error: This is the default setting. Any document that does not meet the validation rules will cause the insertion or update operation to fail with an error.
• warn: Instead of causing an error, the operation will proceed, but a warning will be logged. This can be useful for tracking validation rule violations without rejecting data.

You can specify these options when creating or modifying a collection with schema validation.

Let's jump into a practical example. Suppose we're creating a new collection called journals. We want each journal document to have three fields: title, author, and published_year. Also, we want published_year to be no earlier than 1440 (the year the Gutenberg Press was invented!).

This is how we do it:

use library_db

db.createCollection("journals", {
  validator: {  // Start of the validator object
    $jsonSchema: {
      bsonType: "object",  // The documents must be objects
      required: ["title", "author", "published_year"],  // These fields are required
      properties: {
        title: {
          bsonType: "string",  // The 'title' field must be a string
          description: "must be a string and is required"
        },
        author: {
          bsonType: "string",  // The 'author' field must be a string
          description: "must be a string and is required"
        },
        published_year: {
          bsonType: "int",  // The 'published_year' field must be an integer
          minimum: 1440,  // 'published_year' cannot be earlier than 1440
          description: "must be an integer no less than 1440 and is required"
        }
      }
    }
  },
  validationLevel: "strict",  // Setting validation level to 'strict'
  validationAction: "error"  // Setting validation action to 'error'
});

This command establishes a new collection in our library_db database with a schema validator. Now, every new document in the journals collection must meet the rules we've set.

But what if we want to add schema validation to a collection that already exists? Or we want to change the validation rules we defined for a specific collection? MongoDB has us covered. Let's say we want to update the validation rules for the journals collection and we want to make sure that published year is at least 1800 instead of 1450 as we defined earlier. Here's how we can do that:

use library_db

db.runCommand({
  collMod: "journals",
  validator: {
    $jsonSchema: {
      bsonType: "object",
      required: ["title", "author", "published_year"],
      properties: {
        title: {
          bsonType: "string",
          description: "must be a string and is required"
        },
        author: {
          bsonType: "string",
          description: "must be a string and is required"
        },
        published_year: {
          bsonType: "int",
          minimum: 1800,  // This field was updated
          description: "must be an integer and is required"
        }
      }
    }
  },
  validationLevel: "strict",
  validationAction: "error"
});

In this command, we use db.runCommand() with the collMod operation, effectively updating schema validation of an existing collection. Notice that published_year now should have a value no less than 1800.

MongoDB allows you to validate basic BSON types such as strings, numbers, booleans, and more. Here is how you can validate a document's structure for various basic fields:

use library_db

db.createCollection("journals", {
  validator: {
    $jsonSchema: {
      bsonType: "object",
      required: ["title", "pages", "isPublished", "publishDate", "createdAt", "metadata", "price", "views"],
      properties: {
        title: { bsonType: "string", description: "must be a string and is required" },
        pages: { bsonType: "int", minimum: 1, description: "must be a positive integer and is required" },
        isPublished: { bsonType: "bool", description: "must be a boolean and is required" },
        publishDate: { bsonType: "date", description: "must be a Date object and is required" },
        createdAt: { bsonType: "timestamp", description: "must be a Timestamp object and is required" },
        metadata: { bsonType: ["null", "object"], description: "must be either null or an object" },
        price: { bsonType: "decimal", description: "must be a decimal and is required" },
        views: { bsonType: "long", description: "must be a long integer and is required" }
      }
    }
  }
})

In this example, the journals collection requires fields title, pages, isPublished, publishDate, createdAt, metadata,price, and views with specific BSON types. For metadata, using an array of BSON types ["null", "object"] means that it can be either of those types.

When dealing with nested objects, you can define validation rules for each nested field. Here's an example of validating an object structure:

use library_db

db.createCollection("journals", {
  validator: {
    $jsonSchema: {
      bsonType: "object",
      required: ["title", "editor"],
      properties: {
        title: { bsonType: "string", description: "must be a string and is required" },
        editor: {
          bsonType: "object",
          required: ["name", "years_of_experience"],
          properties: {
            name: { bsonType: "string", description: "must be a string and is required" },
            years_of_experience: { bsonType: "int", minimum: 1, description: "must be an integer and at least 1" }
          },
          description: "must be an object and is required"
        }
      }
    }
  }
});

Here, the journals collection requires an editor field that must be an object containing name and years_of_experience with their respective rules.

MongoDB also supports validation of arrays and their contents. Below is an example of how to enforce rules on array fields:

use library_db

db.createCollection("journals", {
  validator: {
    $jsonSchema: {
      bsonType: "object",
      required: ["keywords", "ratings"],
      properties: {
        keywords: {
          bsonType: "array",
          items: { bsonType: "string", description: "each item must be a string" },
          description: "must be an array of strings and is required"
        },
        ratings: {
          bsonType: "array",
          items: { bsonType: "int", minimum: 1, maximum: 5, description: "each rating must be an integer between 1 and 5" },
          description: "must be an array of integers and is required"
        }
      }
    }
  }
});

In this snippet, the journals collection requires keywords to be an array of strings and ratings to be an array of integers between 1 and 5, inclusive.

Great job making it this far! We've covered the concept of schema validation in MongoDB: what it is, why it's useful, and how to apply it to new and existing collections with the journals example.

Next, you'll be designing your own schemas and applying them to various collections through hands-on exercises. This practice will solidify your skills and prepare you for our next topic: data querying and retrieval. Keep up the impressive work as we continue to dive deeper into MongoDB's capabilities.