Database Migrations in NestJS

Welcome to the lesson on database migrations! You’ve already learned how to integrate a database into your NestJS application.

Now, it's time to understand how to manage changes in your database schema effectively using migrations. This will allow you to modify your database schema incrementally and track changes over time, ensuring consistency and reliability.

In this lesson, you'll gain a deep understanding of database migrations and their role in maintaining a robust, scalable application, including:

• What database migrations are and why they matter
• How to create and run migrations in a NestJS application
• A practical example of adding a new field to an existing entity
• Managing migrations programmatically to track and apply changes

Database migrations are a way to handle changes to your database schema over time. Instead of manually altering your database structure, migrations allow you to define these changes programmatically. Once defined, migrations can be applied automatically, ensuring that your database schema evolves in a controlled and predictable manner.

Imagine you have a to-do application, and you need to add a "completed" field to the todo table. Instead of making manual updates to the database, migrations let you define this change in a file and then execute it to modify the database automatically. This ensures that changes to your schema are versioned and applied consistently across different environments.

In most applications, migrations are created and executed using CLI commands provided by TypeORM, the ORM used in NestJS. Let’s walk through the usual setup and then explain how we’ll apply these steps programmatically in the practice section.

Before setting up migrations, ensure you have the required dependencies installed. In a typical NestJS project using TypeORM, you need the following packages:

npm install @nestjs/typeorm typeorm sqlite3
npm install ts-node typeorm-ts-node-esm --save-dev

• typeorm: The ORM that will handle your migrations.
• sqlite3: The database driver (or another database driver, depending on your setup).
• ts-node and typeorm-ts-node-esm: Allow you to run TypeScript files in Node.js, which is necessary for running migration files written in TypeScript.

To make it easier to generate and run migrations, we add custom scripts in package.json. Add these lines to the scripts section:

"scripts": {
  "build": "nest build",
  "start": "nest start",
  "start:dev": "nest start --watch",
  "typeorm": "typeorm-ts-node-esm -d src/typeorm/ormconfig.ts",
  "migration:generate": "npm run typeorm -- migration:generate -n",
  "migration:run": "npm run typeorm -- migration:run"
}

This setup allows you to generate and run migrations using npm run migration:generate and npm run migration:run. The TypeORM CLI is now linked to your NestJS project for easier management of migrations.

After making changes to your entities, you can generate a migration file with:

npx typeorm migration:generate -n AddCompletedColumn

This will create a new migration file that contains instructions to update your database schema based on the most recent changes to your entities.

Once the migration is generated, you can apply it with:

npx typeorm migration:run

This command will apply all pending migrations to your database, updating the schema as defined in the migration files.

If something goes wrong, you can undo the last migration using:

npx typeorm migration:revert

This will revert the most recent database schema change, rolling your database back to the previous state.

Now that we’ve covered the usual approach, let’s see how we can manage migrations programmatically in the practice section to track and apply changes directly in the code.

To understand migrations programmatically, let's consider an example where we add a completed field to the todo table. Our goal is to add a boolean completed field to track whether a task is done or not. The todo table currently contains fields like id, title, and description.

Here is an example migration file to add a new field to your todo table:

export class AddCompleteFieldToTodo1631020456781 implements MigrationInterface {
  public async up(queryRunner: QueryRunner): Promise<void> {
    await queryRunner.addColumn(
      'todo',
      new TableColumn({
        name: 'completed',
        type: 'boolean',
        isNullable: false,
        default: false,
      }),
    );
  }

  public async down(queryRunner: QueryRunner): Promise<void> {
    await queryRunner.dropColumn('todo', 'completed');
  }
}

The timestamp in the class name (AddCompleteFieldToTodo1631020456781) serves a critical role in ensuring migration order. By including a timestamp, each migration has a unique identifier that reflects the time of its creation.

This helps TypeORM determine the correct sequence to apply migrations, ensuring that changes are applied in the intended order, and avoiding conflicts or missing dependencies between migrations.

Let’s break down the key components of this migration file:

1. MigrationInterface: This interface is implemented by all migration classes in TypeORM. It requires you to define two methods: up() and down(), which handle how to apply and revert the schema changes.

   • up() method: The up() method is responsible for applying the schema changes. In this case, it adds a new column to the todo table. The column is called completed, and it has the following properties:

     • name: The name of the column, which is completed.
     • type: The data type of the column, which is boolean. This means it can only hold true or false values.
     • isNullable: Set to false, which means this column is required and cannot be left empty (null).
     • default: The default value for this column is set to false. So, if no value is provided when creating or updating a todo record, this field will automatically be set to false.

     In this way, the up() method programmatically defines the structure of the new column, ensuring it is added to the database table with the specified properties.

   • down() method: This method is just as important as the up() method because it defines how to undo the changes made by the up() method. In our example, the down() method removes the completed column from the todo table. This is useful when rolling back a migration if something goes wrong or when reverting to a previous version of the database schema. Without a proper down() method, reverting database changes would be difficult or impossible.

2. QueryRunner: This is the main object that TypeORM uses to run queries inside migrations. The QueryRunner allows the migration to execute SQL statements that modify the database schema. In our example, we use queryRunner.addColumn() to add the completed column and queryRunner.dropColumn() to remove it.

3. TableColumn: This is a TypeORM class that describes the details of the column we want to add. In this case, we define the column name, type, and other options (such as whether it's nullable or has a default value).

• Migration as Version Control for Your Database: Migrations allow you to track and apply incremental changes to your database schema over time. Each migration represents a single change, and they are typically applied in sequence. This version control-like behavior ensures that your database schema is consistent across environments (development, staging, production).

• Predictable and Reversible Changes: The up() and down() methods in the migration file let you apply and revert changes safely. This means you can confidently introduce changes to your schema, knowing you can roll back if necessary. Reversible migrations are essential for maintaining a stable application and preventing data corruption.

• Automated Deployment: When you integrate migrations into your workflow, they can be applied automatically during application deployment, ensuring that the database schema is always in sync with your codebase. This eliminates the need for manual database updates and reduces the risk of errors.

In the practice section, we’ll see how you can apply migrations programmatically instead of using CLI commands. By integrating migration logic directly into your NestJS application, you can control when and how migrations are run, ensuring flexibility and control.

Here’s how we’ll handle it:

• Tracking migrations: We will manually track whether a migration has been run or not.
• Running migrations: Once the migration logic is in place, you’ll see how to programmatically apply the migration in the application lifecycle, typically at startup.

export const ormConfig: TypeOrmModuleOptions = {
  type: 'sqlite',
  database: 'database.sqlite',
  entities: [__dirname + '/../**/*.entity{.ts,.js}'],
  synchronize: false, // We use migrations instead of auto-sync
  migrations: [__dirname + '/../migrations/*{.ts,.js}'], // Path to migration files
  migrationsRun: false, // We’ll trigger migrations manually
};

This configuration:

• synchronize: false: Disables automatic schema synchronization to ensure migrations are used to control schema changes.
• migrations: Points to the directory where migration files are stored.
• migrationsRun: false: Prevents automatic migration execution on startup; we’ll manage it programmatically.

Understanding and using database migrations is important for several reasons:

• Consistency: Migrations ensure that database schema changes are applied consistently across different environments, preventing discrepancies.
• Version Control: Schema changes are versioned, making it easy to track and manage over time.
• Automation: Migrations automate the process of updating your database schema, reducing the risk of human error.

Now that you've learned how to manage database migrations both typically and programmatically, let's move to the practice section and apply these concepts in action. You’ll programmatically track and apply migrations in your NestJS app, ensuring you have full control over database changes.