Transcription Routing and Audio Clipping with OpenAI Whisper

Lesson 2: Segment-Based Transcription with Whisper API

In the previous lesson, you learned how to load and play audio files using Howler.js in the browser, while controlling playback state from a TypeScript backend. This laid the foundation for interactive audio applications. Now it's time to take the next step: extracting meaningful text from audio using OpenAI Whisper . But instead of transcribing full audio files, we’ll focus on clipping and transcribing only the audio segments the user listens to—based on when they press start and stop. This will prepare the way for precise, user-driven transcription workflows.

What You’ll Learn

By the end of this lesson, you will be able to: Clip a specific portion of an audio file using ffmpeg . Prepare the clipped audio segment for Whisper transcription. Send a transcription request to the backend with start and duration parameters. Use the OpenAI Whisper API to convert speech to text. In this lesson, we focus purely on the backend logic. In the upcoming, you’ll learn how the browser tracks segment timing and sends that to the backend.

Prerequisite: Installing ffmpeg

To clip audio files before transcription, we rely on ffmpeg, a powerful command-line tool for processing multimedia files. While ffmpeg is widely supported, it may not be installed by default on your system. You’ll need to install it manually if you haven’t already. ffmpeg is a cross-platform utility for handling audio, video, and other media files. In this project, we use it to: Extract a segment of an audio file (-ss and -t) Convert the audio to mono and 16kHz sample rate (as required by Whisper) Here are common installation methods by platform: macOS (using Homebrew) Shell brew install ffmpeg brew install ffmpeg Ubuntu Shell sudo apt update sudo apt install ffmpeg sudo apt update sudo apt install ffmpeg Windows Download from: https://ffmpeg.org/download.html Unzip the package. Add the bin directory to your system's PATH environment variable. To verify installation, run: Shell ffmpeg -version ffmpeg -version You should see version details printed in the terminal. ⚠️ If ffmpeg is not installed or not in your PATH, the clipping step will fail with an error like ffmpeg: command not found.

Backend: Clipping Audio Segments with ffmpeg

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Shell

brew install ffmpeg

Shell

sudo apt update
sudo apt install ffmpeg

Shell

ffmpeg -version

TypeScript

// src/routes/transcribe.ts
exec(`ffmpeg -y -ss ${start} -t ${duration} -i "${fullPath}" -ac 1 -ar 16000 "${clippedPath}"`, async (err) => {
  if (err) return res.status(500).json({ error: 'Clipping failed' });
  const transcription = await transcribe(clippedPath);
  res.json({ transcription });
});

TypeScript

// src/routes/transcribe.ts
router.post('/', async (req, res) => {
  // Extract input values from request body
  const { filePath, start = 0, duration } = req.body;

  // Validate presence of required fields
  if (!filePath || typeof duration !== 'number') {
    return res.status(400).json({ error: 'Missing filePath or duration' });
  }

  // Resolve the full absolute path to the input audio file
  const fullPath = path.resolve(process.cwd(), filePath);

  // Check if the file actually exists on disk
  if (!fs.existsSync(fullPath)) {
    return res.status(404).json({ error: 'File not found' });
  }

  // Define output path for the clipped segment (same as input with .clip.mp3 extension)
  const clippedPath = fullPath + `.clip.mp3`;

  try {
    // Run ffmpeg command to extract the desired segment
    await new Promise((resolve, reject) => {
      exec(
        // Clip using start and duration values, convert to mono (1 channel) and 16kHz sample rate
        `ffmpeg -y -ss ${start} -t ${duration} -i "${fullPath}" -ac 1 -ar 16000 "${clippedPath}"`,
        (err) => (err ? reject(err) : resolve())
      );
    });

    // Transcribe the clipped segment using Whisper API
    const transcription = await transcribe(clippedPath);

    res.json({ transcription });

  } catch (err) {
    console.error('[Transcribe] ❌', err);
    res.status(500).json({ error: 'Transcription failed' });
  }
});

TypeScript

// src/services/transcriber.ts
import { OpenAI } from 'openai';
import fs from 'fs';

const client = new OpenAI();

export async function transcribe(filePath: string): Promise<string> {
  try {
    if (!fs.existsSync(filePath)) {
      throw new Error(`File not found: ${filePath}`);
    }

    const stream = fs.createReadStream(filePath);

    const response = await client.audio.transcriptions.create({
      model: "whisper-1",
      file: stream
    });

    return response.text;
  } catch (err) {
    console.error('[Whisper Error]', err);
    throw new Error('Transcription failed');
  }
}

// src/services/transcriber.ts
import { OpenAI } from 'openai';
import fs from 'fs';

const client = new OpenAI();

export async function transcribe(filePath: string): Promise<string> {
  try {
    if (!fs.existsSync(filePath)) {
      throw new Error(`File not found: ${filePath}`);
    }

    const stream = fs.createReadStream(filePath);

    const response = await client.audio.transcriptions.create({
      model: "whisper-1",
      file: stream
    });

    return response.text;
  } catch (err) {
    console.error('[Whisper Error]', err);
    throw new Error('Transcription failed');
  }
}

TypeScript

// src/services/transcriber.ts
import { OpenAI } from 'openai';
import fs from 'fs';

const client = new OpenAI();

export async function transcribe(filePath: string): Promise<string> {
  try {
    if (!fs.existsSync(filePath)) {
      throw new Error(`File not found: ${filePath}`);
    }

    const stream = fs.createReadStream(filePath);

    const response = await client.audio.transcriptions.create({
      model: "whisper-1",
      file: stream
    });

    return response.text;
  } catch (err) {
    console.error('[Whisper Error]', err);
    throw new Error('Transcription failed');
  }
}

TypeScript

// src/routes/transcribe.ts
exec(`ffmpeg -y -ss ${start} -t ${duration} -i "${fullPath}" -ac 1 -ar 16000 "${clippedPath}"`, async (err) => {
  if (err) return res.status(500).json({ error: 'Clipping failed' });
  const transcription = await transcribe(clippedPath);
  res.json({ transcription });
});

TypeScript

// src/routes/transcribe.ts
router.post('/', async (req, res) => {
  // Extract input values from request body
  const { filePath, start = 0, duration } = req.body;

  // Validate presence of required fields
  if (!filePath || typeof duration !== 'number') {
    return res.status(400).json({ error: 'Missing filePath or duration' });
  }

  // Resolve the full absolute path to the input audio file
  const fullPath = path.resolve(process.cwd(), filePath);

  // Check if the file actually exists on disk
  if (!fs.existsSync(fullPath)) {
    return res.status(404).json({ error: 'File not found' });
  }

  // Define output path for the clipped segment (same as input with .clip.mp3 extension)
  const clippedPath = fullPath + `.clip.mp3`;

  try {
    // Run ffmpeg command to extract the desired segment
    await new Promise((resolve, reject) => {
      exec(
        // Clip using start and duration values, convert to mono (1 channel) and 16kHz sample rate
        `ffmpeg -y -ss ${start} -t ${duration} -i "${fullPath}" -ac 1 -ar 16000 "${clippedPath}"`,
        (err) => (err ? reject(err) : resolve())
      );
    });

    // Transcribe the clipped segment using Whisper API
    const transcription = await transcribe(clippedPath);

    res.json({ transcription });

  } catch (err) {
    console.error('[Transcribe] ❌', err);
    res.status(500).json({ error: 'Transcription failed' });
  }
});

Lesson 2: Segment-Based Transcription with Whisper API

What You’ll Learn

Prerequisite: Installing ffmpeg

Backend: Clipping Audio Segments with ffmpeg

Join the 1M+ learners on CodeSignal

Route: /transcribe

Whisper Integration: transcriber.ts

Summary