Transcription Routing and Audio Clipping with OpenAI Whisper | CodeSignal Learn

Transcription Routing and Audio Clipping with OpenAI Whisper | CodeSignal Learn

Lesson 2: Segment-Based Transcription with Whisper API

In the previous lesson, you learned how to load and play audio files using Howler.js in the browser, while controlling playback state from a TypeScript backend. This laid the foundation for interactive audio applications. Now it's time to take the next step: extracting meaningful text from audio using OpenAI Whisper . But instead of transcribing full audio files, we’ll focus on clipping and transcribing only the audio segments the user listens to—based on when they press start and stop. This will prepare the way for precise, user-driven transcription workflows.

What You’ll Learn

By the end of this lesson, you will be able to: Clip a specific portion of an audio file using ffmpeg . Prepare the clipped audio segment for Whisper transcription. Send a transcription request to the backend with start and duration parameters. Use the OpenAI Whisper API to convert speech to text. In this lesson, we focus purely on the backend logic. In the upcoming, you’ll learn how the browser tracks segment timing and sends that to the backend.

Prerequisite: Installing ffmpeg

To clip audio files before transcription, we rely on ffmpeg, a powerful command-line tool for processing multimedia files. While ffmpeg is widely supported, it may not be installed by default on your system. You’ll need to install it manually if you haven’t already. ffmpeg is a cross-platform utility for handling audio, video, and other media files. In this project, we use it to: Extract a segment of an audio file (-ss and -t) Convert the audio to mono and 16kHz sample rate (as required by Whisper) Here are common installation methods by platform: macOS (using Homebrew) Shell brew install ffmpeg Ubuntu Shell sudo apt update sudo apt install ffmpeg Windows Download from: https://ffmpeg.org/download.html Unzip the package. Add the bin directory to your system's PATH environment variable. To verify installation, run: Shell ffmpeg -version You should see version details printed in the terminal. ⚠️ If ffmpeg is not installed or not in your PATH, the clipping step will fail with an error like ffmpeg: command not found.

Sign up

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Whisper Integration: transcriber.ts

Once the clipped file is ready, we use the Whisper API to transcribe it. We are already familiar with it from the previous course, // src/services/transcriber.ts import { OpenAI } from 'openai'; import fs from 'fs'; const client = new OpenAI(); export async function transcribe(filePath: string): Promise<string> {try {if (!fs.existsSync(filePath)) {throw new Error(`File not found: ${filePath}`);} const stream = fs.createReadStream(filePath); const response = await client.audio.transcriptions.create({model: "whisper-1", file: stream}); return response.text;} catch (err) {console.error('[Whisper Error]', err); throw new Error('Transcription failed');}} Explanation: A readable stream is created from the clipped file. The Whisper API is called with model: "whisper-1" . The resulting text is returned to the route handler. This service can be reused anywhere in your app that needs transcription.

To transcribe just part of a file, we need to extract a time-based audio segment. This is done using ffmpeg, a powerful command-line tool for media processing.

TypeScript

// src/routes/transcribe.ts
exec(`ffmpeg -y -ss ${start} -t ${duration} -i "${fullPath}" -ac 1 -ar 16000 "${clippedPath}"`, async (err) => {
  if (err) return res.status(500).json({ error: 'Clipping failed' });
  const transcription = await transcribe(clippedPath);
  res.json({ transcription });
});

Explanation:

-ss ${start}: Start time (in seconds)
-t ${duration}: How long the clip should last
-ac 1: Force mono audio (required by Whisper)
-ar 16000: Set sample rate to 16kHz (also required by Whisper)
-i "${fullPath}": Input file path
Output is saved to a .clip.mp3 file

This command ensures the audio segment is formatted properly before sending it to Whisper.

Let’s look at the full backend route for segment-based transcription.

TypeScript

// src/routes/transcribe.ts
router.post('/', async (req, res) => {
  // Extract input values from request body
  const { filePath, start = 0, duration } = req.body;

  // Validate presence of required fields
  if (!filePath || typeof duration !== 'number') {
    return res.status(400).json({ error: 'Missing filePath or duration' });
  }

  // Resolve the full absolute path to the input audio file
  const fullPath = path.resolve(process.cwd(), filePath);

  // Check if the file actually exists on disk
  if (!fs.existsSync(fullPath)) {
    return res.status(404).json({ error: 'File not found' });
  }

  // Define output path for the clipped segment (same as input with .clip.mp3 extension)
  const clippedPath = fullPath + `.clip.mp3`;

  try {
    // Run ffmpeg command to extract the desired segment
    await new Promise((resolve, reject) => {
      exec(
        // Clip using start and duration values, convert to mono (1 channel) and 16kHz sample rate
        `ffmpeg -y -ss ${start} -t ${duration} -i "${fullPath}" -ac 1 -ar 16000 "${clippedPath}"`,
        (err) => (err ? reject(err) : resolve())
      );
    });

    // Transcribe the clipped segment using Whisper API
    const transcription = await transcribe(clippedPath);

    res.json({ transcription });

  } catch (err) {
    console.error('[Transcribe] ❌', err);
    res.status(500).json({ error: 'Transcription failed' });
  }
});

This endpoint does three things:

Validates the request (ensures file exists and duration is valid)
Clips the audio file using ffmpeg
Sends the clipped audio to the Whisper transcription function

TypeScript

// src/services/transcriber.ts
import { OpenAI } from 'openai';
import fs from 'fs';

const client = new OpenAI();

export async function transcribe(filePath: string): Promise<string> {
  try {
    if (!fs.existsSync(filePath)) {
      throw new Error(`File not found: ${filePath}`);
    }

    const stream = fs.createReadStream(filePath);

    const response = await client.audio.transcriptions.create({
      model: "whisper-1",
      file: stream
    });

    return response.text;
  } catch (err) {
    console.error('[Whisper Error]', err);
    throw new Error('Transcription failed');
  }
}