Implementing Response Streaming for Large Audio Transcription

Introduction: Why Stream Transcription Results?

Welcome back! In the previous lesson, you learned how to make your transcriptions more accurate by customizing the language and prompt parameters. Now, let's address a common challenge: what if you need to transcribe a very large audio file, such as a long meeting or a podcast episode? Transcribing large files all at once can be slow and resource-intensive. Instead, you can break the audio into smaller pieces (chunks), transcribe each chunk separately, and stream the results as soon as they are ready. This approach is called response streaming . It allows you to see parts of the transcription sooner, making the process faster and more interactive. In this lesson, you will learn how to implement response streaming in Java using the OpenAI GPT-4o Mini model. You will see how to split audio, process each chunk in parallel, and handle the results efficiently.

Understanding Different Streaming Approaches

Before we dive into the implementation, it's important to understand that there are two main approaches to streaming transcription results:

1. Chunk-Based Streaming (This Lesson's Approach)

How it works: Split large audio files into smaller chunks, process them in parallel, and return results as each chunk completes. Best for: Very large files (hours long), when you want maximum parallelization, or when you need to process different parts with different parameters. Pros: Full control over chunking strategy, can process multiple chunks simultaneously, works with any audio length. Cons: Requires manual audio splitting, potential for slight gaps or overlaps between chunks, more complex setup.

2. OpenAI Native Streaming (SSE-Style)

How it works: Send the entire audio file to OpenAI's API with streaming enabled, receive partial transcription results as the model processes the audio sequentially. Best for: Medium-sized files where you want real-time partial results without manual chunking. Pros: Simpler implementation, no need to split audio, maintains natural flow and context across the entire file. Cons: Limited to OpenAI's internal chunking strategy, may be slower for very large files, single-threaded processing. When to choose each approach: Use chunk-based streaming (this lesson) for very large files, when you need maximum control, or when processing multiple hours of audio. Use native streaming for most other cases where you want simplicity and don't need custom chunking logic. This lesson focuses on chunk-based streaming because it gives you more control and better performance for very large files. However, for many use cases, OpenAI's native streaming might be sufficient and simpler to implement.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Java

// Split the large audio file into 10-minute chunks for parallel processing
List<File> chunks = AudioChunkSplitter.splitAudioBySeconds("large_audio.mp3", 600);

// Split the large audio file into 10-minute chunks for parallel processing
List<File> chunks = AudioChunkSplitter.splitAudioBySeconds("large_audio.mp3", 600);

Java

// Create HTTP client for making API requests to OpenAI
HttpClient httpClient = HttpClient.newHttpClient();

// Load environment variables securely (API key should never be hardcoded)
Dotenv dotenv = Dotenv.configure().ignoreIfMissing().load();
String apiKey = dotenv.get("OPENAI_API_KEY");
String baseUrl = dotenv.get("OPENAI_BASE_URL");

// Create HTTP client for making API requests to OpenAI
HttpClient httpClient = HttpClient.newHttpClient();

// Load environment variables securely (API key should never be hardcoded)
Dotenv dotenv = Dotenv.configure().ignoreIfMissing().load();
String apiKey = dotenv.get("OPENAI_API_KEY");
String baseUrl = dotenv.get("OPENAI_BASE_URL");

Java

// Create a thread pool with 5 threads to process multiple chunks simultaneously
// This allows up to 5 chunks to be transcribed at the same time
ExecutorService executor = Executors.newFixedThreadPool(5);

// Create a thread pool with 5 threads to process multiple chunks simultaneously
// This allows up to 5 chunks to be transcribed at the same time
ExecutorService executor = Executors.newFixedThreadPool(5);

Java

private String transcribeAudioChunk(File audioFile) throws Exception {
    // Create unique boundary for multipart form data
    String boundary = "----boundary" + System.currentTimeMillis();
    
    // Build the multipart form data with the audio file
    byte[] formData = buildFormData(audioFile, boundary);
    
    // Construct the HTTP request to OpenAI's transcription endpoint
    HttpRequest request = HttpRequest.newBuilder()
        .uri(URI.create(baseUrl + "/v1/audio/transcriptions"))
        .header("Authorization", "Bearer " + apiKey)  // Authentication
        .header("Content-Type", "multipart/form-data; boundary=" + boundary)
        .POST(HttpRequest.BodyPublishers.ofByteArray(formData))
        .build();
    
    // Send the request and get the response
    HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
    
    // Check if the request was successful
    if (response.statusCode() == 200) {
        // Simple JSON parsing to extract the transcribed text
        // Note: In production, consider using a JSON library like Jackson or Gson
        String responseBody = response.body();
        int textStart = responseBody.indexOf("\"text\":\"") + 8;
        int textEnd = responseBody.indexOf("\"", textStart);
        return responseBody.substring(textStart, textEnd);
    } else {
        // Handle API errors
        throw new RuntimeException("API request failed: " + response.body());
    }
}

private String transcribeAudioChunk(File audioFile) throws Exception {
    // Create unique boundary for multipart form data
    String boundary = "----boundary" + System.currentTimeMillis();
    
    // Build the multipart form data with the audio file
    byte[] formData = buildFormData(audioFile, boundary);
    
    // Construct the HTTP request to OpenAI's transcription endpoint
    HttpRequest request = HttpRequest.newBuilder()
        .uri(URI.create(baseUrl + "/v1/audio/transcriptions"))
        .header("Authorization", "Bearer " + apiKey)  // Authentication
        .header("Content-Type", "multipart/form-data; boundary=" + boundary)
        .POST(HttpRequest.BodyPublishers.ofByteArray(formData))
        .build();
    
    // Send the request and get the response
    HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
    
    // Check if the request was successful
    if (response.statusCode() == 200) {
        // Simple JSON parsing to extract the transcribed text
        // Note: In production, consider using a JSON library like Jackson or Gson
        String responseBody = response.body();
        int textStart = responseBody.indexOf("\"text\":\"") + 8;
        int textEnd = responseBody.indexOf("\"", textStart);
        return responseBody.substring(textStart, textEnd);
    } else {
        // Handle API errors
        throw new RuntimeException("API request failed: " + response.body());
    }
}

Java

// Process each chunk in parallel and handle results as they arrive
for (File chunk : chunks) {
    CompletableFuture.supplyAsync(() -> {
        try {
            // Transcribe this chunk (runs in background thread)
            return transcribeAudioChunk(chunk);
        } catch (Exception e) {
            // Convert checked exceptions to runtime exceptions for CompletableFuture
            throw new RuntimeException("Transcription failed: " + e.getMessage(), e);
        }
    }, executor).thenAccept(transcription -> {
        // This callback runs as soon as this chunk is transcribed
        // Results stream in as they're ready, not all at once
        System.out.println("\n--- New Chunk Transcribed ---");
        System.out.println(transcription);
    });
}

// Process each chunk in parallel and handle results as they arrive
for (File chunk : chunks) {
    CompletableFuture.supplyAsync(() -> {
        try {
            // Transcribe this chunk (runs in background thread)
            return transcribeAudioChunk(chunk);
        } catch (Exception e) {
            // Convert checked exceptions to runtime exceptions for CompletableFuture
            throw new RuntimeException("Transcription failed: " + e.getMessage(), e);
        }
    }, executor).thenAccept(transcription -> {
        // This callback runs as soon as this chunk is transcribed
        // Results stream in as they're ready, not all at once
        System.out.println("\n--- New Chunk Transcribed ---");
        System.out.println(transcription);
    });
}

Java

// Create futures for all chunks and collect them in an array
CompletableFuture<?>[] futures = chunks.stream()
    .map(chunk -> CompletableFuture.supplyAsync(() -> {
        try {
            // Each chunk gets processed in its own thread
            return transcribeAudioChunk(chunk);
        } catch (Exception e) {
            throw new RuntimeException("Transcription failed: " + e.getMessage(), e);
        }
    }, executor).thenAccept(transcription -> {
        // Print results immediately as each chunk completes
        System.out.println("\n--- New Chunk Transcribed ---");
        System.out.println(transcription);
    }))
    .toArray(CompletableFuture[]::new);

// Wait for all chunks to complete before proceeding
// This ensures the program doesn't exit while transcriptions are still running
CompletableFuture.allOf(futures).join();

// Create futures for all chunks and collect them in an array
CompletableFuture<?>[] futures = chunks.stream()
    .map(chunk -> CompletableFuture.supplyAsync(() -> {
        try {
            // Each chunk gets processed in its own thread
            return transcribeAudioChunk(chunk);
        } catch (Exception e) {
            throw new RuntimeException("Transcription failed: " + e.getMessage(), e);
        }
    }, executor).thenAccept(transcription -> {
        // Print results immediately as each chunk completes
        System.out.println("\n--- New Chunk Transcribed ---");
        System.out.println(transcription);
    }))
    .toArray(CompletableFuture[]::new);

// Wait for all chunks to complete before proceeding
// This ensures the program doesn't exit while transcriptions are still running
CompletableFuture.allOf(futures).join();

--- New Chunk Transcribed ---
Transcribed text for large_audio.mp3

--- New Chunk Transcribed ---
Transcribed text for large_audio.mp3

--- New Chunk Transcribed ---
Transcribed text for large_audio.mp3

--- New Chunk Transcribed ---
Transcribed text for large_audio.mp3

Java

// Enhanced version with cleanup after each chunk
.thenAccept(transcription -> {
    // Print the transcription result
    System.out.println("\n--- New Chunk Transcribed ---");
    System.out.println(transcription);
    
    // Clean up the temporary chunk file to save disk space
    cleanupTempFile(chunk);
})

// Enhanced version with cleanup after each chunk
.thenAccept(transcription -> {
    // Print the transcription result
    System.out.println("\n--- New Chunk Transcribed ---");
    System.out.println(transcription);
    
    // Clean up the temporary chunk file to save disk space
    cleanupTempFile(chunk);
})

Java

private void cleanupTempFile(File file) {
    try {
        // Only delete files that are temporary chunks (safety check)
        if (file.exists() && file.getName().contains("_chunk")) {
            Files.delete(file.toPath());
            System.out.println("Cleaned up: " + file.getName());
        }
    } catch (IOException e) {
        // Log cleanup failures but don't stop processing
        System.err.println("Failed to cleanup file: " + file.getName());
    }
}

private void cleanupTempFile(File file) {
    try {
        // Only delete files that are temporary chunks (safety check)
        if (file.exists() && file.getName().contains("_chunk")) {
            Files.delete(file.toPath());
            System.out.println("Cleaned up: " + file.getName());
        }
    } catch (IOException e) {
        // Log cleanup failures but don't stop processing
        System.err.println("Failed to cleanup file: " + file.getName());
    }
}

Java

// Properly shut down the thread pool to release system resources
// This is important to prevent resource leaks
executor.shutdown();

// Properly shut down the thread pool to release system resources
// This is important to prevent resource leaks
executor.shutdown();

Java

// Split the large audio file into 10-minute chunks for parallel processing
List<File> chunks = AudioChunkSplitter.splitAudioBySeconds("large_audio.mp3", 600);

Java

// Create HTTP client for making API requests to OpenAI
HttpClient httpClient = HttpClient.newHttpClient();

// Load environment variables securely (API key should never be hardcoded)
Dotenv dotenv = Dotenv.configure().ignoreIfMissing().load();
String apiKey = dotenv.get("OPENAI_API_KEY");
String baseUrl = dotenv.get("OPENAI_BASE_URL");

Java

// Create a thread pool with 5 threads to process multiple chunks simultaneously
// This allows up to 5 chunks to be transcribed at the same time
ExecutorService executor = Executors.newFixedThreadPool(5);

Java

private String transcribeAudioChunk(File audioFile) throws Exception {
    // Create unique boundary for multipart form data
    String boundary = "----boundary" + System.currentTimeMillis();
    
    // Build the multipart form data with the audio file
    byte[] formData = buildFormData(audioFile, boundary);
    
    // Construct the HTTP request to OpenAI's transcription endpoint
    HttpRequest request = HttpRequest.newBuilder()
        .uri(URI.create(baseUrl + "/v1/audio/transcriptions"))
        .header("Authorization", "Bearer " + apiKey)  // Authentication
        .header("Content-Type", "multipart/form-data; boundary=" + boundary)
        .POST(HttpRequest.BodyPublishers.ofByteArray(formData))
        .build();
    
    // Send the request and get the response
    HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
    
    // Check if the request was successful
    if (response.statusCode() == 200) {
        // Simple JSON parsing to extract the transcribed text
        // Note: In production, consider using a JSON library like Jackson or Gson
        String responseBody = response.body();
        int textStart = responseBody.indexOf("\"text\":\"") + 8;
        int textEnd = responseBody.indexOf("\"", textStart);
        return responseBody.substring(textStart, textEnd);
    } else {
        // Handle API errors
        throw new RuntimeException("API request failed: " + response.body());
    }
}

Java

// Process each chunk in parallel and handle results as they arrive
for (File chunk : chunks) {
    CompletableFuture.supplyAsync(() -> {
        try {
            // Transcribe this chunk (runs in background thread)
            return transcribeAudioChunk(chunk);
        } catch (Exception e) {
            // Convert checked exceptions to runtime exceptions for CompletableFuture
            throw new RuntimeException("Transcription failed: " + e.getMessage(), e);
        }
    }, executor).thenAccept(transcription -> {
        // This callback runs as soon as this chunk is transcribed
        // Results stream in as they're ready, not all at once
        System.out.println("\n--- New Chunk Transcribed ---");
        System.out.println(transcription);
    });
}

Java

// Create futures for all chunks and collect them in an array
CompletableFuture<?>[] futures = chunks.stream()
    .map(chunk -> CompletableFuture.supplyAsync(() -> {
        try {
            // Each chunk gets processed in its own thread
            return transcribeAudioChunk(chunk);
        } catch (Exception e) {
            throw new RuntimeException("Transcription failed: " + e.getMessage(), e);
        }
    }, executor).thenAccept(transcription -> {
        // Print results immediately as each chunk completes
        System.out.println("\n--- New Chunk Transcribed ---");
        System.out.println(transcription);
    }))
    .toArray(CompletableFuture[]::new);

// Wait for all chunks to complete before proceeding
// This ensures the program doesn't exit while transcriptions are still running
CompletableFuture.allOf(futures).join();

--- New Chunk Transcribed ---
Transcribed text for large_audio.mp3

--- New Chunk Transcribed ---
Transcribed text for large_audio.mp3

Java

// Enhanced version with cleanup after each chunk
.thenAccept(transcription -> {
    // Print the transcription result
    System.out.println("\n--- New Chunk Transcribed ---");
    System.out.println(transcription);
    
    // Clean up the temporary chunk file to save disk space
    cleanupTempFile(chunk);
})

Java

private void cleanupTempFile(File file) {
    try {
        // Only delete files that are temporary chunks (safety check)
        if (file.exists() && file.getName().contains("_chunk")) {
            Files.delete(file.toPath());
            System.out.println("Cleaned up: " + file.getName());
        }
    } catch (IOException e) {
        // Log cleanup failures but don't stop processing
        System.err.println("Failed to cleanup file: " + file.getName());
    }
}

Java

// Properly shut down the thread pool to release system resources
// This is important to prevent resource leaks
executor.shutdown();

Introduction: Why Stream Transcription Results?

Understanding Different Streaming Approaches

1. Chunk-Based Streaming (This Lesson's Approach)

2. OpenAI Native Streaming (SSE-Style)

Join the 1M+ learners on CodeSignal

Quick Recall: Customizing Transcription Requests

Setting Up for Streaming Transcription

1. Splitting the Audio File

2. Setting Up the HTTP Client and Configuration

3. Preparing for Parallel Processing

Streaming Logic: Processing and Returning Chunks

1. Creating the Transcription Method

2. Submitting Chunks for Transcription

3. Handling All Chunks Together

Cleanup and Resource Management

1. Cleaning Up Temporary Files

2. Shutting Down the Executor

Summary and What's Next