Welcome back! In the previous lesson, you learned how to make your transcriptions more accurate by customizing the language and prompt parameters. Now, let's address a common challenge: what if you need to transcribe a very large audio file, such as a long meeting or a podcast episode?

Transcribing large files all at once can be slow and resource-intensive. Instead, you can break the audio into smaller pieces (chunks), transcribe each chunk separately, and stream the results as soon as they are ready. This approach is called response streaming. It allows you to see parts of the transcription sooner, making the process faster and more interactive.

In this lesson, you will learn how to implement response streaming in Java using the OpenAI GPT-4o Mini model. You will see how to split audio, process each chunk in parallel, and handle the results efficiently.

Before we dive into the implementation, it's important to understand that there are two main approaches to streaming transcription results:

• How it works: Split large audio files into smaller chunks, process them in parallel, and return results as each chunk completes.
• Best for: Very large files (hours long), when you want maximum parallelization, or when you need to process different parts with different parameters.
• Pros: Full control over chunking strategy, can process multiple chunks simultaneously, works with any audio length.
• Cons: Requires manual audio splitting, potential for slight gaps or overlaps between chunks, more complex setup.

• How it works: Send the entire audio file to OpenAI's API with streaming enabled, receive partial transcription results as the model processes the audio sequentially.
• Best for: Medium-sized files where you want real-time partial results without manual chunking.
• Pros: Simpler implementation, no need to split audio, maintains natural flow and context across the entire file.
• Cons: Limited to OpenAI's internal chunking strategy, may be slower for very large files, single-threaded processing.

When to choose each approach:

• Use chunk-based streaming (this lesson) for very large files, when you need maximum control, or when processing multiple hours of audio.
• Use native streaming for most other cases where you want simplicity and don't need custom chunking logic.

This lesson focuses on chunk-based streaming because it gives you more control and better performance for very large files. However, for many use cases, OpenAI's native streaming might be sufficient and simpler to implement.

Before we dive in, let's briefly remind ourselves of what you learned in the last lesson. You saw how to:

• Set the language parameter to tell the model what language to expect.
• Use a prompt to give the model extra context about the audio.

These customizations help improve transcription quality. In this lesson, we will focus on handling large files, but you can still use those parameters when transcribing each chunk.

To stream transcription results, you need a few key components:

• Audio Chunking Tool: This splits your large audio file into smaller, manageable pieces.
• HTTP Client: This sends each chunk to the OpenAI API for transcription using manual HTTP requests.
• Executor Service: This allows you to process multiple chunks at the same time (in parallel), making the process faster.

Let's look at how to set up these components step by step.

First, you need to split your audio file into chunks. Here's how you might do this using a helper method:

// Split the large audio file into 10-minute chunks for parallel processing
List<File> chunks = AudioChunkSplitter.splitAudioBySeconds("large_audio.mp3", 600);

Note: The AudioChunkSplitter utility class and its implementation were covered in detail in the previous courses on audio processing. If you need a refresher on how to split audio files, refer back to those lessons.

• splitAudioBySeconds takes the path to your audio file and the chunk size in seconds (here, 600 seconds = 10 minutes).
• It returns a list of File objects, each representing a chunk of the original audio.

Next, you need to set up the HTTP client and load your API configuration:

// Create HTTP client for making API requests to OpenAI
HttpClient httpClient = HttpClient.newHttpClient();

// Load environment variables securely (API key should never be hardcoded)
Dotenv dotenv = Dotenv.configure().ignoreIfMissing().load();
String apiKey = dotenv.get("OPENAI_API_KEY");
String baseUrl = dotenv.get("OPENAI_BASE_URL");

• This creates an HTTP client for making requests to the OpenAI API.
• Environment variables are loaded for secure API key management.

To process multiple chunks at once, you use an ExecutorService:

// Create a thread pool with 5 threads to process multiple chunks simultaneously
// This allows up to 5 chunks to be transcribed at the same time
ExecutorService executor = Executors.newFixedThreadPool(5);

• This sets up a pool of 5 threads, so up to 5 chunks can be processed at the same time.

Now, let's put it all together to process each chunk and stream the results as soon as they are ready.

You need a method that can send each chunk to the OpenAI API:

private String transcribeAudioChunk(File audioFile) throws Exception {
    // Create unique boundary for multipart form data
    String boundary = "----boundary" + System.currentTimeMillis();
    
    // Build the multipart form data with the audio file
    byte[] formData = buildFormData(audioFile, boundary);
    
    // Construct the HTTP request to OpenAI's transcription endpoint
    HttpRequest request = HttpRequest.newBuilder()
        .uri(URI.create(baseUrl + "/v1/audio/transcriptions"))
        .header("Authorization", "Bearer " + apiKey)  // Authentication
        .header("Content-Type", "multipart/form-data; boundary=" + boundary)
        .POST(HttpRequest.BodyPublishers.ofByteArray(formData))
        .build();
    
    // Send the request and get the response
    HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
    
    // Check if the request was successful
    if (response.statusCode() == 200) {
        // Simple JSON parsing to extract the transcribed text
        // Note: In production, consider using a JSON library like Jackson or Gson
        String responseBody = response.body();
        int textStart = responseBody.indexOf("\"text\":\"") + 8;
        int textEnd = responseBody.indexOf("\"", textStart);
        return responseBody.substring(textStart, textEnd);
    } else {
        // Handle API errors
        throw new RuntimeException("API request failed: " + response.body());
    }
}

This method handles the manual construction of multipart form data and extracts the transcription text from the JSON response.

You want to process each chunk in parallel and handle the result as soon as it's done. Here's how you can do that:

// Process each chunk in parallel and handle results as they arrive
for (File chunk : chunks) {
    CompletableFuture.supplyAsync(() -> {
        try {
            // Transcribe this chunk (runs in background thread)
            return transcribeAudioChunk(chunk);
        } catch (Exception e) {
            // Convert checked exceptions to runtime exceptions for CompletableFuture
            throw new RuntimeException("Transcription failed: " + e.getMessage(), e);
        }
    }, executor).thenAccept(transcription -> {
        // This callback runs as soon as this chunk is transcribed
        // Results stream in as they're ready, not all at once
        System.out.println("\n--- New Chunk Transcribed ---");
        System.out.println(transcription);
    });
}

Let's break this down:

• For each chunk, you use CompletableFuture.supplyAsync to start the transcription in a separate thread.
• When the transcription is ready, thenAccept is called, and you print the result right away.
• This means you don't have to wait for all chunks to finish before seeing results.

To make sure you wait for all chunks to finish before shutting down, you can collect all the futures and wait for them:

// Create futures for all chunks and collect them in an array
CompletableFuture<?>[] futures = chunks.stream()
    .map(chunk -> CompletableFuture.supplyAsync(() -> {
        try {
            // Each chunk gets processed in its own thread
            return transcribeAudioChunk(chunk);
        } catch (Exception e) {
            throw new RuntimeException("Transcription failed: " + e.getMessage(), e);
        }
    }, executor).thenAccept(transcription -> {
        // Print results immediately as each chunk completes
        System.out.println("\n--- New Chunk Transcribed ---");
        System.out.println(transcription);
    }))
    .toArray(CompletableFuture[]::new);

// Wait for all chunks to complete before proceeding
// This ensures the program doesn't exit while transcriptions are still running
CompletableFuture.allOf(futures).join();

• This code starts all chunk transcriptions in parallel.
• As each chunk finishes, its transcription is printed.
• CompletableFuture.allOf(futures).join(); waits for all chunks to finish before moving on.

Example Output:

--- New Chunk Transcribed ---
Transcribed text for large_audio.mp3

--- New Chunk Transcribed ---
Transcribed text for large_audio.mp3

(You would see one output per chunk.)

After processing each chunk, it's important to clean up any temporary files and shut down the executor service to free up resources.

You can add a cleanup step after each chunk is processed:

// Enhanced version with cleanup after each chunk
.thenAccept(transcription -> {
    // Print the transcription result
    System.out.println("\n--- New Chunk Transcribed ---");
    System.out.println(transcription);
    
    // Clean up the temporary chunk file to save disk space
    cleanupTempFile(chunk);
})

Where cleanupTempFile removes any temporary files:

private void cleanupTempFile(File file) {
    try {
        // Only delete files that are temporary chunks (safety check)
        if (file.exists() && file.getName().contains("_chunk")) {
            Files.delete(file.toPath());
            System.out.println("Cleaned up: " + file.getName());
        }
    } catch (IOException e) {
        // Log cleanup failures but don't stop processing
        System.err.println("Failed to cleanup file: " + file.getName());
    }
}

Once all work is done, shut down the executor:

// Properly shut down the thread pool to release system resources
// This is important to prevent resource leaks
executor.shutdown();

• This ensures all threads are closed and resources are released.

In this lesson, you learned how to stream transcription results for large audio files by:

• Splitting audio into chunks using manual file operations.
• Processing each chunk in parallel using HTTP client requests.
• Streaming each chunk's transcription as soon as it's ready.
• Cleaning up resources after processing.

This approach helps you get faster feedback and makes it easier to handle long recordings. In the next practice exercises, you'll get hands-on experience with streaming transcription and see how it works in real scenarios. Good luck!