Streaming Microphone Input with the Web Audio API

In this lesson, you'll learn how to record audio from your browser in real time and use the OpenAI Whisper API to transcribe it. We'll walk through the full logic from initiating the recording in the browser to returning the transcription from the backend.

This lesson will guide you through:

• Setting up the browser to record microphone audio in real time.
• Uploading recorded audio files from the frontend to the backend.
• Processing those files using the OpenAI Whisper API.
• Displaying the transcription result in the browser.
• Cleaning up files after use to manage server storage efficiently.

Each of these steps contributes to building a fluid real-time transcription interface directly from the browser.

We'll start in public/app.js, which handles browser audio recording and the UI.

async function startRecording() {
  try {
    const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
    mediaRecorder = new MediaRecorder(stream);
    audioChunks = [];

    mediaRecorder.ondataavailable = (e) => audioChunks.push(e.data);
    mediaRecorder.start();

    resultPanel.classList.add('hidden');
    textArea.textContent = '';

    startBtn.textContent = 'Recording...';
    stopBtn.disabled = false;
    console.log('Recording started...');
  } catch (err) {
    alert('Microphone access denied or unavailable.');
    console.error(err);
  }
}

• getUserMedia({ audio: true }): Requests access to the user's microphone using the WebRTC API.
• MediaRecorder: This browser API lets you capture media streams such as audio or video; here, we use it specifically to record audio.
• mediaRecorder.ondataavailable: This event is triggered periodically during recording. We push each audio chunk into audioChunks, which is an array that will hold all segments of the final recording.
• UI updates ensure a clean user experience:
  • textArea.textContent = '' clears any previous transcriptions.
  • resultPanel.classList.add('hidden') hides the results panel so users don’t see stale output.
  • Button states are updated to reflect that recording has started.

function stopRecordingAndTranscribe() {
  if (mediaRecorder && mediaRecorder.state === 'recording') {
    mediaRecorder.stop();

    mediaRecorder.onstop = async () => {
      startBtn.textContent = 'Start Recording';
      stopBtn.disabled = true;

      const blob = new Blob(audioChunks, { type: 'audio/webm' });

• When the user decides to stop recording, mediaRecorder.stop() halts the recording session.
• The onstop event triggers once the recording has fully stopped, allowing us to safely process the audio data.
• We create a Blob object from the collected chunks. This blob acts like a file in memory, with the type set to 'audio/webm' so that we retain format compatibility when sending to the server.

A Blob (Binary Large Object) represents raw immutable binary data. In this context, the Blob acts like an in-memory file containing all the audio chunks we recorded. The second argument ({ type: 'audio/webm' }) defines the MIME type, helping both the browser and the server interpret the content format. Blobs are commonly used for handling file-like objects, such as media streams or generated documents, in frontend JavaScript.

FormData is a built-in web API that mimics a form submission and is used to build a set of key/value pairs for HTTP requests—especially useful when sending files. In this case, we’re appending the Blob under the key 'audio', which acts like an <input type="file" name="audio"> in an HTML form. The FormData object ensures the request uses the multipart/form-data content type automatically, which is required when sending binary files in HTTP POST requests.

      try {
        const formData = new FormData();
        formData.append('audio', blob, 'recording.webm');

        const uploadRes = await fetch('/recordings/upload', {
          method: 'POST',
          body: formData,
        });
        const { path } = await uploadRes.json();

• The audio blob is wrapped in a FormData object, allowing us to send it via multipart/form-data—a format suitable for file uploads.
• We send a POST request to /recordings/upload, where the server will store our recording.
• The response includes the server path to the uploaded file, which we will use for transcription.

        resultPanel.classList.remove('hidden');
        textArea.textContent = '⏳ Transcribing...';

        const transcriptRes = await fetch('/transcribe', {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify({ filePath: path }),
        });
        const { transcription } = await transcriptRes.json();
        textArea.textContent = transcription || '(No text returned)';
      } catch (err) {
        console.error('Transcription failed:', err);
        alert('Transcription failed. Check console for details.');
      }
    };
  }
}

• Once the file has been uploaded, we call /transcribe with the file path.
• The backend then processes the file using Whisper API and returns the text.
• The transcription (or a fallback message) is displayed in the text area.
• Errors during the upload or transcription phase are caught and presented via an alert, helping the user debug issues.