Implementing LinkedIn Video Downloader with TypeScript

Welcome back to our journey in video scraping! In previous lessons, you've learned how to transcribe videos using external APIs and download videos from public Google Drive links. In this lesson, we'll take things further by downloading videos from LinkedIn using TypeScript and Node.js. This approach simplifies accessing video content across multiple platforms.

In this lesson, you will:

• Identify and validate a range of LinkedIn URLs.
• Discover how to use TypeScript and Node.js to download videos from LinkedIn.
• Manage temporary files and address potential legal concerns when downloading videos.

Our objective is to leverage TypeScript and Node.js to download videos from LinkedIn. The key is recognizing valid LinkedIn URLs and downloading videos efficiently.

LinkedIn URLs can be in formats such as:

• Full length: https://www.linkedin.com/feed/update/urn:li:activity:VIDEO_ID
• Post based: https://www.linkedin.com/posts/USERNAME_activity-VIDEO_ID

Understanding these structures is crucial for initiating the download process.

We'll start by verifying if a URL belongs to LinkedIn using TypeScript's URL class:

static isLinkedInUrl(urlStr: string): boolean {
  try {
    const parsed = new URL(urlStr);
    const validPaths = [
      '/feed/update/urn:li:activity:',  // Existing format
      '/posts/'  // New format to support
    ];
    return parsed.hostname.includes('linkedin.com') && 
      validPaths.some(path => parsed.pathname.includes(path));
  } catch (error) {
    return false;
  }
}

This method checks for linkedin.com in the URL's hostname and confirms if a recognizable path is present, ensuring accurate URL validation.

Once the URL is validated, we proceed with the download. For LinkedIn videos, we'll use a command-line tool called yt-dlp which is powerful for downloading videos from various platforms:

import * as fs from 'fs';
import * as path from 'path';
import * as url from 'url';
import * as util from 'util';
import { exec as execCallback } from 'child_process';
import * as os from 'os';

const exec = util.promisify(execCallback);

export class LinkedInService {
  // Previous isLinkedInUrl method...

  static async downloadVideo(urlStr: string): Promise<string> {
    console.log("Downloading LinkedIn video...");
    
    try {
      // Create temporary directory
      const tempDir = path.join(os.tmpdir(), 'media-transcriber');
      fs.mkdirSync(tempDir, { recursive: true });
      
      // Generate a temporary filename
      const timestamp = Date.now();
      const outputTemplate = path.join(tempDir, `linkedin_${timestamp}.%(ext)s`);
      
      // Create the yt-dlp command
      const command = `yt-dlp "${urlStr}" -o "${outputTemplate}" -f "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best" --merge-output-format mp4 --no-playlist`;
      
      console.log(`Executing command: ${command}`);
      
      // Execute the command
      const { stdout, stderr } = await exec(command);
      
      if (stderr && !stderr.includes('has already been downloaded')) {
        console.error(`yt-dlp stderr: ${stderr}`);
      }
      
      // Find the downloaded file (should match our template but with actual extension)
      const files = fs.readdirSync(tempDir)
        .filter(file => file.startsWith(`linkedin_${timestamp}`) && file.endsWith('.mp4'));
      
      if (files.length === 0) {
        throw new Error("No file downloaded. This may indicate that yt-dlp is not installed or couldn't access the video.");
      }
      
      const outputPath = path.join(tempDir, files[0]);
      
      // Verify the download
      if (fs.statSync(outputPath).size === 0) {
        fs.unlinkSync(outputPath);
        throw new Error("Downloaded file is empty");
      }
      
      console.log(`Successfully downloaded to: ${outputPath}`);
      return outputPath;
    } catch (error) {
      console.error(`Error downloading video: ${error}`);
      throw new Error(
        "Failed to download LinkedIn video. Make sure:\n" +
        "1. yt-dlp is installed on your system (https://github.com/yt-dlp/yt-dlp)\n" +
        "2. The URL is correct and the video is accessible\n" +
        "3. Original error: " + error.message
      );
    }
  }
}

Here's how it works:

• First, we create a temporary directory using Node.js's file system functions.
• We generate a unique output template using a timestamp to avoid file conflicts.
• We build a command that uses yt-dlp with specific options:
  • -o specifies the output filename pattern
  • -f selects the best available video and audio formats
  • --merge-output-format mp4 ensures we get an MP4 file
  • --no-playlist prevents downloading entire playlists if the URL is part of one
• We execute the command using Node's child_process.exec function (promisified for async/await)
• After downloading, we verify the file exists and has content
• Finally, we return the path to the downloaded video file

This approach is more robust than using TypeScript libraries like axios that we used before for several reasons:

1. yt-dlp is actively maintained to adapt to platform changes
2. It handles authentication challenges and browser emulation
3. It automatically selects the best quality available

In our application, this LinkedIn downloader fits into our URL processing route. When a user submits a URL, our service detects whether it's a LinkedIn URL and processes it accordingly:

// From url.ts route handler
if (LinkedInService.isLinkedInUrl(url)) {
  videoPath = await LinkedInService.downloadVideo(url);
} else if (GoogleDriveService.isGoogleDriveUrl(url)) {
  videoPath = await GoogleDriveService.downloadFile(url);
} else {
  return res.status(400).json({ error: 'Unsupported URL format. Please use LinkedIn or Google Drive URLs.' });
}

Once downloaded, the video is saved to a session-specific directory, and the path is returned to the frontend for playback and transcription.

Mastering LinkedIn video downloads with TypeScript enables the collection of educational videos, supports offline access, and aids in backing up personal content. Always be aware of potential legal issues, ensuring compliance with terms of service and copyright laws.

Now that you understand the downloader's potential, take the upcoming practice section as an opportunity to solidify your knowledge with hands-on tasks.