Secure File Operations

Introduction

Welcome to the third lesson of the "Secure Data Handling and Integrity in FastAPI" course! In the previous lessons, we explored the fundamentals of data integrity and file checksum verification. Now, we will delve into secure file operations, a crucial aspect of web application security. This lesson will focus on protecting files from unauthorized access and ensuring their integrity. By the end of this lesson, you'll be equipped to implement secure file upload and download endpoints in a FastAPI application, enhancing the security of your applications. Let's get started! 🚀

Understanding Secure File Handling

Previously, we discussed secure file handling within the application. However, files uploaded to or downloaded from an application also require secure management to prevent unauthorized access and ensure data integrity. Improper file handling can lead to vulnerabilities, such as unauthorized file access or malicious file uploads. By understanding secure file handling, you can protect sensitive data and maintain the trustworthiness of your application. In this lesson, we'll explore techniques to secure file storage and transfer, ensuring that your application remains robust against potential threats.

Understanding the Risks

To understand the importance of secure file handling, it's crucial to recognize the potential risks associated with malicious file uploads. Execute Arbitrary Code: Malicious files can contain scripts or code that, if executed by the server, could lead to unauthorized access or control over the server. Data Breach: Uploaded files may be designed to access, steal, or manipulate sensitive data stored on the server, leading to data breaches. Spread Malware: Files can contain malware that infects the server or other connected systems, potentially spreading to users who download the file. Deface Websites: Attackers can upload files that alter the appearance or content of a website, damaging its reputation. Denial of Service (DoS): Malicious files can be crafted to consume excessive server resources, leading to service disruptions. Privilege Escalation: Attackers can exploit vulnerabilities to gain higher-level access, allowing them to perform actions beyond their original permissions. Backdoor Installation: Malicious files can create backdoors, providing attackers with persistent access to the server for future exploitation.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Implementing Secure File Storage

To protect against file upload vulnerabilities, we need to implement secure file storage. We'll use FastAPI and Python's built-in file handling to configure a secure file storage system. This involves setting up secure directories and ensuring files are stored with appropriate permissions.

Python

import os
import secrets
from pathlib import Path
from fastapi import UploadFile, HTTPException

# Configure upload directory
UPLOAD_DIR = Path("uploads")
UPLOAD_DIR.mkdir(exist_ok=True)

def generate_secure_filename(original_filename: str) -> str:
    """Generate a secure filename to prevent overwriting"""
    ext = Path(original_filename).suffix.lower()
    random_name = secrets.token_hex(16)
    return f"{random_name}{ext}"

# File size limit (5MB)
MAX_FILE_SIZE = 5 * 1024 * 1024
ALLOWED_EXTENSIONS = {'.jpg', '.jpeg', '.png', '.pdf'}

def validate_file(file: UploadFile) -> None:
    """Validate file type and size"""
    if not file.filename:
        raise HTTPException(status_code=400, detail="No file provided")
    
    # Check file extension
    ext = Path(file.filename).suffix.lower()
    if ext not in ALLOWED_EXTENSIONS:
        raise HTTPException(
            status_code=400, 
            detail=f"Invalid file type. Allowed types: {', '.join(ALLOWED_EXTENSIONS)}"
        )

import os
import secrets
from pathlib import Path
from fastapi import UploadFile, HTTPException

# Configure upload directory
UPLOAD_DIR = Path("uploads")
UPLOAD_DIR.mkdir(exist_ok=True)

def generate_secure_filename(original_filename: str) -> str:
    """Generate a secure filename to prevent overwriting"""
    ext = Path(original_filename).suffix.lower()
    random_name = secrets.token_hex(16)
    return f"{random_name}{ext}"

# File size limit (5MB)
MAX_FILE_SIZE = 5 * 1024 * 1024
ALLOWED_EXTENSIONS = {'.jpg', '.jpeg', '.png', '.pdf'}

def validate_file(file: UploadFile) -> None:
    """Validate file type and size"""
    if not file.filename:
        raise HTTPException(status_code=400, detail="No file provided")
    
    # Check file extension
    ext = Path(file.filename).suffix.lower()
    if ext not in ALLOWED_EXTENSIONS:
        raise HTTPException(
            status_code=400, 
            detail=f"Invalid file type. Allowed types: {', '.join(ALLOWED_EXTENSIONS)}"
        )

In this code, we configure secure file storage using Python's pathlib for path handling and secrets for generating random filenames. We specify a directory for uploads and ensure it exists. Filenames are randomized to prevent overwriting and potential security issues. This setup forms the foundation for secure file storage in your application.

Secure File Upload Endpoint

Now, let's implement a secure file upload endpoint in a FastAPI application. We'll use file validation and upload limits to enhance security.

Python

from fastapi import APIRouter, UploadFile, File, HTTPException
import aiofiles

router = APIRouter()

@router.post('/upload')
async def upload_file(file: UploadFile = File(...)):
    """Secure file upload endpoint"""
    # Validate file
    validate_file(file)
    
    try:
        # Generate secure filename
        secure_filename = generate_secure_filename(file.filename)
        file_path = UPLOAD_DIR / secure_filename
        
        # Read and validate file size
        content = await file.read()
        if len(content) > MAX_FILE_SIZE:
            raise HTTPException(
                status_code=400,
                detail=f"File too large. Maximum size: {MAX_FILE_SIZE} bytes"
            )
        
        # Save file securely
        async with aiofiles.open(file_path, 'wb') as f:
            await f.write(content)
        
        # Store file metadata (in a real app, you'd save this to database)
        file_info = {
            'original_name': file.filename,
            'stored_name': secure_filename,
            'content_type': file.content_type,
            'size': len(content)
        }
        
        return {
            'success': True,
            'file_id': secrets.token_hex(8)
        }
        
    except Exception as e:
        # Clean up file if something went wrong
        if file_path.exists():
            file_path.unlink()
        raise HTTPException(status_code=500, detail="Upload failed")

from fastapi import APIRouter, UploadFile, File, HTTPException
import aiofiles

router = APIRouter()

@router.post('/upload')
async def upload_file(file: UploadFile = File(...)):
    """Secure file upload endpoint"""
    # Validate file
    validate_file(file)
    
    try:
        # Generate secure filename
        secure_filename = generate_secure_filename(file.filename)
        file_path = UPLOAD_DIR / secure_filename
        
        # Read and validate file size
        content = await file.read()
        if len(content) > MAX_FILE_SIZE:
            raise HTTPException(
                status_code=400,
                detail=f"File too large. Maximum size: {MAX_FILE_SIZE} bytes"
            )
        
        # Save file securely
        async with aiofiles.open(file_path, 'wb') as f:
            await f.write(content)
        
        # Store file metadata (in a real app, you'd save this to database)
        file_info = {
            'original_name': file.filename,
            'stored_name': secure_filename,
            'content_type': file.content_type,
            'size': len(content)
        }
        
        return {
            'success': True,
            'file_id': secrets.token_hex(8)
        }
        
    except Exception as e:
        # Clean up file if something went wrong
        if file_path.exists():
            file_path.unlink()
        raise HTTPException(status_code=500, detail="Upload failed")

In this snippet, we configure the upload endpoint to accept only specific file types and limit the file size to 5MB. This prevents unauthorized file types and excessively large files from being uploaded, enhancing the security of the application.

Secure File Download Endpoint

Next, let's implement a secure file download endpoint. This ensures that files are downloaded securely and with the correct metadata.

Python

from fastapi.responses import FileResponse

@router.get('/download/{file_id}')
async def download_file(file_id: str):
    """Secure file download endpoint"""
    # Get file info from database (not implemented)
    # In a real app, you would look up the file by ID
    file_info = {
        'stored_name': '123abc456def.jpg',
        'original_name': 'profile.jpg',
        'content_type': 'image/jpeg'
    }
    
    file_path = UPLOAD_DIR / file_info['stored_name']
    
    # Verify file exists
    if not file_path.exists():
        raise HTTPException(status_code=404, detail="File not found")
    
    # Return file with proper headers
    return FileResponse(
        path=file_path,
        filename=file_info['original_name'],
        media_type=file_info['content_type']
    )

from fastapi.responses import FileResponse

@router.get('/download/{file_id}')
async def download_file(file_id: str):
    """Secure file download endpoint"""
    # Get file info from database (not implemented)
    # In a real app, you would look up the file by ID
    file_info = {
        'stored_name': '123abc456def.jpg',
        'original_name': 'profile.jpg',
        'content_type': 'image/jpeg'
    }
    
    file_path = UPLOAD_DIR / file_info['stored_name']
    
    # Verify file exists
    if not file_path.exists():
        raise HTTPException(status_code=404, detail="File not found")
    
    # Return file with proper headers
    return FileResponse(
        path=file_path,
        filename=file_info['original_name'],
        media_type=file_info['content_type']
    )

This code snippet demonstrates a secure file download endpoint. It verifies the existence of the requested file and sets appropriate headers before streaming the file to the client. This ensures that files are downloaded securely and with the correct metadata.

Secure File Deletion Code Example

Python

import os
from pathlib import Path

def secure_delete(file_path: Path) -> None:
    """Securely delete a file by overwriting with random data"""
    if not file_path.exists():
        print("File not found")
        return
    
    try:
        # Get file size
        file_size = file_path.stat().st_size
        
        # Overwrite with random data
        with open(file_path, 'r+b') as f:
            random_data = os.urandom(file_size)
            f.seek(0)
            f.write(random_data)
            f.flush()
            os.fsync(f.fileno())
        
        # Delete the file
        file_path.unlink()
        
        print("File securely deleted")
        
    except Exception as error:
        print(f"Error during secure file deletion: {error}")

# Example usage
file_path = Path("uploads/fileToDelete.txt")
secure_delete(file_path)

import os
from pathlib import Path

def secure_delete(file_path: Path) -> None:
    """Securely delete a file by overwriting with random data"""
    if not file_path.exists():
        print("File not found")
        return
    
    try:
        # Get file size
        file_size = file_path.stat().st_size
        
        # Overwrite with random data
        with open(file_path, 'r+b') as f:
            random_data = os.urandom(file_size)
            f.seek(0)
            f.write(random_data)
            f.flush()
            os.fsync(f.fileno())
        
        # Delete the file
        file_path.unlink()
        
        print("File securely deleted")
        
    except Exception as error:
        print(f"Error during secure file deletion: {error}")

# Example usage
file_path = Path("uploads/fileToDelete.txt")
secure_delete(file_path)

In this code snippet, the secure_delete function opens the file for writing, overwrites its contents with random data, and then deletes the file. This approach ensures that the file is securely deleted in a manner consistent with real-world applications.

Note that overwriting a file on modern SSDs or cloud-hosted storage may not always guarantee physical erasure due to wear leveling and storage abstractions. When using such environments, rely on encrypted storage and delete the encryption key instead of just overwriting the file.

Introduction

Understanding Secure File Handling

Understanding the Risks

Join the 1M+ learners on CodeSignal

Scenarios Where Uploaded Files Can Pose Risks

Implementing Secure File Storage

Secure File Upload Endpoint

Secure File Download Endpoint

Implementing Secure File Deletion

Secure File Deletion Code Example

Mitigating Common Vulnerabilities

Conclusion and Next Steps