---
title: "How to design a resumable, chunked file upload system for web applications?"  
description: "How to design a resumable, chunked file upload system for web applications?"  
author: "Hemant Patel"  
published: 2026-08-23  
updated: 2026-08-25  
canonical: https://answers.mindstick.com/qa/117106/how-to-design-a-resumable-chunked-file-upload-system-for-web-applications  
category: "Cloud Storage"  
tags: ["AWS S3", "File System", "Blob Storage", "System Design"]  
reading_time: 4 minutes  

---

# How to design a resumable, chunked file upload system for web applications?

## Overview

Uploading large files (e.g., several gigabytes) directly through standard HTTP POST requests is prone to network timeouts and connection drops. A robust system design requires chunking files on the browser client, uploading parts concurrently, and stitching them on cloud blob storage.

### Key Considerations

- **Resumability:** Record uploaded chunk hashes in a database so failed uploads resume from the last successful chunk.
- **Direct Cloud Uploads:** Bypass application web servers by generating presigned S3 URLs to save web tier bandwidth.
- **Deduplication:** Hash file content to avoid storing duplicate files.

### Frontend Chunking JavaScript Implementation

Below is a JavaScript snippet demonstrating client-side file slicing into distinct byte-array chunks:

```javascript
async function uploadFileInChunks(file, chunkSize = 5 * 1024 * 1024) {
    const totalChunks = Math.ceil(file.size / chunkSize);
    for (let index = 0; index < totalChunks; index++) {
        const start = index * chunkSize;
        const end = Math.min(file.size, start + chunkSize);
        const chunk = file.slice(start, end);

        const formData = new FormData();
        formData.append('chunkIndex', index);
        formData.append('data', chunk);

        await fetch('/api/upload-chunk', {
            method: 'POST',
            body: formData
        });
    }
}
```

## Answers

### Answer by Ravi Vishwakarma

Uploading large files over HTTP poses significant challenges, including server timeouts, high memory usage, and dropped network connections. Designing a **resumable, chunked file upload system** solves these issues by breaking a file into smaller segments (chunks), uploading them independently, and allowing interrupted uploads to resume seamlessly from the last successful chunk.

## Core Concepts of Chunked Uploads

A chunked file upload architecture relies on four main pillars:

- **File Slicing:** Using modern browser APIs like `Blob.prototype.slice()` to split files into fixed-size chunks on the client side.
- **Unique File Identification:** Generating a unique identifier (e.g., using file metadata or a hash like MD5/SHA-256) to track upload progress across sessions.
- **State Tracking:** Maintaining metadata on the server to record which chunks have been received.
- **Chunk Merging:** Stitching all individual chunk files back into the original file once all parts are uploaded.

## System Workflow

### 1. Upload Initialization

Before sending file data, the frontend requests an upload session from the server by sending file metadata (name, size, mime type, hash). The server returns an `uploadId` and a list of already uploaded chunk indexes if an upload session already exists.

### 2. Client-Side Chunked Uploading

The client iterates over the file in chunk sizes (e.g., 5MB each) and uploads each chunk along with metadata such as `uploadId`, `chunkIndex`, and `totalChunks`.

```javascript
async function uploadFileInChunks(file) {
    const CHUNK_SIZE = 5 * 1024 * 1024; // 5 MB
    const totalChunks = Math.ceil(file.size / CHUNK_SIZE);
    const fileId = `${file.name}-${file.size}-${file.lastModified}`;

    // Check uploaded chunks from server
    const response = await fetch(`/api/upload/status?fileId=${fileId}`);
    const { uploadedChunks } = await response.json();

    for (let chunkIndex = 0; chunkIndex < totalChunks; chunkIndex++) {
        if (uploadedChunks.includes(chunkIndex)) {
            console.log(`Chunk ${chunkIndex} already uploaded. Skipping...`);
            continue;
        }

        const start = chunkIndex * CHUNK_SIZE;
        const end = Math.min(file.size, start + CHUNK_SIZE);
        const chunk = file.slice(start, end);

        const formData = new FormData();
        formData.append('chunk', chunk);
        formData.append('chunkIndex', chunkIndex);
        formData.append('totalChunks', totalChunks);
        formData.append('fileId', fileId);

        await fetch('/api/upload/chunk', {
            method: 'POST',
            body: formData
        });
    }

    // Notify server to merge chunks
    await fetch('/api/upload/merge', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ fileId, fileName: file.name, totalChunks })
    });
}
```

### 3. Server-Side Chunk Handling & Merging

The server saves individual chunk files into a temporary directory named after the `fileId`. Once all chunks are present, it concatenates them in order into the final file location and removes the temporary files.

```javascript
const fs = require('fs');
const path = require('path');

async function mergeChunks(fileId, fileName, totalChunks) {
    const tempDir = path.join(__dirname, 'uploads', fileId);
    const targetPath = path.join(__dirname, 'completed', fileName);
    const writeStream = fs.createWriteStream(targetPath);

    for (let i = 0; i < totalChunks; i++) {
        const chunkPath = path.join(tempDir, `chunk-${i}`);
        const chunkBuffer = fs.readFileSync(chunkPath);
        writeStream.write(chunkBuffer);
        fs.unlinkSync(chunkPath);
    }

    writeStream.end();
    fs.rmdirSync(tempDir);
}
```

## Key Best Practices

- **Concurrency Control:** Upload 2 to 4 chunks in parallel to optimize network throughput while avoiding browser connection limits.
- **Automated Cleanup:** Implement background jobs (e.g., cron jobs) to clean up abandoned temporary chunks older than a specified threshold (e.g., 24 hours).
- **Checksum Verification:** Send an MD5 or SHA-256 hash along with each chunk and the final merge request to verify data integrity.
- **Leverage Cloud Storage:** For large scale architectures, leverage native cloud mechanisms like AWS S3 Multipart Upload or Azure Block Blobs directly from the client using pre-signed URLs.


---

Original Source: https://answers.mindstick.com/qa/117106/how-to-design-a-resumable-chunked-file-upload-system-for-web-applications

Copyright © MindStick Software Pvt. Ltd. This Markdown version is provided for developers, AI systems, and offline reading.