Streaming Large ZIP Downloads Without Exhausting Server Memory

A customer clicks “Download all” on a gallery containing 800 high-resolution images.

The naive backend implementation fetches every object into memory, creates a ZIP buffer, and only then sends the response. It works in development. In production, a few concurrent downloads push memory usage into gigabytes and the process gets killed.

The fix is to stream the archive instead of building it in RAM.

Stream object storage directly into the archive

A Node.js endpoint can pipe each object stream into a ZIP writer and pipe the ZIP output straight to the HTTP response.

import archiver from "archiver";

app.get("/galleries/:id/download", async (req, res) => {
  const gallery = await authorizeGalleryDownload(
    req.user,
    req.params.id
  );

  res.setHeader("Content-Type", "application/zip");
  res.setHeader(
    "Content-Disposition",
    'attachment; filename="gallery.zip"'
  );

  const archive = archiver("zip", { zlib: { level: 6 } });
  archive.pipe(res);

  for (const item of gallery.files) {
    const stream = await storage.getObjectStream(item.key);

    archive.append(stream, {
      name: sanitizeFilename(item.filename)
    });
  }

  await archive.finalize();
});

The important part is that the application never holds the complete archive—or even a complete image—in memory.

Node streams also provide backpressure. If the client or network is slow, downstream writes naturally slow upstream reads instead of letting buffers grow without limit.

Do authorization before sending bytes

Once a response has started, changing your mind is difficult.

Resolve permissions before piping anything:

if (!viewer.canDownloadOriginals) {
  return res.sendStatus(403);
}

Never accept storage keys directly from the browser. Resolve downloadable objects from server-side gallery state, otherwise an attacker may try to request files belonging to another tenant.

Handle disconnects

Users close tabs and mobile connections disappear.

Abort upstream reads when the HTTP connection closes:

req.on("close", () => {
  archive.abort();
});

Your storage SDK should receive the same cancellation signal when possible. Otherwise the server may continue pulling gigabytes that nobody will receive.

Avoid unbounded parallelism

Fetching 800 objects simultaneously is not faster in practice. It can exhaust sockets, storage limits, and memory.

Use sequential reads or small bounded batches. The right concurrency depends on object size, storage latency, and infrastructure limits.

Large media downloads are primarily a streaming problem, not a ZIP problem. Keep authorization server-side, respect backpressure, cancel abandoned work, and avoid buffering entire archives. Those choices let the same endpoint handle a small photo set and a multi-gigabyte client delivery without changing the architecture.