Subham Haldar
personal

StreamVault — Short-Form Video Platform

Solo, end to end · Jun 2026 – Aug 2026

StreamVault — a short-form video platform: upload, transcode, stream. Ingest, async and degrade pillars over a TypeScript, Fastify, Postgres, Redis, BullMQ, S3 and Mux stack.

A short-form video platform — upload, transcode, stream, view-count, auth — built end to end on free-tier managed services, to work through real distributed-systems problems rather than to look busy.

StreamVault system architecture: React SPA, Fastify API, Neon Postgres, Upstash Redis, BullMQ ephemeral worker, Backblaze B2 storage, Mux transcoding and Sentry.
System architecture

Video bytes never touch the API

Uploads go straight to object storage through a presigned URL. On completion the API does a ranged read of the first bytes and verifies real MP4, WebM or AVI magic bytes rather than trusting the client's Content-Type header.

Sequence diagram of the upload, transcode and playback flow between browser, API, B2, BullMQ worker, Mux and Postgres.
Upload → transcode → playback

Transcoding off the request path

A BullMQ job hands the source to a transcoding service; a signature-verified webhook flips the status when it is ready, and a WebSocket pushes progress to the client. Nothing polls.

Video lifecycle state machine: pending, processing, ready, failed, rejected and deleted, each transition owned by exactly one actor.
Video lifecycle — every transition has one owner

The incident worth talking about

A free-tier Redis command quota ran out during development. The always-on queue worker retried on every Redis error with no backoff, producing a tight loop hitting the exhausted quota about ten times a second — a self-sustaining outage.

I fixed it with a circuit breaker: burst detection, cooldown, exponential backoff. It worked. Then I realised it was treating the symptom. The durable fix was making the worker ephemeral — it spawns when a job is enqueued and exits a couple of minutes after the queue drains. With no idle connection, the failure mode mostly stops being possible, and the circuit breaker became unnecessary. So I deleted it.

Graceful degradation

View counts buffer in Redis and batch-flush to PostgreSQL on a schedule. Reads prefer the fresher Redis value and fall back to the last flushed Postgres value rather than failing. Every Redis-touching path degrades this way; the guest upload limit fails open rather than blocking uploads.

One session mechanism, not two

Guests and OAuth users share a single JWT cookie, separated only by a provider column. Pure reads provision nothing. The first upload silently issues a guest identity. Signing in resolves through a user_identities table so the same verified email across Google and GitHub lands on one account instead of two, and any guest videos are reassigned.

Sessions and identity flow: guest session on first upload, Google or GitHub sign-in resolved through user_identities, and guest videos reassigned on upgrade.
Sessions & identity
StreamVault data model: users, user_identities, videos and video_events tables with their enums and relationships.
Data model

What it does not do

No automated end-to-end coverage, and a single API instance with no horizontal scaling. Both are written up in the repository rather than left out.

Built with

TypeScriptNode.jsFastifyPostgreSQLDrizzle ORMRedisBullMQMuxBackblaze B2ReactViteVitestDocker