Resilient Job Queue

A small production-style background job service built with TypeScript, Express, PostgreSQL, Redis, BullMQ, Prisma, Docker Compose, and Vitest.

It demonstrates the reliability concerns that disappear in a synchronous demo: duplicate submissions, transient failures, timeouts, bounded retries, durable state, and manual replay.

Architecture

flowchart LR
  Client -->|"POST /jobs + Idempotency-Key"| API
  API --> DB[(PostgreSQL)]
  API --> Queue[(Redis / BullMQ)]
  Queue --> Worker
  Worker --> DB
  Client -->|"GET /jobs/:id"| API

Loading

PostgreSQL owns user-visible job state. Redis transports work and schedules retries, but queue retention is intentionally bounded.

Run the complete stack

docker compose up --build

The API is available at http://localhost:4000; PostgreSQL and Redis are mapped to 5433 and 6380 to avoid collisions with common local services.

Create and inspect an idempotent job:

curl -X POST http://localhost:4000/jobs \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: portfolio-demo-001" \
  -d '{"type":"delay","payload":{"delayMs":500},"timeoutMs":2000}'

curl http://localhost:4000/jobs/JOB_ID

Repeat the same POST and the API returns the original job without publishing duplicate work. Use type: "fail" to observe retries, then POST /jobs/{id}/retry for a controlled manual replay.

API contract

The full contract is in openapi.yaml. Supported demo tasks are deliberately allow-listed: echo, delay, and fail. This avoids turning the service into an arbitrary command runner.

Verification

npm install
npm run lint
npm run typecheck
npm test
npm run build

My contributions

• Designed the durable job schema and idempotent submission boundary.
• Implemented exponential retries, per-job timeouts, final-failure state, and manual replay.
• Separated API and worker lifecycles with structured operational events.
• Added an OpenAPI contract, unit tests, CI, and a one-command Docker environment.

Design notes

See failure model for failure scenarios and production extensions.

This repository is a focused extraction of queue and worker patterns used in a larger AI-content platform. It is intentionally generic and contains no private product data or domain-specific assets.