Decision in brief
Start with SQLite when application state and jobs can live on one host and relational updates need a local transaction. Start with Redis for a shared cache with expiry or a queue that multiple hosts must reach and that fits your memory and recovery budgets. Keep the authoritative database for records whose loss or duplication would have business consequences.

Choose based on what the data represents
SQLite runs in the application process and stores data in a file. Write-ahead logging allows readers and a writer to operate concurrently on the same host, with one writer at a time. A queue stored beside business records in the same database can insert a job in the same transaction as the state that triggered it. Your own queue code still needs safe claims, retry limits, and a way to recover jobs left running after a crash.
Redis is reached over a network and offers key expiry, configurable eviction, and structures such as streams. Redis can persist to disk using snapshots or an append-only file. The chosen settings affect recovery and possible loss. Ensure your Redis queue library's account of acknowledgment, redelivery, and failure matches what the application expects.
| Workload | First option to test | Decision check |
|---|---|---|
| Authoritative records and a local job created by the same write | SQLite tables sharing a transaction | Confirm job creation is atomic and simulate a worker crash and repeat delivery. |
| Rebuildable cache used by several application instances | Redis with expiry and a deliberate eviction policy | Measure peak memory and database load during cold starts and evictions. |
| Distributed workers consuming jobs across hosts | Redis with a queue library or Streams consumer groups | Measure oldest job age, retries, acknowledged-work loss, and duplicate side effects at restart. |
| Local read-through cache in a single application | Benchmark SQLite or an in-process cache before operating another service | Compare p95 latency and time to rebuild the cache under realistic contention. |
What two HN queue discussions add
In a BunQueue submission, kernelvoid shared a SQLite-backed job queue built for Bun as an option without a Redis service. For a similar queue, compare the actual job claim, execution, and retry path. A local benchmark cannot exercise worker communication across machines.
In a separate discussion about moving away from Redis for jobs, ashniu123 reported moving their own workload to BullMQ and Redis after PostgreSQL became a bottleneck. Their starting point was PostgreSQL. Treat it as a reminder to drive your trial at burst concurrency and separate SQL contention from worker runtime.
Test backlog, crash recovery, and rollback
Use an isolated workload with synthetic job IDs. Agree on a p95 enqueue limit, a maximum backlog age, and tolerances for missing and repeated outcomes before running a test.
- Replay the same steady and burst rates with matching work durations. Record p95 enqueue and processing latency, backlog age, disk writes, and memory. Include background writers on the SQLite host or competing cache keys on Redis if they exist in the intended design.
- Stop one test worker after claiming a job and before acknowledging completion. Restart the backing service or application separately, then reconcile completed job IDs with downstream writes. Verify that an application idempotency key prevents a repeated write.
- Restore the configured SQLite backup or Redis persistence artifact into a clean staging instance. Count lost or replayed jobs and time the recovery. Rehearse switching back while preserving the record of which jobs may run again.
- Promote only if latency and recovery stay within the agreed limits and every unprocessed or retried job is accounted for. Stop if the new design can silently lose authoritative state or repeat an effect that cannot be made safe.