Comparison

SQLite vs Redis: Cache, Queues and Durable State

Compare SQLite and Redis for local application data, caches, and background jobs. Test contention, recovery, and replay against the same workload.

Editorial update

Decision in brief

Start with SQLite when application state and jobs can live on one host and relational updates need a local transaction. Start with Redis for a shared cache with expiry or a queue that multiple hosts must reach and that fits your memory and recovery budgets. Keep the authoritative database for records whose loss or duplication would have business consequences.

Separate data roles: SQLite — Local state. Redis — Shared cache. Before launch — Test recovery.
Choose based on data ownership, expiry and what happens after loss. Keep authoritative records where their recovery needs can be met.

See discussion trends ↓

Choose based on what the data represents

SQLite runs in the application process and stores data in a file. Write-ahead logging allows readers and a writer to operate concurrently on the same host, with one writer at a time. A queue stored beside business records in the same database can insert a job in the same transaction as the state that triggered it. Your own queue code still needs safe claims, retry limits, and a way to recover jobs left running after a crash.

Redis is reached over a network and offers key expiry, configurable eviction, and structures such as streams. Redis can persist to disk using snapshots or an append-only file. The chosen settings affect recovery and possible loss. Ensure your Redis queue library's account of acknowledgment, redelivery, and failure matches what the application expects.

WorkloadFirst option to testDecision check
Authoritative records and a local job created by the same writeSQLite tables sharing a transactionConfirm job creation is atomic and simulate a worker crash and repeat delivery.
Rebuildable cache used by several application instancesRedis with expiry and a deliberate eviction policyMeasure peak memory and database load during cold starts and evictions.
Distributed workers consuming jobs across hostsRedis with a queue library or Streams consumer groupsMeasure oldest job age, retries, acknowledged-work loss, and duplicate side effects at restart.
Local read-through cache in a single applicationBenchmark SQLite or an in-process cache before operating another serviceCompare p95 latency and time to rebuild the cache under realistic contention.

What two HN queue discussions add

In a BunQueue submission, kernelvoid shared a SQLite-backed job queue built for Bun as an option without a Redis service. For a similar queue, compare the actual job claim, execution, and retry path. A local benchmark cannot exercise worker communication across machines.

In a separate discussion about moving away from Redis for jobs, ashniu123 reported moving their own workload to BullMQ and Redis after PostgreSQL became a bottleneck. Their starting point was PostgreSQL. Treat it as a reminder to drive your trial at burst concurrency and separate SQL contention from worker runtime.

Test backlog, crash recovery, and rollback

Use an isolated workload with synthetic job IDs. Agree on a p95 enqueue limit, a maximum backlog age, and tolerances for missing and repeated outcomes before running a test.

  1. Replay the same steady and burst rates with matching work durations. Record p95 enqueue and processing latency, backlog age, disk writes, and memory. Include background writers on the SQLite host or competing cache keys on Redis if they exist in the intended design.
  2. Stop one test worker after claiming a job and before acknowledging completion. Restart the backing service or application separately, then reconcile completed job IDs with downstream writes. Verify that an application idempotency key prevents a repeated write.
  3. Restore the configured SQLite backup or Redis persistence artifact into a clean staging instance. Count lost or replayed jobs and time the recovery. Rehearse switching back while preserving the record of which jobs may run again.
  4. Promote only if latency and recovery stay within the agreed limits and every unprocessed or retried job is accounted for. Stop if the new design can silently lose authoritative state or repeat an effect that cannot be made safe.

Related articles