Durable background jobs for Postgres

Creek runs background work — emails, reports, batch processing, scheduled cleanups — using the Postgres database you already have. No new infrastructure, no Redis, no separate queue service. Jobs survive restarts, retries are automatic, and your transactional state and your job queue live in the same database.

Why Creek #

Most background-job libraries fall into one of two camps. The first is Redis-backed (Sidekiq, BullMQ, Celery) — fast, but you now operate a second datastore whose durability model differs from your primary database. A job can be marked "done" in Redis while the transaction that enqueued it rolls back, or vice versa. The second camp polls your database directly (a jobs table with SELECT … FOR UPDATE SKIP LOCKED), which keeps everything in one database but ships without retries, scheduling, concurrency control, or observability — you rebuild all of it.

Creek closes that gap. It uses SKIP LOCKED for job claiming (so it scales horizontally without contention), but layers on the operational machinery you'd otherwise hand-roll: exponential backoff with jitter, per-queue concurrency caps, cron-style scheduling, dead-letter routing, and structured logging that correlates a job back to the request that enqueued it.

Quickstart #

A job is an async function with a typed payload. Register it, enqueue it from anywhere in your transaction, and Creek handles the rest.

import { Creek } from "creek";

const creek = new Creek({
  connectionString: process.env.DATABASE_URL,
});

// 1. Define a job
creek.job("send-welcome-email", async (ctx, payload) => {
  await mailer.send({
    to: payload.email,
    template: "welcome",
    name: payload.name,
  });
});

// 2. Enqueue inside a transaction
await db.transaction(async (tx) => {
  const user = await tx.users.create({ email, name });
  await creek.enqueue("send-welcome-email", {
    email: user.email,
    name: user.name,
  });
});

// 3. Start the worker (one process, or many)
creek.start();

from creek import Creek

creek = Creek(connection_string=os.environ["DATABASE_URL"])

# 1. Define a job
@creek.job("send-welcome-email")
async def send_welcome(ctx, payload):
    await mailer.send(
        to=payload["email"],
        template="welcome",
        name=payload["name"],
    )

# 2. Enqueue inside a transaction
async with db.transaction() as tx:
    user = await tx.users.create(email=email, name=name)
    await creek.enqueue("send-welcome-email", {
        "email": user.email,
        "name": user.name,
    })

# 3. Start the worker
await creek.start()

import "github.com/creek/creek-go"

creek := creek.New(creek.Config{
    ConnectionString: os.Getenv("DATABASE_URL"),
})

// 1. Define a job
creek.Register("send-welcome-email", func(ctx creek.Context, p WelcomePayload) error {
    return mailer.Send(ctx, p.Email, "welcome", p.Name)
})

// 2. Enqueue inside a transaction
tx.Users().Create(&user)
creek.Enqueue("send-welcome-email", WelcomePayload{
    Email: user.Email,
    Name:  user.Name,
})

// 3. Start the worker
creek.Start()

That's a production-grade worker. It will retry failed jobs with exponential backoff, respect per-queue concurrency limits, and persist its state in your existing database. Scale by running more processes — SKIP LOCKED prevents double-processing without a coordination service.

How it works #

Creek creates a single creek_jobs table in your database on first run:

CREATE TABLE creek_jobs (
  id            bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  queue         text      NOT NULL DEFAULT 'default',
  handler       text      NOT NULL,
  payload       jsonb     NOT NULL,
  priority      smallint  NOT NULL DEFAULT 0,
  attempts      smallint  NOT NULL DEFAULT 0,
  max_attempts  smallint  NOT NULL DEFAULT 25,
  run_at        timestamptz NOT NULL DEFAULT now(),
  locked_by     text,
  locked_at     timestamptz,
  last_error    text,
  created_at    timestamptz NOT NULL DEFAULT now()
);

CREATE INDEX creek_jobs_claim_idx ON creek_jobs
  (priority DESC, run_at) WHERE locked_by IS NULL;

A worker claims the next available row with a single query — no advisory locks, no polling coordination, no leader election:

UPDATE creek_jobs
   SET locked_by  = $1,      -- worker id
       locked_at = now()
  WHERE id IN (
    SELECT id FROM creek_jobs
     WHERE queue = $2
       AND run_at <= now()
       AND locked_by IS NULL
     ORDER BY priority DESC, run_at
     LIMIT $3         -- batch size
     FOR UPDATE SKIP LOCKED
  )
RETURNING *;

FOR UPDATE SKIP LOCKED is the Postgres feature that makes this safe under concurrency. If worker A has locked a row, worker B's query silently skips it and grabs the next available one — no contention, no deadlocks, no double-processing. This is the same primitive Stripe, Shopify, and GitHub use for their internal job systems.

Scheduling #

Any job can be scheduled for the future by passing a run_at timestamp. For recurring work, Creek reads a creek_schedules table that a maintenance loop populates — no separate cron process, no sidecar.

// Run in 5 minutes
await creek.enqueue("generate-report", { userId }, {
  run_at: new Date(Date.now() + 5 * 60_000),
});

// Every Monday at 09:00 UTC
creek.schedule("weekly-summary", "0 9 * * 1", {
  generateForAllTeams: true,
});

REST API #

If you enqueue jobs from outside your application process — a webhook receiver, a different service, a cron trigger — use the HTTP API. Every endpoint accepts JSON and returns the created job.

Enqueue one or more jobs. Jobs are durable the moment the response returns 201 — they live in Postgres and will survive a worker restart.

curl https://api.creek.dev/v2/jobs \
  -H "Authorization: Bearer $CREEK_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "handler": "send-welcome-email",
    "payload": { "email": "ada@example.com", "name": "Ada" },
    "priority": 10
  }'

{
  "id":            8492104,
  "queue":         "default",
  "handler":       "send-welcome-email",
  "priority":      10,
  "attempts":      0,
  "run_at":        "2026-07-29T14:03:11.482Z",
  "created_at":    "2026-07-29T14:03:11.482Z"
}

When to reach for Creek — and when not to #

Creek is the right default when your jobs relate to data in Postgres and you value operational simplicity over raw throughput at the extreme tail. Specifically:

• Transactional enqueue. The job must run if and only if the database write succeeds — welcome emails after signup, receipts after payment, thumbnail generation after upload.
• Moderate throughput. Up to a few thousand jobs per second per database. Beyond that, a dedicated queue (Kafka, Redis Streams) offloads work from your primary database.
• Scheduled and recurring work. Nightly rollups, hourly cleanup, per-user digests — without a separate cron fleet.
• Teams that already run Postgres. No new operational surface. The queue is a table; the worker is a process; the dashboard is a SQL query.

It is not the right tool for event streams at Kafka scale (millions of events/second), sub-millisecond latency requirements (the database round-trip is ~1ms), or workloads where you deliberately want the queue decoupled from your transactional database for isolation.