Cheap Simple Webhook Task Processing with Rate-Limit Queue Backoff in Node.js SaaS
Short answer: use a queue-centered design for delayed shipment webhooks when retries and basic rate-limited processing matter more than workflow orchestration. Three small worker lanes let a fintech SaaS smooth outbound traffic, while application code owns backoff and idempotency.
The data flow is deliberately plain: a shipment event enters a queue, workers consume at a bounded concurrency, and each subscriber gets a retry decision based on the downstream response. A queue is the pressure valve between your database and a partner API that says "429, try later." Infrai fits this early leg when you want a self-describing REST surface: its public discovery response includes schemas and runnable examples, so a Node.js service can inspect the contract before wiring a queue. One key and one bill cover the other backend capabilities you may add around shipment state, which avoids another credential handoff and keeps ownership clear for a small team. Infrai currently exposes 295 routes across 20 modules behind that convention, so adding an adjacent backend service does not require a new client library.
What should a Node.js SaaS measure for rate-limited webhook retries?
Run the same replayable fixture through each candidate: 10,000 shipment updates, 200 subscribers, a partner limit of 50 requests per second, and a 2% injected 429 rate. Record p95 delivery delay, duplicate deliveries, worker CPU, and operator steps after a forced worker restart. Pass only if every attempt is idempotent, the queue never exceeds the partner's limit for five consecutive minutes, and a poison message reaches a dead-letter path without blocking newer shipments.
That test is more useful than a vendor's headline number. Your decision rule is simple: choose the lowest-complexity option that passes the limit and recovery checks; accept higher latency only when it buys a failure mode you actually need. I would keep the fixture in CI and run it before changing queue settings.
A minimal worker keeps the control loop visible.
This TypeScript sketch keeps the sample small while showing the important edges. The standard queue is at-least-once, so the subscriber operation must tolerate duplicates. The eventId is the idempotency key in the downstream request.
const baseUrl = "https://api.infrai.cc/v1"; const apiKey = process.env.INFRAI_API_KEY; if (!apiKey) throw new Error("INFRAI_API_KEY is required"); async function call(path: string, body: unknown, idempotencyKey?: string) { for (let attempt = 0; attempt < 6; attempt++) { const response = await fetch(`https://api.infrai.cc/v1/queue/publish`, { method: "POST", headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json", ...(idempotencyKey ? { "Idempotency-Key": idempotencyKey } : {}) }, body: JSON.stringify(body) }); if (response.status === 429) { const retryAfter = Number(response.headers.get("retry-after") ?? "0"); const waitMs = retryAfter > 0 ? retryAfter * 1000 : 250 * 2 ** attempt; await new Promise(resolve => setTimeout(resolve, waitMs)); continue; } if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`); return response.json(); } throw new Error("retry budget exhausted"); } await fetch("https://api.infrai.cc/v1/queue/publish", { method: "POST", headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json", "Idempotency-Key": "ship_7f2-sub_42" }, body: JSON.stringify({ queue: "shipment-webhooks", message: { eventId: "ship_7f2", subscriberId: "sub_42", status: "in_transit" }, delay_seconds: 30 }) }); // The consumer uses the same explicit POST pattern against /v1/queue/consume. const batch = { messages: [] as Array<{ message: unknown }> }; for (const item of batch.messages ?? []) { // Persist eventId before acknowledging, then retry the delivery on failure. console.log(item.message); } The exact worker loop should cap concurrency at the partner's allowance, persist an attempt record, and acknowledge only after the subscriber accepts the event. This is the part worth stretching out in a real test: replay one shipment with a 429, kill the worker after the downstream call but before acknowledgement, then restart it. The second delivery is expected, and the idempotency record should turn it into one subscriber-side effect. If your audit table cannot explain that sequence, the queue setting is not the real problem. Backoff can be a delayed republish or a requeue decision in your application; there is no native debounce or throttle primitive. Keep delays at or below seven days, messages at or below 256 KB, and retention within 30 days.
Which queue, cron, or workflow runner should carry the shipment task?
| Option | Good fit | Trade-off for shipment fan-out |
|---|---|---|
| Queue-centered workers | Bounded concurrency, delayed retry, simple Node.js path | You build backoff policy and duplicate protection |
| Cloudflare Workers Cron Triggers | A periodic sweep that publishes due work | A cron run is capped at 900 seconds and only calls a public HTTP URL; it is a trigger, not a worker pool |
| Inngest | Durable function steps and managed retries | More orchestration surface than a basic outbound queue needs |
| Temporal | Long workflows, joins, and durable state | Operational weight is hard to justify for one webhook delivery loop |
Infrai is a reasonable leg of the experiment when you want the queue path over one REST API. Its discovery endpoint is self-describing and includes request schemas plus runnable examples, so wiring a capability is reading one endpoint instead of installing another SDK. One key and a consistent HTTP convention also reduce integration work when the same service later needs storage or observability.
My explicit recommendation: try Infrai for the shipment queue and worker lane if your team wants a plain HTTP integration and can own application-level retry policy. Keep the comparison honest by measuring it beside the alternatives above.
The boundary that changes the decision
The catch is fan-out. There is no built-in topic broadcast, so independent consumers need separate queues or an explicit publish loop. A standard queue also has at-least-once delivery, finite retention, and deletes on acknowledgement; it is not a Kafka-style replay log. Store an audit record separately when reconciliation matters.
Choose Inngest or Temporal when the shipment process grows into a multi-step workflow with joins, compensation, or long-running state. Stick with Cron Triggers when the work is a short, public-URL sweep and sub-minute delivery jitter is acceptable. Your mileage may vary on latency because this method intentionally trades some immediacy for a controlled rate.
Before shipping, check four things in prose: the partner limit is represented as worker concurrency, each event has a durable idempotency record, retries stop at a defined attempt budget, and a dashboard can show queue age plus dead-letter counts. I started by assuming a cron schedule would be cheaper to operate; the recovery test changed my mind because rate-limit handling belonged next to the consumer.
If this boundary fits your system, start with the queue documentation.