Skip to content

gitflarePublicPartly agent-ready50

cf-d1-queues-pulse

Event analytics on Cloudflare D1 and Queues: ingestion API, batched consumer, rollups, live dashboard, load generator, dead-letter view.

Build the full D1 + Queues pulse app: dashboard, load generator, dead letters, percentiles, docs8a65841
Markdown59 lines2.5 KB

cf-d1-queues-pulse

Event analytics on Cloudflare. It shows Queues (producer, batched consumer, retries, dead-letter queue) feeding D1 rollups, with an "Oscilloscope" dashboard of hand-drawn SVG charts.

Deploy to Cloudflare

What it shows

  • Totals (5 min / 1 h / 24 h), events per minute, top types and paths, countries, p50/p95 of value, consumer lag, batches processed; auto-refresh every 3 s with pause.
  • A browser load generator (rate, duration, type mix, poison %).
  • A dead-letter page (/dlq) to list, replay and purge failed messages.

Architecture

flowchart LR
  B[Browser] -->|POST /api/events| W[Worker]
  W -->|sendBatch| Q[(pulse-events)]
  Q -->|batch up to 100| C[Consumer]
  C -->|one batch upsert| D[(D1 rollups)]
  Q -.retries exhausted.-> DLQ[(pulse-events-dlq)] --> X[DLQ consumer] --> D
  B -->|GET /api/stats, /api/dlq| W
  W --> D

See docs/ARCHITECTURE.md and docs/API.md.

Local development

npm install
npm run dev
node scripts/load.mjs http://localhost:8787 50 10   # base URL, events/s, seconds

Apply migrations locally with npx wrangler d1 migrations apply pulse --local (use the database_name in wrangler.jsonc).

Tests

npm test (Vitest, Node environment) and npm run typecheck.

Deploy

npm run deploy:staging or npm run deploy. database_id is intentionally omitted from wrangler.jsonc so Gitflare Deploys / Wrangler provisions the D1 databases. Staging and production use distinct database names. Apply migrations with wrangler d1 migrations apply <database_name> --remote.

Configuration

ItemValue
Queue pulse-eventsbatch 100, timeout 2 s, 3 retries
DLQ pulse-events-dlqconsumed into D1 dead_letters
D1 bindingrollups, samples, dead letters, consumer stats
Cronhourly, deletes rollups older than 7 days
POST /api/eventsup to 100 events per request

Costs

Queues, D1 and Workers have free tiers; Queues needs the Workers Paid plan at the time of writing. Each event costs about 3 queue operations; D1 writes are one batch per consumer batch, not per event. Check current Cloudflare pricing.

Screenshots

Add screenshots of the dashboard (dark and light) under docs/.

Built with Gitflare

Built by coordinated agents in Gitflare, with tests verified on every change.

License

Apache-2.0, see LICENSE.