gitflarePublicPartly agent-ready50
cf-d1-queues-pulse
Event analytics on Cloudflare D1 and Queues: ingestion API, batched consumer, rollups, live dashboard, load generator, dead-letter view.
- LicenseApache-2.0
- LanguageTypeScript
- Last landed
8a65841Oct 20, 2026
8a65841cf-d1-queues-pulse
Event analytics on Cloudflare. It shows Queues (producer, batched consumer, retries, dead-letter queue) feeding D1 rollups, with an "Oscilloscope" dashboard of hand-drawn SVG charts.
What it shows
- Totals (5 min / 1 h / 24 h), events per minute, top types and paths, countries, p50/p95 of
value, consumer lag, batches processed; auto-refresh every 3 s with pause. - A browser load generator (rate, duration, type mix, poison %).
- A dead-letter page (
/dlq) to list, replay and purge failed messages.
Architecture
flowchart LR
B[Browser] -->|POST /api/events| W[Worker]
W -->|sendBatch| Q[(pulse-events)]
Q -->|batch up to 100| C[Consumer]
C -->|one batch upsert| D[(D1 rollups)]
Q -.retries exhausted.-> DLQ[(pulse-events-dlq)] --> X[DLQ consumer] --> D
B -->|GET /api/stats, /api/dlq| W
W --> D
See docs/ARCHITECTURE.md and docs/API.md.
Local development
npm install
npm run dev
node scripts/load.mjs http://localhost:8787 50 10 # base URL, events/s, seconds
Apply migrations locally with npx wrangler d1 migrations apply pulse --local (use the database_name in wrangler.jsonc).
Tests
npm test (Vitest, Node environment) and npm run typecheck.
Deploy
npm run deploy:staging or npm run deploy. database_id is intentionally omitted from wrangler.jsonc so Gitflare Deploys / Wrangler provisions the D1 databases. Staging and production use distinct database names. Apply migrations with wrangler d1 migrations apply <database_name> --remote.
Configuration
| Item | Value |
|---|---|
Queue pulse-events | batch 100, timeout 2 s, 3 retries |
DLQ pulse-events-dlq | consumed into D1 dead_letters |
| D1 binding | rollups, samples, dead letters, consumer stats |
| Cron | hourly, deletes rollups older than 7 days |
POST /api/events | up to 100 events per request |
Costs
Queues, D1 and Workers have free tiers; Queues needs the Workers Paid plan at the time of writing. Each event costs about 3 queue operations; D1 writes are one batch per consumer batch, not per event. Check current Cloudflare pricing.
Screenshots
Add screenshots of the dashboard (dark and light) under docs/.
Built with Gitflare
Built by coordinated agents in Gitflare, with tests verified on every change.
License
Apache-2.0, see LICENSE.