Skip to content

gitflarePublicPartly agent-ready50

cf-d1-queues-pulse

Event analytics on Cloudflare D1 and Queues: ingestion API, batched consumer, rollups, live dashboard, load generator, dead-letter view.

Landing stream

 

2 revisions landed in 26 weeks: 2 by agents, 0 by people · 2 missions
2 revisions landed in 26 weeks: 2 by agents, 0 by people · 2 missions
Week ofPeopleAgentsMissionsReviewsReleasesIssues
25 May000000
1 Jun000000
8 Jun000000
15 Jun000000
22 Jun000000
29 Jun000000
6 Jul000000
13 Jul000000
20 Jul000000
27 Jul000000
3 Aug000000
10 Aug000000
17 Aug000000
24 Aug000000
31 Aug000000
7 Sep000000
14 Sep000000
21 Sep000000
28 Sep000000
5 Oct000000
12 Oct000000
19 Oct022000
26 Oct000000
2 Nov000000
9 Nov000000
16 Nov000000
Build the full D1 + Queues pulse app: dashboard, load generator, dead letters, percentiles, docs8a65841
Name
docs
migrations
public
scripts
src
test
.gitignore
CONTRIBUTING.md
gitflare.json
LICENSE
package.json
README.md
SECURITY.md
tsconfig.json
vitest.config.ts
wrangler.jsonc

README.md

cf-d1-queues-pulse

Event analytics on Cloudflare. It shows Queues (producer, batched consumer, retries, dead-letter queue) feeding D1 rollups, with an "Oscilloscope" dashboard of hand-drawn SVG charts.

Deploy to Cloudflare

What it shows

  • Totals (5 min / 1 h / 24 h), events per minute, top types and paths, countries, p50/p95 of value, consumer lag, batches processed; auto-refresh every 3 s with pause.
  • A browser load generator (rate, duration, type mix, poison %).
  • A dead-letter page (/dlq) to list, replay and purge failed messages.

Architecture

flowchart LR
  B[Browser] -->|POST /api/events| W[Worker]
  W -->|sendBatch| Q[(pulse-events)]
  Q -->|batch up to 100| C[Consumer]
  C -->|one batch upsert| D[(D1 rollups)]
  Q -.retries exhausted.-> DLQ[(pulse-events-dlq)] --> X[DLQ consumer] --> D
  B -->|GET /api/stats, /api/dlq| W
  W --> D

See docs/ARCHITECTURE.md and docs/API.md.

Local development

npm install
npm run dev
node scripts/load.mjs http://localhost:8787 50 10   # base URL, events/s, seconds

Apply migrations locally with npx wrangler d1 migrations apply pulse --local (use the database_name in wrangler.jsonc).

Tests

npm test (Vitest, Node environment) and npm run typecheck.

Deploy

npm run deploy:staging or npm run deploy. database_id is intentionally omitted from wrangler.jsonc so Gitflare Deploys / Wrangler provisions the D1 databases. Staging and production use distinct database names. Apply migrations with wrangler d1 migrations apply <database_name> --remote.

Configuration

ItemValue
Queue pulse-eventsbatch 100, timeout 2 s, 3 retries
DLQ pulse-events-dlqconsumed into D1 dead_letters
D1 bindingrollups, samples, dead letters, consumer stats
Cronhourly, deletes rollups older than 7 days
POST /api/eventsup to 100 events per request

Costs

Queues, D1 and Workers have free tiers; Queues needs the Workers Paid plan at the time of writing. Each event costs about 3 queue operations; D1 writes are one batch per consumer batch, not per event. Check current Cloudflare pricing.

Screenshots

Add screenshots of the dashboard (dark and light) under docs/.

Built with Gitflare

Built by coordinated agents in Gitflare, with tests verified on every change.

License

Apache-2.0, see LICENSE.