Skip to content

gitflarePublicPartly agent-ready50

cf-workflows-pipeline

Durable content pipeline on Cloudflare Workflows: ingest, extract, summarise with Workers AI via AI Gateway, human approval, publish.

Fix pipeline runs failing at ingest (steps.attempt NOT NULL)629b2fa
Markdown20 lines1.5 KB

Architecture

Flow

ingest (fetch, 2 MB cap, 10 s timeout) → extract (pure HTML to title and text) → summarise (Workers AI via AI Gateway) → approval → publish (D1 published).

Retries

Each step.do uses retries: { limit: 3, delay: "2 seconds", backoff: "exponential" } and timeout: "30 seconds". A thrown error retries the step; after the limit the step and run fail. The UI shows retrying n/3.

waitForEvent

After summarising, the workflow calls step.waitForEvent("approval", { type: "approval", timeout: "24 hours" }). The approve and reject endpoints call instance.sendEvent with that type and a payload (decision, note). Reject ends the run as rejected; timeout fails the wait. No compute is held while waiting.

Idempotent steps

Step bodies may run more than once, so they only do safe-to-repeat work: D1 writes are upserts keyed by run id and step name, and publish uses the run id as the key. Step return values are persisted by Workflows and not recomputed on replay.

Failure injection

Run params carry a per-step count N (extract, summarise, publish). A step throws while its attempt number ≤ N. The attempt is recorded in D1 before throwing, so the UI shows it. Counts of 3 or less recover via retries; higher counts exhaust them.

D1 mirror

Steps write state into runs and steps (id, run_id, name, state, attempt, started_at, ended_at, output_excerpt, error). The UI polls GET /api/runs/:id every second and stops when the run finishes.