Pipelines

Pipelines for scheduled runs and background work

Run pipelines on a schedule, from a webhook or for background processing. Connect the same functions that serve your HTTP API into steps with retries, branching and fan-out.

Related guides and docs

Get started →

How it works

How serverless pipelines work

Define steps

Drop existing functions onto the visual canvas as pipeline steps, then wire up their inputs, outputs, and the dependencies between them.

Connect and schedule

Link steps into a DAG or a sequential chain, then add a cron schedule or expose an HTTP trigger so runs start automatically.

Monitor in real time

Watch each step's status, logs, and output live, and replay a failed step without re-running the whole pipeline.

Features

Built for real workflows

DAG execution

Steps run in parallel whenever their dependencies allow, with full dependency resolution handled out of the box.

HTTP triggers

Give a pipeline its own HTTP endpoint with API-key auth — the POST payload lands in the first step. Manual runs from the UI or API still work.

Branching and fan-out

Route on step output with if conditions (input.intent === 'billing'), and fan work out and back in with explicit parallel and merge nodes.

Manual approval

Pause execution at any step and require a human sign-off before the pipeline proceeds.

Pipeline schedules

Schedule pipelines with a plain cron expression — daily reports, batch jobs, cleanup tasks.

Data shaping

Reshape payloads between steps with mapper and set nodes — no extra function deploys just to glue two steps together.

Pipelines reuse the same functions you expose through the gateway.

Decision guide

Pipelines vs direct functions vs async jobs

Use pipelines

Use pipelines for recurring runs, webhook-triggered workflows and background processing that needs retries, branching, fan-out or several function calls.

Use direct functions

Use a direct function when one HTTP request can finish in a single gateway handler without a schedule or multiple steps.

Use async jobs

Use async jobs when callers should return immediately and all you need is a queued background job, not a full DAG across many steps.

Example

Branching DAG pipeline (HTTP trigger)

customer-support-pipeline.json

{
  "schemaVersion": 1,
  "nodes": [
    { "id": "t1", "kind": "httpTrigger", "name": "Webhook", "position": { "x": 0, "y": 0 }, "config": { "method": "POST", "authType": "apiKey" } },
    { "id": "n1", "kind": "lambda", "name": "Classify intent", "position": { "x": 240, "y": 0 }, "config": { "functionId": "intent-classifier", "onError": "failPipeline" } },
    { "id": "i1", "kind": "if", "name": "Billing?", "position": { "x": 480, "y": 0 }, "config": { "expression": "input.intent === 'billing'" } },
    { "id": "n2", "kind": "lambda", "name": "Billing agent", "position": { "x": 720, "y": -60 }, "config": { "functionId": "billing-agent", "onError": "failPipeline" } },
    { "id": "n3", "kind": "lambda", "name": "Support agent", "position": { "x": 720, "y": 60 }, "config": { "functionId": "support-agent", "onError": "failPipeline" } },
    { "id": "rB", "kind": "respond", "name": "Done", "position": { "x": 960, "y": -60 }, "config": { "statusCode": 200, "bodyMapping": { "ok": true } } },
    { "id": "rS", "kind": "respond", "name": "Done", "position": { "x": 960, "y": 60 }, "config": { "statusCode": 200, "bodyMapping": { "ok": true } } }
  ],
  "edges": [
    { "id": "e1", "sourceNodeId": "t1", "targetNodeId": "n1", "sourceHandle": "default" },
    { "id": "e2", "sourceNodeId": "n1", "targetNodeId": "i1", "sourceHandle": "success" },
    { "id": "e3", "sourceNodeId": "i1", "targetNodeId": "n2", "sourceHandle": "true" },
    { "id": "e4", "sourceNodeId": "i1", "targetNodeId": "n3", "sourceHandle": "false" },
    { "id": "e5", "sourceNodeId": "n2", "targetNodeId": "rB", "sourceHandle": "success" },
    { "id": "e6", "sourceNodeId": "n3", "targetNodeId": "rS", "sourceHandle": "success" }
  ]
}

Sequential example (nightly processing)

nightly-sequential-pipeline.json

{
  "schemaVersion": 1,
  "nodes": [
    { "id": "cron", "kind": "cronTrigger", "name": "Nightly 02:00", "position": { "x": 0, "y": 0 }, "config": { "cron": "0 2 * * *", "timezone": "UTC" } },
    { "id": "extract", "kind": "lambda", "name": "Extract", "position": { "x": 240, "y": 0 }, "config": { "functionId": "nightly-extract", "onError": "failPipeline" } },
    { "id": "transform", "kind": "lambda", "name": "Transform", "position": { "x": 480, "y": 0 }, "config": { "functionId": "nightly-transform", "onError": "failPipeline" } },
    { "id": "publish", "kind": "lambda", "name": "Publish", "position": { "x": 720, "y": 0 }, "config": { "functionId": "nightly-publish", "onError": "failPipeline" } }
  ],
  "edges": [
    { "id": "e1", "sourceNodeId": "cron", "targetNodeId": "extract", "sourceHandle": "default" },
    { "id": "e2", "sourceNodeId": "extract", "targetNodeId": "transform", "sourceHandle": "success" },
    { "id": "e3", "sourceNodeId": "transform", "targetNodeId": "publish", "sourceHandle": "success" }
  ]
}

Get started

Create an application or deploy a Lambda. Start building your project.

Get started →