utilitiesintermediate10 min setupby Mantle

Delay, retry and dead-letter

Call a flaky endpoint with backoff retries; on final failure wait, try a fallback endpoint, and if that also fails email an operator.

Request this template in a demo Talk to sales

Drag to pan, pinch to zoom. This is the real graph the template installs.

Purpose

Call a flaky endpoint with backoff retries; on final failure wait, try a fallback endpoint, and if that also fails email an operator.

How it works

What it does

Primary endpoint retries 4× with exponential backoff (1 s → 20 s cap).
Only if it still fails does the dashed error edge fire: Wait 60s → Fallback endpoint (2 fixed retries).
If the fallback fails too, Dead-letter to operator emails the payload.

Nodes 5

01Job intrigger.webhookTrigger
02Primary endpointhttp.requestHTTP
03Wait 60scontrol.delayLogic
04Fallback endpointhttp.requestHTTP
05Dead-letter to operatormail.sendDestination

View template JSON

{
  "format": "mantle-workflow/v1",
  "id": "util-delay-retry-webhook",
  "name": "Delay, retry and dead-letter",
  "graph": {
    "nodes": [
      {
        "id": "webhook",
        "type": "trigger.webhook",
        "label": "Job in",
        "config": {
          "slug": "flaky-job",
          "idempotency_path": "job_id"
        },
        "worker_type": "generic-worker",
        "position": {
          "x": 60,
          "y": 220
        }
      },
      {
        "id": "primary_call",
        "type": "http.request",
        "label": "Primary endpoint",
        "config": {
          "method": "POST",
          "url": "https://primary.example.com/jobs",
          "body_from_input": true,
          "timeout_seconds": 10
        },
        "worker_type": "generic-worker",
        "position": {
          "x": 380,
          "y": 220
        },
        "retry": {
          "max_attempts": 4,
          "backoff": "exp",
          "initial_ms": 1000,
          "max_ms": 20000
        }
      },
      {
        "id": "cool_down",
        "type": "control.delay",
        "label": "Wait 60s",
        "config": {
          "seconds": 60
        },
        "worker_type": "generic-worker",
        "position": {
          "x": 700,
          "y": 410
        }
      },
      {
        "id": "fallback_call",
        "type": "http.request",
        "label": "Fallback endpoint",
        "config": {
          "method": "POST",
          "url": "https://fallback.example.com/jobs",
          "body_from_input": true,
          "timeout_seconds": 10
        },
        "worker_type": "generic-worker",
        "position": {
          "x": 1020,
          "y": 410
        },
        "retry": {
          "max_attempts": 2,
          "backoff": "fixed",
          "initial_ms": 5000,
          "max_ms": 5000
        }
      },
      {
        "id": "dead_letter",
        "type": "mail.send",
        "label": "Dead-letter to operator",
        "config": {
          "to": "ops@example.com",
          "subject": "Job {{ steps.webhook.job_id }} failed on primary and fallback",
          "body_text": "Payload:\n{{ steps.webhook }}"
        },
        "worker_type": "generic-worker",
        "position": {
          "x": 1340,
          "y": 600
        }
      }
    ],
    "edges": [
      {
        "from": "webhook",
        "to": "primary_call"
      },
      {
        "from": "primary_call",
        "to": "cool_down",
        "branch": "error",
        "label": "primary failed"
      },
      {
        "from": "cool_down",
        "to": "fallback_call"
      },
      {
        "from": "fallback_call",
        "to": "dead_letter",
        "branch": "error",
        "label": "fallback failed"
      }
    ]
  }
}

Requirements

The connections and permissions each step needs, from the node contracts in the catalog.

SendGridAPI key
Used by Dead-letter to operator.
Credential: {"api_key"} or {"username", "password"}

Setup

Set both urls and the operator address (SendGrid/SMTP connection). Success paths end silently — add downstream nodes to either call as needed.

Sample input and output

Synthetic examples: node-contract samples or placeholders, never customer data.

Input

The fields the template declares or reads from the request body. Values are placeholders.

{
  "job_id": "<job_id>"
}

Output

Primary endpoint http.request

{
  "body": null,
  "headers": {},
  "status_code": 0
}

Fallback endpoint http.request

{
  "body": null,
  "headers": {},
  "status_code": 0
}

Failure handling

What the graph does when a step fails, read from its retry settings, error branches and error policies.

Primary endpoint: up to 4 attempts, exponential backoff from 1 s up to 20 s.
Primary endpoint: the request times out after 10 s.
Fallback endpoint: up to 2 attempts, 5 s between attempts.
Fallback endpoint: the request times out after 10 s.
If Primary endpoint still fails, the error branch runs Wait 60s with the error and the step's input.
If Fallback endpoint still fails, the error branch runs Dead-letter to operator with the error and the step's input.
Any other step that fails, after its retries, fails the run. The failed step is kept with its input and error.