Circuit Breaker Notifications
When a circuit breaker trips (automatically or manually), your team needs to know immediately. AxonFlow supports three notification channels: webhook, Slack, and PagerDuty.
Circuit breaker notifications are available on licensed tiers. The circuit breaker auto-trip mechanism itself is available more broadly, but notification routing is part of the higher-tier operating model.
What Triggers Notifications
Notifications fire when a circuit breaker trips, whether from threshold breach or manual trip. All active notification configurations for the organization receive the event.
Where This Fits In Operations
Teams usually treat circuit breaker notifications as one part of a larger production loop:
- detect a provider outage, runaway tool call pattern, or manual safety stop
- notify the owning team through Slack, PagerDuty, or an internal webhook
- inspect recent executions, approvals, and policy events to understand impact
- decide whether to retry, reset, re-route, or keep the breaker open
That is what turns the circuit breaker from a simple emergency stop into a real operating control for production AI systems.
Setup
Webhook
Webhooks send a POST request to your endpoint with the event payload. You can optionally configure HMAC signing for payload verification.
curl -X POST https://your-axonflow-host/api/v1/circuit-breaker/notifications \
-H "X-Org-ID: your-org-id" \
-H "Content-Type: application/json" \
-d '{
"type": "webhook",
"url": "https://ops.example.com/hooks/circuit-breaker",
"secret": "whsec_your_signing_secret",
"active": true
}'
HMAC Verification
When secret is configured, every webhook request includes these headers:
| Header | Description |
|---|---|
X-AxonFlow-Event | Event type (e.g., circuit_breaker.tripped) |
X-AxonFlow-Delivery | Unique delivery UUID |
X-AxonFlow-Timestamp | Delivery timestamp (RFC 3339) |
X-AxonFlow-Signature-256 | sha256=<HMAC-SHA256 hex digest> |
Verify the signature by recomputing the HMAC on the raw request body:
import hmac
import hashlib
def verify_webhook(raw_body: bytes, signature_header: str, secret: str) -> bool:
expected = hmac.new(
secret.encode(), raw_body, hashlib.sha256
).hexdigest()
received = signature_header.removeprefix("sha256=")
return hmac.compare_digest(expected, received)
Always use constant-time comparison (hmac.compare_digest or equivalent) to prevent timing attacks.
Slack
Slack notifications use Incoming Webhooks. Create a webhook in your Slack workspace, then configure it in AxonFlow:
curl -X POST https://your-axonflow-host/api/v1/circuit-breaker/notifications \
-H "X-Org-ID: your-org-id" \
-H "Content-Type: application/json" \
-d '{
"type": "slack",
"url": "https://hooks.slack.com/services/T00/B00/xxx",
"active": true
}'
Slack messages use Block Kit formatting with a header ("Circuit Breaker Tripped"), structured fields showing the scope, reason, organization, and time, plus a details section.
PagerDuty
PagerDuty notifications use the Events API v2. Create a service integration in PagerDuty. The secret field is used as the PagerDuty routing key:
curl -X POST https://your-axonflow-host/api/v1/circuit-breaker/notifications \
-H "X-Org-ID: your-org-id" \
-H "Content-Type: application/json" \
-d '{
"type": "pagerduty",
"url": "https://events.pagerduty.com/v2/enqueue",
"secret": "your-pagerduty-routing-key",
"active": true
}'
Trip events send a trigger action with severity critical. The dedup_key is axonflow-cb-<org_id>-<scope_id>, which allows PagerDuty to deduplicate related incidents.
Managing Notification Configurations
List All Configurations
curl https://your-axonflow-host/api/v1/circuit-breaker/notifications \
-H "X-Org-ID: your-org-id"
Response:
{
"success": true,
"data": {
"notifications": [
{
"id": "notif-abc123",
"org_id": "your-org-id",
"tenant_id": "",
"type": "webhook",
"url": "https://ops.example.com/hooks/circuit-breaker",
"secret": "***",
"active": true,
"created_at": "2026-03-16T08:00:00Z",
"updated_at": "2026-03-16T08:00:00Z"
}
],
"count": 1
}
}
Secrets are masked in list responses.
Update a Configuration
curl -X PUT https://your-axonflow-host/api/v1/circuit-breaker/notifications/notif-abc123 \
-H "X-Org-ID: your-org-id" \
-H "Content-Type: application/json" \
-d '{
"active": false
}'
All fields are accepted (type, url, secret, active, tenant_id). Only provided fields are updated.
Delete a Configuration
curl -X DELETE https://your-axonflow-host/api/v1/circuit-breaker/notifications/notif-abc123 \
-H "X-Org-ID: your-org-id"
Tenant-Scoped Notifications
You can scope a notification to a specific tenant by including tenant_id in the create or update request. When set, the notification only fires for circuit breaker events affecting that tenant.
curl -X POST https://your-axonflow-host/api/v1/circuit-breaker/notifications \
-H "X-Org-ID: your-org-id" \
-H "Content-Type: application/json" \
-d '{
"type": "webhook",
"url": "https://ops.example.com/hooks/tenant-alerts",
"tenant_id": "tenant-001",
"active": true
}'
Testing Notifications
To verify your notification setup, manually trip the circuit breaker and confirm you receive the notification:
# Trip the circuit breaker (triggers notification)
curl -X POST https://your-axonflow-host/api/v1/circuit-breaker/trip \
-H "X-Org-ID: your-org-id" \
-H "X-User-ID: your-user-id" \
-H "Content-Type: application/json" \
-d '{
"scope": "client",
"scope_id": "test-client",
"reason": "Testing notification setup"
}'
# Reset it
curl -X POST https://your-axonflow-host/api/v1/circuit-breaker/reset \
-H "X-Org-ID: your-org-id" \
-H "X-User-ID: your-user-id" \
-H "Content-Type: application/json" \
-d '{
"scope": "client",
"scope_id": "test-client"
}'
SSRF Protection
Webhook URLs are validated to prevent Server-Side Request Forgery (SSRF):
- Private/internal IP ranges are blocked:
10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.0/8 - Link-local addresses are blocked:
169.254.0.0/16 - IPv6 private ranges are blocked:
::1/128,fc00::/7,fe80::/10 - Both
http://andhttps://URLs are accepted - DNS resolution is performed at request delivery time (not at configuration time)
API Reference
For complete request/response schemas, see the HITL & Circuit Breaker API Reference in the Enterprise API documentation.
