Choosing an Integration Mode

AxonFlow offers three integration approaches. This guide helps you choose the right one for your application.

TL;DR

Starting Point	Recommended Mode
New project	Proxy Mode - Full compliance, Multi-Agent Planning
Existing LangChain/LangGraph/CrewAI workflows	Workflow Control Plane - Step-level governance gates
Existing LLM calls (no workflow)	Gateway Mode - Add governance without changing LLM calls

Quick Decision Guide

Feature Comparison

Feature	Proxy Mode	Gateway Mode	Workflow Control Plane
Compliance
Policy enforcement	✅ Automatic	✅ Automatic (pre-check)	✅ Per-step gates
Audit logging	✅ 100% automatic	⚠️ Manual (call audit API)	✅ Automatic per step
Response filtering (PII redaction)	✅ Yes	❌ No	❌ No (your LLM calls)
Policies
System policies (PII, SQL injection)	✅ Yes	✅ Yes	✅ Yes
Tenant policies for LLM calls	✅ Yes	❌ No	❌ No
Workflow-scoped policies	❌ N/A	❌ N/A	✅ Yes
UI policy changes take effect (LLM)	✅ Immediate	❌ No effect	❌ No effect
Features
Multi-Agent Planning (MAP)	✅ Yes	❌ No	❌ No
Workflow step gates	❌ N/A	❌ No	✅ Yes
MCP Connectors	✅ Yes	✅ Yes	✅ Yes
MCP audit logging	✅ Automatic	✅ Automatic	✅ Automatic
MCP dynamic policies	✅ Yes	✅ Yes	✅ Yes
LLM Provider Routing	✅ Automatic	❌ You handle	❌ You handle
Integration
Code changes	Minimal	Moderate (pre-check + audit)	Moderate (gate calls per step)
Framework support	Good	Best (single LLM calls)	Best (LangChain, LangGraph, CrewAI)
Performance
Total latency	Sub-30ms typical	~10ms on top of framework	~10ms per step gate
Request flow	App → AxonFlow → LLM	App → Framework → LLM	App → Framework → (AxonFlow gate) → LLM

Policy Support Differences for LLM Calls

Gateway Mode only evaluates system policies for LLM calls (built-in security patterns like PII detection, SQL injection blocking). Custom tenant policies created in the Customer Portal UI or via the API will NOT be enforced for LLM calls in Gateway Mode.

If you need custom tenant policies for LLM calls, use Proxy Mode.

Note: MCP connectors have independent policy evaluation. Both system and tenant policies are fully supported for MCP queries regardless of which mode you use for LLM calls. See MCP Policy Enforcement.

When to Choose Proxy Mode

Proxy Mode is strongly recommended for new projects. You get:

100% Compliance Coverage
- Every request automatically logged for audit
- Response filtering catches PII in LLM outputs
- No risk of forgetting to call audit API
Multi-Agent Planning (MAP)
- Generate and execute complex multi-step plans
- Only available in Proxy Mode
- Powers advanced AI workflows
Simpler Integration
- Single API call handles everything
- No need to manage pre-check → LLM → audit flow
- Fewer lines of code, fewer bugs

Ideal Use Cases

Greenfield applications - Start with governance built-in
Compliance-heavy industries - Healthcare (HIPAA), Finance (SOX, PCI, RBI)
Multi-Agent workflows - Travel planning, complex research, orchestrated tasks
Teams new to AI governance - Let AxonFlow handle the complexity

Code Example

import { AxonFlow } from '@axonflow/sdk';

const axonflow = new AxonFlow({
  licenseKey: process.env.AXONFLOW_LICENSE_KEY,
  endpoint: process.env.AXONFLOW_ENDPOINT
});

// Single call - policy, LLM routing, and audit handled automatically
const response = await axonflow.executeQuery({
  userToken: 'user-123',
  query: 'Analyze customer churn patterns',
  requestType: 'chat',
  context: { provider: 'openai', model: 'gpt-4' }
});

if (response.blocked) {
  console.log('Blocked:', response.blockReason);
} else {
  console.log('Response:', response.data);
}

Multi-Agent Planning Example

// Generate a complex plan (Proxy Mode only)
const plan = await axonflow.generatePlan(
  'Plan a 3-day business trip to Tokyo with meetings and hotels',
  'travel'
);

console.log(`Generated ${plan.steps.length} step plan`);

// Execute the plan
const result = await axonflow.executePlan(plan.planId);
console.log('Trip itinerary:', result.result);

When to Choose Gateway Mode

Gateway Mode is recommended if you already have LLM integrations you don't want to change, or need the absolute lowest latency.

Ideal Use Cases

Existing LangChain/CrewAI/LlamaIndex apps - Add governance without rewriting
Performance-critical applications - Sub-50ms latency requirements
Multi-provider setups - You manage provider failover/routing yourself
Gradual adoption - Start with governance, consider full migration later

Code Example

import { AxonFlow } from '@axonflow/sdk';
import OpenAI from 'openai';

const axonflow = new AxonFlow({
  licenseKey: process.env.AXONFLOW_LICENSE_KEY,
  endpoint: process.env.AXONFLOW_ENDPOINT
});
const openai = new OpenAI();

// 1. Pre-check policies
const ctx = await axonflow.getPolicyApprovedContext({
  userToken: 'user-123',
  query: prompt
});

if (!ctx.approved) {
  throw new Error(`Blocked: ${ctx.blockReason}`);
}

// 2. Your existing LLM call (unchanged)
const start = Date.now();
const response = await openai.chat.completions.create({
  model: 'gpt-4',
  messages: [{ role: 'user', content: prompt }]
});

// 3. Audit the call (don't forget this!)
await axonflow.auditLLMCall({
  contextId: ctx.contextId,
  responseSummary: response.choices[0].message.content?.substring(0, 100) || '',
  provider: 'openai',
  model: 'gpt-4',
  tokenUsage: {
    promptTokens: response.usage?.prompt_tokens || 0,
    completionTokens: response.usage?.completion_tokens || 0,
    totalTokens: response.usage?.total_tokens || 0
  },
  latencyMs: Date.now() - start
});

LangChain Integration Example

import { AxonFlow } from '@axonflow/sdk';
import { ChatOpenAI } from '@langchain/openai';

const axonflow = new AxonFlow({ licenseKey: process.env.AXONFLOW_LICENSE_KEY });
const llm = new ChatOpenAI({ model: 'gpt-4' });

// Wrap your LangChain calls with governance
async function governedLangChainCall(prompt: string, userToken: string) {
  // Pre-check
  const ctx = await axonflow.getPolicyApprovedContext({ userToken, query: prompt });
  if (!ctx.approved) throw new Error(ctx.blockReason);

  // Your existing LangChain code
  const start = Date.now();
  const response = await llm.invoke(prompt);

  // Audit
  await axonflow.auditLLMCall({
    contextId: ctx.contextId,
    responseSummary: response.content.toString().substring(0, 100),
    provider: 'openai',
    model: 'gpt-4',
    latencyMs: Date.now() - start
  });

  return response;
}

When to Choose Workflow Control Plane

Workflow Control Plane is the recommended approach for adding governance to existing multi-step agent workflows like LangChain, LangGraph, and CrewAI.

How It Differs from Gateway Mode

Gateway Mode adds a single pre-check before your LLM call. Workflow Control Plane adds governance gates at each step of your workflow, giving you:

Step-level decisions - Allow, block, or require approval per step
Workflow-scoped policies - Policies that apply specifically to workflow steps
Automatic audit trail - Every step decision is logged
LangGraph adapter - Native Python adapter for LangGraph workflows

Ideal Use Cases

LangGraph workflows - Add gates between graph nodes
LangChain agents - Gate each tool call or LLM invocation
CrewAI crews - Control which tasks can execute
Custom orchestrators - Any multi-step AI workflow

Code Example

from axonflow import AxonFlow
from axonflow.adapters import AxonFlowLangGraphAdapter

async with AxonFlow(endpoint="http://localhost:8080") as client:
    adapter = AxonFlowLangGraphAdapter(client, "my-workflow")

    await adapter.start_workflow(total_steps=3)

    # Gate before each step
    if await adapter.check_gate("generate_code", "llm_call", model="gpt-4"):
        result = await generate_code(state)
        await adapter.step_completed("generate_code")

    if await adapter.check_gate("review_code", "tool_call"):
        result = await review_code(state)
        await adapter.step_completed("review_code")

    await adapter.complete_workflow()

See the full Workflow Control Plane guide for API reference and SDK examples in all languages.

MCP Connectors - Available in All Modes

MCP (Model Context Protocol) connectors work in Proxy Mode, Gateway Mode, and Workflow Control Plane:

// List available connectors
const connectors = await axonflow.listConnectors();

// Query a connector (works in both modes)
const flights = await axonflow.queryConnector(
  'amadeus-prod',
  'Find flights from NYC to London on Jan 15',
  { origin: 'JFK', destination: 'LHR', date: '2025-01-15' }
);

Latency Comparison

Mode	Overhead	Best For
Proxy (Public)	50-100ms	Most applications
Proxy (VPC)	20-40ms	Enterprise deployments
Gateway	10-20ms	Latency-critical apps

Audit Logging Comparison

Both modes provide comprehensive audit logging, but capture different information at different points in the request flow.

What Gets Captured

Audit Aspect	Proxy Mode	Gateway Mode
Policy decisions	`orchestrator_audit_logs` (automatic)	`gateway_contexts` (automatic)
LLM call details	`orchestrator_audit_logs` (automatic)	`llm_call_audits` (requires SDK call)
MCP connector access	`mcp_query_audits` (automatic)	`mcp_query_audits` (automatic)
Response content	Captured (with PII redacted)	Not captured (you make direct LLM calls)
Token usage & cost	Automatic tracking	Manual reporting via `auditLLMCall()`
Decision chain tracing	Full step-by-step trace	Pre-check decision only
User attribution	Full (email, role from context)	Token hash (privacy-preserving)

Audit Tables by Mode

Proxy Mode:
  App → Orchestrator → orchestrator_audit_logs
       ↓
       LLM Provider ← full request/response logged
       ↓
       Agent MCP Handler → mcp_query_audits

Gateway Mode:
  App → Agent → gateway_contexts (pre-check)
       ↓
  App → LLM Provider (direct) ← you control this
       ↓
  App → Agent → llm_call_audits (manual report)
       ↓
       Agent MCP Handler → mcp_query_audits

Compliance Implications

Requirement	Proxy Mode	Gateway Mode
100% audit coverage	✅ Guaranteed (automatic)	⚠️ Depends on SDK usage
Response sampling	✅ Built-in	❌ Not available
Cost attribution	✅ Automatic	⚠️ Manual reporting
EU AI Act Article 12	✅ Full compliance	✅ With proper SDK integration
SEBI/RBI audit trail	✅ Complete	✅ With proper SDK integration

MCP Is Mode-Independent

MCP connectors are first-class citizens in AxonFlow with full policy support regardless of which mode you use for LLM calls:

Static policies: SQLi blocking, PII redaction, dangerous operation blocking
Dynamic policies: Rate limiting, budgets, time-based access, role-based access (when MCP_DYNAMIC_POLICIES_ENABLED=true)
Exfiltration detection: Row/volume limits
Audit logging: Complete audit trail in mcp_query_audits

This means you can use Gateway Mode for LLM calls (lowest latency) while still getting full dynamic policy evaluation on MCP connector access.

Trade-offs Summary

Proxy Mode

Strengths:

Complete audit trail - Every request automatically logged with full context
Multi-Agent Planning - Complex workflows with parallel execution
Response filtering - PII automatically redacted from LLM outputs
Tenant policies - Custom policies from UI/API enforced in real-time
Simpler integration - Single API call, less code to maintain
Decision chain tracing - Step-by-step audit for complex workflows

Considerations:

Additional latency - 20-50ms overhead for in-VPC, 50-100ms for public
Orchestrator dependency - Requests flow through AxonFlow infrastructure
Less control over LLM calls - AxonFlow manages provider routing

Gateway Mode

Strengths:

Lowest latency - Only 10-20ms overhead for policy pre-check
Direct LLM control - You manage provider selection and calls
Framework compatibility - Works with LangChain, CrewAI, LlamaIndex unchanged
Gradual adoption - Add governance without major code changes
No orchestration dependency - Policy evaluation is lightweight

Considerations:

Manual audit reporting - Must call auditLLMCall() after each LLM call
No response filtering - PII redaction must be handled separately
System policies only - Custom tenant policies not enforced
No MAP support - Multi-Agent Planning requires Proxy Mode
Compliance risk - Forgetting audit calls creates gaps in audit trail

Migration Paths

Gateway → Proxy (Recommended)

If you started with Gateway Mode and want full AxonFlow benefits:

// Before: Gateway Mode (3 calls)
const ctx = await axonflow.getPolicyApprovedContext({ userToken, query });
const response = await openai.chat.completions.create({ ... });
await axonflow.auditLLMCall({ contextId: ctx.contextId, ... });

// After: Proxy Mode (1 call, plus MAP access)
const response = await axonflow.executeQuery({
  userToken,
  query,
  requestType: 'chat'
});

Staying with Gateway Mode

If you're happy with Gateway Mode for governance but want to explore MAP:

Consider Proxy Mode for new features that need multi-agent planning
Keep Gateway Mode for existing latency-critical paths
Hybrid approach works well (see below)

Hybrid Approach

You can use both modes in the same application:

// Use Proxy Mode for complex workflows (gets MAP)
app.post('/api/plan-trip', async (req, res) => {
  const plan = await axonflow.generatePlan(req.body.request, 'travel');
  const result = await axonflow.executePlan(plan.planId);
  res.json(result);
});

// Use Gateway Mode for simple, latency-critical chat
app.post('/api/quick-chat', async (req, res) => {
  const ctx = await axonflow.getPolicyApprovedContext({
    userToken: req.user.id,
    query: req.body.message
  });

  if (!ctx.approved) return res.status(403).json({ error: ctx.blockReason });

  const response = await openai.chat.completions.create({ ... });

  // Fire-and-forget audit
  axonflow.auditLLMCall({ contextId: ctx.contextId, ... }).catch(console.error);

  res.json(response);
});

Decision Matrix

Requirement	Recommended Mode
New project	Proxy
Custom policies (UI or API)	Proxy (required)
Multi-Agent Planning	Proxy (required)
100% automatic audit	Proxy
Response filtering (PII)	Proxy
Existing LangGraph/LangChain/CrewAI workflows	Workflow Control Plane
Step-level governance in workflows	Workflow Control Plane
Existing single LLM calls	Gateway
Sub-50ms latency requirement	Gateway
Already managing LLM providers	Gateway
Only need built-in security patterns	Gateway (acceptable)

Summary

Start with Proxy Mode for new projects - you get compliance, MAP, and simpler code
Use Workflow Control Plane for existing multi-step agent workflows (LangGraph, LangChain, CrewAI)
Use Gateway Mode for existing single LLM calls without workflow orchestration
MCP Connectors work in all three modes
You can migrate from Gateway to Proxy over time to unlock more features
Hybrid approach lets you use multiple modes where they make sense

Next Steps

Proxy Mode Deep Dive
Gateway Mode Deep Dive
Workflow Control Plane Guide - Step-level governance for LangChain/LangGraph/CrewAI
Audit Logging Architecture - Detailed audit table schemas and compliance
Multi-Agent Planning Guide
MCP Connectors

TL;DR​

Quick Decision Guide​

Feature Comparison​

When to Choose Proxy Mode​

Ideal Use Cases​

Code Example​

Multi-Agent Planning Example​

When to Choose Gateway Mode​

Ideal Use Cases​

Code Example​

LangChain Integration Example​

When to Choose Workflow Control Plane​

How It Differs from Gateway Mode​

Ideal Use Cases​

Code Example​

MCP Connectors - Available in All Modes​

Latency Comparison​

Audit Logging Comparison​

What Gets Captured​

Audit Tables by Mode​

Compliance Implications​

Trade-offs Summary​

Proxy Mode​

Gateway Mode​

Migration Paths​

Gateway → Proxy (Recommended)​

Staying with Gateway Mode​

Hybrid Approach​

Decision Matrix​

Summary​

Next Steps​

TL;DR

Quick Decision Guide

Feature Comparison

When to Choose Proxy Mode

Ideal Use Cases

Code Example

Multi-Agent Planning Example

When to Choose Gateway Mode

Ideal Use Cases

Code Example

LangChain Integration Example

When to Choose Workflow Control Plane

How It Differs from Gateway Mode

Ideal Use Cases

Code Example

MCP Connectors - Available in All Modes

Latency Comparison

Audit Logging Comparison

What Gets Captured

Audit Tables by Mode

Compliance Implications

Trade-offs Summary

Proxy Mode

Gateway Mode

Migration Paths

Gateway → Proxy (Recommended)

Staying with Gateway Mode

Hybrid Approach

Decision Matrix

Summary

Next Steps