Skip to main content

OpenAI-Compatible Gateway

Route your existing OpenAI SDK calls through AxonFlow by changing a single line of code. AxonFlow enforces policies, records audit logs, and tracks token usage — then forwards the request to your upstream provider.

Your app → POST /v1/chat/completions (AxonFlow) → Policy check → OpenAI API → Audit → Response

Quick Start​

Python​

import os
from openai import OpenAI

client = OpenAI(
    api_key="any-value",                          # AxonFlow auth (community mode: any value)
    base_url="http://localhost:8080/v1",           # Point at AxonFlow instead of OpenAI
    default_headers={
        "X-Provider-Key": os.environ["OPENAI_API_KEY"]  # Your real OpenAI key
    }
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What is 2+2?"}],
    max_tokens=100
)

print(response.choices[0].message.content)  # "4"
print(response.usage.total_tokens)          # e.g., 21

TypeScript​

import OpenAI from 'openai';

const client = new OpenAI({
    apiKey: 'any-value',
    baseURL: 'http://localhost:8080/v1',
    defaultHeaders: {
        'X-Provider-Key': process.env.OPENAI_API_KEY!
    }
});

const response = await client.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'What is 2+2?' }],
    max_tokens: 100
});

console.log(response.choices[0].message.content);  // "4"
console.log(response.usage?.total_tokens);          // e.g., 21

curl​

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-Provider-Key: $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "What is 2+2?"}],
    "max_tokens": 100
  }'

What Happens Behind the Scenes​

When your request hits /v1/chat/completions, AxonFlow:

  1. Authenticates the request (same auth as all AxonFlow endpoints)
  2. Evaluates policies against the message content — PII detection, SQL injection blocking, dangerous query prevention, compliance rules
  3. Blocks the request if any policy triggers a deny (returns an OpenAI-compatible error)
  4. Forwards the request to the upstream provider using your X-Provider-Key
  5. Records audit — model, provider, token usage, cost estimate, latency, policy decision
  6. Returns the provider's response unchanged, plus AxonFlow headers

The response you receive is identical to what OpenAI would return directly, with two additional headers:

HeaderDescription
X-AxonFlow-Decision-IdUUID for this request in audit logs
X-AxonFlow-Trace-IdW3C-compatible 32-hex trace ID for observability

Provider Key​

Pass your upstream provider API key in the X-Provider-Key header. AxonFlow does not store this key — it is forwarded to the upstream provider for the duration of the request only.

# Python: set via default_headers
client = OpenAI(
    base_url="http://localhost:8080/v1",
    default_headers={"X-Provider-Key": os.environ["OPENAI_API_KEY"]}
)

# Or per-request via extra_headers
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[...],
    extra_headers={"X-Provider-Key": os.environ["OPENAI_API_KEY"]}
)

If the header is missing, AxonFlow returns a clear error:

{
  "error": {
    "message": "X-Provider-Key header is required. Pass your upstream provider API key in this header.",
    "type": "invalid_request_error",
    "code": "missing_provider_key"
  }
}

Policy Enforcement​

AxonFlow evaluates the same policy categories as Gateway Mode and Decision Mode:

  • PII detection — SSN, credit cards, Aadhaar, PAN, email, phone numbers
  • SQL injection — UNION-based injection, tautology attacks, stacked queries
  • Dangerous queries — attempts to access system tables, credentials, admin functions
  • Compliance — RBI, SEBI, EU AI Act, MAS FEAT framework rules

When a policy denies a request, AxonFlow returns an OpenAI-compatible error that the SDK parses correctly:

from openai import BadRequestError

try:
    client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "My SSN is 123-45-6789"}]
    )
except BadRequestError as e:
    print(e.status_code)  # 400
    print(e.body)
    # {'error': {'message': 'Request blocked by policy: Social Security Number detected',
    #            'type': 'policy_violation', 'code': 'policy_denied'}}

The error envelope shape ({error: {message, type, code}}) matches OpenAI's, so the OpenAI SDK parses it as BadRequestError and your existing error handling works unchanged. The type and code values (policy_violation, policy_denied) are AxonFlow-specific — check for them explicitly to branch on policy denials.

Authentication​

The endpoint uses the same authentication as all AxonFlow endpoints:

ModeAuthentication
Community (self-hosted)No auth required — api_key can be any value
Enterprise (self-hosted)Basic Auth with client_id:client_secret
Community SaaS (try.getaxonflow.com)Registration credentials

In enterprise mode, set your AxonFlow credentials as the api_key:

client = OpenAI(
    api_key="your-axonflow-client-id:your-axonflow-client-secret",
    base_url="https://your-axonflow-host/v1",
    default_headers={"X-Provider-Key": os.environ["OPENAI_API_KEY"]}
)

Audit and Cost Tracking​

Every request is recorded in the llm_call_audits table with:

FieldDescription
providerUpstream provider (e.g., openai)
modelModel used (e.g., gpt-4o-mini)
prompt_tokensInput tokens
completion_tokensOutput tokens
total_tokensTotal tokens
estimated_cost_usdEstimated cost based on published pricing
latency_msProvider response time
metadata.verdictPolicy decision (allow or deny)
metadata.decision_idCorrelation ID (matches X-AxonFlow-Decision-Id header)

Denied requests are also audited with verdict: "deny" and zero token counts (the request never reached the provider).

Comparison with Other Modes​

FeatureOpenAI-Compatible GatewayProxy ModeGateway ModeDecision Mode
SDK changebaseURL onlyAxonFlow SDKAxonFlow SDKCustom PEP
Request formatOpenAI nativeAxonFlow formatAxonFlow formatAxonFlow format
Provider callAxonFlow forwardsAxonFlow forwardsYour code callsYour code calls
Policy enforcementPre-requestPre-requestPre-checkPre-request
AuditAutomaticAutomaticTwo-step (pre-check + audit)Per-decision
StreamingNot yet supportedSupportedN/AN/A
Custom policiesSystem policiesSystem + tenantSystem policiesSystem policies

Choose OpenAI-Compatible Gateway when you already use the OpenAI SDK and need policy enforcement + audit without changing your request format. No new SDK to learn.

Choose Proxy Mode when you need tenant-specific custom policies or your workflow requires AxonFlow's multi-step execution model.

Limitations​

This is the first release of the OpenAI-Compatible Gateway:

  • Non-streaming only. Requests with stream: true return HTTP 400 with a clear error message. Streaming support is planned for a future release.
  • Caller-supplied provider key. You pass your API key in the X-Provider-Key header. Server-side provider key management is planned.
  • OpenAI routing only. This release routes every request to OpenAI regardless of the model value. Model prefixes (gpt-, o1, o3, o4, chatgpt-) all map to api.openai.com. Anthropic, Gemini, and Bedrock routing are planned.
  • No upstream header passthrough. AxonFlow forwards only the request body. Provider-specific headers in your request (OpenAI-Beta, OpenAI-Organization, OpenAI-Project) are not forwarded — pass equivalent configuration in the request body where the provider supports it.
  • System policies only. Custom tenant policies created in the Portal UI require Proxy Mode.
  • Chat Completions only. The /v1/chat/completions endpoint is supported. Other OpenAI endpoints (embeddings, images, audio) are not proxied.

Troubleshooting​

"X-Provider-Key header is required"​

Pass your upstream provider API key in the X-Provider-Key header. See the Provider Key section.

"Streaming is not supported in this release"​

Remove stream: true from your request. Streaming support is planned for a future release.

Upstream provider returns an error​

If your provider key is invalid or the model doesn't exist, the upstream error is forwarded to your SDK as-is. Check the error message — it comes directly from the provider.

Policy denials​

If your request is blocked, the error body contains the policy violation reason. Adjust your prompt to avoid the detected pattern, or configure your AxonFlow deployment's policy actions (PII_ACTION, SQLI_ACTION) if the detection is too strict for your use case.

Last updated on