Skip to main content

LLM Providers Overview

AxonFlow lets you run governed LLM traffic in two main ways:

  • Proxy Mode: AxonFlow evaluates policy and makes the model call for you.
  • Gateway Mode: AxonFlow approves and audits the request, while your application calls the model directly.

Proxy Mode and MAP rely on AxonFlow's configured providers. Gateway Mode and WCP let your application keep direct provider control while AxonFlow handles governance, approvals, and audit.

This page focuses on the provider types, configuration surfaces, and routing behavior that are actually present in the current codebase.

ProviderTypical ModelsCommunityEnterpriseNotes
OpenAIgpt-4o, gpt-4o-mini✅✅Default community cloud provider
Anthropicclaude-sonnet-4-20250514, claude-opus-4-20250514✅✅Strong long-context and reasoning workflows
Google Geminigemini-2.0-flash, gemini-2.5-pro, gemini-2.5-flash✅✅Community-available
Mistral AImistral-small-latest, mistral-large-latest, codestral-latest✅✅EU data residency, competitive pricing
Azure OpenAIYour deployment names, such as gpt-4o-mini✅✅Good fit for Azure-first estates
Ollamallama3.2:latest, mistral:latest✅✅Self-hosted and air-gapped deployments
AWS Bedrockanthropic.claude-sonnet-4-20250514-v1:0, meta.llama3-1-70b-instruct-v1:0❌✅Paid tier required
Custom ProviderYour own adapter❌✅Paid tier required

How Provider Configuration Works​

AxonFlow loads provider configuration in this order:

  1. Enterprise database-backed runtime config
  2. YAML config file
  3. Environment variables

That means community deployments usually start with environment variables or axonflow.yaml, while enterprise deployments can move provider management into the customer portal and runtime APIs.

Quick Start​

Environment variables​

# OpenAI
export OPENAI_API_KEY=sk-...
export OPENAI_MODEL=gpt-4o

# Anthropic
export ANTHROPIC_API_KEY=sk-ant-...
export ANTHROPIC_MODEL=claude-sonnet-4-20250514

# Gemini
export GOOGLE_API_KEY=...
export GOOGLE_MODEL=gemini-2.0-flash

# Mistral
export MISTRAL_API_KEY=...
export MISTRAL_MODEL=mistral-small-latest

# Azure OpenAI
export AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com
export AZURE_OPENAI_API_KEY=...
export AZURE_OPENAI_DEPLOYMENT_NAME=gpt-4o-mini
export AZURE_OPENAI_API_VERSION=2024-08-01-preview

# Ollama
export OLLAMA_ENDPOINT=http://localhost:11434
export OLLAMA_MODEL=llama3.2:latest

YAML​

version: "1.0"

llm_providers:
  openai:
    enabled: true
    credentials:
      api_key: ${OPENAI_API_KEY}
    config:
      model: ${OPENAI_MODEL:-gpt-4o}

  anthropic:
    enabled: true
    credentials:
      api_key: ${ANTHROPIC_API_KEY}
    config:
      model: ${ANTHROPIC_MODEL:-claude-sonnet-4-20250514}

  gemini:
    enabled: true
    credentials:
      api_key: ${GOOGLE_API_KEY}
    config:
      model: ${GOOGLE_MODEL:-gemini-2.0-flash}

  mistral:
    enabled: true
    credentials:
      api_key: ${MISTRAL_API_KEY}
    config:
      model: ${MISTRAL_MODEL:-mistral-small-latest}

  ollama:
    enabled: true
    config:
      endpoint: ${OLLAMA_ENDPOINT:-http://localhost:11434}
      model: ${OLLAMA_MODEL:-llama3.2:latest}

Proxy Mode Example​

cURL​

curl -X POST http://localhost:8080/api/request \
  -H "Content-Type: application/json" \
  -d '{
    "user_token": "user-123",
    "client_id": "demo-client",
    "request_type": "chat",
    "query": "Explain the difference between symmetric and asymmetric encryption.",
    "context": {
      "provider": "openai",
      "model": "gpt-4o"
    }
  }'

TypeScript​

import { AxonFlow } from '@axonflow/sdk';

const axonflow = new AxonFlow({
  endpoint: 'http://localhost:8080',
  clientId: process.env.AXONFLOW_CLIENT_ID,
  clientSecret: process.env.AXONFLOW_CLIENT_SECRET,
});

const response = await axonflow.proxyLLMCall({
  userToken: 'user-123',
  query: 'Explain the difference between symmetric and asymmetric encryption.',
  requestType: 'chat',
  context: {
    provider: 'openai',
    model: 'gpt-4o',
  },
});

console.log(response.data);

Go​

package main

import (
    "fmt"

    axonflow "github.com/getaxonflow/axonflow-sdk-go/v5"
)

func main() {
    client := axonflow.NewClient(axonflow.AxonFlowConfig{
        Endpoint:     "http://localhost:8080",
        ClientID:     "demo-client",
        ClientSecret: "demo-secret",
    })

    result, err := client.ProxyLLMCall(
        "user-123",
        "Explain the difference between symmetric and asymmetric encryption.",
        "chat",
        map[string]interface{}{
            "provider": "openai",
            "model":    "gpt-4o",
        },
    )
    if err != nil {
        panic(err)
    }

    fmt.Println(result.Data)
}

Provider Selection and Routing​

Provider routing is decided server-side.

  • context.provider is a provider preference by default.
  • context.strict_provider=true makes that choice a hard pin for the request.
  • Routing strategy can be configured at the platform level with weighted, round-robin, failover, and enterprise-only cost-optimized routing.

See Provider Routing for the full routing guide. Use Provider And Credential Matrix when you need the fastest comparison of runtime provider names, portal-managed names, and credential patterns.

Choosing the Right Provider​

NeedGood Starting Point
Fastest community cloud startOpenAI or Anthropic
EU data residency or cost-optimized workloadsMistral
Azure-first enterprise stackAzure OpenAI
Google ecosystem or multimodal-heavy workflowsGemini
Local inference or air-gapped deploymentOllama
Regulated AWS estates and Bedrock governanceBedrock

Production Guidance​

  • Start with one or two providers in community mode and validate policy behavior end to end.
  • Move to evaluation when you need more headroom, more realistic limits, and pre-production testing.
  • Move to enterprise when provider operations, Bedrock, custom providers, runtime credential handling, and governance guarantees become part of the deployment requirement.

Next Steps​

Last updated on