Enterprise Provider Operations

Provider operations are the enterprise surface for deciding which model backends are available, how traffic is routed between them, and how those decisions are audited inside the current tenant or environment.

Use the public setup pages for baseline provider configuration. Use this page when the rollout needs portal-managed provider records, secret references, cost metadata, or day-two routing changes.

Portal-Managed Provider Names

The current Customer Portal provider-management API accepts these runtime-managed provider names:

• bedrock
• ollama
• openai
• anthropic

The broader runtime can support additional providers through environment or config-file paths. Keep that distinction clear: a provider can be runtime-supported without being a portal-managed provider type today.

Provider Record Shape

Each provider record can hold:

• display_name
• config
• credentials_secret_arn
• priority
• weight
• enabled
• cost_per_1k_input_tokens
• cost_per_1k_output_tokens

This supports common enterprise tasks:

• primary and fallback provider configuration
• phased migration between providers
• tenant-scoped cost attribution
• provider disablement during incidents

Portal API Surface

The portal registers these provider-management endpoints:

Method	Path	Purpose
GET	/api/v1/llm-providers	List providers for the current session tenant
POST	/api/v1/llm-providers	Create a provider record
PUT	/api/v1/llm-providers/routing	Update priority, weight, and enabled across providers
GET	/api/v1/llm-providers/{providerName}	Fetch one provider
PUT	/api/v1/llm-providers/{providerName}	Update one provider
DELETE	/api/v1/llm-providers/{providerName}	Delete one provider

These are portal-authenticated endpoints. For CLI or scripted examples, model them with the axonflow_session cookie rather than a bearer-token flow.

Example: Create a Provider

curl -X POST https://portal.example.com/api/v1/llm-providers \
  --cookie "axonflow_session=$AXONFLOW_SESSION" \
  -H "Content-Type: application/json" \
  -d '{
    "provider_name": "bedrock",
    "display_name": "Bedrock Primary",
    "config": {
      "region": "us-east-1",
      "model": "anthropic.claude-sonnet-4-20250514-v1:0"
    },
    "credentials_secret_arn": "arn:aws:secretsmanager:us-east-1:123456789012:secret:axonflow/bedrock-primary",
    "priority": 10,
    "weight": 0.7,
    "cost_per_1k_input_tokens": 0.003,
    "cost_per_1k_output_tokens": 0.015
  }'

Important behavior:

• provider_name is required
• weight must be between 0 and 1
• default priority is 5
• default weight is 1.0

Example: Update a Provider

Use partial updates when a provider remains the same logical backend but the operating configuration changes:

curl -X PUT https://portal.example.com/api/v1/llm-providers/bedrock \
  --cookie "axonflow_session=$AXONFLOW_SESSION" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "Bedrock Primary - EU workloads",
    "config": {
      "region": "eu-central-1",
      "model": "eu.anthropic.claude-sonnet-4-5-20250929-v1:0"
    },
    "priority": 20,
    "weight": 0.8,
    "enabled": true
  }'

This is the right operation when you are shifting traffic to a new region, changing a model, temporarily disabling a provider, or updating chargeback metadata.

Routing Updates

The portal routing contract is provider-level weighting and ordering, not a separate strategy resource:

curl -X PUT https://portal.example.com/api/v1/llm-providers/routing \
  --cookie "axonflow_session=$AXONFLOW_SESSION" \
  -H "Content-Type: application/json" \
  -d '{
    "providers": [
      {"provider_name": "bedrock", "priority": 10, "weight": 0.7, "enabled": true},
      {"provider_name": "openai", "priority": 5, "weight": 0.3, "enabled": true}
    ]
  }'

Use this for migration cutovers, canary traffic, controlled fallback posture, and provider maintenance windows.

Credential Model

For production enterprise deployments, store only the secret reference in the provider record. The routine provider response masks the credentials_secret_arn, which lets platform teams confirm the right secret is wired without exposing raw credentials.

The current provider-management APIs store:

• provider metadata in llm_provider_configs
• an optional credentials_secret_arn
• audit records for provider create, update, delete, and routing changes

Keep a clear split between:

• config: model, region, endpoint, and similar runtime settings
• credentials_secret_arn: the secret reference used to obtain credentials

That distinction matters when you need to review whether a routing change was about economics or security, whether a provider failure was caused by auth or model config, and whether a tenant should reuse a secret across multiple provider records.

Recommended credential pattern

For production enterprise deployments, use a secret manager and store only the secret reference in AxonFlow provider records. In AWS deployments, that usually means AWS Secrets Manager ARNs.

This gives you:

• separation of duties between platform engineering and secret owners
• a cleaner audit trail
• easier credential rotation
• less accidental exposure in portal workflows

Good rotation patterns are:

• rotate the secret value behind the same ARN when the provider record should stay stable
• create a new secret ARN and update the provider when you need explicit cutover traceability

The API does not expose a dedicated credential-rotation endpoint today. Treat rotation as a secret-manager workflow plus a provider-record update when the ARN changes.

Secret rotation runbook

Use this default rotation workflow:

1. Create a new secret or rotate the existing one in your secret manager.
2. Validate the provider configuration in a lower environment if the provider model, region, endpoint, or permission boundary also changed.
3. Update the provider record only if the ARN or secret reference changed.
4. Verify provider health, routing, and cost tagging after cutover.
5. Confirm the expected provider-management audit entry exists.

When the consuming provider record should stay stable, rotate the secret value behind the same ARN. When you need explicit cutover traceability, create a new secret ARN and update the provider:

curl -X PUT https://portal.example.com/api/v1/llm-providers/openai \
  --cookie "axonflow_session=$AXONFLOW_SESSION" \
  -H "Content-Type: application/json" \
  -d '{
    "credentials_secret_arn": "arn:aws:secretsmanager:us-east-1:123456789012:secret:axonflow/openai-prod-v2"
  }'

Credential guidance by provider

Bedrock

Bedrock usually relies on AWS runtime credentials, account or role access, and region/model configuration rather than raw third-party API keys. In enterprise environments, the key review questions are:

• which AWS account or role provides Bedrock access
• which region the provider is configured for
• whether the workload uses a model ID or inference profile
• whether the role boundary matches the deployment environment

See AWS Bedrock Setup.

Ollama

Ollama typically does not need an external vendor API key. The sensitive value is usually the reachable endpoint and the network boundary around the serving cluster.

Credential-management reviews for Ollama often become:

• network reachability reviews
• TLS and internal service identity reviews
• host isolation and GPU node hardening reviews
• checks that the configured endpoint matches the intended private serving cluster

See Ollama Setup.

OpenAI and Anthropic

For vendor-hosted APIs, prefer dedicated production secrets per environment or tenant grouping. That makes chargeback cleaner, incident isolation easier, and procurement or renewal reviews easier to audit.

Health and Cost Fields

Provider records can expose operational fields such as:

• health_status
• last_health_check
• last_error
• cost_per_1k_input_tokens
• cost_per_1k_output_tokens

Use those fields for enterprise cost reviews, production readiness checks, incident response, and finance-facing comparisons of provider economics.

Common Misconceptions

The current portal provider-management surface does not provide:

• a separate credential-rotation endpoint
• a separate health-check policy endpoint
• a separate routing-strategy endpoint

Treat those capabilities as runbook concerns built around create, update, and routing calls rather than assuming a dedicated API exists.

Provider-Specific Notes

Provider	Enterprise operating note
bedrock	Requires a region in provider config. If model is omitted, the runtime uses its Bedrock default. Model-family detection supports Anthropic, Amazon, Meta, and Mistral Bedrock model IDs.
ollama	Defaults to a local endpoint and model if omitted. Health status is inferred from successful completions; the portal API does not run a dedicated Ollama tags health check.
openai	Managed through the same provider record shape as other portal-managed providers.
anthropic	Managed through the same provider record shape as other portal-managed providers.

Azure OpenAI Boundary

Azure OpenAI is available through the public runtime provider path, and it is often the provider enterprises want politically and operationally. The current protected Customer Portal provider-management allowlist does not expose Azure OpenAI as a distinct portal-managed provider type.

Read Azure OpenAI as:

• a supported provider path in the runtime
• an enterprise operating pattern for Azure-first organizations
• not a separate azure-openai portal-managed provider record in the current portal API

Use Azure OpenAI Setup for baseline setup.

Azure OpenAI is usually chosen when an enterprise needs Azure-first governance, regional alignment with a broader Azure estate, Microsoft procurement standardization, or internal AI rollout behind a platform team.

For higher-risk workloads, the operating pattern is:

1. apply policy and governance in AxonFlow
2. capture the approved context and audit boundary
3. invoke the Azure-hosted model through the configured runtime path
4. record the resulting call in the enterprise audit posture

That is stronger than allowing every application team to call Azure OpenAI directly with its own credentials and reporting model.

Deployment Scope

The current portal provider-management handlers filter provider records by the authenticated session tenant. They do not expose a cross-tenant provider listing endpoint.

Mode	Behavior
SaaS	Providers are isolated to the current tenant
In-VPC	Providers are still read through the current session tenant; single-tenant deployments often behave like one shared enterprise routing posture

Recommended Naming Pattern

Use provider names and display names that make operational intent obvious:

• bedrock-primary-us
• bedrock-eu-regulated
• ollama-private-research
• openai-fallback
• anthropic-eval

That makes routing changes, audits, and incident response easier to reason about.

Related Docs

• Provider And Credential Matrix
• Provider Routing
• LLM Provider Management API
• Azure OpenAI Setup
• AWS Bedrock Setup