Audit Logging

AxonFlow provides comprehensive audit logging for every AI interaction, capturing complete request/response data for compliance, debugging, and observability.

Community Examples

See complete working examples in the AxonFlow examples repository.

Architecture Overview

AxonFlow implements a multi-layer logging architecture:

Request Flow:
┌─────────────────────────────────────────────────────────────────┐
│  Your Application                                               │
└─────────────────┬───────────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────────────────────────┐
│  AxonFlow Agent                                                 │
│  ┌─────────────────┐    ┌─────────────────────────────────────┐ │
│  │ Policy Engine   │───▶│ Agent Audit Queue                   │ │
│  │ (Gateway Mode)  │    │ - Pre-check decisions               │ │
│  └─────────────────┘    │ - Policy violations                 │ │
│                         │ - PII detections                    │ │
│  ┌─────────────────┐    │                                     │ │
│  │ MCP Handler     │───▶│ - MCP query audits (NEW)            │ │
│  │                 │    │ - Connector access logs             │ │
│  └─────────────────┘    └─────────────────────────────────────┘ │
└─────────────────┬───────────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────────────────────────┐
│  AxonFlow Orchestrator                                          │
│  ┌─────────────────┐    ┌─────────────────────────────────────┐ │
│  │ LLM Routing     │───▶│ Orchestrator Audit Logger           │ │
│  │ MAP Planning    │    │ - LLM call details                  │ │
│  └─────────────────┘    │ - Token usage & cost                │ │
│                         │ - Response metadata                 │ │
│                         └─────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘

What Gets Logged

Gateway Context (Pre-Check)

Every policy pre-check creates a gateway_context record:

Field	Description
`context_id`	Unique identifier for correlation
`user_id`	User making the request
`client_id`	Application or service identifier
`query`	Original prompt (with optional PII redaction)
`approved`	Whether policy allowed the request
`policies_applied`	List of policies evaluated
`policy_violations`	Any violations detected
`pii_detected`	Types of PII found (SSN, credit card, etc.)
`created_at`	Timestamp
`metadata`	Custom context (department, project, etc.)

LLM Call Audit

After LLM processing, an llm_call_audit record is created:

Field	Description
`context_id`	Links to gateway_context
`provider`	LLM provider (openai, anthropic, bedrock)
`model`	Specific model used (gpt-4, claude-opus-4)
`input_tokens`	Token count for prompt
`output_tokens`	Token count for response
`total_tokens`	Combined token usage
`latency_ms`	Response time in milliseconds
`response_summary`	Truncated response (configurable)
`cost_usd`	Estimated cost based on token pricing
`created_at`	Timestamp

Agent Audit Logs

Policy enforcement events captured by the Agent (via audit_queue.go):

Entry Type	Data Captured
`violation`	Severity (critical/high/medium/low), policy name, block reason
`metric`	Performance metrics, latency, throughput
`audit`	General audit trail entries
`gateway_context`	SDK pre-check context (synchronous write)
`llm_call_audit`	SDK LLM call audit (synchronous write)

Orchestrator Audit Logs

All requests captured by the Orchestrator (via audit_logger.go):

Field	Description
`user_id`, `user_email`, `user_role`	User identity
`client_id`, `tenant_id`	Client/tenant isolation
`request_type`	Type of operation
`query`, `query_hash`	Request content (hashed for deduplication)
`policy_decision`	allowed, blocked, redacted, error
`provider`, `model`	LLM provider and model used
`response_time_ms`, `tokens_used`, `cost`	Performance and billing
`compliance_flags`	hipaa_relevant, gdpr_applicable, sox_relevant, pii_access
`security_metrics`	risk_score, query_complexity (low/medium/high)

MCP Query Audits

All MCP connector operations are logged in the mcp_query_audits table with complete policy evaluation results. This applies to both Gateway and Proxy mode operations.

Added in v3.3.0

MCP audit logging was added in AxonFlow v3.3.0 to provide a complete audit trail for connector access.

Three-Phase Policy Evaluation

MCP audit logging captures results from three policy evaluation phases:

Phase	What's Captured	Example
REQUEST	SQL injection blocking, dangerous operation detection	Query blocked due to DROP TABLE pattern
RESPONSE	PII detection and redaction	SSN and credit card fields redacted
EXFILTRATION	Row count and volume limit violations	Query exceeded 10K row limit

Audit Fields

Field	Description
`audit_id`	Unique identifier (matches SDK `PolicyInfo.audit_id`)
`tenant_id`, `client_id`, `user_id`	Identity context
`connector_name`	MCP connector accessed
`operation`	Operation type (query, execute, etc.)
`statement_hash`	SHA256 hash of the query (privacy-preserving)
`request_blocked`	Whether request was blocked before execution
`request_block_reason`	Block reason (e.g., "SQL injection detected")
`request_matched_policies`	Array of policy IDs that matched
`response_redacted`	Whether PII was redacted from response
`response_redacted_fields`	JSONPath of redacted fields
`exfil_rows_returned`	Number of rows returned
`exfil_exceeded`	Whether exfiltration limit was exceeded
`exfil_limit_type`	Type of limit exceeded (row_count, byte_size)
`duration_ms`	Total operation duration
`success`	Whether operation completed successfully
`created_at`	Timestamp

Privacy by Design

MCP audit logs protect sensitive data:

Statement hashing: Only SHA256 hash of SQL stored, not the raw query
No response data: Only metadata captured, not actual query results
Field paths only: Redacted field paths logged, not the original values

SDK Correlation

The audit_id field in PolicyInfo correlates SDK responses with audit entries:

// Go SDK
resp, _ := client.MCPQuery(ctx, req)
if resp.PolicyInfo != nil {
    // audit_id can be used to look up the mcp_query_audits record
    fmt.Printf("Audit ID: %s\n", resp.PolicyInfo.AuditID)
}

Example Queries

Daily MCP operations summary:

SELECT
    DATE_TRUNC('day', created_at) AS date,
    connector_name,
    COUNT(*) AS total_queries,
    COUNT(*) FILTER (WHERE request_blocked = true) AS blocked,
    COUNT(*) FILTER (WHERE response_redacted = true) AS redacted,
    COUNT(*) FILTER (WHERE exfil_exceeded = true) AS exfil_violations
FROM mcp_query_audits
WHERE tenant_id = 'your-tenant'
GROUP BY 1, 2
ORDER BY date DESC;

Security incident investigation (blocked SQLi attempts):

SELECT created_at, client_id, request_block_reason, statement_hash
FROM mcp_query_audits
WHERE request_blocked = true
  AND request_block_reason LIKE '%SQL injection%'
ORDER BY created_at DESC
LIMIT 100;

PII redaction report:

SELECT
    connector_name,
    UNNEST(response_redacted_fields) AS redacted_field,
    COUNT(*) AS redaction_count
FROM mcp_query_audits
WHERE response_redacted = true
GROUP BY 1, 2
ORDER BY redaction_count DESC;

Workflow Audit Logs

Workflow Control Plane (WCP) operations are logged to provide a complete audit trail for external orchestrators (LangChain, LangGraph, CrewAI). Every workflow lifecycle event is recorded in the audit_logs table.

Added in v3.4.0

Workflow audit logging was added in AxonFlow v3.4.0 as part of the Workflow Control Plane feature.

Workflow Operations Logged

Request Type	When Logged	Data Captured
`workflow_created`	Workflow registration	workflow_id, workflow_name, source, total_steps, metadata
`workflow_step_gate`	Step gate check	workflow_id, step_id, step_name, step_type, decision, policies_evaluated, policies_matched
`workflow_step_completed`	Step marked complete	workflow_id, step_id, output
`workflow_completed`	Workflow finishes	workflow_id, final_status
`workflow_aborted`	Workflow cancelled	workflow_id, reason

SDK Correlation

Use SDK audit search methods to query workflow audit logs:

# Python SDK
from datetime import datetime, timedelta, timezone
from axonflow.types import AuditSearchRequest, AuditQueryOptions

# Search for workflow audit logs from the last hour
response = await client.search_audit_logs(
    AuditSearchRequest(
        start_time=datetime.now(timezone.utc) - timedelta(hours=1),
        limit=100,
    )
)

# Filter for specific workflow
workflow_id = "wf_abc123"
for entry in response.entries:
    if entry.request_id == workflow_id:
        print(f"{entry.request_type}: {entry.query_summary}")

// Go SDK
auditLogs, err := client.SearchAuditLogs(ctx, &axonflow.AuditSearchRequest{
    StartTime: &startTime,
    Limit:     100,
})
for _, entry := range auditLogs.Entries {
    if entry.RequestID == workflowID {
        fmt.Printf("%s: %s\n", entry.RequestType, entry.QuerySummary)
    }
}

// TypeScript SDK
const response = await client.searchAuditLogs({
  clientId: "workflow-service",
  limit: 100,
});

for (const entry of response.entries) {
  if (entry.requestId === workflowId) {
    console.log(`${entry.requestType}: ${entry.querySummary}`);
  }
}

// Java SDK
AuditSearchResponse response = client.searchAuditLogs(
    AuditSearchRequest.builder()
        .clientId("workflow-service")
        .limit(100)
        .build()
);

for (AuditLogEntry entry : response.getEntries()) {
    if (workflowId.equals(entry.getRequestId())) {
        System.out.println(entry.getRequestType() + ": " + entry.getQuerySummary());
    }
}

# Python SDK - tenant scoped query
tenant_logs = await client.get_audit_logs_by_tenant(
    "workflow-service",
    AuditQueryOptions(limit=25, offset=0),
)
print(f"Tenant entries: {len(tenant_logs.entries)}")

// TypeScript SDK - tenant scoped query
const tenantLogs = await client.getAuditLogsByTenant("workflow-service", {
  limit: 25,
  offset: 0,
});
console.log(`Tenant entries: ${tenantLogs.entries.length}`);

// Java SDK - tenant scoped query
AuditSearchResponse tenantLogs = client.getAuditLogsByTenant(
    "workflow-service",
    AuditQueryOptions.builder().limit(25).offset(0).build()
);
System.out.println("Tenant entries: " + tenantLogs.getEntries().size());

Example Queries

Daily workflow operations summary:

SELECT
    DATE_TRUNC('day', timestamp) AS date,
    request_type,
    COUNT(*) AS count
FROM audit_logs
WHERE request_type LIKE 'workflow_%'
GROUP BY 1, 2
ORDER BY date DESC, count DESC;

Blocked workflow steps (policy violations):

SELECT timestamp, request_id, query_summary, metadata->>'reason' AS reason
FROM audit_logs
WHERE request_type = 'workflow_step_gate'
  AND metadata->>'decision' = 'block'
ORDER BY timestamp DESC
LIMIT 100;

Media Audit Records

When media (images, documents) is submitted alongside a request, the audit trail captures media-specific metadata for traceability and compliance.

Field	Description	Edition
`sha256_hash`	SHA-256 hash of the submitted file	Community
`mime_type`	Detected MIME type (e.g., `image/png`)	Community
`file_size`	File size in bytes	Community
`has_pii` / `pii_types`	Whether PII was detected, and which types, via OCR extraction	Community
`content_safe`	Boolean: whether the image passed content safety checks	Community
`warnings`	Validation warnings (format, dimensions)	Community
`has_faces`	Whether faces were detected in the image	Enterprise
`has_biometric_data`	Whether biometric indicators were found	Enterprise
`nsfw_score` / `violence_score`	Content safety confidence scores	Enterprise
`document_classification`	Document type label from classifier	Enterprise
`analyzer_details`	Per-analyzer results (Rekognition, Vision, etc.)	Enterprise

Community records cover validation, OCR-based PII, and SHA-256 hashing. Enterprise records add cloud analyzer results, biometric detection, and content safety scores. See Media Governance for configuration details.

Plan (MAP) Audit Logs

Multi-Agent Planning (MAP) operations are logged to provide visibility into plan creation, execution, and completion. All plan lifecycle events are recorded in the audit_logs table.

Added in v3.4.0

Plan audit logging was added in AxonFlow v3.4.0 to enhance observability for multi-agent workflows.

Plan Operations Logged

Request Type	When Logged	Data Captured
`plan_created`	Plan stored	plan_id, query, domain, step_count, tenant_id, client_id
`plan_execution_started`	Execution begins	plan_id, domain, step_count
`plan_completed`	Execution succeeds	plan_id, domain, final_status
`plan_failed`	Execution fails	plan_id, domain, error_message

Example Queries

Plan execution success rate:

SELECT
    DATE_TRUNC('day', timestamp) AS date,
    COUNT(*) FILTER (WHERE request_type = 'plan_completed') AS succeeded,
    COUNT(*) FILTER (WHERE request_type = 'plan_failed') AS failed,
    ROUND(
        100.0 * COUNT(*) FILTER (WHERE request_type = 'plan_completed') /
        NULLIF(COUNT(*) FILTER (WHERE request_type IN ('plan_completed', 'plan_failed')), 0),
        2
    ) AS success_rate
FROM audit_logs
WHERE request_type IN ('plan_completed', 'plan_failed')
GROUP BY 1
ORDER BY date DESC;

Recent failed plans with error details:

SELECT timestamp, request_id AS plan_id, metadata->>'error' AS error
FROM audit_logs
WHERE request_type = 'plan_failed'
ORDER BY timestamp DESC
LIMIT 20;

Query Examples

Common audit log queries for day-to-day operations:

Search by date range and user:

curl -X POST http://localhost:8080/audit/search \
  -H "Content-Type: application/json" \
  -d '{
    "user_email": "[email protected]",
    "start_time": "2026-01-01T00:00:00Z",
    "end_time": "2026-01-31T23:59:59Z",
    "limit": 50
  }'

Search by policy action (all blocked requests):

curl -X POST http://localhost:8080/audit/search \
  -H "Content-Type: application/json" \
  -d '{
    "request_type": "policy_violation",
    "start_time": "2026-02-01T00:00:00Z",
    "end_time": "2026-02-07T23:59:59Z",
    "limit": 100
  }'

Search by client ID (specific application):

curl -X POST http://localhost:8080/audit/search \
  -H "Content-Type: application/json" \
  -d '{
    "client_id": "customer-support-agent",
    "limit": 25
  }'

SQL: Top users by blocked requests in the last 7 days:

SELECT user_email, COUNT(*) AS blocked_count
FROM audit_logs
WHERE policy_decision = 'blocked'
  AND timestamp >= NOW() - INTERVAL '7 days'
GROUP BY user_email
ORDER BY blocked_count DESC
LIMIT 20;

Retention Policies

AxonFlow supports configurable retention periods to meet regulatory requirements:

Framework	Required Retention	AxonFlow Support
SEBI AI/ML	5 years (1825 days)	✅ Enterprise
RBI FREE-AI	7 years (2555 days)	✅ Enterprise
EU AI Act	7 years (2555 days) for decision_chain	✅ Enterprise
HIPAA	6 years (2190 days)	✅ Enterprise
Custom	Configurable	✅ All editions

Configuration

Retention is configured per-log-type via YAML config or environment variables:

# axonflow-config.yaml
audit:
  retention:
    gateway_contexts: 365     # Days to retain pre-check logs
    llm_call_audits: 365      # Days to retain LLM call logs
    agent_audit_logs: 2555    # 7 years for RBI compliance
    orchestrator_logs: 1825   # 5 years for SEBI compliance

Environment variable equivalents:

Environment Variable	Default	Description
`AXONFLOW_AUDIT_RETENTION_GATEWAY`	`365`	Days to retain gateway context logs
`AXONFLOW_AUDIT_RETENTION_LLM`	`365`	Days to retain LLM call audit logs
`AXONFLOW_AUDIT_RETENTION_AGENT`	`365`	Days to retain agent audit logs
`AXONFLOW_AUDIT_RETENTION_ORCHESTRATOR`	`365`	Days to retain orchestrator logs
`AXONFLOW_AUDIT_RETENTION_MCP`	`365`	Days to retain MCP query audit logs
`AXONFLOW_AUDIT_CLEANUP_INTERVAL`	`24h`	How often the retention cleanup job runs
`AXONFLOW_AUDIT_FALLBACK_PATH`	`/var/log/axonflow/audit_fallback.jsonl`	Path for fallback JSONL file

Export Formats

Community Edition

Format	Use Case
JSON	Programmatic access, API responses
CSV	Spreadsheet analysis, basic reporting

Enterprise Edition

Format	Use Case
JSON	Programmatic access, API responses
CSV	Spreadsheet analysis, basic reporting
PDF	Compliance reports, board presentations
XLSX	Advanced Excel analysis with formatting
XML	SEBI regulatory submission format

Export API

# Export last 30 days of audit logs (JSON)
curl -X GET "http://localhost:8080/api/audit/export?days=30&format=json" \
  -H "Authorization: Bearer $AXONFLOW_API_KEY"

# Export for SEBI compliance (Enterprise)
curl -X GET "http://localhost:8080/api/v1/sebi/audit-export?start=2024-01-01&end=2024-12-31" \
  -H "Authorization: Bearer $AXONFLOW_LICENSE_KEY"

Fallback Mechanism

If the database is temporarily unavailable, audit logs are written to a local JSONL file:

/var/log/axonflow/audit_fallback.jsonl

On service restart, RecoverFromFallback() automatically replays entries back to the database:

Reads JSONL file line-by-line
Retries failed entries with exponential backoff
Truncates fallback file when recovery succeeds

SIEM Integration

AxonFlow integrates with enterprise security information and event management (SIEM) systems:

Platform	Integration Method	Status
CloudWatch	Native (AWS deployments)	Community + Enterprise
Splunk	HTTP Event Collector	Enterprise
Datadog	Log forwarding	Enterprise
Elastic/ELK	Logstash integration	Enterprise

CloudWatch Integration (AWS)

Audit logs are automatically forwarded to CloudWatch when deployed in AWS:

# View audit logs in CloudWatch
aws logs filter-log-events \
  --log-group-name "/ecs/axonflow-agent" \
  --filter-pattern '"audit"'

Prometheus Metrics

AxonFlow exposes 80+ Prometheus metrics for audit observability:

Key Audit Metrics

Metric	Type	Description
`axonflow_audit_logs_total`	Counter	Total audit log entries
`axonflow_audit_write_latency_ms`	Histogram	Time to write audit log
`axonflow_audit_fallback_active`	Gauge	1 if using fallback mode
`axonflow_policy_violations_total`	Counter	Total policy violations
`axonflow_pii_detections_total`	Counter	PII detections by type

Example PromQL Queries

# Audit write latency P95
histogram_quantile(0.95, rate(axonflow_audit_write_latency_ms_bucket[5m]))

# Policy violations per minute
rate(axonflow_policy_violations_total[1m])

# PII detections by type
sum by (pii_type) (increase(axonflow_pii_detections_total[1h]))

API Reference

Query Audit Logs

# Search audit logs (POST with criteria)
POST /audit/search
{
  "user_email": "[email protected]",
  "client_id": "client-123",
  "start_time": "2024-01-01T00:00:00Z",
  "end_time": "2024-01-31T23:59:59Z",
  "request_type": "llm_call",
  "limit": 100
}

# Get tenant-specific audit logs
GET /audit/tenant/{tenant_id}

Response Example

{
  "logs": [
    {
      "id": "audit_abc123",
      "context_id": "ctx_xyz789",
      "user_id": "user-123",
      "event_type": "policy_violation",
      "policy_name": "pii_ssn_detection",
      "query_hash": "sha256:abc...",
      "pii_types": ["ssn"],
      "blocked": true,
      "created_at": "2024-01-15T10:30:00Z"
    }
  ],
  "total": 1523,
  "limit": 100,
  "offset": 0
}

Community vs Enterprise

Feature	Community	Enterprise
Basic audit logging	✅	✅
Gateway context capture	✅	✅
LLM call auditing	✅	✅
MCP query auditing	✅	✅
Workflow audit logging	✅	✅
Plan (MAP) audit logging	✅	✅
Media audit records (hash, MIME, PII, safety)	✅	✅
Media audit records (face, biometric, NSFW, classifiers)		✅
JSON/CSV export	✅	✅
JSONL fallback	✅	✅
CloudWatch integration	✅	✅
Prometheus metrics	✅	✅
Custom retention policies		✅
PDF/XLSX/XML export		✅
SEBI audit export		✅
RBI audit export		✅
Splunk/Datadog SIEM		✅
Compliance dashboards		✅

Best Practices

1. Enable Correlation IDs

Always pass context_id through your application flow for end-to-end tracing:

# Python SDK
ctx = await client.get_policy_approved_context(
    user_id="user-123",
    query="...",
    metadata={"request_id": "your-trace-id"}
)

# Use ctx.context_id for downstream correlation

2. Set Appropriate Retention

Match retention to your compliance requirements:

Minimum 5 years for financial services (SEBI)
Minimum 7 years for Indian banking (RBI FREE-AI)
Minimum 7 years for EU AI Act decision_chain records

3. Monitor Audit Queue Depth

Set alerts on audit queue metrics to catch database issues early:

# Prometheus alert rule
groups:
  - name: axonflow-audit
    rules:
      - alert: AuditFallbackActive
        expr: axonflow_audit_fallback_active == 1
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "AxonFlow audit using fallback storage"

4. Regular Export for Compliance

Schedule regular exports for compliance archives:

# Weekly SEBI compliance export (cron)
0 0 * * 0 curl -X POST "http://localhost:8080/api/v1/sebi/audit-export" \
  -H "Authorization: Bearer $LICENSE_KEY" \
  -d '{"format": "xml", "period": "last_week"}'

MCP Policy Enforcement - MCP connector security policies
Monitoring Overview - Prometheus metrics and dashboards
Security Best Practices - PII detection configuration
Compliance Overview - Regulatory framework support
API Reference - Full API documentation

Architecture Overview​

What Gets Logged​

Gateway Context (Pre-Check)​

LLM Call Audit​

Agent Audit Logs​

Orchestrator Audit Logs​

MCP Query Audits​

Three-Phase Policy Evaluation​

Audit Fields​

Privacy by Design​

SDK Correlation​

Example Queries​

Workflow Audit Logs​

Workflow Operations Logged​

SDK Correlation​

Example Queries​

Media Audit Records​

Plan (MAP) Audit Logs​

Plan Operations Logged​

Example Queries​

Query Examples​

Retention Policies​

Configuration​

Export Formats​

Community Edition​

Enterprise Edition​

Export API​

Fallback Mechanism​

SIEM Integration​

CloudWatch Integration (AWS)​

Prometheus Metrics​

Key Audit Metrics​

Example PromQL Queries​

API Reference​

Query Audit Logs​

Response Example​

Community vs Enterprise​

Best Practices​

1. Enable Correlation IDs​

2. Set Appropriate Retention​

3. Monitor Audit Queue Depth​

4. Regular Export for Compliance​

Related Documentation​

Architecture Overview

What Gets Logged

Gateway Context (Pre-Check)

LLM Call Audit

Agent Audit Logs

Orchestrator Audit Logs

MCP Query Audits

Three-Phase Policy Evaluation

Audit Fields

Privacy by Design

SDK Correlation

Example Queries

Workflow Audit Logs

Workflow Operations Logged

SDK Correlation

Example Queries

Media Audit Records

Plan (MAP) Audit Logs

Plan Operations Logged

Example Queries

Query Examples

Retention Policies

Configuration

Export Formats

Community Edition

Enterprise Edition

Export API

Fallback Mechanism

SIEM Integration

CloudWatch Integration (AWS)

Prometheus Metrics

Key Audit Metrics

Example PromQL Queries

API Reference

Query Audit Logs

Response Example

Community vs Enterprise

Best Practices

1. Enable Correlation IDs

2. Set Appropriate Retention

3. Monitor Audit Queue Depth

4. Regular Export for Compliance

Related Documentation