Audit Logging

AxonFlow provides comprehensive audit logging for every AI interaction, capturing complete request/response data for compliance, debugging, and observability.

Architecture Overview

AxonFlow implements a dual-logging architecture:

Request Flow:
┌─────────────────────────────────────────────────────────────────┐
│  Your Application                                               │
└─────────────────┬───────────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────────────────────────┐
│  AxonFlow Agent                                                 │
│  ┌─────────────────┐    ┌─────────────────────────────────────┐ │
│  │ Policy Engine   │───▶│ Agent Audit Queue                   │ │
│  │ (Gateway Mode)  │    │ - Pre-check decisions               │ │
│  └─────────────────┘    │ - Policy violations                 │ │
│                         │ - PII detections                    │ │
│                         └─────────────────────────────────────┘ │
└─────────────────┬───────────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────────────────────────┐
│  AxonFlow Orchestrator                                          │
│  ┌─────────────────┐    ┌─────────────────────────────────────┐ │
│  │ LLM Routing     │───▶│ Orchestrator Audit Logger           │ │
│  │ MCP Connectors  │    │ - LLM call details                  │ │
│  │ MAP Planning    │    │ - Token usage & cost                │ │
│  └─────────────────┘    │ - Response metadata                 │ │
│                         └─────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘

What Gets Logged

Gateway Context (Pre-Check)

Every policy pre-check creates a gateway_context record:

Field	Description
`context_id`	Unique identifier for correlation
`user_id`	User making the request
`client_id`	Application or service identifier
`query`	Original prompt (with optional PII redaction)
`approved`	Whether policy allowed the request
`policies_applied`	List of policies evaluated
`policy_violations`	Any violations detected
`pii_detected`	Types of PII found (SSN, credit card, etc.)
`created_at`	Timestamp
`metadata`	Custom context (department, project, etc.)

LLM Call Audit

After LLM processing, an llm_call_audit record is created:

Field	Description
`context_id`	Links to gateway_context
`provider`	LLM provider (openai, anthropic, bedrock)
`model`	Specific model used (gpt-4, claude-3-opus)
`input_tokens`	Token count for prompt
`output_tokens`	Token count for response
`total_tokens`	Combined token usage
`latency_ms`	Response time in milliseconds
`response_summary`	Truncated response (configurable)
`cost_usd`	Estimated cost based on token pricing
`created_at`	Timestamp

Agent Audit Logs

Policy enforcement events captured by the Agent (via audit_queue.go):

Entry Type	Data Captured
`violation`	Severity (critical/high/medium/low), policy name, block reason
`metric`	Performance metrics, latency, throughput
`audit`	General audit trail entries
`gateway_context`	SDK pre-check context (synchronous write)
`llm_call_audit`	SDK LLM call audit (synchronous write)

Orchestrator Audit Logs

All requests captured by the Orchestrator (via audit_logger.go):

Field	Description
`user_id`, `user_email`, `user_role`	User identity
`client_id`, `tenant_id`	Client/tenant isolation
`request_type`	Type of operation
`query`, `query_hash`	Request content (hashed for deduplication)
`policy_decision`	allowed, blocked, redacted, error
`provider`, `model`	LLM provider and model used
`response_time_ms`, `tokens_used`, `cost`	Performance and billing
`compliance_flags`	hipaa_relevant, gdpr_applicable, sox_relevant, pii_access
`security_metrics`	risk_score, query_complexity (low/medium/high)

Retention Policies

AxonFlow supports configurable retention periods to meet regulatory requirements:

Framework	Required Retention	AxonFlow Support
SEBI AI/ML	5 years (1825 days)	✅ Enterprise
RBI FREE-AI	7 years (2555 days)	✅ Enterprise
EU AI Act	7 years (2555 days) for decision_chain	✅ Enterprise
HIPAA	6 years (2190 days)	✅ Enterprise
Custom	Configurable	✅ All editions

Configuration

Retention is configured per-log-type:

# axonflow-config.yaml
audit:
  retention:
    gateway_contexts: 365     # Days to retain pre-check logs
    llm_call_audits: 365      # Days to retain LLM call logs
    agent_audit_logs: 2555    # 7 years for RBI compliance
    orchestrator_logs: 1825   # 5 years for SEBI compliance

Export Formats

Community Edition

Format	Use Case
JSON	Programmatic access, API responses
CSV	Spreadsheet analysis, basic reporting

Enterprise Edition

Format	Use Case
JSON	Programmatic access, API responses
CSV	Spreadsheet analysis, basic reporting
PDF	Compliance reports, board presentations
XLSX	Advanced Excel analysis with formatting
XML	SEBI regulatory submission format

Export API

# Export last 30 days of audit logs (JSON)
curl -X GET "http://localhost:8080/api/audit/export?days=30&format=json" \
  -H "Authorization: Bearer $AXONFLOW_API_KEY"

# Export for SEBI compliance (Enterprise)
curl -X GET "http://localhost:8081/api/v1/sebi/audit-export?start=2024-01-01&end=2024-12-31" \
  -H "Authorization: Bearer $AXONFLOW_LICENSE_KEY"

Fallback Mechanism

If the database is temporarily unavailable, audit logs are written to a local JSONL file:

/var/log/axonflow/audit_fallback.jsonl

On service restart, RecoverFromFallback() automatically replays entries back to the database:

Reads JSONL file line-by-line
Retries failed entries with exponential backoff
Truncates fallback file when recovery succeeds

SIEM Integration

AxonFlow integrates with enterprise security information and event management (SIEM) systems:

Platform	Integration Method	Status
CloudWatch	Native (AWS deployments)	Community + Enterprise
Splunk	HTTP Event Collector	Enterprise
Datadog	Log forwarding	Enterprise
Elastic/ELK	Logstash integration	Enterprise

CloudWatch Integration (AWS)

Audit logs are automatically forwarded to CloudWatch when deployed in AWS:

# View audit logs in CloudWatch
aws logs filter-log-events \
  --log-group-name "/ecs/axonflow-agent" \
  --filter-pattern '"audit"'

Prometheus Metrics

AxonFlow exposes 80+ Prometheus metrics for audit observability:

Key Audit Metrics

Metric	Type	Description
`axonflow_audit_logs_total`	Counter	Total audit log entries
`axonflow_audit_write_latency_ms`	Histogram	Time to write audit log
`axonflow_audit_fallback_active`	Gauge	1 if using fallback mode
`axonflow_policy_violations_total`	Counter	Total policy violations
`axonflow_pii_detections_total`	Counter	PII detections by type

Example PromQL Queries

# Audit write latency P95
histogram_quantile(0.95, rate(axonflow_audit_write_latency_ms_bucket[5m]))

# Policy violations per minute
rate(axonflow_policy_violations_total[1m])

# PII detections by type
sum by (pii_type) (increase(axonflow_pii_detections_total[1h]))

API Reference

Query Audit Logs

# Search audit logs (POST with criteria)
POST /audit/search
{
  "user_email": "[email protected]",
  "client_id": "client-123",
  "start_time": "2024-01-01T00:00:00Z",
  "end_time": "2024-01-31T23:59:59Z",
  "request_type": "llm_call",
  "limit": 100
}

# Get tenant-specific audit logs
GET /audit/tenant/{tenant_id}

Response Example

{
  "logs": [
    {
      "id": "audit_abc123",
      "context_id": "ctx_xyz789",
      "user_id": "user-123",
      "event_type": "policy_violation",
      "policy_name": "pii_ssn_detection",
      "query_hash": "sha256:abc...",
      "pii_types": ["ssn"],
      "blocked": true,
      "created_at": "2024-01-15T10:30:00Z"
    }
  ],
  "total": 1523,
  "limit": 100,
  "offset": 0
}

Community vs Enterprise

Feature	Community	Enterprise
Basic audit logging	✅	✅
Gateway context capture	✅	✅
LLM call auditing	✅	✅
JSON/CSV export	✅	✅
JSONL fallback	✅	✅
CloudWatch integration	✅	✅
Prometheus metrics	✅	✅
Custom retention policies		✅
PDF/XLSX/XML export		✅
SEBI audit export		✅
RBI audit export		✅
Splunk/Datadog SIEM		✅
Compliance dashboards		✅

Best Practices

1. Enable Correlation IDs

Always pass context_id through your application flow for end-to-end tracing:

# Python SDK
ctx = await client.get_policy_approved_context(
    user_id="user-123",
    query="...",
    metadata={"request_id": "your-trace-id"}
)

# Use ctx.context_id for downstream correlation

2. Set Appropriate Retention

Match retention to your compliance requirements:

Minimum 5 years for financial services (SEBI)
Minimum 7 years for Indian banking (RBI FREE-AI)
Minimum 7 years for EU AI Act decision_chain records

3. Monitor Audit Queue Depth

Set alerts on audit queue metrics to catch database issues early:

# Prometheus alert rule
groups:
  - name: axonflow-audit
    rules:
      - alert: AuditFallbackActive
        expr: axonflow_audit_fallback_active == 1
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "AxonFlow audit using fallback storage"

4. Regular Export for Compliance

Schedule regular exports for compliance archives:

# Weekly SEBI compliance export (cron)
0 0 * * 0 curl -X POST "http://localhost:8081/api/v1/sebi/audit-export" \
  -H "Authorization: Bearer $LICENSE_KEY" \
  -d '{"format": "xml", "period": "last_week"}'

Monitoring Overview - Prometheus metrics and dashboards
Security Best Practices - PII detection configuration
Compliance Overview - Regulatory framework support
API Reference - Full API documentation

Architecture Overview​

What Gets Logged​

Gateway Context (Pre-Check)​

LLM Call Audit​

Agent Audit Logs​

Orchestrator Audit Logs​

Retention Policies​

Configuration​

Export Formats​

Community Edition​

Enterprise Edition​

Export API​

Fallback Mechanism​

SIEM Integration​

CloudWatch Integration (AWS)​

Prometheus Metrics​

Key Audit Metrics​

Example PromQL Queries​

API Reference​

Query Audit Logs​

Response Example​

Community vs Enterprise​

Best Practices​

1. Enable Correlation IDs​

2. Set Appropriate Retention​

3. Monitor Audit Queue Depth​

4. Regular Export for Compliance​

Related Documentation​