Skip to main content

Policy Simulation, Impact Reports & Conflict Detection

Policy Simulation and Impact Report are the safest way to answer a question every serious team eventually asks:

"If we tighten this policy, what will actually happen to our traffic?"

That matters because the hardest policy mistakes are rarely syntax errors. They are:

  • false positives that block legitimate requests
  • false negatives that let risky traffic through
  • over-broad rules that surprise downstream teams
  • changes that look correct in review but behave differently at runtime

AxonFlow gives you three tools for this:

  • Policy Simulation runs all active policies against one input as a dry run.
  • Impact Report tests one policy against a batch of representative inputs and returns match and block rates.
  • Policy Conflict Detection checks active policies for contradictory actions, shadowed policies, and redundant rules.

All three features are available starting with the Evaluation tier.

Why Teams Use This Before Production​

Policy simulation is especially useful when:

  • a security team is rolling out stricter data-loss prevention rules
  • a platform team is adding tenant-specific policy overrides
  • a regulated workflow is introducing require_approval or stricter blocking
  • a product team wants evidence that a governance change will not break legitimate prompts

For senior engineers, this is the feature that turns policy work from guesswork into repeatable validation.

Policy Simulation​

Policy Simulation evaluates the current active policy set against a single input without enforcing the result on live traffic and without writing audit records for that simulated run.

Endpoint​

POST /api/v1/policies/simulate

Request​

curl -X POST "http://localhost:8080/api/v1/policies/simulate" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Transfer $500,000 from account 4532-XXXX-XXXX-1234 to offshore account",
    "request_type": "finance",
    "context": {
      "user_id": "analyst-42",
      "department": "finance"
    }
  }'

Request Fields​

FieldRequiredDescription
queryYesInput text to evaluate
request_typeNoRequest classification used during policy evaluation
userNoUser context for policy evaluation
clientNoClient or application context
contextNoArbitrary key-value context passed into evaluation

Response​

{
  "allowed": false,
  "applied_policies": ["sys_pii_credit_card", "high-value-transaction-oversight"],
  "risk_score": 0.92,
  "required_actions": ["redact", "require_approval"],
  "processing_time_ms": 4,
  "total_policies": 83,
  "dry_run": true,
  "simulated_at": "2026-03-01T10:30:00Z",
  "tier": "evaluation",
  "daily_usage": {
    "used": 12,
    "limit": 300
  }
}

What The Response Tells You​

FieldMeaning
allowedwhether the simulated request would be allowed
applied_policiespolicies that matched
risk_scoreaggregate risk score
required_actionsactions the runtime would have taken
processing_time_msevaluation latency
total_policiesactive policies evaluated during the dry run
dry_runalways true for this endpoint
simulated_attimestamp of the run
tiercurrent license tier
daily_usagequota usage when the tier has limits

Impact Report​

Impact Report is for a different question:

"If I target this one policy against a representative sample, how often will it match and how often will it block?"

Endpoint​

POST /api/v1/policies/impact-report

Request​

curl -X POST "http://localhost:8080/api/v1/policies/impact-report" \
  -H "Content-Type: application/json" \
  -d '{
    "policy_id": "high-value-transaction-oversight",
    "inputs": [
      {"query": "Transfer $50,000 to offshore account in Cayman Islands"},
      {"query": "Move funds to our shell company subsidiary"},
      {"query": "What is the weather in Tokyo?"},
      {"query": "Transfer $200 to savings account"}
    ]
  }'

Response​

{
  "policy_id": "high-value-transaction-oversight",
  "policy_name": "High Value Transaction Oversight",
  "total_inputs": 4,
  "matched": 2,
  "blocked": 2,
  "match_rate": 0.5,
  "block_rate": 0.5,
  "results": [
    {
      "input_index": 0,
      "matched": true,
      "blocked": true,
      "actions": ["block"]
    },
    {
      "input_index": 1,
      "matched": true,
      "blocked": true,
      "actions": ["block"]
    },
    {
      "input_index": 2,
      "matched": false,
      "blocked": false
    },
    {
      "input_index": 3,
      "matched": false,
      "blocked": false
    }
  ],
  "processing_time_ms": 12,
  "generated_at": "2026-03-01T10:35:00Z",
  "tier": "evaluation"
}

This report is valuable because it gives you both the aggregate rates and the per-input result list. Teams usually need both: the summary for decision-making and the per-input detail for debugging false positives.

Policy Conflict Detection​

Policy Conflict Detection answers a third rollout question:

"Do any of our active policies contradict, shadow, or duplicate each other?"

This is useful after a team adds tenant-specific policies, changes priorities, or introduces a stricter blocking rule on top of existing logging or alerting policies.

Endpoint​

POST /api/v1/policies/conflicts

The Agent proxies this endpoint to the Orchestrator, so teams can call it through the same runtime entry point they use for simulation.

Request​

The request body is optional. Send an empty body to compare the active policy set, or pass policy_id to return only conflicts involving one policy.

curl -X POST "http://localhost:8080/api/v1/policies/conflicts" \
  -H "Content-Type: application/json" \
  -H "X-Tenant-ID: tenant_123" \
  -d '{
    "policy_id": "high-value-transaction-oversight"
  }'

Response​

{
  "conflicts": [
    {
      "policy_a": {
        "id": "policy_block_high_value_transfer",
        "name": "Block High Value Transfers",
        "type": "tenant"
      },
      "policy_b": {
        "id": "policy_log_high_value_transfer",
        "name": "Log High Value Transfers",
        "type": "tenant"
      },
      "conflict_type": "contradictory_action",
      "description": "Policies have contradictory actions on overlapping field \"query\": one blocks while the other allows or logs",
      "severity": "high",
      "overlapping_field": "query"
    }
  ],
  "total_policies": 42,
  "conflict_count": 1,
  "checked_at": "2026-03-01T10:40:00Z",
  "tier": "evaluation"
}

The service checks enabled policies for the tenant. The current implementation evaluates the first 1,000 active policies returned by the policy service.

Conflict Types​

TypeSeverityWhat It Means
contradictory_actionHighPolicies overlap on a field and value, but one blocks while the other only logs or alerts.
shadowMediumA higher-priority policy has a less restrictive matching condition that can make a lower-priority policy unreachable.
redundantLowTwo policies have identical conditions and identical actions.

Auth, Tier, And Limits​

Conflict detection uses the same feature gate and daily quota as policy simulation:

  • Community receives FEATURE_REQUIRES_EVALUATION_LICENSE.
  • Evaluation can run 300 total simulation or conflict-check requests per day.
  • Enterprise has an unlimited simulation and conflict-check quota.

The request must include X-Tenant-ID unless the tenant has already been placed in request context by the runtime. If the body is present, it must be valid JSON.

Tier Limits​

CapabilityCommunityEvaluationEnterprise
Policy Simulation❌300/dayUnlimited
Policy Conflict Detection❌Shared 300/day quotaUnlimited
Impact Report inputs per run❌50100
Scheduled or deeper operational workflows❌❌✅

These limits are verified against the platform tier support. Community does not expose these features. Evaluation is intentionally strong enough for meaningful validation work. Enterprise is the fit once simulations and impact analysis become part of an ongoing governance program.

Teams usually get the best results with a three-step loop:

  1. Run Policy Simulation against known risky and known-safe prompts.
  2. Run Policy Conflict Detection after changing priorities or adding overlapping policies.
  3. Run Impact Report against a curated input set that looks like real traffic.
  4. Adjust the rule before rollout if match or block rates are too broad.

That sounds simple, but it is one of the most important habits for avoiding "security policy broke the product" incidents.

Building A Useful Test Input Set​

Impact reports are only as good as the inputs you feed them. The strongest test sets usually include:

  • clearly unsafe prompts you expect to catch
  • clearly legitimate prompts you must not break
  • borderline prompts that often cause false positives
  • examples from several teams or applications if the platform is shared

That mix is what makes the report useful to both security and product owners. A report built only from obviously bad prompts can make a policy look perfect when it is actually too broad for real traffic.

Example Engineering Review Pattern​

A solid internal review process often looks like this:

  • security or platform proposes a new rule
  • the owning application team supplies representative prompts
  • simulation confirms that the highest-risk cases are caught
  • conflict detection confirms the new rule does not contradict or shadow existing active policies
  • impact report confirms the rule does not over-match legitimate cases
  • the team only then enables the rule for production traffic

That process is useful whether you are running one sensitive workflow or a company-wide AI platform.

Why This Often Triggers An Upgrade​

Policy simulation is one of the easiest features for teams to justify internally because it removes avoidable governance risk:

  • security gets evidence instead of assumptions
  • product teams get fewer surprise breakages
  • platform teams get a repeatable rollout pattern

Once that workflow becomes part of normal release practice, Evaluation often becomes the minimum viable operating tier and Enterprise becomes attractive when teams want the rest of the governance operating model around it.

Last updated on