Skip to main content

Policy Testing

Tutorial. Follow this to prove a policy behaves correctly before it is in force. It is the last step in the policy learning path, after you have written a policy with Typed Policy Authoring.

Policy testing is the difference between "the condition looks right" and "we know this control behaves correctly under real traffic."

In v11 that means three layers:

  • the test cases your document declares, which publication runs;
  • policy simulation against realistic inputs, before you activate;
  • end-to-end requests once the document is active.

If you skip any of those layers, it is easy to activate a policy that looks correct in a review but causes false positives, misses real violations, or behaves differently once it is part of the full request flow.

What You Should Test​

Good policy validation usually covers:

  • known-bad inputs you expect to catch
  • known-good inputs that must not be refused
  • borderline inputs that often produce false positives
  • realistic workload context from the actual applications using the platform

That mix matters more than having a large number of toy examples.

1. Test Cases in the Document​

Publication runs the test cases your document declares against the bundle it has just compiled, and refuses a document that declares none: a document nobody tested has not been shown to do anything. Each case supplies the attribute values your policies read and says what each policy should return for them; in the portal's publish dialog that is MATCH, NO_MATCH or UNKNOWN.

Validate first, to see every finding without publishing anything:

curl -X POST http://localhost:8080/api/v1/typed-policies/validate \
  -H "Content-Type: application/json" \
  -H "Authorization: Basic $(echo -n 'CLIENT_ID:CLIENT_SECRET' | base64)" \
  --data @my-document.json

Then publish. The response carries the digest that activation names. Typed Policy Authoring walks through both steps.

Keep each document and its test cases in source control, beside the code that depends on the policy, so a change to either is reviewed together.

2. Policy Simulation Before Activation​

With an Evaluation or Enterprise licence, Policy Simulation & Impact Report runs one input, or an impact report runs a batch, through the policies that govern your organization without affecting live traffic. In v11 it answers with the engine that enforces, so a dry run cannot disagree with enforcement.

This is especially useful when:

  • the rule is broad
  • the application is shared across teams
  • a false positive would break important workflows

3. End-To-End Request Testing​

Test cases and simulation are not enough by themselves. Once the document is active, check how it behaves in the real request pipeline.

For Proxy Mode requests, use the agent entry point:

POST /api/request
curl -X POST http://localhost:8080/api/request \
  -H "Content-Type: application/json" \
  -H "Authorization: Basic $(echo -n 'CLIENT_ID:CLIENT_SECRET' | base64)" \
  -d '{
    "query": "My SSN is 123-45-6789",
    "request_type": "chat",
    "client_id": "my-tenant"
  }'

With no user_token, the request is decided for the client credential that authenticated it. On a deployment that verifies user identity, a user_token must be a real token for a user: one that does not verify is refused with 401 before any policy runs.

A refused request says so and names the policy that decided, and its decision record carries the engine and the policy bundle that decided it; Decision Mode lists those fields. That is the level you care about in production, because it proves the policy is not just valid but effective in the live request path.

Removed in v11: pattern-test routes​

v10.x tested a regular expression on its own with POST /api/v1/static-policies/test (also served as /api/v1/system-policies/test), and the SDK testPattern / test_pattern helpers belonged to that API. The route is removed in v11: it tested a pattern outside the engine that decides. The shipped detection patterns are part of the platform, and a policy you write is tested by its document's test cases and by simulation.

A Practical Review Workflow​

A strong policy review cycle usually looks like this:

  1. validate the document and fix every finding
  2. publish it with test cases that cover known-bad, known-good and borderline inputs
  3. use simulation or an impact report if the change is broad or risky
  4. activate it (on Enterprise, someone other than its author does)
  5. run end-to-end requests through /api/request and read the decision records

That sequence gives product, platform, and security teams a much better chance of agreeing on the real effect of the rule before it reaches production users.

Rollout Checklist​

Use this page as one layer of the broader governance rollout:

Last updated on