Advisory Risk Scoring for Agentic Payments
The Fraud & Risk Add-on for Agentic Payments includes an ML risk scoring layer that augments the deterministic FinCrime Policy Pack. It is advisory by design: scores escalate transactions for human review, and the deterministic layer keeps its authority regardless of what the model says. This page states exactly what scoring does, what it records, and what its published numbers do and do not mean.
Advisory risk scoring ships with the Fraud & Risk Add-on for Agentic Payments, a separately priced add-on to AxonFlow Enterprise. It is not included in the base Enterprise license, the Evaluation tier, or the source-available Community edition. It requires platform v9.18.0 or later and is currently available through the early-access program.
Posture and verdict mapping
- A request that carries the fincrime transaction context and passes validation is forwarded to the scoring service, which returns per-model risk scores, the threshold applied, and the top contributing features.
- On the decision API, a score above the configured threshold produces verdict
needs_approvaland a pending entry in the approval queue: the transaction is routed to human review. A below-threshold score does not change the verdict. - On tool-call planes (MCP query, execute, and check-input), an above-threshold score is recorded as an attributed detection on the audit record. Nothing pauses; the plane's enforcement contract is detection and attribution, as laid out in the per-plane outcome table.
- There is no path where the model alone denies or approves a transaction. Scoring can escalate toward review; only deterministic policies deny. Deterministic blocks apply regardless of the score.
Degradation is visible, never silent
If the scoring service is unreachable, exceeds its latency budget (100 milliseconds by default, configurable), or returns an error, the decision proceeds on deterministic controls alone and the audit record is stamped ml_inference_layer_status: "unavailable". A successful score stamps scored. An audit reviewer can therefore distinguish "scored and below threshold" from "not scored at all" on every decision; scoring degradation never silently looks like a clean pass.
What a scored decision records
Every scored decision writes a structured risk_score object into the standard decision record, alongside the policy attribution id fincrime_ml_fraud_score:
- the overall score and the per-model scores,
- the threshold that was applied to it,
- the top contributing features, each with its direction (
increases_riskordecreases_risk) and the engineered value it was computed from, - the feature coverage fraction, and
- the model version.
So a reviewer sees why a transaction was escalated, not just that it was, and the record lands in the same audit surface as every other decision; see audit logging and decision explainability.
Two reading notes. Feature contributions are SHAP values in the model's margin (log-odds) space, not percentages of risk; treat a near-zero magnitude as neutral regardless of its direction label. A feature's value can be the empty string when the field was not supplied, which is itself an explanation: a score can rise because something is missing.
Feature coverage is the fraction of the fields this model version can actually consume that your request supplied with parse-valid values. It is declared by the model artifact, so it cannot drift from the model. Use it to weigh score confidence: a low coverage score was computed from less evidence.
Threshold and operating point
The operating threshold ships in the model artifact and can be overridden per deployment on the scoring service (AXONFLOW_FINCRIME_SCORER_THRESHOLD, a value in 0 to 1). The v0 artifact's default threshold was tuned on the Sparkov chronological validation partition to flag at most 1 percent of transactions, a stated capacity assumption, not a measured one. The review-rate table below shows what other operating points buy; the right one is a policy decision your team makes from your own review capacity. Score thresholds will be calibrated with design partners against their real agent traffic before the defaults are treated as anything more than a starting point.
The model, honestly
The v0 scoring model (fincrime-fraud, version 0.1.0) is a gradient-boosted tree ensemble trained in house from a published method on a public dataset: the Sparkov Credit Card Transactions Fraud Detection corpus (CC0 1.0 Public Domain; 1,852,394 simulated consumer card transactions, 0.52 percent fraudulent). Every number below is measured on that corpus.
Every metric below was measured on the Sparkov corpus, which is simulated consumer card transactions. None of it was measured on agent traffic, and none of it predicts what this model will do on agent traffic. No public labeled corpus of agent-initiated transaction fraud exists yet, and we have no agent-traffic training data either. That gap is why scoring is advisory by default, why feature coverage is reported per score, and why design-partner calibration precedes any accuracy claim. We publish no detection-rate or accuracy claims for agent traffic.
Headline metrics, with the label that keeps them honest
Measured on the Sparkov corpus, on both a chronological split (train on the past, test on the future; the setting production resembles) and a random split (reported for comparison only, because it lets the model interpolate across transactions it has effectively seen):
| Split (Sparkov corpus) | Partition | Rows | Fraud rows | PR-AUC | ROC-AUC | Fraud precision | Fraud recall | Review rate | Accuracy |
|---|---|---|---|---|---|---|---|---|---|
| chronological | test | 370,479 | 1,349 | 0.4678 | 0.9680 | 22.06% | 61.16% | 1.01% | 99.07% |
| chronological | validation | 185,239 | 796 | 0.4143 | 0.9604 | 23.27% | 54.15% | 1.00% | 99.04% |
| random | test | 370,479 | 1,930 | 0.9231 | 0.9985 | 48.15% | 94.87% | 1.03% | 99.44% |
| random | validation | 185,239 | 965 | 0.9067 | 0.9984 | 48.81% | 93.68% | 1.00% | 99.46% |
The shipped artifact is the chronological one. A random split roughly doubles this model's apparent PR-AUC (0.9231 against 0.4678 on the Sparkov test partitions), which is exactly why the chronological numbers are the ones to plan around.
Why the accuracy column is nearly meaningless here
On a corpus that is 0.52 percent fraud, accuracy measures the majority class and almost nothing else:
| Sparkov test partitions | Model accuracy | Baseline accuracy | Model fraud recall | Baseline fraud recall |
|---|---|---|---|---|
| chronological split | 99.07% | 99.64% | 61.16% | 0.00% |
| random split | 99.44% | 99.48% | 94.87% | 0.00% |
The baseline is the model that predicts "not fraud" always. On the chronological split it beats the shipped model on accuracy while catching nothing. That is the entire argument for reading the minority-class columns (fraud precision and recall) first, and for distrusting any fraud product's unlabeled accuracy headline.
The review-rate trade-off
The threshold is a capacity dial. On the Sparkov chronological test partition (370,479 rows, 1,349 fraud), different target review rates buy:
| Target review rate | Flagged | Fraud caught | Fraud precision | Fraud recall |
|---|---|---|---|---|
| 0.10% | 370 | 313 | 84.59% | 23.20% |
| 0.50% | 1,852 | 693 | 37.42% | 51.37% |
| 1.00% | 3,704 | 825 | 22.27% | 61.16% |
| 2.00% | 7,409 | 962 | 12.98% | 71.31% |
| 5.00% | 18,523 | 1,126 | 6.08% | 83.47% |
The trade-off is steep and worth stating to anyone who will operate a review queue: at a 0.1 percent review rate the queue is 85 percent fraud but three quarters of fraud is missed; at 5 percent almost everything is caught and nineteen of every twenty reviews is a false positive.
What the v0 model actually consumes
The v0 model's consumable-field list, declared in the artifact and used as the feature-coverage denominator, is four fields: amount, timestamp, and the two cohort aggregates (historical_mean_amount, txn_frequency_1h). On traffic conforming to the documented context formats, the v0 model's discriminative signal comes from those four fields only: its geography and merchant features were fitted on the training corpus's own location and merchant vocabulary, which does not overlap the ISO 3166 and ISO 18245 formats the context schema specifies, so conforming values in those fields are accepted but carry no learned signal. The metrics above were measured on corpus-format inputs where those features are live; no number on this page quantifies the reduced conforming-traffic setting. A retrain aligning the geography and merchant vocabularies with the context formats is planned.
Training integrity
The training pipeline deliberately deviates from its published source method where the published construction inflates results: aggregate features and categorical encodings are fitted on the training partition only (fitting them on the whole corpus, as published, roughly doubles apparent PR-AUC on this corpus: 0.9403 against 0.4678 on the Sparkov chronological test partition), model selection never touches the test partition, and the shipped chronological headline is reported next to the flattering random split rather than instead of it. Columns carrying personal attributes in the training CSVs (names, addresses, birth dates, and similar) are never loaded; every model feature is a pure function of the documented wire contract, which has no field that could carry them.
Data handling and performance
- The scoring service runs inside your deployment and is called by the platform over an internal HTTP contract authenticated with internal service credentials. Transaction context is not sent to any third-party service; see the enablement page for the wiring.
- The platform sends the scorer transaction metadata from the documented context objects, agent runtime identifiers (agent, tool, session), and the caller-supplied cohort aggregates. Raw prompt text is not sent to the scoring service; prompt-derived signals stay in the in-process deterministic checks.
- Server-side inference latency for the shipped v0 artifact measured 4.49 milliseconds at p99 (3,000-request benchmark of the scoring service's inference path, Apple M4 Pro), against the 100 millisecond default end-to-end budget on the decision path. Network transport between the platform and the service is the deployment's share of that budget.
