Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.complyhat.ai/llms.txt

Use this file to discover all available pages before exploring further.

Production data distributions change over time — the people your model scores today may look different from the people in your training set. ComplyHat’s drift engine compares production snapshots against a baseline distribution using Population Stability Index (PSI) and the Kolmogorov-Smirnov test to quantify how much the input distribution has shifted. When drift exceeds configurable thresholds, ComplyHat raises an alert so your team can investigate before a regulator does. For the mathematical definitions of PSI and KS, see methodology.

​
PSI thresholds at a glance

PSI rangeInterpretationRecommended action
< 0.10No material changeContinue normal monitoring
0.10 – 0.25Moderate driftIncrease monitoring frequency; document in model risk log
>= 0.25Significant driftInvestigate root cause; consider revalidation

​
Create a drift monitor

Call drift with mode: "monitors_create". Specify the model_id, the monitor_type, the metric_name (the engine statistic), and your warning and critical thresholds. 
{
  "tool": "drift",
  "arguments": {
    "mode": "monitors_create",
    "model_id": "mdl_01j9z...",
    "name": "Input distribution — fraud-detector-v2",
    "monitor_type": "data_drift",
    "metric_name": "psi",
    "warning_threshold": 0.10,
    "critical_threshold": 0.25,
    "window_size_hours": 168,
    "check_interval_minutes": 60,
    "auto_revalidate": true
  }
}
The monitor_type field selects the drift family:
ValueWhat it tracks
data_driftInput feature distribution shift
concept_driftShift in the relationship between inputs and outputs
performance_driftDegradation in model accuracy or calibration
behavior_driftChanges in prediction distribution without labeled ground truth
fairness_driftChanges in bias metric values over time
Set auto_revalidate: true to automatically queue a revalidation workflow when the critical threshold is breached. The response includes a monitor_id (mon_...) you use for subsequent status checks.

​
Submit a production snapshot

Your host agent submits snapshots from production. Call mode: "snapshots_submit" with the monitor_id and the distribution data for the current window: 
{
  "tool": "drift",
  "arguments": {
    "mode": "snapshots_submit",
    "monitor_id": "mon_01p7...",
    "snapshot": {
      "feature_distributions": {
        "age": { "mean": 41.2, "std": 12.8, "bins": [0.12, 0.24, 0.31, 0.22, 0.11] },
        "income_band": { "counts": { "low": 312, "mid": 891, "high": 204 } }
      },
      "row_count": 1407,
      "window_start": "2026-04-01T00:00:00Z",
      "window_end": "2026-04-07T23:59:59Z"
    }
  }
}
The drift engine evaluates the snapshot against the baseline, computes PSI and KS statistics, and raises an alert if any threshold is breached.

​
Check monitor status

Retrieve the current status of a monitor — including the latest PSI value and any active alerts — with mode: "monitors_status": 
{
  "tool": "drift",
  "arguments": {
    "mode": "monitors_status",
    "monitor_id": "mon_01p7..."
  }
}

​
List drift alerts for a model

See all drift alerts across all monitors for a given model with mode: "alerts_list_by_model": 
{
  "tool": "drift",
  "arguments": {
    "mode": "alerts_list_by_model",
    "model_id": "mdl_01j9z..."
  }
}
Each alert includes the monitor that triggered it, the metric value that crossed the threshold, and a timestamp.

​
Create alert rules

Use the alerts tool to configure rules that govern when and how you are notified. Create a rule with mode: "rules_create": 
{
  "tool": "alerts",
  "arguments": {
    "mode": "rules_create",
    "model_id": "mdl_01j9z...",
    "trigger": "drift_critical",
    "channel": "email",
    "recipients": ["model-risk@example.com"],
    "severity": "high"
  }
}
List existing rules with mode: "rules_list" and acknowledge or resolve triggered alerts with mode: "update": 
{
  "tool": "alerts",
  "arguments": {
    "mode": "update",
    "alert_id": "alrt_02r...",
    "state": "acknowledged"
  }
}
The KS test uses a dual gate — drift is flagged only when p < 0.05 and KS > 0.10. This prevents large production samples from triggering spurious alerts on trivially small but statistically significant shifts.
Next steps: Read the methodology page for the full PSI and KS formulas, or see all drift and alerts tool modes in the tool reference.