Section 05 · Monitoring & Drift
Monitor AI in production.
AI governance frameworks frequently underspecify what happens after deployment. Controls are defined at design time, but operational visibility decays. AI systems are data-dependent, environment-sensitive, behaviorally evolving, and vulnerable to adversarial interaction. Monitoring is not a single metric; it is a layered discipline.
Operating principle
Continuous governance, not one-time approval. The executive dashboard shows exposure, not model metrics. Leaders don’t need confusion; they need a defensible read on where the enterprise is exposed.
Four monitoring layers
Each layer has a clear primary owner and oversight contract. The governance posture layer is the differentiator: it watches the governance system itself.
Model Performance Integrity
Primary owner
Product / AI team
Oversight
Governance visibility
What it monitors
- Accuracy / precision / recall
- False positive / false negative rates
- Calibration shifts
- Performance degradation trends
- Model confidence variance
Trigger examples
- Performance drops below defined threshold
- Sudden distribution shift
- Confidence collapse in specific segments
Data & Drift Signals
Primary owner
Platform + Product
Oversight
Security notified
What it monitors
- Input distribution shifts
- Feature drift
- Data pipeline anomalies
- Upstream data source integrity
- Training data contamination indicators
Trigger examples
- Distribution divergence beyond tolerance
- New categorical feature values
- Anomalous spikes in specific classes
Drift is not just performance degradation; it is also a potential adversarial signal.
Security & Abuse
Primary owner
Security + Platform
Oversight
Governance informed
What it monitors
- Abnormal inference patterns
- Query volume anomalies
- Model extraction patterns
- Input manipulation attempts
- Unauthorized model artifact access
- Vendor model version changes
Trigger examples
- High-frequency probing
- Repeated adversarial input sequences
- Suspicious API key usage
- Unscheduled vendor model update
Governance Posture
Primary owner
Governance
Oversight
Executive visibility (Tier 4)
What it monitors
- Risk tier registry accuracy
- Models operating without required documentation
- Unreviewed vendor integrations
- Expired risk acceptances
- Monitoring coverage gaps
- Lack of explainability documentation (Tier 3–4)
Trigger examples
- Documented risk acceptance expires without review
- Vendor changes model without notification
- Coverage gap detected on Tier 3+ system
This layer ensures governance itself doesn't decay over time.
Escalation pathway
Product → Platform → Security → Governance → Executive (if Tier 4). Escalate when Tier 3–4 systems breach drift thresholds, when automated decisions produce anomalous outcomes, when bias or fairness anomalies are detected, or when monitoring suggests potential extraction or poisoning.
Executive risk dashboard
For Tier 3–4 systems. Six signals show exposure, not model metrics.